run-device-tests

Run Device Tests Skill

Build and run .NET MAUI device tests locally on iOS simulators, MacCatalyst, Android emulators, or Windows.

Platform Support

Host OS	Supported Platforms
macOS	ios, maccatalyst, android
Windows	android, windows

Tools Required

This skill uses bash together with pwsh (PowerShell 7+) to run the PowerShell scripts. Requires:

• xharness - Global dotnet tool for running tests on iOS/MacCatalyst/Android
• dotnet - .NET SDK with platform workloads installed
• iOS/MacCatalyst: Xcode with iOS simulators
• Android: Android SDK with emulator
• Windows: Windows SDK

Dependencies

This skill uses shared infrastructure scripts:

• .github/scripts/shared/Start-Emulator.ps1 - Detects and boots iOS simulators / Android emulators
• .github/scripts/shared/shared-utils.ps1 - Common utility functions

These are automatically loaded by the Run-DeviceTests.ps1 script.

When to Use

• User wants to run device tests locally
• User wants to verify iOS/MacCatalyst/Android/Windows compatibility
• User wants to test on a specific iOS version (e.g., iOS 26)
• User asks "run device tests for Controls/Core/Essentials/Graphics"
• User asks "test on iOS simulator" or "test on Android emulator"
• User asks "run device tests on MacCatalyst"
• User wants to run only specific test categories (e.g., "run Button tests")

Available Test Projects

Project	Path
Controls	src/Controls/tests/DeviceTests/Controls.DeviceTests.csproj
Core	src/Core/tests/DeviceTests/Core.DeviceTests.csproj
Essentials	src/Essentials/test/DeviceTests/Essentials.DeviceTests.csproj
Graphics	src/Graphics/tests/DeviceTests/Graphics.DeviceTests.csproj
BlazorWebView	src/BlazorWebView/tests/DeviceTests/MauiBlazorWebView.DeviceTests.csproj

Scripts

All scripts are in .github/skills/run-device-tests/scripts/

Run Device Tests (Full Workflow)

# Run Controls device tests on iOS simulator (default on macOS)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

# Run Core device tests on MacCatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform maccatalyst

# Run Controls device tests on Android emulator
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

# Run Controls device tests on Windows (default on Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

# Run on specific iOS version
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform ios -iOSVersion 26

# Run with test filter
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Run one exact Core test class on Windows
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform windows -IncludeClasses "Microsoft.Maui.DeviceTests.WindowHandlerTests"

# Run other test projects
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Essentials -Platform android
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Graphics -Platform maccatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project BlazorWebView -Platform ios

Build Only (No Test Run)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -BuildOnly

Force a Full Rebuild

Use -Rebuild when source files changed after an earlier build in the same worktree, such as an A/B verification run:

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform windows -Rebuild

List Available Simulators/Emulators

# iOS simulators
xcrun simctl list devices available

# Android emulators
emulator -list-avds

Workflow

1. Run tests: scripts/Run-DeviceTests.ps1 -Project <name> -Platform <platform> automatically detects/boots device, builds, and runs tests
2. Check results: Look at the console output or artifacts/log/ directory for detailed test results

Output

• Build artifacts: artifacts/bin/<Project>.DeviceTests/<Configuration>/<tfm>/<rid>/
• Test logs: artifacts/log/
• Test results summary is printed to console

Prerequisites

• xharness global tool: dotnet tool install --global Microsoft.DotNet.XHarness.CLI
• .NET SDK with platform workloads
• iOS/MacCatalyst: Xcode with simulators installed
• Android: Android SDK with emulator configured
• Windows: Windows SDK
• For iOS 26: macOS Tahoe (26) with Xcode 26

Examples

# Quick test run for Controls on iOS (default on macOS)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

# Test on MacCatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform maccatalyst

# Test on Android emulator (works on both macOS and Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

# Test on Windows (default on Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

# Test on iOS 26 specifically
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -iOSVersion 26

# Run only Button category tests on iOS
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Build for Android without running tests
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform android -BuildOnly

Notes

• The script automatically detects and boots an iOS simulator / Android emulator using the shared Start-Emulator.ps1 infrastructure
• Default iOS simulator is iPhone Xs with iOS 18.5 (same as UI tests)
• Default Android emulator priority: API 30 Nexus > API 30 > Nexus > First available
• MacCatalyst runs directly on the Mac (no simulator needed)
• Windows tests run directly on the local machine
• Simulator/emulator selection and boot logic is handled by .github/scripts/shared/Start-Emulator.ps1
• xharness manages test execution and reporting for iOS/MacCatalyst/Android
• Class-filtered XHarness retries use isolated child output directories; Android also uses a per-run result filename so stale diagnostics or device-side XML cannot be reused
• Windows runs the built device-test app directly and reads its xUnit XML results, matching eng/devices/windows.cake

Test Filtering

The -TestFilter parameter allows running specific test categories instead of all tests. This is useful for quick iteration during development.

Filter Syntax

Format	Description	Example
Category=X	Run only category X	Category=Button
SkipCategories=X,Y,Z	Skip categories X, Y, Z	SkipCategories=Shell,CollectionView

Examples

# Run only Button tests on iOS
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Run only Button tests on Android  
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android -TestFilter "Category=Button"

# Skip heavy test categories
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "SkipCategories=Shell,CollectionView,HybridWebView"

How Test Filtering Works

Test filtering is implemented in src/Core/tests/DeviceTests.Shared/DeviceTestSharedHelpers.cs:

Platform	How Filter is Passed	How Filter is Read
iOS/MacCatalyst	--set-env=TestFilter=...	NSProcessInfo.ProcessInfo.Environment["TestFilter"]
Android	--arg TestFilter=...	MauiTestInstrumentation.Current.Arguments.GetString("TestFilter")
Windows Controls	App argument selects discovered category index	ControlsHeadlessTestRunner category loop
Windows non-Controls class filter	Per-child NUNIT_SKIPPED_CLASSES plus the normal runner	XHarness ApplicationOptions class include

The Copilot Gate combines -TestFilter with -IncludeClasses when it knows the exact test class. On Windows, Controls still requires category discovery, while Core, Essentials, Graphics, and BlazorWebView bypass discovery and use XHarness's native class include. The result parser rejects output containing any unrelated class. Exact-class Windows runs, including scoped Controls category runs, are capped at 10 minutes per attempt; the Gate retries a timeout three times before treating repeated target-only timeouts as deterministic evidence.

Available Test Categories

Common categories in Controls.DeviceTests:

• Button, Label, Entry, Editor - Individual control tests
• CollectionView, ListView, CarouselView - Collection controls (heavy)
• Shell, Navigation, TabbedPage - Navigation tests (heavy)
• Layout, FlexLayout - Layout tests
• Memory - Memory leak tests
• Accessibility - Accessibility tests
• Gesture - Gesture recognition tests

To see all categories, check src/Controls/tests/DeviceTests/TestCategory.cs.

Troubleshooting

Xcode Version Mismatch

If you see errors about Xcode version mismatch (e.g., "Xcode 26.2 installed but 26.0 required"):

# Use -SkipXcodeVersionCheck to bypass validation
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -SkipXcodeVersionCheck

MacCatalyst App Bundle Not Found

MacCatalyst apps use display names (e.g., "Controls Tests.app") not assembly names. The script handles this automatically by searching for .app bundles in the outpu

---

Content truncated.