-
Notifications
You must be signed in to change notification settings - Fork 1.9k
[ci] Run xharness DeviceTests on dnceng-public Helix #30544
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
rmarinho
wants to merge
108
commits into
main
Choose a base branch
from
devicetests-dnceng
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rmarinho
commented
Jul 10, 2025
@@ -3,8 +3,8 @@ namespace Microsoft.Maui.TestUtils.DeviceTests.Runners | |||
{ | |||
public class HeadlessRunnerOptions | |||
{ | |||
public string TestResultsFilename { get; set; } = "TestResults.xml"; | |||
public string TestResultsFilename { get; set; } = "testResults.xml"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Helix looks at this to grab the results
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Change
Enable a way to run device tests on Helix with xharness.
Right now we going to support both modes, helix and non helix as we are doing the move.
About helix
Check available queues https://helix.dot.net (right now we just going to use open)
Documentation about xharness on Helix https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/Readme.md#android-apk-payloads
Run locally from the Mac
The arcade system uses ./build.sh and /build.cmd as the start point, and it will look at the Build.props file for information of what projects to build
This pull request introduces several updates to streamline device test configurations and address project path inconsistencies across multiple solution files. Key changes include modifying solution file references for the
MauiBlazorWebView.DeviceTests
project, enhancing build and test pipelines for device tests, and adding support for Helix and XHarness testing.Updates to solution files:
MauiBlazorWebView.DeviceTests
in multiple solution files (Microsoft.Maui-dev.sln
,Microsoft.Maui-mac.slnf
,Microsoft.Maui-vscode.sln
,Microsoft.Maui-windows.slnf
,Microsoft.Maui.LegacyControlGallery.sln
, andMicrosoft.Maui.sln
) to usesrc\BlazorWebView\tests\DeviceTests\MauiBlazorWebView.DeviceTests.csproj
instead ofsrc\BlazorWebView\tests\MauiDeviceTests\MauiBlazorWebView.DeviceTests.csproj
. [1] [2] [3] [4] [5] [6]Enhancements to build and test pipelines:
eng/Build.props
to conditionally include device test projects in the build process whenBuildDeviceTests
is set totrue
. This includes projects forControls
,Core
,Graphics
,Essentials
, andMauiBlazorWebView
device tests.eng/helix_xharness.proj
, to support Helix and XHarness testing for iOS and Android platforms. This includes configurations for test scenarios, payload directories, and handling app bundles and APKs.eng/pipelines/arcade/stage-device-tests.yml
, for running device tests on iOS and Android using Helix. This includes parameters for pool configurations, build settings, and artifact publishing.Pipeline variable and configuration updates:
eng/pipelines/arcade/variables.yml
to update the_buildScriptMacOS
variable, changing-warnAsError 0
to-warnAsError false
.eng/pipelines/common/variables.yml
to include a new condition for themaui-pr-devicetests
build definition.eng/pipelines/device-tests.yml
for pool providers.These changes collectively enhance the project's ability to manage device tests more efficiently while ensuring consistency in project paths across solution files.
Issues Fixed
Fixes #30466