xUnit v3 upgrade #960

aaronpowell · 2025-11-11T05:04:05Z

Upgrading the tests to use xUnit v3, which is what is used by aspire now too.

Close #961

Co-authored-by: aaronpowell <[email protected]>

…ET 10 Co-authored-by: aaronpowell <[email protected]>

Co-authored-by: aaronpowell <[email protected]>

… but the MassTransit.ActiveMQ package isn't updated yet Tracked: MassTransit/MassTransit#6133

…ss ambiguous referencing

…able

I feel dirty doing that...

Youssef1313 · 2025-11-16T22:11:32Z

.github/workflows/tests.yaml

      - name: Run tests
        run: >-
-          dotnet test ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }}
+          dotnet run ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }}


You need to:

Remove --logger "console;verbosity=normal".

Replace --logger trx with --report-trx (and ensure Microsoft.Testing.Extensions.TrxReport is referenced)

Replace all the blame-related args with --crashdump, --hangdump, and --hangdump-timeout 7m

--results-directory is changing the meaning a little bit with this PR. Previously it would be relative to current working directory ${{ github.workspace }} while with this PR it will be relative to the test executable. So either pass the full path, or keep using dotnet test which will handle the path transformation for you.

--collect "XPlat Code Coverage" needs to be replaced with --coverage (and ensure Microsoft.Testing.Extensions.CodeCoverage is referenced)

Youssef1313 · 2025-11-16T22:11:51Z

.github/workflows/tests.yaml

+          --report-trx
+          --report-trx-filename "${{ matrix.name }}-${{ matrix.os }}.trx"
+          --ignore-exit-code 8
+          -- RunConfiguration.CollectSourceInformation=true


This shouldn't be needed.

Youssef1313 · 2025-11-16T22:12:31Z

Directory.Packages.props

-    <PackageVersion Include="xunit.runner.visualstudio" Version="2.8.2" />
-    <PackageVersion Include="xunit.extensibility.execution" Version="2.9.3" />
-    <PackageVersion Include="Microsoft.DotNet.XUnitExtensions" Version="9.0.0-beta.24568.1" />
+    <PackageVersion Include="xunit.v3" Version="3.2.0" />


nit: there is now xunit.v3.mtp-v2 package.

Youssef1313 · 2025-11-16T22:12:48Z

Directory.Packages.props

+    <PackageVersion Include="xunit.v3" Version="3.2.0" />
+    <PackageVersion Include="xunit.v3.assert" Version="3.2.0" />
+    <PackageVersion Include="xunit.v3.extensibility.core" Version="3.2.0" />
+    <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />


nit: no longer needed if you don't care about VSTest support.

Suggested change

<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />

Youssef1313 · 2025-11-16T22:13:33Z

tests/.runsettings

  <RunConfiguration>
    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=unsupported-platform</TestCaseFilter>
+    <TestCaseFilter>category!=failing</TestCaseFilter>


runsettings isn't supported with MTP.

Youssef1313 · 2025-11-16T22:14:49Z

tests/Directory.Build.props

+         https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --ignore-exit-code 8</TestRunnerAdditionalArguments>
+
+    <!-- <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner> -->


Youssef1313

You need to also update global.json to specify the test runner as MTP, and probably ensure it requires .NET 10.

…with MTP support

It seems there are two different paths for resolving the reference expression that _should_ return the same, but in the tests they don't seem to, so we'll force eval

Evangelink · 2025-11-26T15:00:08Z

@aaronpowell please feel free ping @Youssef1313 or I so we can move forward here.

…/Aspire into xunit-3-upgrade

…ng the app host

Evangelink · 2025-12-11T12:50:29Z

examples/powershell/CommunityToolkit.Aspire.PowerShell.AppHost/Program.cs

 // outputs "the sum of 2 and 3 is 5"
 var script2 = ps.AddScript("script2", """
-    & ./scripts/script.ps1 @args
+    & ./Scripts/script.ps1 @args


I don't think that Powershell is case sensitive (cc @nohwnd)

This is up to the file system not up to powershell. If this needs to run on linux and the filesystem is case sensitive then ./scripts and ./Scripts are two different paths. On windows we typically don't use case sensitive file systems.

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_case-sensitivity?view=powershell-7.5

Evangelink · 2025-12-11T12:51:35Z

tests/CommunityToolkit.Aspire.Hosting.McpInspector.Tests/AppHostTests.cs


 // Forcing linux only due to: https://github.com/modelcontextprotocol/inspector/issues/893
-[RequiresLinux]
+[RequiresWindows]


How was this working before?

I have no idea why I made that change, I'm going to revert it 🤣

Evangelink

@aaronpowell the latest errors are weird, it's complaining about a file that doesn't exist when I pull this branch.

I have also noticed the following missing changes:

In global.json, add:

"test": {
    "runner": "Microsoft.Testing.Platform"
  }

In Directory.Packages.props and tests/Directory.Build.props remove:

coverlet.collector -> not compatible with MTP, instead use MS coverage
Microsoft.NET.Test.Sdk -> VSTest testhost part, no longer needed as you are moving to MTP
xunit.runner.visualstudio -> VSTest adapter for xUnit, also no longer needed.

I'd suggest to be careful with the --ignore-exit-code 8 set on all test projects because you are allowing all projects not to run tests without failure. I don't know if you have some other guard in place but if you don't then you would potentially no longer run any test without noticing.

Copilot AI and others added 19 commits November 6, 2025 05:00

Initial plan

4e9036e

Add .NET 10 (preview) as a TFM for multi-targeting builds

7236d38

Co-authored-by: aaronpowell <[email protected]>

Change default TFM to .NET 10 and update GitHub Actions to install .N…

303a76f

…ET 10 Co-authored-by: aaronpowell <[email protected]>

Update devcontainer to install .NET 10 as preview

c6b19fc

Co-authored-by: aaronpowell <[email protected]>

Add .NET 10 to setup-runtimes-caching action

1e4e20d

Co-authored-by: aaronpowell <[email protected]>

Fixing some versioning

ac50650

Hard coding an update to use [email protected] as 2.2.0 has a CVE…

9441794

… but the MassTransit.ActiveMQ package isn't updated yet Tracked: MassTransit/MassTransit#6133

Fixing some projects that specified their own TFM and trying to addre…

ef89379

…ss ambiguous referencing

Using an improved approach approach to adding system.linq.asyncenumer…

e66de71

…able

Updating how dev certs are setup

51e57ba

Turns out you can't upgrade around the CVE, so we now suppress it

f3ff1ff

I feel dirty doing that...

Update to xunit v3

df16921

Disabling JSON schema tests as we don't generate those files

1ac2ad3

Some adjustments to not be so aspire configured

5840c2e

Oh, some of that was used

dcfbd73

Adding filter for failing tests to be skipped

102aaab

using dotnet <assembly> to run tests rather than dotnet test

733964b

forgot I made that change

f10b449

updating a few more deps

425a39e

aaronpowell self-assigned this Nov 11, 2025

aaronpowell added the awaiting response Waiting for the author of the issue to provide more information or answer a question label Nov 11, 2025

Base automatically changed from copilot/add-dotnet-10-tfm-support to main November 15, 2025 20:52

Youssef1313 reviewed Nov 16, 2025

View reviewed changes

Merge branch 'main' into xunit-3-upgrade

58b4b33

aaronpowell added 11 commits November 24, 2025 05:47

Updating based on comments and using the new GitHubActionsTestLogger …

270aa58

…with MTP support

Missing a few packages

cb145d8

fixing ignore

31b4c71

Fixing coverage arguments

d319e2f

Running built binary

587cace

Don't need configuration on a run

874567a

Removing an artificial bail out

837d9ef

Removing flag

db83468

going back to dotnet run

6d7a778

Removing reference to runsettings

239aca3

Performing an earlier eval of the connection string

0ab87ce

It seems there are two different paths for resolving the reference expression that _should_ return the same, but in the tests they don't seem to, so we'll force eval

aaronpowell added 6 commits November 27, 2025 10:39

Merge branch 'main' into xunit-3-upgrade

70133ad

Merge branch 'main' into xunit-3-upgrade

f925398

Merge branch 'xunit-3-upgrade' of https://github.com/CommunityToolkit…

d7d256d

…/Aspire into xunit-3-upgrade

Adjusting msbuild props

8273771

Linux file systems are case sensitive - why has this ever worked?

fb3743d

The path when the scripts run during tests is different to just runni…

91576a2

…ng the app host

Evangelink reviewed Dec 11, 2025

View reviewed changes

Allowing mcp tests to run on all OS's

3a0368b

Evangelink reviewed Dec 18, 2025

View reviewed changes

aaronpowell added 3 commits December 19, 2025 10:10

Merge branch 'main' into xunit-3-upgrade

51d39e6

Fixing copiler error

0b37f60

Updated traits

c3fab49

xUnit v3 upgrade #960

Are you sure you want to change the base?

xUnit v3 upgrade #960

Uh oh!

Conversation

aaronpowell commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Youssef1313 left a comment

Choose a reason for hiding this comment

Uh oh!

Evangelink commented Nov 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Evangelink left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aaronpowell commented Nov 11, 2025 •

edited

Loading