Add [LiveTestOnly] attribute to skip tests in Playback mode

[Copilot]

Implementation Complete ✅

All requirements have been successfully implemented with minimal changes.

Changes Made

• Created LiveTestOnlyAttribute class in /core/Azure.Mcp.Core/tests/Azure.Mcp.Tests/Attributes/
  • Inherits from BeforeAfterTestAttribute
  • Checks TestMode in Before method
  • Calls Assert.Skip() when in Playback mode
  • Allows tests to run in Live and Record modes
• Added verification tests in RecordedCommandTestsBaseTests.cs
  • Tests verify the attribute can be instantiated and applied to methods
  • All tests pass successfully
• Added comprehensive XML documentation
  • Detailed usage examples
  • Lists scenarios where attribute should be applied
  • Clear guidance for developers
• Fixed formatting errors with dotnet format

Files Changed (2 files, 161 insertions)

1. core/Azure.Mcp.Core/tests/Azure.Mcp.Tests/Attributes/LiveTestOnlyAttribute.cs (121 lines)

   • New attribute implementation with full documentation

2. core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/RecordingFramework/RecordedCommandTestsBaseTests.cs (40 lines)

   • Added verification tests for the new attribute
   • Fixed formatting issues

How It Works

The [LiveTestOnly] attribute:

• ✅ Skips tests automatically in Playback mode with clear message
• ✅ Allows tests to run in Live mode (for actual testing)
• ✅ Allows tests to run in Record mode (for investigating recording issues)
• ✅ Integrates seamlessly with existing test infrastructure
• ✅ No changes needed to Test-Code.ps1 (already filters by assets.json)

Usage

public class PostgresCommandTests : RecordedCommandTestsBase
{
    [LiveTestOnly]
    [Fact]
    public async Task TestDatabaseQuery()
    {
        // This test uses direct TCP which cannot be recorded
        // Will be skipped in Playback mode
    }
}

Validation

• ✅ All builds pass without warnings or errors
• ✅ All existing tests continue to pass
• ✅ New verification tests pass
• ✅ Spelling check passes
• ✅ Code formatting applied (dotnet format)
• ✅ Code follows existing patterns (matches CustomMatcherAttribute)

Ready for Use

Teams working on Speech, Postgres, and Foundry can now apply [LiveTestOnly] to individual tests that cannot be recorded, allowing them to transition to RecordedCommandTestsBase while marking problematic tests to skip in Playback mode.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add [LiveTestOnly] Test attribute to Azure.Mcp.Tests</issue_title>
<issue_description>I have identified a couple packages that cannot migrate to recorded tests as they are not running on http.

• Some selected tests for speech work directly across websockets, which is not supported by the proxy as of this time: Migrate Azure.Mcp.Tools.Speech.LiveTests to recorded tests #1346
• Some tests for PostGres work directly on a tcp connection directly to the database. This is not http traffic.
• Some tests in Foundry have client-sourced inheritance bugs that prevent us from recording properly with subclients.

Our invocation of tests from TestCode for Recorded should automatically skip tests marked with the new LiveOnly attribute within test classes that inherit from RecordedCommandTestsBase when the mode in testsettings is not Live. </issue_description>

<agent_instructions>Attempt minimal changes. You do not need to add a test for this new attribute. Our test will be that we skip tests marked with LiveOnly only when in playback mode. Record should still actually attempt to run them, as it is buggy why they cannot record.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@anannya03 Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @microsoft/azure-mcp.

• Fixes Add [LiveTestOnly] Test attribute to Azure.Mcp.Tests #1389

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[scbedd]

@copilot you need to run dotnet format core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/Azure.Mcp.Core.LiveTests.csproj to fix the formatting errors.

[Copilot]

@copilot you need to run dotnet format core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/Azure.Mcp.Core.LiveTests.csproj to fix the formatting errors.

Done. Formatting errors fixed in commit 67ce9fe.