ci(accessibility-report): generate a report of full and partial accessibility violations

.github/workflows/playwright.yml

@@ -79,11 +79,19 @@ jobs:
           merge-multiple: true
 
       - name: Merge into HTML Report
-        run: npx playwright merge-reports --config=playwright-ct.config.ts --reporter=html ./all-blob-reports
+        run: npx playwright merge-reports --config=playwright-ct.config.ts --reporter=html --reporter=./playwright/support/accessibility-reporter.ts ./all-blob-reports
 
       - name: Upload HTML report
         uses: actions/upload-artifact@v6
         with:
           name: html-report--attempt-${{ github.run_attempt }}
           path: playwright-report
           retention-days: 14
+
+      - name: Upload Accessibility report
+        if: always()
+        uses: actions/upload-artifact@v6
+        with:
+          name: accessibility-report--attempt-${{ github.run_attempt }}
+          path: playwright/test-report/accessibility-report.*
+          retention-days: 14

playwright-ct.config.ts

@@ -23,8 +23,15 @@ export default defineConfig({
   maxFailures: process.env.CI ? 10 : undefined,
 
   reporter: process.env.CI
-    ? "blob"
-    : [["html", { outputFolder: resolve(playwrightDir, "./test-report") }]],
+    ? [
+        ["blob"],
+        [resolve(playwrightDir, "./support/accessibility-reporter.ts")],
+      ]
+    : [
+        ["list"],
+        ["html", { outputFolder: resolve(playwrightDir, "./test-report") }],
+        [resolve(playwrightDir, "./support/accessibility-reporter.ts")],
+      ],
 
   use: {
     trace: "retain-on-failure",

playwright/README.md

@@ -30,6 +30,33 @@ To run the tests locally:
 4. To run specific Playwright tests at the command line run `npm run test:ct -- ./src/components/[component]/*.pw.tsx`. Test results can be seen in the console run summary.
 5. We have specified three browsers to run tests on. So to run Playwright tests in a specific browser run `npm run test:ct -- --project=chromium` or `npm run test:ct -- --project=firefox`. If you want to run Playwright tests on a specific browser using `UI` runner you need to manually select which browser you want to use (or all available in `playwright-ct.config.ts`).
 
+### Accessibility Report
+
+When tests are run, an **Accessibility Report** is automatically generated alongside the standard Playwright test report. This report consolidates all accessibility violations and incomplete checks detected during the test run.
+
+**Accessing the Report:**
+
+- **Locally**: After running `npm run test:ct`, view the report at `playwright/test-report/accessibility-report.html`
+- **Via command**: Run `npm run test:ct:report` to open the standard Playwright HTML report. The accessibility report is in the same directory (`playwright/test-report/`)
+- **In CI**: The accessibility report is included in the HTML report artifacts uploaded to GitHub Actions
+
+**Report Contents:**
+
+The accessibility report includes:
+
+- **Summary statistics**: Count of incomplete checks and violations by component
+- **Filtering**: Search by component name, test name, or issue description; filter by type (incomplete/violation) or severity (critical/serious/moderate/minor)
+- **Detailed issue information**: Each issue includes the rule ID, impact level, description, help text, and a link to detailed documentation
+- **Affected elements**: For each issue, see which DOM elements are affected with their selectors and HTML
+- **Export options**: Download the data as CSV or JSON for bug tracking systems
+
+**Understanding the Results:**
+
+- **Incomplete Checks** (Primary Focus): Accessibility rules that require manual verification because they cannot be fully automated (e.g., color contrast checks that need visual confirmation, ARIA attribute values that need contextual validation). These are displayed as console warnings during local test runs and captured in the report. They do NOT cause test failures but should be reviewed and logged as bugs if needed.
+- **Violations**: Accessibility rules that definitively failed automated testing. These WILL cause test failures and must be fixed before merging.
+
+The report is particularly valuable for tracking **incomplete checks** that require manual review by Test Analysts. While tests continue to pass with incomplete checks, the report provides a centralized view for logging and tracking these issues.
+
 > If you use VSCode as your code editor, you can also run tests via the official [Playwright Test for VSCode](https://playwright.dev/docs/getting-started-vscode) extension.
 
 ## Continuous Integration (CI)