Feature/centralized test reporting issue 77

.github/workflows/tests.yml

@@ -16,7 +16,24 @@ jobs:
           python-version: "3.11"
 
       - name: Install dependencies
-        run: pip install -r requirements.txt
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install --group test
 
-      - name: Run tests
-        run: pytest
+      # Validates tests/test_manifest.yaml, regenerates tests/README.md, then
+      # runs pytest with pytest-html. Exits with pytest's own exit code, so a
+      # test failure still fails this job -- after the report is uploaded.
+      - name: Run tests and generate report
+        run: python scripts/generate_test_dashboard.py
+
+      - name: Upload test report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: pyrpod-test-report
+          path: |
+            reports/pyrpod-pytest-report.html
+            reports/logs/
+          retention-days: 14
+          if-no-files-found: warn

.gitignore

@@ -7,6 +7,10 @@ data/stl/tcd/
 
 results/
 
+# Run-specific test reports (HTML report + failed-test logs). The generated
+# tests/README.md and tests/test_manifest.yaml ARE tracked.
+reports/
+
 results.txt
 *.pyc
 *.png

CONTRIBUTING.md

@@ -48,8 +48,15 @@ PyRPOD uses [pytest](https://docs.pytest.org/) to ensure stability and prevent r
 
 1. Write tests for your contributions in the `tests` directory, following the existing `<group>_<category>_test_NN.py` naming convention (e.g. `rpod_unit_test_04.py`).
 2. Run the full suite locally with `pytest` from the repository root before submitting a pull request.
-3. Tests are automatically tagged with `unit`, `integration`, `verification`, and subsystem (`mdao`, `mission`, `plume`, `rpod`) markers based on their filename, so you can run a subset with e.g. `pytest -m unit` or `pytest -m rpod`.
-4. CI runs the same `pytest` suite automatically on every push and pull request via [GitHub Actions](.github/workflows/tests.yml).
+3. Tests are automatically tagged with `unit`, `integration`, `verification`, and subsystem (`logging`, `mdao`, `mission`, `plume`, `rpod`, `tooling`) markers based on their filename, so you can run a subset with e.g. `pytest -m unit` or `pytest -m rpod`.
+4. **Add a matching entry to [`tests/test_manifest.yaml`](tests/test_manifest.yaml)** describing the new file, then regenerate the inventory:
+   ```bash
+   python -m pip install "pytest-html>=4.2,<5" "PyYAML>=6"   # once
+   python scripts/generate_test_dashboard.py
+   ```
+   Commit the regenerated `tests/README.md` alongside your test. The generator fails with an explicit list of missing or stale entries if the manifest and pytest's collection disagree.
+5. Record the *development* status (`implemented`, `placeholder`, `needs_review`, `blocked`, `archived`, `deprecated`) in the manifest — never a pass/fail result. Execution outcomes live only in the run-specific `reports/pyrpod-pytest-report.html`, which is git-ignored. A test with no assertions is a `placeholder`; skip it explicitly so it cannot be mistaken for coverage.
+6. CI runs the same command automatically on every push and pull request via [GitHub Actions](.github/workflows/tests.yml), and uploads the HTML report as the `pyrpod-test-report` artifact even when tests fail.
 
 ## Code Formatting
 

README.md

@@ -47,6 +47,46 @@ PyRPOD utilizies scientific libraries such as NumPy, SciPy, Matplotlib, and SymP
    python -m pytest tests/rpod          # or just point pytest at a directory/file directly
    ```
 
+## Test Reporting
+
+Test information lives in two places, and the split is deliberate:
+
+| | Where | Owned by | Committed? |
+| --- | --- | --- | --- |
+| **Execution outcome** (passed / failed / skipped, for one run) | `reports/pyrpod-pytest-report.html` | pytest | No — `reports/` is git-ignored |
+| **Development status** (implemented / placeholder / blocked / ...) | [`tests/test_manifest.yaml`](tests/test_manifest.yaml) → [`tests/README.md`](tests/README.md) | maintainers | Yes |
+
+A passing test is *not* evidence that a test is finished: several placeholder
+tests contain no assertions and are explicitly skipped so they can never be
+mistaken for coverage.
+
+1. Install the reporting dependencies once (declared as the `test` dependency
+   group in `pyproject.toml`):
+   ```bash
+   python -m pip install "pytest-html>=4.2,<5" "PyYAML>=6"
+   ```
+   With pip 25.1 or newer you can instead run `python -m pip install --group test`.
+
+2. Generate the dashboard and the HTML report:
+   ```bash
+   python scripts/generate_test_dashboard.py
+   ```
+   This validates the manifest, cross-checks it against pytest's current
+   collection, regenerates `tests/README.md`, then runs the suite and writes a
+   self-contained `reports/pyrpod-pytest-report.html`. It exits with pytest's
+   own exit code, and the report is written even when tests fail. Use
+   `--inventory-only` to refresh `tests/README.md` without running the suite,
+   or `--check` to verify it is up to date.
+
+3. When you add a test, add a matching entry to `tests/test_manifest.yaml`.
+   The generator fails with an explicit list of missing or stale entries
+   otherwise.
+
+CI runs the same command on every pull request and every push to `master`, and
+uploads the report as the **`pyrpod-test-report`** artifact (retained 14 days),
+including when tests fail. Download it from the *Artifacts* section of the
+workflow run summary on the Actions tab.
+
 ## Logging
 
 PyRPOD ships a formal, opt-in operational logging system built entirely on the

pyproject.toml

@@ -1,3 +1,15 @@
+# Optional dependencies for generating the test dashboard (PEP 735). PyRPOD is
+# not installed as a distribution, so there is no [project] table to hang
+# optional-dependencies off; a dependency group adds these without introducing
+# packaging metadata or changing how the repository is installed.
+#   python -m pip install --group test        (pip >= 25.1)
+[dependency-groups]
+test = [
+    "pytest",
+    "pytest-html>=4.2,<5",
+    "PyYAML>=6",
+]
+
 [tool.black]
 line-length = 88
 target-version = ["py311"]
@@ -39,4 +51,5 @@ markers = [
     "mission: mission subsystem tests",
     "plume: plume subsystem tests",
     "rpod: rpod subsystem tests",
+    "tooling: test-infrastructure tests",
 ]