Repository navigation
test(smoke): sentinel invariants gate (plan A / task #175) - #78
Merged
Merged
Conversation
Adds a fast (<5s) operator-boundary smoke gate that runs on every PR and
push to develop. Guards against the class of regressions caught by the
tutorial-walk smoke on 2026-07-04 (S1/S3/O1/O3 in .factory/STATE.md drift
register): --help exiting non-zero to stderr, missing --version flag,
hardcoded version banner literals, missing ldflags wiring.
Contract (per Murat's RISK-1 from BMAD party 2026-07-04):
- Assertions are behavioral only — exit code, stream direction, substring
presence. Cosmetic diffs (whitespace, ordering, colour) are forbidden.
Reviewers reject cosmetic sentinels in PR review.
- New invariants require a paired update to docs/architecture.md §Smoke
invariants. An invariant without a documented rationale is a phantom.
- Exit codes: 0 all-pass, 1 regression, 2 harness broken. CI distinguishes
"smoke found a bug" from "smoke can't run."
- Isolated: runs in mktemp -d with trap cleanup; never touches user state.
- Failure-with-a-receipt: JSONL report written under .smoke/<ts>/, uploaded
by CI on failure as artifact `smoke-report` (7-day retention).
Files:
- test/smoke/invariants.sh — the harness, 12 invariants INV-1..INV-8
(INV-7 parameterised over access|router|console|control subcommands;
INV-8 asserts both switchboard and sbctl banners).
- justfile — `just smoke-quick` inlines both binary builds with a
computed VERSION stamp so INV-8 (ldflags wiring) asserts the injected
value flows through both banners. Inlined rather than using a
build-both dependency because just evaluates `VERSION := env(...)` at
invocation time and the shell-computed STAMP would not flow into a
dependency recipe.
- .github/workflows/ci.yml — new step runs after Validate build with
stamp `smoke-ci-${GITHUB_SHA::7}`; smoke report uploaded on failure.
Also fixes latent defect: Validate build only built switchboard, now
builds sbctl too.
- docs/architecture.md — new §Smoke invariants section with contract
rules, invariants table INV-1..INV-8 with defect-class column,
add-a-new-invariant procedure, and historical context.
- .gitignore — .smoke/ (report artifacts).
Verified locally: 12/12 pass in <1s. INV-8 correctly fails when ldflags
wiring is removed (regression-proven).
Refs #175 #163 #146
Coverage ReportTotal coverage: 81.8% Package breakdown |
arcaven
added
the
type.test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a fast (<5s) operator-boundary smoke gate that runs on every PR and push to
develop. Guards the class of regressions caught by the tutorial-walk smoke on 2026-07-04 (S1/S3/O1/O3 in.factory/STATE.mddrift register):--helpexiting non-zero to stderr, missing--versionflag, hardcoded version banner literals, missing ldflags wiring.Plan A of 4 from the BMAD party-mode planning session on smoke-test strategy (task #173/#174). Plan A is the sentinel invariants gate; C is the blast-radius declaration process gate; B (stratified harness) and #179 (orchestrator smoke-before-green doc edit) chain after A merges. Plan D (spec-as-test) deferred.
What the gate asserts (12 invariants, all behavioral)
switchboard --help→ exit 0, stdout non-empty, stderr emptyswitchboard --version→ exit 0, stdout starts withswitchboardargs[0]-derived basenamesbctl --help→ exit 0, stdout non-empty, stderr emptysbctl --version→ exit 0, stdout starts withsbctlsbctl --versionflag was missing entirely pre-#77sbctl(no args) → exit 2, stderr containsavailable subcommands:interface-definitions.md§174 usage-error contractsbctl <unknown>→ exit 2, stderr containsunknown subcommandinterface-definitions.md§174 unknown-subcommand contractswitchboard {access,router,console,control} --helpshort-circuits before I/O with exit 0--versionbanners contain the CI-injected${VERSION}substring-ldflags "-X main.version=..."wiring — the task #163 sbctl-a packaging defect at pre-merge timeContract rules (per Murat's RISK-1 from BMAD 2026-07-04)
docs/architecture.md §Smoke invariants. An invariant without a documented rationale is a phantom.smoke-report(7-day retention) for post-mortem.mktemp -dtmpdir withtrapcleanup. Never touches~/.switchboard,~/.sbctl, or user state.0all-pass,1regression,2harness broken. CI distinguishes "smoke found a bug" from "smoke can't run."Files
test/smoke/invariants.sh— the harness (213 lines, executable), 12 checks including INV-7 parameterised over 4 subcommands and INV-8 over 2 binaries.justfile—just smoke-quickinlines both binary builds with a computed VERSION stamp so INV-8 asserts the injected value flows through both banners. Inlined rather than depending on abuild-bothrecipe becausejustevaluatesVERSION := env(...)at invocation time and shell-computed values don't back-propagate to dependency recipes..github/workflows/ci.yml— new "Sentinel invariants (smoke gate)" step after Validate build, stampsVERSION=smoke-ci-${GITHUB_SHA::7}. Report upload on failure viaactions/upload-artifact@v7(SHA-pinned). Also fixes latent defect: Validate build only builtswitchboard— now buildssbctltoo.docs/architecture.md— new §Smoke invariants section with contract rules, invariants table, add-a-new-invariant procedure, historical context..gitignore—.smoke/(report artifacts).Regression-proof
Locally verified INV-8 correctly fails when ldflags wiring is removed. All 12 invariants pass in <1s under
just smoke-quick.Test plan
just smoke-quickpasses 12/12go vet ./...cleangolangci-lint run ./...— 0 issuesWhat this unblocks
per-story-delivery.mdRefs #175 #163 #146