Skip to content

CI never runs test-pg-both, so pg-shipped-only-check cannot catch PG_SHIPPED_ONLY_PKGS drift #667

Description

@mzaitzeff

pg-shipped-only-check exists to keep PG_SHIPPED_ONLY_PKGS honest — the Makefile says the equivalence argument behind test-pg-both "is checked rather than trusted". But nothing in CI runs it, so in practice it is trusted after all.

ci.yml has three PostgreSQL-ish lanes and none of them reach the guard:

  • Test (PostgreSQL)make test-pg-shipped
  • Test (pgvector-tagged)go test -tags "fts5 sqlite_vec pgvector" ./internal/vector/... ./internal/scheduler/... ./cmd/msgvault/cmd/... (a second hand-maintained package list)
  • nothing invokes make test-pg-both, which is the only target that depends on pg-shipped-only-check

docs/internal/PG_STATUS.md tells contributors that test-pg-both "is the one to run", so the target most likely to be run by hand is the one whose prerequisite CI never exercises.

It has already drifted once, for six days

c5b8e30 (#589, Aug 13) added test imports of internal/vector/embed to internal/store/embed_gen_group_test.go and scripts/contextual-retrieval-eval/eval_test.go. That makes both test binaries pgvector-tag-sensitive, so both belong in PG_SHIPPED_ONLY_PKGS, and neither was added. From that commit until d6a7b49 (Aug 19), make test-pg-both failed immediately on any tree:

PG_SHIPPED_ONLY_PKGS is stale ('-' expected, '+' actual). Update it in the Makefile; test-pg-both's coverage argument depends on it.
--- expected
+++ actual
@@ -3,7 +3,9 @@
 ./internal/api
 ./internal/mcp
 ./internal/scheduler
+./internal/store
 ./internal/vector/chunkmatch
 ./internal/vector/embed
 ./internal/vector/hybrid
 ./internal/vector/pgvector
+./scripts/contextual-retrieval-eval
make: *** [Makefile:149: pg-shipped-only-check] Error 1

Reproducible in a clean worktree at either c5b8e30 or any commit before d6a7b49. During that window the target was unusable, and the two packages were not covered in the shipped-build configuration that the guard is meant to protect.

The list was corrected inside #621, an unrelated vcard PR — by hand, not by the check. That is the part worth fixing: the guard works, it just never gets to speak.

Suggestion

Run make pg-shipped-only-check in CI. It needs no database and runs no tests — two go list invocations over the module — so it can hang off an existing job cheaply, and it would have failed on #589 itself.

The Test (pgvector-tagged) job's hardcoded package list has the same drift-by-hand property, if you want a second thing to derive rather than maintain.

Found downstream: a fork here runs make test-pg-both nightly, which is how the six-day window showed up at all.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions