Release pipeline can publish a package whose tests are failing

[adityamukho]

Summary

release.yml never runs this binding's tests, and nothing connects it to ci.yml. The publish path can therefore ship a package whose tests are failing at that very moment.

This is not hypothetical — it happened during the v1.2.2 cascade, in minigraf-node. Every binding repo has the same pipeline shape, so filing here too.

The gap

The two workflows are triggered independently and share no dependency:

• release.yml ← repository_dispatch: [core-release] (from the core's Release Cascade)
• ci.yml ← push / pull_request on main

release.yml's jobs are prepare → build → publish. There is no test job in that chain.

The subtlety is that CI does run during a release, which makes it look covered: prepare's "Commit and push tag" step pushes the version-pin commit to main, and that push triggers ci.yml. So the two race, and publish does not wait.

What it looked like in practice

minigraf 1.2.2 (project-minigraf/minigraf#304) made a second handle on an already-open file an error. minigraf-node had no way to release a handle — JavaScript has no deterministic destructor, and unlike the UniFFI bindings (destroy()) or the C API (minigraf_close) it exposed no close(). Its file-backed roundtrip test broke on all four platforms.

time (UTC)	event
05:21:57	cascade dispatches core-release → Release starts
05:22:15	prepare pushes the version-pin commit → triggers CI
05:22:51	CI fails on all 4 platforms
05:25:38	npm publish succeeds

CI had been red for nearly three minutes. minigraf@1.2.2 went to the registry broken and is still there.

Suggested fix

Add a test job to release.yml and make publish depend on it:

jobs:
  prepare:      # unchanged
  test:
    needs: prepare
    # run the same suite ci.yml runs, on the pinned core version
  build:
    needs: [prepare, test]
  publish:
    needs: [prepare, build, test]

Testing inside release.yml rather than reusing the ci.yml run is the point: the release pins an exact core version (minigraf = "=X.Y.Z"), and that pinned combination is what must be proven green — not whatever main happened to contain.

Reusing ci.yml via workflow_call would also work, provided publish genuinely waits on the result.

Related

A second, smaller thing surfaced alongside: the publish step swallows npm's "cannot publish over the previously published versions" and reports success. That is a reasonable re-run guard, but it means a release that published nothing still goes green. Worth distinguishing "already published, skipping" from "published" in the job summary so a no-op re-run is not mistaken for a delivery.