Skip to content

SQLR-12 — make release.yml publish jobs idempotent (skip-if-already-published) #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joaoh82 merged 1 commit into from
Jun 1, 2026
Merged

SQLR-12 — make release.yml publish jobs idempotent (skip-if-already-published) #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 152 additions & 7 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,33 @@ jobs:
with:
shared-key: publish-crate

# Idempotency guard (SQLR-12). A re-dispatch of release.yml at a
# version where some-but-not-all artifacts already published (the
# v0.11.0 partial failure: the engine 413'd but every other channel
# had already shipped) must SKIP what's done and publish only what's
# missing — instead of erroring with "crate version already uploaded".
# crates.io returns HTTP 200 for an existing crate@version, 404 for a
# missing one. The `User-Agent` header is mandatory: crates.io 403s
# any request without one. On a transport error (curl fails → "000")
# we fall through to publish, preserving the old fail-loud behavior.
- name: Skip if already on crates.io
id: published_check
run: |
V="${{ needs.detect.outputs.version }}"
CODE=$(curl -sS -o /dev/null -w '%{http_code}' \
-H 'User-Agent: sqlrite-release-ci (https://github.com/joaoh82/rust_sqlite)' \
"https://crates.io/api/v1/crates/sqlrite-engine/$V" || true)
CODE=${CODE:-000}
if [ "$CODE" = "200" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-engine@$V already on crates.io (HTTP 200) — skipping cargo publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-engine@$V not found on crates.io (HTTP $CODE) — will publish"
fi

- name: cargo publish
if: steps.published_check.outputs.skip != 'true'
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
run: |
Expand Down Expand Up @@ -236,7 +262,26 @@ jobs:
with:
shared-key: publish-ask

# Idempotency guard (SQLR-12) — see publish-crate for the full
# rationale. crates.io: 200 = already published (skip), else publish.
- name: Skip if already on crates.io
id: published_check
run: |
V="${{ needs.detect.outputs.version }}"
CODE=$(curl -sS -o /dev/null -w '%{http_code}' \
-H 'User-Agent: sqlrite-release-ci (https://github.com/joaoh82/rust_sqlite)' \
"https://crates.io/api/v1/crates/sqlrite-ask/$V" || true)
CODE=${CODE:-000}
if [ "$CODE" = "200" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-ask@$V already on crates.io (HTTP 200) — skipping cargo publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-ask@$V not found on crates.io (HTTP $CODE) — will publish"
fi

- name: cargo publish
if: steps.published_check.outputs.skip != 'true'
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
# `--no-verify` mirrors `publish-crate` — Release-PR CI
Expand Down Expand Up @@ -302,7 +347,26 @@ jobs:
with:
shared-key: publish-mcp

# Idempotency guard (SQLR-12) — see publish-crate for the full
# rationale. crates.io: 200 = already published (skip), else publish.
- name: Skip if already on crates.io
id: published_check
run: |
V="${{ needs.detect.outputs.version }}"
CODE=$(curl -sS -o /dev/null -w '%{http_code}' \
-H 'User-Agent: sqlrite-release-ci (https://github.com/joaoh82/rust_sqlite)' \
"https://crates.io/api/v1/crates/sqlrite-mcp/$V" || true)
CODE=${CODE:-000}
if [ "$CODE" = "200" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-mcp@$V already on crates.io (HTTP 200) — skipping cargo publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-mcp@$V not found on crates.io (HTTP $CODE) — will publish"
fi

- name: cargo publish
if: steps.published_check.outputs.skip != 'true'
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
# `--no-verify` mirrors `publish-crate` / `publish-ask` —
Expand Down Expand Up @@ -827,17 +891,40 @@ jobs:
- name: List files about to be uploaded
run: ls -la dist/

# Single atomic upload of all wheels + sdist. If any file
# fails to upload, none are published — no partial wave
# on PyPI.
# Idempotency log line (SQLR-12). PyPI's JSON API returns HTTP 200
# for an existing project/version, 404 otherwise. Unlike the
# single-artifact crates.io / npm jobs, PyPI's publish unit is a
# *set* of files (one wheel per platform + the sdist), so we do NOT
# gate the whole step on this check: a partial wave (some wheels
# landed before the job died) would leave the version "present" yet
# incomplete, and a version-level skip would strand the missing
# wheels. Instead this step just records the state in the run log;
# `skip-existing: true` below does the actual file-granular skipping.
- name: Report PyPI publish state
run: |
V="${{ needs.detect.outputs.version }}"
CODE=$(curl -sS -o /dev/null -w '%{http_code}' \
"https://pypi.org/pypi/sqlrite/$V/json" || true)
CODE=${CODE:-000}
if [ "$CODE" = "200" ]; then
echo "::notice::sqlrite==$V already present on PyPI (HTTP 200) — upload will skip files already there (skip-existing)"
else
echo "::notice::sqlrite==$V not found on PyPI (HTTP $CODE) — uploading all wheels + sdist"
fi

# Upload all wheels + sdist aggregated from the build matrix.
# `skip-existing: true` (SQLR-12) is what makes this job idempotent:
# a re-dispatch (full or partial-wave recovery) uploads only the
# files not yet on PyPI and silently skips the rest, instead of the
# old `skip-existing: false` which failed loudly the moment any one
# file already existed. File-granular skipping is the right unit for
# PyPI's multi-file publish — it cleanly handles both a fully-shipped
# version (every file skipped → no-op) and a half-shipped one.
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist
# Keep `skip-existing: false` so a re-run of this job
# (after a partial-failure scenario) fails loudly rather
# than silently ignoring already-uploaded files.
skip-existing: false
skip-existing: true

- name: GitHub Release
uses: softprops/action-gh-release@v2
Expand Down Expand Up @@ -1080,6 +1167,26 @@ jobs:
# devDep pulled in by accident would be visible here.
npm pack --dry-run

# Idempotency guard (SQLR-12) — see publish-crate for the full
# rationale. `npm view <pkg>@<version> version` prints the version
# on a hit and exits non-zero with empty stdout on a miss; the
# `|| true` keeps the `-e` shell from aborting on the miss path.
# npm's publish unit is a single tarball per version, so a present
# version means a complete publish — a clean version-level skip.
- name: Skip if already on npm
id: published_check
working-directory: sdk/nodejs
run: |
V="${{ needs.detect.outputs.version }}"
EXISTING=$(npm view "@joaoh82/sqlrite@$V" version 2>/dev/null || true)
if [ -n "$EXISTING" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::@joaoh82/sqlrite@$V already on npm ($EXISTING) — skipping npm publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::@joaoh82/sqlrite@$V not found on npm — will publish"
fi

# Single atomic publish via OIDC trusted publisher.
#
# The `--provenance` flag is what tells npm CLI to use the
Expand Down Expand Up @@ -1111,6 +1218,7 @@ jobs:
# diagnosable from the run log without re-running with
# debug-on. Cheap insurance against another silent failure.
- name: Publish to npm
if: steps.published_check.outputs.skip != 'true'
working-directory: sdk/nodejs
run: npm publish --access public --provenance --loglevel verbose

Expand Down Expand Up @@ -1226,7 +1334,24 @@ jobs:
# node_modules).
npm pack --dry-run

# Idempotency guard (SQLR-12) — see publish-crate / publish-nodejs.
# Package is the unscoped `sqlrite-notes`.
- name: Skip if already on npm
id: published_check
working-directory: examples/nodejs-notes
run: |
V="${{ needs.detect.outputs.version }}"
EXISTING=$(npm view "sqlrite-notes@$V" version 2>/dev/null || true)
if [ -n "$EXISTING" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-notes@$V already on npm ($EXISTING) — skipping npm publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::sqlrite-notes@$V not found on npm — will publish"
fi

- name: Publish to npm
if: steps.published_check.outputs.skip != 'true'
working-directory: examples/nodejs-notes
run: npm publish --access public --provenance --loglevel verbose

Expand Down Expand Up @@ -1386,12 +1511,32 @@ jobs:
echo "ACTIONS_ID_TOKEN_REQUEST_TOKEN is set: ${ACTIONS_ID_TOKEN_REQUEST_TOKEN:+yes}${ACTIONS_ID_TOKEN_REQUEST_TOKEN:-NO}"
npm pack --dry-run

# Idempotency guard (SQLR-12) — see publish-crate / publish-nodejs.
# Package is the scoped `@joaoh82/sqlrite-wasm`. (The wasm-pack build
# above still runs on a skip — gating only the publish keeps this
# consistent with the other channels; the build is cheap relative to
# the rest of the release wave.)
- name: Skip if already on npm
id: published_check
working-directory: sdk/wasm/pkg
run: |
V="${{ needs.detect.outputs.version }}"
EXISTING=$(npm view "@joaoh82/sqlrite-wasm@$V" version 2>/dev/null || true)
if [ -n "$EXISTING" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::notice::@joaoh82/sqlrite-wasm@$V already on npm ($EXISTING) — skipping npm publish"
else
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "::notice::@joaoh82/sqlrite-wasm@$V not found on npm — will publish"
fi

# Atomic OIDC publish. Same flag combo proven in
# publish-nodejs: `--provenance` to trigger OIDC code path,
# `--access public` because scoped packages default to
# private, `--loglevel verbose` to keep error logs
# diagnosable.
- name: Publish to npm
if: steps.published_check.outputs.skip != 'true'
working-directory: sdk/wasm/pkg
run: npm publish --access public --provenance --loglevel verbose

Expand Down
53 changes: 39 additions & 14 deletions docs/release-plan.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,9 +231,11 @@ The "publish" half. Auto-fires on the release commit.
commit: `sqlrite-vX.Y.Z`, `sqlrite-ffi-vX.Y.Z`,
`sqlrite-py-vX.Y.Z`, `sqlrite-node-vX.Y.Z`, `sqlrite-wasm-vX.Y.Z`,
`sdk/go/vX.Y.Z`, `sqlrite-desktop-vX.Y.Z`, `vX.Y.Z`. Pushes
them. Runs *before* the publish jobs — if a tag already exists
(accidental re-run, cosmic ray), the whole workflow aborts
cleanly.
them. Runs *before* the publish jobs. Idempotent on re-run: if a
tag already exists (partial-failure re-dispatch, accidental
re-trigger), that tag is skipped with a `::notice::` rather than
failing, so a re-dispatch at the same version proceeds to the
publish jobs instead of aborting.
- **publish-crate** — `cargo publish -p sqlrite-engine` the root
crate to crates.io. (The crates.io name is `sqlrite-engine`, not
`sqlrite`, because the short name was already taken by an
Expand Down Expand Up @@ -284,18 +286,41 @@ The "publish" half. Auto-fires on the release commit.
in parallel. Umbrella GitHub Release finalizes. No branch-
protection bypass needed, no deploy keys, no admin override.
- **Sad path — publish fails after tag push**: say
`publish-python` fails on wheel upload. The tag
`sqlrite-py-vX.Y.Z` is already on the remote. **Convention:
never reuse a tag, always bump past.** Next release is
`v0.2.1`, not a re-try of `v0.2.0`. Partial success is visible
— the `sqlrite-vX.Y.Z` crate *did* publish, the Python wheels
didn't, and both facts are recorded. Operators can fix the
Python SDK and re-dispatch `release.yml` in manual mode at
`v0.2.1`.
`publish-crate` fails while the other channels succeed (this is
exactly the v0.11.0 wave — the engine crate hit a crates.io 413
but `sqlrite-ask`, npm, PyPI, FFI, Go and desktop had all
already shipped). The publish jobs are **idempotent** (SQLR-12):
each one probes its registry first and skips with a `::notice::`
when the version is already there. So the recovery is to fix the
failing channel and **re-dispatch `release.yml` at the same
version** — `tag-all` skips the existing tags, the
already-published channels skip their `publish` step, and only
the missing artifact actually publishes. No tag bump required;
the old "never reuse a tag, always bump past" workaround is
retired. Per-registry guards:
- **crates.io** (`publish-crate` / `-ask` / `-mcp`): `GET
crates.io/api/v1/crates/<name>/<version>` (with a mandatory
`User-Agent`) → HTTP 200 skips, 404 publishes.
- **npm** (`publish-nodejs` / `-wasm` / `-notes-example`):
`npm view <pkg>@<version> version` — non-empty skips.
- **PyPI** (`publish-python`): `GET
pypi.org/pypi/sqlrite/<version>/json` is logged for
visibility, and `skip-existing: true` does the actual
file-granular skipping — the right unit for PyPI's
multi-wheel wave (a partial wave fills in the missing wheels
without erroring on the ones already there).
- **GitHub Releases** (`publish-ffi` / `-desktop` / `-go` /
`build-mcp-binaries`): `softprops/action-gh-release` is
create-or-update, so re-runs refresh the release in place.
- **Sad path — a fully-successful release re-dispatched at the
same version**: a clean no-op. Every tag is skipped, every
`publish` step is skipped, GitHub Releases refresh in place —
no wall of "already exists" failures.
- **Sad path — an accidental `release: v…` commit message**: the
auto-trigger fires. `tag-all` runs and finds the tags already
exist (because the real release happened weeks ago). Workflow
aborts with a clear "tag already exists" error. No damage.
auto-trigger fires at a version that shipped weeks ago.
`tag-all` finds every tag present and skips them; each publish
job finds its artifact already on the registry and skips. The
run is a green no-op. No damage.

## Pinned binaryen / wasm-opt

Expand Down
Loading