Skip to content

ci: simplify Rust release automation - #891

Merged
Xuanwo merged 17 commits into
mainfrom
codex/simplify-rust-release
Oct 5, 2026
Merged

ci: simplify Rust release automation#891
Xuanwo merged 17 commits into
mainfrom
codex/simplify-rust-release

Conversation

@tisonkun

@tisonkun tisonkun commented Sep 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

The release helpers repeat Cargo metadata validation, implement their own topological sorter, and audit bootstrap state through several layers. This PR removes 1,268 lines while retaining dependency-ordered publishing, fresh OIDC credentials per attempt, rate-limit recovery, and resumable bootstrap.

The changes are split into 16 ordered commits for review. Each commit message explains its purpose, behavioral impact, and validation. The final file tree is identical to the previously tested head 74ca948; this update only restructures history.

Step Commit Review focus
1 df09792 Remove YAML, literal, and current-workspace snapshot assertions; runtime unchanged.
2 fc5e50d Replace manual topological sorting; independent-crate tie order may change.
3 d5bdaad Trust Cargo metadata fields; assert unpublished local dependency errors.
4 539f5e6 Keep OIDC acquisition, masking, and revocation in one context manager.
5 df1a238 Stop on ordinary API failures instead of retrying them automatically.
6 21e0f08 Assert crate identity and exact publisher fields; retain historical crate metadata.
7 7ad1df6 Use existing job logs instead of maintaining a second summary report.
8 b922df6 Remove unused alternate-project and alternate-registry entry points.
9 4bd0911 Pass crate names directly to placeholder packaging; verify an offline build.
10 b00cecc Consolidate metadata waits; let the final audit check publisher persistence.
11 3a3bf63 Use one preflight/reconcile/final-audit flow and behavior-boundary tests.
12 0183a52 Remove unused plan paths and intermediate planner interfaces.
13 85df013 Remove the unconsumed discovery job; approval and authenticated preflight remain.
14 c5f49d5 Rely on the formal-tag push filter; PR/manual runs still validate only.
15 db9b01e Obtain the exact run ID from the versioned GitHub dispatch response.
16 14475d2 Express local preconditions directly; retain reviewer and run-SHA checks.

Behavior changes to inspect especially closely: ordinary API failures stop immediately (5); publisher matching is exact (6); failed publisher visibility is handled by rerunning rather than a separate polling loop (10); discovery is no longer displayed before environment approval (13). Assertions require normal Python execution, as documented.

Validation performed at every intermediate commit:

  • The complete release-helper test suite passed, reducing from 29 tests after the first deletion to 11 at the final head.
  • The current 15-crate publish plan, actionlint, Bash syntax, and git diff --check passed.
  • Both helper commits (15 and 16) separately passed eight isolated command-stub scenarios: success, dirty checkout, wrong main, missing reviewers, dispatch failure, missing run ID, wrong run SHA, and failed watch. Invalid preconditions prevented dispatch; a mismatched run SHA prevented watch and verification. No permanent test framework was added for these checks.

To check out an individual commit and rerun its common checks:

python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
python3 .github/scripts/release_rust/plan.py
bash -n release/scripts/bootstrap-rust-crates.sh
actionlint .github/workflows/release.yml .github/workflows/bootstrap_rust_crates.yml
git diff --check

The identical final tree previously passed cargo publish --workspace --dry-run --allow-dirty, public bootstrap.py verify for all 15 crates, and the full PR CI. Final Ruff and Hawkeye checks passed again after splitting. CI will run again for the rewritten head.

Validation limits: OIDC HTTP exchanges and registry mutations are mocked in the behavior tests. No live publication, authenticated crates.io mutation, or protected bootstrap dispatch was performed. Public verification checks existence and trustpub_only, not authenticated publisher configuration.

@tisonkun
test(release): remove configuration snapshot assertions
df09792 
Drop tests that compare workflow YAML text, fixed publisher constants,
the current AWS package list, and literal Cargo command arrays. They
mostly repeat configuration and couple tests to unrelated edits.

Keep real placeholder packaging, dependency-order tests, token renewal,
and failure/recovery coverage. No production code or workflow changes.

Validation: ran 29 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): delegate dependency ordering to graphlib
fc5e50d 
Replace the hand-written heap, indegree bookkeeping, and cycle detection
with Python's TopologicalSorter. Non-dev local dependencies still precede
their consumers; independent packages no longer have a path-sorted tie
break, which is not part of the publishing contract.

Assert dependency relationships in the test instead of one arbitrary
linear order. Remove the cycle test now covered by the standard library;
leave metadata validation and script interfaces unchanged.

Validation: ran 28 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): trust the Cargo metadata schema
d5bdaad 
Read the documented metadata fields directly instead of repeating type,
identity, and workspace-membership checks after Cargo succeeds. Cargo
--no-deps already returns the workspace package set.

Keep the meaningful invariant: a publishable package must not have a
non-dev dependency on an unpublished workspace member. Express that as
an assertion and document that these scripts require normal Python,
without optimization flags that disable assertions.

Validation: ran 28 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): keep the OIDC token lifecycle in one context
539f5e6 
Keep GitHub token acquisition, crates.io exchange, log masking, and
revocation together in one context manager. Remove configurable registry
URLs, generic URL/error adapters, and separate one-call wrappers.

Preserve a fresh token for each publish attempt and revocation on failure.
Consolidate four helper tests into one HTTP-boundary test covering query
preservation, credentials, masking, and cleanup when publication raises.

Validation: ran 25 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

The HTTP exchange is mocked; this does not exercise live GitHub OIDC.
@tisonkun
refactor(release): fail fast on bootstrap API errors
df1a238 
Reduce the crates.io client to direct requests and response fields.
Keep 404 handling for missing crate names, authentication, and request
timeouts. Remove the custom exception type, error-body adapters, and
automatic retries for GET/PATCH transport failures.

This intentionally changes recovery: ordinary API errors stop the run;
the operator can rerun after fixing the problem. Cargo upload rate-limit
retries and metadata propagation waits remain. Drop the test for the
removed HTTP-retry policy and document the new behavior.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): assert bootstrap identity and publisher invariants
21e0f08 
Use direct assertions for the crate name, accepted repository metadata,
placeholder description, and the single expected Trusted Publisher.
Remove the second lookup that only rechecks the known 0.0.0 version.

Historical repository/placeholder metadata is still accepted. Publisher
configuration must exactly match the expected fields; case variants no
longer receive special handling. Keep rejection tests for unrelated
repositories and old or unexpected publisher configurations.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): use bootstrap logs without a second report
7ad1df6 
Remove GITHUB_STEP_SUMMARY generation and the catch/rethrow block used
only to populate it. Keep discovery output, per-crate progress, preflight
and final-audit messages, and the original exception traceback.

The write sequence and audit checks are unchanged. Completed operations
are visible in the job log, so retaining a second result collection and
formatting success/failure reports adds no recovery capability.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): fix helper entry points to this repository
b922df6 
Remove the unused --project-dir and --registry-url CLI options. These
helpers publish this repository to crates.io; no workflow or documented
invocation supplies alternative values.

Use PROJECT_DIR directly for publishing and the fixed crates.io endpoint
for bootstrap. Keep discover/apply/verify commands and the existing token,
retry, and already-published behavior. Internal planner data cleanup is
left to a later commit so the interface change can be reviewed separately.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): pass only the crate name to placeholder packaging
4bd0911 
The placeholder writer needs a crate name and destination directory,
not a project path plus a PlannedCrate wrapper. Read LICENSE and NOTICE
from PROJECT_DIR, and pass the name directly through publication.

Keep the generated manifest, README, source, and Cargo retry behavior.
Strengthen the existing offline packaging test to build the packaged
placeholder as well; it verifies a usable package without publishing it.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): consolidate bootstrap metadata propagation waits
b00cecc 
Use one bounded polling loop for crate visibility and trustpub_only
propagation. Keep the 120-second timeout and two-second interval; express
timeout as an assertion like the other bootstrap invariants.

Remove the separate Trusted Publisher polling loop. Creation responses
are checked immediately, and the complete authenticated final audit still
reads the persisted configuration. If that read is stale or fails, the
run stops and can be retried instead of adding another recovery loop.

Validation: ran 24 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
refactor(release): reduce bootstrap to audit, reconcile, and audit
3a3bf63 
Find bootstrap candidates during the authenticated preflight instead of
running discovery and re-reading the same metadata first. Reuse the audit
for the final ready-state check, and pass crate names rather than parallel
PlannedCrate/ReconcileResult collections.

Keep the safety boundaries: audit the entire plan before any write,
re-read candidates before modifying them, reject established crates, and
verify the full plan after reconciliation. Discovery and public verify
remain read-only CLI commands.

Replace helper-level and call-order mocks with four behavior tests:
failed preflight causes zero writes; partial bootstrap safely resumes and
repeats; a candidate becoming established is untouched; and a placeholder
packages and builds offline.

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

Registry writes are simulated by a stateful fake; no live bootstrap ran.
@tisonkun
refactor(release): remove unused publish-plan representations
0183a52 
Now that bootstrap consumes names directly, remove Package.path and the
extra metadata-loading and plan_from_metadata entry points. Build the
plan in one function from cargo metadata and emit only name/version.

Consolidate the planner tests at the Cargo-output boundary: local normal
and build dependencies must precede consumers, dev cycles are ignored,
unpublishable packages are excluded, and unpublished local dependencies
are rejected. Update publisher fixtures for the smaller Package value.

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.
@tisonkun
ci(release): remove the redundant bootstrap discovery job
85df013 
The discover job exports no outputs, and the protected bootstrap job
already performs a full authenticated preflight at the checked-out SHA.
Remove the extra runner, checkout, Python setup, and unauthenticated scan.

Keep the input-free dispatch, rust-bootstrap environment, explicit SHA
checkout, and main/SHA checks. Express the main check directly with test.
The tradeoff is that discovery is no longer displayed before environment
approval; the read-only discover CLI remains available when needed.

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

Actionlint and Bash syntax pass. No protected workflow was dispatched.
@tisonkun
ci(release): rely on the existing formal-tag push filter
c5f49d5 
The workflow accepts push events only for v[0-9]+.[0-9]+.[0-9]+ tags.
Therefore the publish job only needs to distinguish push from PR and
manual validation; its extra refs/tags prefix and hyphen checks duplicate
the trigger filter.

Keep validate as a dependency, the release environment, contents: read,
and id-token: write. PR and workflow_dispatch still validate only; RC
and branch pushes do not match this workflow's push trigger.

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

Actionlint validates the workflow. No tag or release was created.
@tisonkun
refactor(release): use the workflow dispatch response run ID
db9b01e 
Call the GitHub dispatch endpoint with API version 2026-03-10 and read
workflow_run_id from its response. This replaces CLI-output parsing and
polling runs by title, timestamp, and commit, which could select another
concurrent dispatch of the same SHA.

Retain the existing local preconditions, returned-run SHA check, watch,
and final public verification. A failed dispatch or missing run ID stops
immediately; the helper never guesses which run to watch.

API contract: https://docs.github.com/en/rest/actions/workflows#create-a-workflow-dispatch-event

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

An isolated command-stub check passed eight paths: success, dirty checkout,
wrong main, missing reviewers, failed dispatch, missing run ID, wrong run
SHA, and failed watch. Only the returned run ID was watched. These checks
made no real GitHub dispatches.
@tisonkun
refactor(release): express bootstrap preconditions directly
14475d2 
Use test for the zero-argument, clean-checkout, current-main, and returned
run SHA requirements. Fetch main from the fixed Apache repository URL
instead of discovering a matching remote name.

Keep the required-reviewer check, failure-log output, waiting for the
exact run, and final public verification. Remove command-existence and
git-object prechecks plus the duplicate final run summary; command errors
now stop at the operation that needs the command.

Validation: ran 11 tests successfully with:
  python3 -m unittest discover -s .github/scripts/release_rust -p 'test_*.py'
The current 15-crate publish plan, actionlint, Bash syntax, and
`git diff --check` also pass.

Re-ran all eight isolated dispatch scenarios after this cleanup. Dirty,
wrong-main, and unprotected cases never dispatch; a wrong run SHA never
reaches watch or verification. No live bootstrap was triggered.
@tisonkun
tisonkun force-pushed the codex/simplify-rust-release branch from 74ca948 to 14475d2 Compare September 29, 2026 23:03
@tisonkun
tisonkun requested review from Xuanwo and a balanced review from Copilot September 29, 2026 23:18
Copilot AI reviewed Sep 29, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The dispatch request omits return_run_details, so it receives no run ID and the bootstrap helper fails after dispatch.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Simplifies Rust release and crates.io bootstrap automation while preserving dependency ordering, OIDC publishing, and resumability.

Changes:

  • Replaces custom planning with TopologicalSorter.
  • Consolidates bootstrap auditing and reconciliation.
  • Streamlines workflows, dispatching, tests, and documentation.
File Description
release/​scripts/​bootstrap-rust-crates.sh Simplifies protected workflow dispatch.
.github/​workflows/​release.yml Simplifies publish gating.
.github/​workflows/​bootstrap_rust_crates.yml Removes the discovery job.
.github/​scripts/​release_rust/​trusted_publishing.py Consolidates OIDC token handling.
.github/​scripts/​release_rust/​publish.py Simplifies publishing interfaces.
.github/​scripts/​release_rust/​plan.py Uses standard topological sorting.
.github/​scripts/​release_rust/​bootstrap.py Consolidates bootstrap operations.
.github/​scripts/​release_rust/​test_trusted_publishing.py Tests credential lifecycle.
.github/​scripts/​release_rust/​test_publish.py Updates publisher tests.
.github/​scripts/​release_rust/​test_plan.py Tests dependency ordering.
.github/​scripts/​release_rust/​test_bootstrap.py Tests bootstrap boundaries.
.github/​scripts/​release_rust/​README.md Documents revised behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread release/scripts/bootstrap-rust-crates.sh Outdated
@tisonkun
Add return_run_details parameter to workflow dispatch
c8e77d4 
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@Xuanwo
Xuanwo merged commit b6fc432 into main Oct 5, 2026
65 checks passed
@Xuanwo
Xuanwo deleted the codex/simplify-rust-release branch October 5, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@Xuanwo Xuanwo Awaiting requested review from Xuanwo

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tisonkun @Xuanwo