ci: add scheduled benchmark workflow#2828
Conversation
Runs both benchmark suites weekly (every Monday at midnight UTC) and on manual dispatch. Criterion results are tracked via benchmark-action/github-action-benchmark and will alert on any regression exceeding 150% of the stored baseline. Transaction cycle counts are uploaded as a workflow artifact each run. Closes 0xMiden#1671
There was a problem hiding this comment.
Pull request overview
Adds a new GitHub Actions workflow to run the repository’s benchmark suites on a weekly schedule and via manual dispatch, and to persist/track results for regression monitoring.
Changes:
- Introduces a scheduled (
cron) + manual (workflow_dispatch) benchmark workflow. - Runs both benchmark suites (
make bench-note-checker,make bench-tx) and uploads transaction cycle counts as an artifact. - Publishes Criterion benchmark results via
benchmark-action/github-action-benchmarkwith regression alerting.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| save-if: true |
There was a problem hiding this comment.
Swatinem/rust-cache is configured to always save the cache (save-if: true). Other CI workflows in this repo intentionally only save the Rust cache on specific pushes to avoid hitting GitHub’s 10GB cache limit and causing evictions (e.g. .github/workflows/build.yml:30-33). Consider using a similar conditional here (e.g. only on schedule runs, or only when github.ref is the default branch) so the weekly benchmark job doesn’t churn caches for the rest of CI.
| save-if: true | |
| save-if: ${{ github.event_name == 'schedule' }} |
| name: performance regression check | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@main |
There was a problem hiding this comment.
This workflow uses floating action refs (actions/checkout@main). Pinning GitHub Actions to a stable version tag (e.g. @v4) or, ideally, a commit SHA reduces supply-chain risk and avoids unexpected behavior changes when upstream updates land on main.
| - uses: actions/checkout@main | |
| - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 |
| path: bin/bench-transaction/bench-tx.json | ||
|
|
||
| - name: Store benchmark results | ||
| uses: benchmark-action/github-action-benchmark@v1 |
There was a problem hiding this comment.
benchmark-action/github-action-benchmark@v1 is also a floating tag. Consider pinning it to a specific commit SHA (or at least a fully-qualified release tag) to reduce supply-chain risk for a workflow that has contents: write permissions and can push data back to the repo.
| uses: benchmark-action/github-action-benchmark@v1 | |
| uses: benchmark-action/github-action-benchmark@v1.20.0 |
|
closing as not meeting our https://github.com/0xMiden/protocol?tab=contributing-ov-file#contribution-quality:
please refrain from opening further PRs unless assigned, thank you! |
Runs both benchmark suites weekly (every Monday at midnight UTC) and on manual dispatch. Criterion results are tracked via benchmark-action/github-action-benchmark and will alert on any regression exceeding 150% of the stored baseline. Transaction cycle counts are uploaded as a workflow artifact each run.
Closes #1671