Skip to content

Commit 877c2d0

Browse files
committed
Make the Avro performance gate stable under shared-runner load
1 parent 209369e commit 877c2d0

5 files changed

Lines changed: 453 additions & 77 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,9 +97,34 @@ jobs:
9797
- run: pip install -e '.[dev]'
9898
- run: ruff check src/ tests/
9999
- run: mypy src/durable_workflow/
100-
- name: Enforce optimized Avro Value regression budget
100+
- name: Enforce load-calibrated Avro Value regression budget
101101
run: python benchmarks/avro_value.py --enforce
102102

103+
avro-benchmark:
104+
name: Avro Value absolute throughput (advisory)
105+
needs: qualification-class
106+
if: ${{ needs.qualification-class.outputs.classification == 'complete' }}
107+
runs-on: ubuntu-latest
108+
timeout-minutes: 5
109+
steps:
110+
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
111+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
112+
with:
113+
python-version: "3.12"
114+
- run: pip install -e .
115+
- name: Measure the bounded absolute throughput signal
116+
continue-on-error: true
117+
run: >-
118+
python benchmarks/avro_value.py --enforce-absolute
119+
--output avro-value-benchmark.json
120+
- name: Publish the measured sample summary
121+
if: ${{ always() && github.server_url == 'https://github.com' }}
122+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
123+
with:
124+
name: avro-value-benchmark-${{ github.sha }}
125+
path: avro-value-benchmark.json
126+
if-no-files-found: error
127+
103128
test:
104129
needs: qualification-class
105130
if: ${{ needs.qualification-class.outputs.classification == 'complete' }}
@@ -225,6 +250,7 @@ jobs:
225250
- qualification-class-report
226251
- regression-corpus
227252
- lint
253+
- avro-benchmark
228254
- test
229255
- package
230256
- cli-parity
@@ -238,6 +264,7 @@ jobs:
238264
CLASSIFICATION_REPORT_RESULT: ${{ needs.qualification-class-report.result }}
239265
QUALIFICATION_CLASS: ${{ needs.qualification-class.outputs.classification }}
240266
LINT_RESULT: ${{ needs.lint.result }}
267+
AVRO_BENCHMARK_RESULT: ${{ needs.avro-benchmark.result }}
241268
CORPUS_RESULT: ${{ needs.regression-corpus.result }}
242269
TEST_RESULT: ${{ needs.test.result }}
243270
PACKAGE_RESULT: ${{ needs.package.result }}
@@ -248,6 +275,7 @@ jobs:
248275
test "$CLASSIFICATION_REPORT_RESULT" = success
249276
if [ "$QUALIFICATION_CLASS" = focused-documentation ]; then
250277
test "$LINT_RESULT" = skipped
278+
test "$AVRO_BENCHMARK_RESULT" = skipped
251279
test "$CORPUS_RESULT" = skipped
252280
test "$TEST_RESULT" = skipped
253281
test "$PACKAGE_RESULT" = skipped
@@ -256,6 +284,7 @@ jobs:
256284
echo "Focused documentation qualification selected; Docs PR checks and Public Boundary remain required."
257285
elif [ "$QUALIFICATION_CLASS" = complete ]; then
258286
test "$LINT_RESULT" = success
287+
test "$AVRO_BENCHMARK_RESULT" = success
259288
test "$CORPUS_RESULT" = success
260289
test "$TEST_RESULT" = success
261290
test "$PACKAGE_RESULT" = success

‎benchmarks/README.md‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,22 @@
11
# Avro Value benchmark
22

33
`python benchmarks/avro_value.py --enforce` loads the checked-in medium corpus
4-
at `schema/avro-value-benchmark-v1.json` (SHA-256
5-
`588771404977f2a95fe7d8969c24a15e1c7dd78fe498af9aa2406f82be54b666`).
6-
The bytes sentinel is adapted only for the fixed typed path; compact JSON and
7-
the removed wrapper use the corpus's documented JSON representation.
4+
at `schema/avro-value-benchmark-v1.json`. It hard-checks the corpus identity,
5+
typed wire identity, round-trip fidelity, and exact wire sizes. The bytes
6+
sentinel is adapted only for the fixed typed path; compact JSON and the removed
7+
wrapper use the corpus's documented JSON representation.
88

99
A release qualification run in the Linux Python 3 worker measured about 65 µs
10-
to encode and 43 µs to decode through `fastavro`. The enforced 125/100 µs
11-
defaults allow about 2x shared-runner variance while remaining far below the
12-
slow recursive reference path. Set `AVRO_VALUE_ENCODE_BUDGET_US` and
13-
`AVRO_VALUE_DECODE_BUDGET_US` to calibrate a different qualification runner.
10+
to encode and 43 µs to decode through `fastavro`. Routine CI warms the codec,
11+
takes seven paired samples in alternating order, and enforces the median ratio
12+
between the production path and direct typed fastavro work from the same run.
13+
The default encode/decode ratio ceilings are 1.75/2.00 and can be calibrated
14+
with `AVRO_VALUE_ENCODE_RATIO_BUDGET` and
15+
`AVRO_VALUE_DECODE_RATIO_BUDGET`.
16+
17+
The dedicated advisory benchmark runs with `--enforce-absolute`, publishes all
18+
samples and median/p95 summaries, and retains the 125/100 µs absolute budgets.
19+
Those findings remain visible without making shared-runner throughput a generic
20+
lint failure. Set `AVRO_VALUE_ENCODE_BUDGET_US` and
21+
`AVRO_VALUE_DECODE_BUDGET_US` only when using a benchmark runner with a known
22+
performance envelope.

0 commit comments

Comments
 (0)