Repository navigation
Per-shard coverage artifacts ship 8 copies of the source tree — 85.7% of the payload is text collect already has checked out #74
Description
Sharded coverage upload is 64× larger than it needs to be, and it exhausted the Actions artifact storage quota on lab-sotashimozono today — every PR in that org went red at Shard labels with
Failed to CreateArtifact: Artifact storage quota has been hit.
The shard jobs had already done their work; only the upload failed. No test ran on any PR after that point.
Where the bytes are
Julia writes *.cov as a 9-character counter column, one space, then a verbatim copy of the source line. Measured on one live testshards-coverage-s1 artifact from lab-sotashimozono/ParaLinearAlgebra.jl (48 files, 965,565 B uncompressed):
| bytes | share | |
|---|---|---|
| counter columns | 136,629 | 14.3 % |
| embedded source text | 815,592 | 85.7 % |
and only 2,134 of 15,181 lines (14.1 %) carry a counter at all — the rest are -.
Each of the 8 shards uploads the whole tree, not the files it happened to touch, so every run ships 8 near-identical copies of the package's source through artifact storage. Live totals in that one repo:
| family | objects | live |
|---|---|---|
| coverage | 625 | 170.84 MB |
| out | 2410 | 28.46 MB |
| records | 503 | 21.80 MB |
| lcov | 298 | 4.63 MB |
| timings | 81 | 0.12 MB |
Per run: 8 coverage artifacts totalling 2.30 MB are merged into one lcov of 0.02 MB. The merged product is 1 % of its inputs.
The source is already at the destination
collect runs actions/checkout before download-artifact — deliberately, and the comment says why (Codecov needs the file network). So the 85.7 % that is source text is already present in that job's workspace, at the same paths, at the same revision. Verified: for all 48 files in that artifact, the text embedded in the .cov is byte-identical to the repo file at the run's head_sha.
Proposed change
Upload a sidecar of counted lines only — <lineno>,<count> — and have restore_counters rebuild the .cov files from the checkout.
Prototyped against the real artifact:
original .cov payload : 965,565 bytes
sidecar payload : 15,049 bytes -> 64.2x smaller
round-trip byte-identical: 48/48 files
Round trip is counter + checked-out source line -> "%9s %s", and it reproduces the original .cov exactly, so everything downstream of restore_counters — CoverageTools, the lcov merge, the Codecov upload — is untouched.
This does not re-open the reason the conversion lives in collect
sharded-tests.yml records it:
and
collectmerges the results anyway, so doing it here does it N times for one answer. Measured on this repository it cost 3-25s per shard, the variance being a 20s JuliaSyntax precompile
That cost is Julia startup, not the conversion. The sidecar is a line filter — awk, no Julia, no depot, no precompile — so the objection does not apply to it. The merge still happens once, in collect, exactly as now.
Where it lands
sharded-tests.yml, "Collect this shard's raw coverage counters": write<path>.cov.idxinstead ofcping the.cov.src/coverage.jl,restore_counters: accept.cov.idxand reconstruct againstdest. Keeping it in the package rather than in the workflow is that file's own stated principle ("a merge written as a step script is unreachable fromPkg.test()").- Backward compatible: a
parts/tree containing real.covfiles still restores by copy, so an in-flight run mid-upgrade is fine.
One new failure mode, and it should be loud
Reconstruction needs the source file present at dest with the same line count. If a counter file has no matching source, or the line counts disagree, that must refuse by name rather than silently emit a short .cov — a truncated counter file would under-report coverage with nothing to announce it. Worth carrying the line count in the sidecar header so the check is exact.
Happy to send the PR.
Activity
Coverage counters can travel without the source they are printed agai…
A full artifact storage quota must not fail a green suite (#74) (#76)
Making the uploads non-fatal moved the failure to the downloads (#74) (…
sotashimozono commentedon Aug 19, 2026
After #76 and #77: the suite runs, and the gate still cannot pass — for a reason worth separating out
Verified on a consumer PR under a full quota. All eight shards now execute and pass (before #76 nothing ran at all). The run then fails at exactly one place:
That is correct and should stay correct:
Every unit ran, exactly onceis a claim, the records did not arrive, so the claim cannot be made. Faking it would be the one failure this package exists to prevent.But it exposes a design point that #76/#77 only worked around:
The completeness records share a transport — and therefore a failure mode — with the coverage payload, and they are four orders of magnitude smaller.
shard-*.tsv/ran-*.tsvare a few KB of TSV; the coverage artifacts in the same run were 170 MB in one repo. When artifact storage is the constrained resource, putting a 2 KB correctness record behind it means a reporting problem can永久ally block a correctness gate, and there is no honest way to unblock it from inside the workflow.Worth considering a different carrier for the completeness data specifically — job outputs, or the summary — so that the one thing that must not degrade does not depend on the resource that ran out. Not proposing a patch here; it is a design decision about where the guarantee lives.
Smaller, and worth knowing while diagnosing
continue-on-error: truereports the step's conclusion assuccessin the jobs API even when the upload failed, so a step list reads as though everything uploaded. The only reliable signal is that the run has zero artifacts. That cost me a wrong reading; a::warning::next to each tolerated upload would make the log say what the step list does not.sotashimozono commentedon Aug 19, 2026
Correction to the storage table; the payload measurement stands.
The "live" column here was summed without filtering
expired, and an expired artifact keeps itssize_in_bytesmetadata row after GitHub deletes the bytes. Re-measured withexpired == false, across the four largest consumers:-out-*-records-lcov-coverage-*So
-coverage-*is not 170 MB of stored bytes — it is zero in steady state, because its retention is one day. The 170 MB was cumulative-created. Account-wide live total is 90.7 MB against a documented 500 MB pool, so this family was never the storage problem, and the claim that it "exhausted the artifact storage quota" is not supported.What is unaffected: the content measurement. Julia writes
*.covas a 9-character counter column plus a verbatim copy of the source line; 815,592 of 965,565 bytes (85.7 %) on a real 48-file artifact are that text, and only 2,134 of 15,181 lines carry a counter. That is a fact about the payload, not about storage totals, and it is why #75 exists.Why this stays open: the reason to fix it changes from "storage" to "the bytes an in-flight run has to create" — which is the thing a quota gate actually blocks on, since the quota is enforced at
CreateArtifactbefore any bytes are sent. #75 shippedcounter_index/restore_counters(~62× measured, round-trip byte-exact) and the workflow still plain-cps the raw.covatsharded-tests.yml:611-624. This issue is the home for that ~5-line wiring.#81 lists it as item 2; treat this issue as authoritative for it.
sotashimozono commentedon Aug 20, 2026
One more data point on the transport question, so the obvious alternative is not re-derived later: job outputs do not work for this. The workflow already uses
GITHUB_OUTPUT(thematrixinlabels,haveincollect), but those are single jobs. A matrix job's outputs collide — N shards writing the same output name leave one value — so per-shard records cannot be aggregated that way.GITHUB_STEP_SUMMARYis human-readable only and not reachable fromcollect.That leaves reading the shard jobs' logs from
collectthrough the Actions API, which works but trades a file for a token and a parse. Recording it as the remaining option rather than proposing it.feat: the completeness evidence gets a second road that a full artifa…
sotashimozono commentedon Oct 14, 2026
Hermes の整理（自動。何も変更していません）
状態: 本文の中心である storage 枯渇の主張は、2026-08-19 のコメントで撤回済みです (
-coverage-*は retention 1 日で定常 0 MB、account 全体 90.7 MB / 500 MB)。ただし本文はそのままなので、初見の人は撤回前の表を読みます。残る論点は payload 計測 (85.7% が source text) だけです。論点: 残作業が何かの食い違いです。
counter_index/restore_counters,src/coverage.jl), A full artifact storage quota must not fail a green suite (#74) #76 / Making the uploads non-fatal moved the failure to the downloads (#74) #77 (upload を non-fatal に), feat: the completeness evidence gets a second road that a full artifact quota cannot close (#74) #85 (completeness の第二経路。cache 経由)。コメント 1 の「completeness を別の carrier に」は feat: the completeness evidence gets a second road that a full artifact quota cannot close (#74) #85 で実現済みと読めます。コメント 3 の通り job outputs は matrix で衝突するので不可。sharded-tests.ymlの "Collect this shard's raw coverage counters" は今も.covをcpしてそのまま upload します (main の該当 step 付近、counter_indexは workflow のどこにも出ません)。コメント 2 が言う「~5 行の配線」は未着手と読めます。次の一手: 本文冒頭に撤回の注記を足す (または題を「
.covの source 複製を送らない」に直す)。残作業を「counter_indexの workflow 配線」1 件に絞ると、close 条件が明確になります。関連: #75, #76, #77, #85 (出荷済みの対応), #81 (CLOSED, 配線を本 issue に委ねた), #78 / #80 / #82 (CLOSED, artifact を使わない設計の議論)。