Sort the full-image runtime stage stable-first - #1545
Merged
Conversation
The runtime stage copied the built Java distribution in as its first step, above the DeLFT install, the JEP build and the glove embeddings preload. That copy changes with every grobid commit, so all three expensive-and-stable layers were rebuilt on every build. Move the grobid COPY to the bottom of the stage. The /data symlink moves up with the preload, since resources-registry.json writes the lmdb to a path that resolves through it. The JEP build bind-mounted the JDK from `builder`, tying it to a stage that is rebuilt per commit; it now mounts from a one-line `jdk` stage on the same pinned base image -- same content, no extra pull. The preload still takes two files from the builder, but BuildKit keys COPY --from on their content, so it rebuilds when the registry changes rather than per commit. Also move the /data permission fixing into the RUN that fills it. A recursive chmod in a later layer copies up every file it touches, which put a second copy of the multi-GB embeddings database in the image. The TensorFlow install already sat above the COPY and is left where it is.
The GitHub Actions cache is capped at 10GB per repository with LRU eviction, and mode=max on this image exports well past that -- the torch/CUDA wheels are ~5GB and the embeddings lmdb ~2GB. Entries were evicted between runs, so layers the Dockerfile ordering makes cacheable were rebuilt anyway. Push the cache to lfoppiano/grobid:buildcache, over the Docker Hub login the job already does. ignore-error keeps a failed export from failing the build.
The build matrix ran `assemble test jacocoTestReport` on all four runners, but only ubuntu-latest publishes coverage. Two kinds of waste followed. jacocoTestReport writes per-subproject reports that nothing downstream reads -- coveralls is fed by the aggregate codeCoverageReport task, which depends on the test tasks and derives from their execution data. Drop it from every leg. The larger cost is the jacoco agent itself, which attaches to each forked test JVM. With forkEvery = 1 and 113 test classes that is 113 attachments, on three legs that discard the result. Add a -PskipCoverage opt-out and pass it everywhere except ubuntu-latest. Verified by probing the realized task graph: the jacoco extension reports enabled=true on grobid-core/service/trainer by default and enabled=false under -PskipCoverage, so the publishing leg is unaffected.
lfoppiano
changed the title
lfoppiano
force-pushed
the
perf/docker-full-image-layer-caching
branch
from
August 13, 2026 15:55
19128d9 to
85120d9
Compare
forkEvery = 1 spawned a fresh JVM for every test class -- 109 of them, run strictly one at a time, since maxParallelForks is 1 and org.gradle.workers.max=1 would cap it regardless. Every start paid JVM boot, a 1g heap, the Mockito -javaagent and java.library.path native library setup, so the suite was dominated by process startup rather than by the tests. Measured locally with `test --rerun-tasks -PskipCoverage`, same machine: forkEvery = 1 9m20s forkEvery = 0 4m07s Both ran an identical 812 tests across 109 classes with no failures, so nothing is being skipped for the speedup. This matters most on the slowest CI runner, where the per-fork constant is largest: macos-15-intel spent 29.6 minutes on build-and-test in the last run against 13.0 for ubuntu-latest, on the same commit and the same task list. The tests now share a JVM, so anything relying on per-class isolation -- static state in GrobidProperties, a native library loaded via LibraryLoader -- would show up as a failure rather than being hidden by a fresh process each time. The full suite passing locally is the evidence that nothing currently does.
lfoppiano
force-pushed
the
perf/docker-full-image-layer-caching
branch
from
August 13, 2026 15:57
85120d9 to
c0ca969
Compare
lfoppiano
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The runtime stage copied the built Java distribution in as its first step, above the DeLFT install, the JEP build and the glove embeddings preload. That copy changes with every grobid commit, so all three expensive-and-stable layers were rebuilt on every build.
Move the grobid COPY to the bottom of the stage. The /data symlink moves up with the preload, since resources-registry.json writes the lmdb to a path that resolves through it. The JEP build bind-mounted the JDK from
builder, tying it to a stage that is rebuilt per commit; it now mounts from a one-linejdkstage on the same pinned base image -- same content, no extra pull. The preload still takes two files from the builder, but BuildKit keys COPY --from on their content, so it rebuilds when the registry changes rather than per commit.Also move the /data permission fixing into the RUN that fills it. A recursive chmod in a later layer copies up every file it touches, which put a second copy of the multi-GB embeddings database in the image.
The TensorFlow install already sat above the COPY and is left where it is.