Skip to content

Load FileWatching before precompilepkgs in ensurecompiled - #1135

Merged
KristofferC merged 3 commits into
masterfrom
kc/ensurecompiled-filewatching
Sep 24, 2026
Merged

KristofferC merged 3 commits into
masterfrom
kc/ensurecompiled-filewatching

Conversation

@KristofferC

Copy link
Copy Markdown
Member

Fixes #1134.

Since #1131, ensurecompiled runs Base.Precompilation.precompilepkgs() under the base sysimage. On Julia 1.11 precompile_pkgs_maybe_cachefile_lock does Base.loaded_modules[FileWatching], which assumes FileWatching is already loaded. That holds for the default sysimage but not for the stdlib-free base sysimage of incremental=false builds, so every precompile task died with a KeyError that is swallowed inside the @async task, and precompilepkgs() printed Precompiling project... and exited 0 having compiled nothing. This is visible in the libtrixi CI logs: v2.4.1 precompiled 273 packages in that step, v2.4.3 spent 3 seconds and compiled none.

The --output-o step then loaded every package from source. Since Julia 1.11 jl_atexit_hook runs Base._atexit before jl_write_compiler_output, so every sysimage carries _atexit_hooks_finished = true, and a nonincremental --output-o process skips the Base.__init__ that resets it. Packages whose top-level code calls atexit, like Downloads (its precompile block calls Curl.__init__()), therefore fail with cannot register new atexit hook; already exiting. when loaded from source there. Loading from a cache file avoids running that top-level code.

Pkg.precompile() did not hit this because loading Pkg loads FileWatching. Julia 1.12+ use Base.trymkpidlock_hook with a return f() fallback when the hooks are undefined, so they are unaffected; loading FileWatching there is harmless.

The new test builds a nonincremental sysimage with Downloads and no precompile execution file, which is the only configuration where the cache files can come from ensurecompiled alone (the existing app/library tests all run a precompile script under the base sysimage, which compiles the caches through regular loading). Verified locally on Julia 1.11.9 with a fresh depot: it fails with the error from the issue without the fix and passes with it.

🤖 Generated with Claude Code

KristofferC and others added 3 commits September 24, 2026 13:59
Fixes #1134.

Since #1131, `ensurecompiled` runs `Base.Precompilation.precompilepkgs()`
under the base sysimage. On Julia 1.11 `precompile_pkgs_maybe_cachefile_lock`
does `Base.loaded_modules[FileWatching]`, which assumes `FileWatching` is
already loaded. That holds for the default sysimage but not for the
stdlib-free base sysimage of `incremental=false` builds, so every precompile
task died with a `KeyError` that is swallowed inside the `@async` task, and
`precompilepkgs()` exited successfully having compiled nothing.

The `--output-o` step then loaded every package from source. Since Julia 1.11
`jl_atexit_hook` runs `Base._atexit` before `jl_write_compiler_output`, so
every sysimage carries `_atexit_hooks_finished = true`, and a nonincremental
`--output-o` process skips the `Base.__init__` that resets it. Packages whose
top-level code calls `atexit`, like `Downloads` (its precompile block calls
`Curl.__init__()`), therefore fail with
`cannot register new atexit hook; already exiting.` when loaded from source
there. Loading from a cache file avoids the top-level code.

`Pkg.precompile()` did not hit this because loading `Pkg` loads
`FileWatching`. Julia 1.12+ use `Base.trymkpidlock_hook` with a fallback, so
they are unaffected; loading `FileWatching` there is harmless.

Adds a nonincremental `create_sysimage(["Downloads"])` test without a
precompile execution file, which is the only way the cache files can come from
`ensurecompiled`. It fails with the error from the issue without the fix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.19%. Comparing base (4e98b47) to head (cf008e5).

❗ There is a different number of reports uploaded between BASE (4e98b47) and HEAD (cf008e5). Click for more details.

HEAD has 6 uploads less than BASE
Flag BASE (4e98b47) HEAD (cf008e5)
7 1
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1135      +/-   ##
==========================================
- Coverage   83.93%   74.19%   -9.75%     
==========================================
  Files           3        3              
  Lines         909      903       -6     
==========================================
- Hits          763      670      -93     
- Misses        146      233      +87     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@KristofferC
KristofferC merged commit 757ccf4 into master Sep 24, 2026
12 of 13 checks passed
@KristofferC
KristofferC deleted the kc/ensurecompiled-filewatching branch September 24, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PackageCompiler v2.4.3 fails with cannot register new atexit hook

1 participant