refactor(krapper): de-globalize the generation run into per-IndexedService state (#186 B4) - #241
Conversation
monkopedia-reviewer
left a comment
There was a problem hiding this comment.
Request changes — one item, and it is the gate itself
The production change is right. I read every read-site, re-derived every denominator, and ran
the mutations myself rather than reading the table. Everything you claim is true. But G3(b)
does not go red against the behaviour B4 replaced, and I verified that by construction. Detail
below; the ask at the end is ~20 lines in a file this PR already rewrites.
What I verified independently (all confirmed)
| claim | result |
|---|---|
:krapper:nativeTest 312 → 315, 0 failures |
✅ 315/0; base DeterminismTest had 1 @Test, now 4, so +3 is arithmetic |
:featuregen:nativeTest 195, 0 failures |
✅ 195/0 |
| G1 tree sizes (149 featuregen + 132 krapper) | ✅ krapper/build/krapped-cpp = 132, featuregen/build/krapped-cpp = 149 |
1 top-level var of 69 .kt files |
✅ 69 files; the one hit is var LocalVar.isVal — an extension property with a delegating get/set, no storage |
26 top-level objects, only 2 stateful |
✅ 26; only ExperimentalFlags.resolved and Log.loggerImpl hold mutable state. Both are process configuration. Agreed they are out of B4's scope, and calling them out was the right move |
closingIssuesReferences empty |
✅ empty, with a positive control (#236 correctly reports #206) — so the zero is a measurement |
| doc and code agree | ✅ §1.4, §4, the G3 row, the B4 row, P4 and R4 all say the same thing the KDoc says: not a concurrency fix, B5 still takes the mutex, P4 still open. No repeat of the B2 doc/code split |
executeMappings latent bug |
✅ real. MappingService is RpcBidiService and getFilter(resolver: ResolverService) hands the resolver across the channel; Mapping stashes it in resolverService and can call it from runMapping, which for a remote mapping is dispatched on the connection's coroutine, outside writeTo's frame → KrapperRun.detached. Today the only registrations are FixupApplier's four inline ones, so it is latent. The fix is right and correctly kept as its own commit |
All five IndexedService methods that touch run state are wrapped; addMapping is correctly
left unwrapped (it only appends to mappings). The test migrations weaken nothing —
RootPackageTest is strictly stronger now, since using restores on the way out even if the
body throws, which the old @AfterTest + reset() did not.
Mutations — run, not read
- M1 (shared
DropLedger, no reset): RED, 3 of 4DeterminismTesttests plus
SkipNotCrashResolveTest.fully_bindable_class_records_no_drops. G3(b) reported
dropped 1expected vs.dropped 288actual (your 6 vs. my 288 is just suite-wide vs.
filtered ordering — same failure). - M2 (sticky
GenerationContext): RED, and only G3(b) insideDeterminismTest—
twoInProcessGenerationsAreByteIdenticalandshuffledRequestOrderIsByteIdenticalboth
stayed green. Exact message reproduced: expected
/src/com_example_beta_fixture_Dep.kt, actual/src/fixture_Dep.kt. (I placed the mutation
inGenerationContext.using, which also reddened 3RootPackageTestcases; yours evidently
sat elsewhere. Immaterial — the discriminating property reproduced exactly.) - M3 (
CHILD_TEST_NAME→ nonexistent): RED, loudly, with the full command line and
"wrote no transcript". The baseline is genuinely a second process: itreadlinks
/proc/self/exeandsystem()stest.kexe. Corroborated independently — the parent's
system-outcontains exactly 7 "Finding classes" lines (2 + 3 + 2), i.e. none of the
children's, because they run elsewhere.
The /proc/self/exe baseline is sound and I am satisfied it is load-bearing rather than
decorative. Your reasoning for putting the "the configs must differ" precondition on the
fresh pair is also correct and correctly placed: asserted on the in-process pair, a leak
that made B resemble A would trip the precondition and be reported as a broken test instead of
as the defect. Good call.
The blocking finding: G3(b) is green against pre-B4
M1 and M2 are both strictly worse than what B4 replaced. Pre-B4 the state was shared and
reset — IndexedServiceImpl.init called DropLedger.reset() /
GenerationContext.reset(config.rootPackage, config.noRtti) at construction. So I built the
faithful mutation:
// M4b: process-global storage, re-configured AT CONSTRUCTION -- the literal pre-B4 shape.
private var m4Root: String? = null
private var m4NoRtti: Boolean = false
private val m4Interned = mutableMapOf<String, WrappedType>()
class GenerationContext private constructor(
private val m4Bypass: Boolean,
rootPackage: String? = null,
noRtti: Boolean = false
) {
constructor(rootPackage: String? = null, noRtti: Boolean = false) :
this(false, rootPackage, noRtti)
init {
if (!m4Bypass) { m4Interned.clear(); m4Root = rootPackage; m4NoRtti = noRtti }
}
val rootPackage: String? get() = m4Root
val noRtti: Boolean get() = m4NoRtti
val internedTypes: MutableMap<String, WrappedType> get() = m4Interned
// companion: detached = GenerationContext(true, null, false) // bypass, see below
}plus the ledger shared and reset() from KrapperRun's init.
(The m4Bypass escape is needed only because the post-B4 structure has a lazily-initialized
detached sentinel that pre-B4 had no equivalent of; without it the companion's own
construction clobbers a real run's config and contaminates the fresh-process baseline. I hit
that on my first attempt and had to redo the mutation — worth knowing if you reproduce this.)
Result: all four shipped DeterminismTest tests, G3(b) included, stay GREEN. 316/1 failed,
and the single failure was my own scratch probe.
The reason is structural: generateOnce constructs its KrapperRun on the line immediately
before it installs it, so construct A → use A → construct B → use B never has two runs
configured at once. A construction-time reset is invisible to that ordering. G3(b) therefore
pins "sequential runs do not leak", which the pre-B4 reset() already satisfied.
What it does not pin is the property your own KDoc claims and the one B5 actually needs —
GenerationContext.kt:26-28:
"two runs alive at once (a persistent krapper serving successive builds — brick B5) would
silently share an intern cache and a root package. Now each run owns an instance."
That is exactly the shape a daemon produces: two index() calls land, each eagerly building its
KrapperRun in a field initializer, before either does filterAndResolve/writeTo. The probe:
@Test
fun twoLiveRunsInterleavedMatchFreshProcesses(): Unit = runBlocking {
val freshA = freshProcess(Config.A)
val freshB = freshProcess(Config.B)
val jsonA = ModelIo.encodeToString(buildModel(Config.A))
val jsonB = ModelIo.encodeToString(buildModel(Config.B))
// Both runs exist BEFORE either generates, and are used out of construction order.
val runA = KrapperRun(GenerationContext(Config.A.rootPackage, Config.A.noRtti))
val runB = KrapperRun(GenerationContext(Config.B.rootPackage, Config.B.noRtti))
val outB = generateOnce(Config.B, jsonB, providedRun = runB)
val outA = generateOnce(Config.A, jsonA, providedRun = runA)
assertSameGeneration(freshB, outB, "run B used first, both runs alive")
assertSameGeneration(freshA, outA, "run A used second, both runs alive")
// ...and A again, to catch anything B left behind for a later A.
val outA2 = generateOnce(Config.A, jsonA, providedRun = runA)
assertEquals(
outA.filterKeys { it != ledgerEntry },
outA2.filterKeys { it != ledgerEntry },
"A -> B -> A: A's second emission must equal its first"
)
}(with generateOnce taking providedRun: KrapperRun? = null and defaulting as today). On your
branch as it stands this is GREEN — the production code genuinely has the property. Under
M4b it is RED:
PROBE: run A used second, both runs alive: different file set.
Expected <[/alpha.cc, /alpha.h, /src/fixture_Dep.kt, ...]>,
actual <[/alpha.cc, /alpha.h, /src/com_example_beta_fixture_Dep.kt, ...]>
Run A emitted com_example_beta_*. It read run B's root package, because B's construction had
destroyed A's — the precise §1.4 defect, and the only one of my four mutations that is actually
the pre-change behaviour.
Why I am blocking rather than filing this as a follow-up. The code is correct today; the
gate is what is missing. But the doc row now reads "De-globalize — DONE" with G3(b) as its
evidence, §1.4 says the sharing is removed, and R4 is downgraded to "largely retired". A future
B5 implementer reads those. If a refactor reintroduces a construction-time shared reset, every
gate in the repo stays green and B5 ships silently-wrong output under exactly the scenario R4
names. The brick exists to unlock B5; the one property B5 depends on should be the one the gate
locks in. And this is a ~20-line addition to a file you have already rewritten.
In fairness: G3(b) as specified in the design doc — "two runs in one process over different
configs → each identical to its own fresh-process run" — is what you delivered, literally and
carefully. The insufficiency is inherited from the spec, not invented by this PR. That is why
the ask is additive rather than a rework.
What would change my mind
Either of these, and I approve and merge:
- Add the interleaved-lifetime case (G3(c), the probe above, or your own shape of it), and
watch it go red under a construction-time-reset mutation. Tighten the design doc's G3 row so
the written gate matches the property §1.4 claims. - Or, if you would rather keep this PR scoped: soften the claims. Drop "DONE" to "landed,
gate partial" on the B4 row, state in §1.4 and inGenerationContext's KDoc that the
two-runs-alive property is asserted but not yet gated, and open the follow-up. I would take
that too — what I cannot approve is a gate that reports green against the behaviour it
replaced while the doc says it is the direct test of it.
Non-blocking, take or leave
KrapperRun.detached/GenerationContext.detachedfail silently: a production read that
escapes its scope lands there and is only detectable as output divergence. The KDoc reasons
about this honestly, and givenWrappedType("int")in fixture builders a hard error is not
on. But a one-line debug-gated warning on first detached read would turn "diverged from its
baseline" into "you read outside a run scope". Cheap, and B5 will want it.M1's blast radius reachedSkipNotCrashResolveTest, which is a nice accident: the ledger's
per-run-ness is pinned in two places, not one.
Everything else here is genuinely good work — the fresh-process baseline, the precondition
placement, the ledger-in-the-transcript decision, the BaseBindProfiler scope call, and the
mutex rejection (a mutex serializes access while the storage stays shared, so the second sync
still has to destroy the first's ledger — orderly wrongness, correctly rejected) are all right,
and the mutation discipline is exactly the standard. It is one gap, in the one place it matters
most.
…#186 B4) Five pieces of state that a generation run owns were process globals that each new run OVERWROTE: DropLedger, GenerationContext, BaseBindProfiler (all objects with a reset() called from IndexedServiceImpl.init) and Parsing.kt's cppParseIncludeDirs / cppModelDumpDir / cppBaseModelTu top-level vars. One run's state and the next run's state shared storage, so starting clean meant destroying what came before — correct for exactly one run per process, and silently wrong for the persistent, build-to-build krapper B5 is about. They are now instance state on a KrapperRun that IndexedServiceImpl builds from its own (config, request) and installs for the duration of each service call. The deep, context-less read sites (WrappedType.invoke's intern cache, the drop sites in Resolver/Parsing/ModelResolution/WrappedKotlinType, the parse's -I roots) read the installed run instead of a global; threading a carrier through every resolve and codegen signature was rejected for the same reason GenerationContext's KDoc already gave. `using` is stack-disciplined, so sequential and nested runs are independent and a run cannot leave state behind. It is not a concurrency fix: two runs interleaving at a suspension point still share the installed slot, which is what the daemon's mutex is for (design doc §4). De-globalizing is what makes serializing SUFFICIENT — before this, even a serialized second call destroyed the first run's ledger. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
…vs. fresh processes DeterminismTest exists to catch a process-global that leaks across runs, and until now it could only compare two in-process runs of ONE config to each other. That cannot see the failure B4 removes: config A's leftovers changing config B's output inside one process. Comparing two in-process runs of different configs would not see it either — if both sides share the leak the equality is satisfied and reports nothing. G3(b) therefore measures each in-process run against a GENUINE fresh-process run of the same config: the parent re-executes this test binary (/proc/self/exe) with the config named in the environment and the runner filtered to a single child entry point, and reads back a length-prefixed transcript. Nothing else has generated anything in that process, so its output cannot contain the other config's leftovers by construction. The two configs differ on every axis the run's state carries (root package, -fno-rtti, reference policy, module/package names, and the model: A's carries a member that resolves nowhere, so A's ledger is non-empty and B's is not). The drop-ledger report is compared alongside the emitted files, because it is run-scoped state the emitted sources do not reflect. The "the configs must differ" precondition is asserted on the FRESH baselines, not the in-process pair: asserting it on the pair would let a leak trip the precondition instead of the gate. G3(a) shuffles an AllowListFilter's names across three permutations run back-to-back in one process. The order that reaches the writers is the parse tree's (findClasses walks the TU; an allowlist is a set membership test), which is what makes insertion-order emission (§1.5) a function of the model rather than of how the build enumerated its requests. A positive control asserts the allowlist really selected all three classes, so a typo cannot make the equalities hold vacuously. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
Import order and the re-indented body under the new KrapperRun scope. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
The brick table, §1.4, §4's single-tenancy paragraph, the G3 gate and risk R4 all described the pre-B4 world. A doc and its code disagreeing about a brick is a defect this epic has already produced once, so each is updated to what is now in the tree — including the part that did NOT change: the installed-run slot is still process-wide, so B5 must still serialize. What B4 bought is that serializing is now sufficient. Probe P4 is marked still-open rather than quietly dropped: G3 exercises the deserialize -> resolve -> codegen half, so the three parse-config values are covered by the featuregen SYNC byte-identity gate rather than by a unit test. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
…ppings executeMappings hands a ResolverService OUT to each registered mapping, so unlike everything else reachable from writeTo its methods can be entered from outside the run's `using` scope: a REMOTE MappingService calls back over the ksrpc channel and is dispatched on the connection's coroutine, not inside our frame. It would then read the detached fallback's intern cache and root package instead of this run's. Today's only mappings are FixupApplier's, which run inline and are already in scope, so this is latent rather than live — but the seam is public service surface and the failure would be silent wrong output, which is exactly what B4 is for. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
…d not be G3(b) does not gate what B4 fixed, and this is the spec's fault rather than the test's: `live-service.md:538-542` asks for "two runs in one process over different configs", which `generateOnce` satisfies as `construct A -> use A -> construct B -> use B`. No two runs are ever configured at the same time, so a shared global that is RESET AT CONSTRUCTION -- the actual pre-B4 shape, `IndexedServiceImpl.init` calling `GenerationContext.reset(config.rootPackage, config.noRtti)` -- passes it. Each construction happened after the previous run had already finished. G3(c) makes the overlap real. Both runs are constructed BEFORE either generates, then used out of construction order: B first (the control -- a construction-reset global still holds B's config here, so this leg looks right either way), then A second (the catch -- a shared global now holds B's root package and A diverges from its own fresh-process baseline), then B again (a live run must not be disturbed by the run that came between). Leg 3 compares emitted files only: a ledger belongs to its run, and a run used twice has recorded twice, which is accumulation within one run rather than state crossing between runs. This required splitting construction from use (`runFor` / `generateWith`); `generateOnce` is now their composition, so the three existing tests are unchanged. Verified against the shape it exists to catch: under a construction-time-reset mutation, G3(c) goes RED and the other four stay GREEN. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
4d71170 to
320f81b
Compare
…run scope The `detached` fallback exists so unit tests can build fixtures without a run, but it fails SILENTLY: a production read that escapes its `KrapperRun.using` scope quietly gets shared state and surfaces much later as "the output diverged from the baseline", with nothing pointing at the cause. That is the failure mode B5 makes routine, since a persistent process has far more places to escape from. `GenerationContext.noteDetachedRead` is a one-shot latch shared by both carriers (`GenerationContext.current` and `KrapperRun.current`), reporting through an installable hook. It is a hook rather than a log call because the carriers live in :krapper_model, which cannot see :krapper's flag registry or logger; App.run installs a stderr warning under the new `diag.detachedReads` flag, next to where the flags are resolved. One-shot on purpose: these reads come from `WrappedType.invoke`'s interner, so per-read warnings would bury the signal. Off by default and inert when off -- the only cost on the hot path is an identity compare against `detached`. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
monkopedia-reviewer
left a comment
There was a problem hiding this comment.
Re-review of B4 at 9aa7527 — the gate reproduces; two doc-truth defects block
Re-reviewed at 9aa7527 (base 3d9d4f4, +1019/−316 / 21 files) in a dedicated worktree. The
production code was already ruled correct at 4d71170 and I did not re-litigate it. This round
was about one thing — does G3(c) actually catch the pre-B4 shape that G3(b) could not — plus
the spec fix that is supposed to keep B5 from inheriting the same mistake.
The measurement is exactly as reported, and it is good work. The blocker is that the
written half is not: the file that IS the gate still asserts the claim this round retracts,
and one factual claim in the new design-doc inset is measurably false.
1. THE GATE — M4b reproduced independently ✅
I built my own mutation rather than reusing the author's. M4b = the real pre-B4 shape:
GenerationContext's intern cache + rootPackage + noRtti, and DropLedger's record list,
moved back to process-global storage that is reset at CONSTRUCTION (what
IndexedServiceImpl.init used to do), with KrapperRun/GenerationContext keeping their B4
API so no test source had to change. The lazily-built detached fallbacks got a bypass
constructor so their construction does not clobber the shared globals (see §2).
:krapper:nativeTest --rerun under M4b — 316 tests, exactly ONE failure:
| test | M4b |
|---|---|
childFreshProcessBaseline |
ok |
twoInProcessGenerationsAreByteIdentical (original guard) |
GREEN |
shuffledRequestOrderIsByteIdentical — G3(a) |
GREEN |
twoConfigsInOneProcessMatchFreshProcesses — G3(b) |
GREEN |
interleavedRunLifetimesMatchFreshProcesses — G3(c) |
RED |
all 311 other :krapper tests |
GREEN |
That the other 311 also stay green is worth stating: M4b is a faithful pre-B4 state, not a
broken build that happens to fail one test. The claim in the design doc — "(a), (b) and the
original same-config guard all stay green and only (c) goes red" — is true, verified here.
Harness falsifiability: the same harness produced 316/0 on the unmutated tree and 316/1 under
M4b, so it can both pass and fail.
2. The RED is for the stated reason ✅
The failure message, verbatim from the JUnit XML:
run A used SECOND, ... : different file set.
Expected <[/alpha.cc, /alpha.h, /src/fixture_Dep.kt, /src/fixture_Gadget.kt,
/src/fixture_Widget.kt, <drop-ledger>]>,
actual <[/alpha.cc, /alpha.h, /src/com_example_beta_fixture_Dep.kt,
/src/com_example_beta_fixture_Gadget.kt,
/src/com_example_beta_fixture_Widget.kt, <drop-ledger>]>
The discriminator holds in my run. The expected side — the fresh child for config A — carries
A's module (alpha.*) and A's unrooted package (fixture_*.kt). A clobbered baseline would
have shown com_example_beta_* on both sides; it does not. The actual side is A's module
carrying B's root package, which is precisely "A read B's construction-time reset".
Both baselines were valid: freshA != freshB and leg 1 (freshB vs bFirst) are asserted
before leg 2 and both passed — the test reached leg 2 to fail there.
3. G3(c)'s construction ✅ — but the leg-3 ledger exemption is not what the docs say (see §4b)
Construction/use split is clean: runFor (construct) / generateWith (use), with generateOnce
their composition. I diffed c18ef68..9aa7527 on DeterminismTest.kt: the extraction is verbatim
— the three pre-existing tests still run construct → use in the same order and were not
touched. The ordering B-first (control) / A-second (catch) / B-again (re-entry) is the right
shape, and B-first genuinely is the ordering a construction-time reset would survive.
4. The spec fix — mostly true, one claim is false
I checked every factual claim in the new G3 inset. All of these are TRUE:
- The original G3 wording did stop at (b) and did call it "the direct test of §1.4" — confirmed
against3d9d4f4:docs/design/live-service.md. - "(a), (b) and the original same-config guard stay green, only (c) goes red" under a
construction-time reset — reproduced, §1. - (a) permutes an
AllowListFilteracross three orders (in-order, reversed, rotated), and the
order that reaches the writers comes from the tree:ParsedResolver.findClassesis
tu.forEachRecursive { if (it.filter()) ret.add(it) }(Parsing.kt:558), not an allowlist walk. - (b)/(c) baselines come from a real second process via
/proc/self/exe, and a missing child
fails loudly (assertTrue(transcript.exists(), ...)). - The lazily-initialised-fallback trap is real — I hit it too and needed the same bypass.
- §1.4's "all six" is six:
generation,includeDirs,dumpModelDir,drops,profiler,
baseModelTu. - P4's "
cppParseIncludeDirs/cppModelDumpDir/cppBaseModelTuare only reachable through a
clang parse":grep -rn '\.(includeDirs|dumpModelDir|baseModelTu)\b'over all 309 non-build
.ktfiles returns read sites only inParsing.kt(dumpModel,parseForcingModel,
parseHeader) — the twoflags.includeDirshits inDefWriter/CppCompilerare the wrapper
compile's flags, a different thing. - R4 now reads "Reduced by B4, NOT retired … making the mutex sufficient, not … optional"; §1.4,
§4, the B4 row andKrapperRun's KDoc all moved to "alive at once", andKrapperRun's KDoc
carries the explicit "do not cite it as the gate for this class."
(4b) — one claim is FALSE, and I measured it. The inset says:
"(c)'s re-entry leg must compare emitted files only, because a ledger belongs to its run
and a run used twice has legitimately recorded twice."
and interleavedRunLifetimesMatchFreshProcesses's KDoc says "Its ledger legitimately differs".
I removed the filterKeys { it != ledgerEntry } from both sides of leg 3 and re-ran
:krapper:nativeTest --rerun on otherwise-unmodified production code:
all five DeterminismTest tests pass, G3(c) included. The ledger does not differ between
bFirst and bAgain. Nor could it, by the fixture's own design: Config.B has
withUnresolvableMember = false and the file's own KDoc says "A's ledger is non-empty while B's
is empty" — an empty ledger accumulated twice is still empty, and requested/bound are
recomputed per call.
So the exemption is not required by anything, and it is the one place a leak could hide behind a
plausible rule — it removes run-scoped state from the only leg that checks "a live run was not
disturbed". I do not insist on removing it (it would be needed the day someone gives Config.B
a drop, and pinning that forward is defensible), but the justification as written is a statement
of fact about this test that is not true, in the inset whose entire job is to be factually
reliable for B5.
5. The detached warning, and the missing test — I agree with the author
The hook itself is sound. noteDetachedRead latches before invoking (re-entrancy safe), the
hook is null by default so an untouched run is byte-identical, and App.run installs it as the
second statement — after ExperimentalFlags.init, before anything can touch WrappedType —
so the one-shot latch cannot be consumed before the hook exists in either the CLI or --service
path. A hook rather than a log call is the right call given :krapper_model cannot see the flag
registry. (Minor irony worth naming: onFirstDetachedRead is a new process-global var in a
brick about removing process globals. It is process-lifetime diagnostic wiring, not run state, so
it is fine — but it is worth the reader knowing that was deliberate.)
Ruling on the absent unit test: the author is right, and I would have declined it too. The
latch is process-global and one-shot and other fixture builders consume it first — I confirmed
DeterminismTest.buildModel calls WrappedType(...) outside any using scope, so it trips the
latch in the parent process before any hypothetical test could. A test for it would therefore be
order-dependent, i.e. flaky, and a flaky test is strictly worse than none: it teaches people to
ignore reds. Adding a test-only latch reset to production code to enable it is a worse trade
again, for an off-by-default diagnostic whose failure mode is "you lose a debugging aid", not
"wrong output".
For the record, the one test I would accept, if anyone wants coverage later: a child-process
test using the /proc/self/exe re-exec harness this very file already owns — a dedicated child
entry point that, in a fresh process, installs onFirstDetachedRead, reads
GenerationContext.current, and writes what it saw. That is deterministic without touching
production code. Not required for this PR; noting it so the option is on record rather than
rediscovered.
6. Regression and hygiene ✅
:krapper:nativeTest(--rerun, reports deleted first): 316 executed, 0 skipped, 0
failures/errors, 19 report files. Consistent with the reported 312 → +4 (DeterminismTest
now holds 5@Tests, 4 of them added by this PR).:featuregen:nativeTest(--rerun, reports deleted first — the previous report was stale from
an earlier head, so I forced it): 195 executed, 0 skipped, 0 failures/errors, 76 files.- Generated-tree denominators match the author's exactly: featuregen
krapped-cpp= 149
files, krapperkrapped-cpp= 132. krapper/build/krapped-cppregenerated during a later run is byte-identical to the head
snapshot (diff -r→ 0); positive control (head-featuregen vs head-krapper) → 1.- G1 vs base
3d9d4f4: PASS. I regeneratedfeaturegen's bindings at the base commit
(:featuregen:kplusplusSync, cpp front-end, full release link of the base tool) and
diff -r'd the two trees: 149 files each, byte-identical, exit 0. Positive control
(append one comment line to one generated.kt) returns 1, so the zero is a measurement. closingIssuesReferencesis empty (0) — correct, #186 is the epic and B4 is 1 of 9.
Positive control: PR #236 returns 1 (#206), so the zero is a measurement, not a null.- CI green: both required checks SUCCESS.
- All mutations restored with
git checkout --(never from a copy); worktree clean.
What blocks
B1 — DeterminismTest.kt still asserts the claim this PR retracts. This is the file that IS
the gate, and it is the first place a B5 author will look. At 9aa7527:
- class KDoc L74: "Three tests, and the third is the one B4 exists for:" — there are now
four (five@Tests with the child entry point), and G3(c) is not in the list at all. - class KDoc L84: "This is the direct test of §1.4 and the gate for B4." — said of G3(b).
That is verbatim the sentencelive-service.md's new inset exists to retract, and which
KrapperRun's KDoc now says "do not cite … as the gate for this class". - L467: "G3(b) — the brick's gate."
- L471-475 (G3(b)'s KDoc): "Before B4 the two shared one
GenerationContext… and B could only
start by destroying A's — so anything A left in place that B did not overwrite became part of
B's output." — this asserts G3(b) would have caught the pre-B4 shape. §1 above measures that
it does not.
Ten lines of comment — but it is the retracted claim surviving in the gate file while the design
doc warns others not to repeat it, and runFor's KDoc ten lines below says the opposite, so the
file now contradicts itself.
B2 — the leg-3 ledger justification is false as written (§4b). Either drop the exemption (it
passes today, measured) or restate the reason as forward-looking ("the ledger would accumulate
if this run dropped anything; excluded so the leg stays honest if Config.B ever gains a drop")
in both live-service.md's inset and the interleavedRunLifetimesMatchFreshProcesses KDoc.
What it must not say is that the ledger does differ, because it does not.
What would change my mind / what I will approve on: B1 and B2 fixed — comment and doc text
only, no behavioural change, no new measurement required. Re-request review and I will confirm
the wording, re-run :krapper:nativeTest, and merge. I am not asking for another mutation round;
§1 stands.
Non-blocking, follow-up
docs/experimental-flags.mddocuments one section per registered flag (diag.timing,
diag.baseBindTiming) and does not mention the newdiag.detachedReads.--list-experimental
covers runtime discoverability, so this is a nit — but a short section would keep the pattern.
Fold it in here if convenient, or file it.DeterminismTest.kt:341'sreadlinkassertion message names only G3(b); G3(c) needs the child
too.
… a false exemption Two text defects, both in the artifact a future agent reads first. B1 -- DeterminismTest's own KDoc still asserted what this PR retracted. It said "Three tests" (there are four), omitted G3(c) from the list, called G3(b) "the direct test of 1.4 and the gate for B4", and its `L467` header repeated "the brick's gate" -- while `runFor`'s KDoc ten lines below said the opposite. The file contradicted itself, and a wrong comment surviving a correct fix is how the defect re-seeds. G3(b)'s KDoc now states plainly that it is NOT the gate, why (the pre-B4 shape reset at construction, so it passes), and what it does still earn its place catching: a global with no reset at all. B2 -- leg 3 exempted the drop ledger and justified it with "a run used twice has recorded twice". That is false of this test: Config.B sets withUnresolvableMember = false, so its ledger is empty on both uses. Rather than re-justify the exemption, it is removed -- comparing the ledger too is strictly stronger, it passes, and there is now no claim left to rot. A forward-looking note records what WOULD make the exemption necessary. Corrected in both places the false reason appeared: the KDoc and live-service.md's G3 inset. Also folded in, both flagged and not filed: `diag.detachedReads` gets the per-flag section every other registered flag has in docs/experimental-flags.md (warning text quoted verbatim from App.kt), and the readlink failure message named only G3(b) though G3(c) spawns the child too. No production behaviour changes; the only non-comment edit is dropping the two `filterKeys` calls. Refs #186. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLWsoRAe7f1RdQrYJ2ZXqm
monkopedia-reviewer
left a comment
There was a problem hiding this comment.
Approve — both text fixes verified, plus the two folded-in extras
Round-3 re-review at 814a2c7 (3 files, +91/−30). Scope held to B1 and B2 as committed;
the measurement settled at 9aa7527 was not reopened.
B1 — the retracted claim is gone from the file that IS the gate
Sweep, with denominator. /usr/bin/grep -rn -F 'G3(b)' . --exclude-dir=.git:
19 textual occurrences across 4 tracked source files (plus 2 stale
krapper/build/** klib binaries, which are build output, not text anyone reads).
Positive control on the same invocation shape: G3(c) returns 15.
| file | occurrences | classification |
|---|---|---|
docs/design/live-service.md |
2 (L146, L725) | both explicitly qualified — "weaker sibling G3(b) only shows that sequential runs do not leak"; "G3(b) is NOT sufficient: it is green against a construction-time reset" |
krapper_model/.../GenerationContext.kt |
1 (L77) | neutral label pair "G3(b)/G3(c) gates measure" |
krapper/.../KrapperRun.kt |
2 (L49, L106) | L49 qualified ("weaker sibling … only sequential"), L106 neutral pair |
krapper/src/nativeTest/kotlin/DeterminismTest.kt |
14 | L87/L89/L95/L96/L483/L535 are the retractions themselves; L82/L353/L480 are neutral labels; L94/L245/L536/L540/L578/L597 are the measured factual statement "the construction-time-reset shape passes G3(b)" |
No surviving text asserts the retracted claim. Targeted sweeps for the exact
retracted phrasings, each printing its own count:
brick's gate→ 0Three tests→ 0emitted FILES only/emitted files only→ 0recorded twice→ 0direct test of→ 1, and it is inside the correction inset at
live-service.md:573— "G3 stopped at (b) and called it 'the direct test of §1.4'.
It is not."gate for B4→ 2, correctly attributed in both:DeterminismTest.kt:87
"This is the gate for B4, and G3(b) is not" (on G3(c)) and:483
"This is NOT the gate for B4" (on G3(b)).
The self-contradiction is resolved in the right direction: the header list now says
four, G3(c)'s bullet is present, L480's KDoc no longer claims the gate and instead
states what G3(b) does earn — catching a global with no reset at all — and it now
agrees with runFor's KDoc ten lines below rather than contradicting it.
Count convention checked: the header says "Four tests" while the class has five
@Test functions, but the fifth is childFreshProcessBaseline, documented at L59
and L411 as the child entry point that is inert in the parent's own run. That is the
same convention the previous "Three tests" used. Not a defect.
B2 — the exemption was DROPPED, and the replacement conditional is TRUE
Both filterKeys calls are gone. /usr/bin/grep -n filterKeys DeterminismTest.kt
→ exit 1, zero matches; positive control on the same file, assertSameGeneration → 9.
ledgerEntry survives at its three legitimate sites (L121 declaration, L289 transcript
insertion, L511 G3(b)'s "A drops, B does not" precondition), so nothing was orphaned.
The tests genuinely pass with the ledger compared. Ran it myself, forced:
./gradlew --no-daemon :krapper:nativeTest --tests '*DeterminismTest*' --rerun-tasks
BUILD SUCCESSFUL — tests="5" skipped="0" failures="0" errors="0"
including interleavedRunLifetimesMatchFreshProcesses, whose leg 3 now compares
bFirst against bAgain in full. That pass is itself the proof of the first half of
the new text: if Config.B's ledger were non-empty with N records, bAgain would
carry 2N and leg 3 would be red.
The conditional was measured, not taken on trust. The claim under review —
"giving Config.B a dropped member WOULD make bAgain's ledger carry two records to
bFirst's one" — is a behavioural claim, so I mutated the fixture
(Config.B.withUnresolvableMember = false → true) and re-ran. Leg 3 goes red with
exactly the shape the note predicts:
run B re-entered after run A generated in between …: <drop-ledger> differs.
Expected <Drop ledger: requested 3 / bound 3 / dropped 1
Dropped (1):
[RESOLVE] fun unresolvable(): fixture::Nowhere — Couldn't resolve return>,
actual <Drop ledger: requested 3 / bound 3 / dropped 2
Dropped (2):
[RESOLVE] fun unresolvable(): fixture::Nowhere — Couldn't resolve return
[RESOLVE] fun unresolvable(): fixture::Nowhere — Couldn't resolve return>.
Two records to one, verbatim. The mechanism matches the source: DropLedger.records
is an append-only mutableListOf with no dedup, report() prints dropped ${records.size} and iterates every record, and leg 3 re-enters the same KrapperRun
— so drops accumulate across uses. Fixture restored with git checkout --; worktree
git status --porcelain is empty at 814a2c7.
One thing the note does not mention, offered as information rather than a request: a
fixture-changer who makes that edit trips G3(b)'s precondition first — "config A must
drop something config B does not, so a leaked ledger is visible" — before ever
reaching leg 3. Both texts are otherwise accurate, and both record that the exemption
was removed rather than re-justified, which is the right disposition: the stronger
comparison passes, so there is no claim left to rot.
The two folded-in extras
docs/experimental-flags.md — warning text quoted verbatim. I did not eyeball
this; I reconstructed the string from App.kt:210-215's concatenated literals with
$site substituted by DETACHED_SITE (GenerationContext.kt:20 =
"GenerationContext.current"), unwrapped the doc's fenced block, and compared:
APP : 'krapper: WARNING — GenerationContext.current read run-scoped state with no
KrapperRun installed; it got the detached fallback, which is SHARED.
Reported once per process.'
DOC : (identical)
MATCH: True
The section's other factual claims hold too: the flag id is diag.detachedReads
(ExperimentalFlags.kt:108); the one-shot latch is shared by both carriers
(GenerationContext.noteDetachedRead at :113, called from
GenerationContext.current at :123 and KrapperRun.current at :121); the hot-path
cost really is one identity compare (installed.also { if (it === detached) … }); the
interner really is the noisy reader (WrappedType.kt:56 reads
GenerationContext.current.internedTypes); and every item in the "belongs to a
KrapperRun" list is a real field — drops, generation.internedTypes,
GenerationContext(rootPackage, noRtti), includeDirs/dumpModelDir/baseModelTu.
Section shape matches its siblings: ## \flag` — short name`, "Inert +
byte-identical when off" (3 of 3 flag sections), "Run it e.g. with …".
DeterminismTest.kt:353 now names G3(b) and G3(c) in the readlink failure
message, which is correct — freshProcess is the shared helper and both spawn the
child.
Merge preconditions
- Both required checks green at
814a2c7:krapper / krapper_model / feature-tests + ktlintpass (4m31s) andfeaturegen / cppfixture + thepass (8m54s).
mergeStateStatusis now CLEAN /MERGEABLE. The featuregen job I was warned
was still running has since finished green. closingIssuesReferences→totalCount: 0,nodes: []. Positive control on the
identical GraphQL query: PR #236 → 1 (#206), #229 → 1 (#222), #228 → 1 (#218).
The zero is a measurement, not a null.- No closing keyword in any of the 6 commit messages:
grep -inE '\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\s+#[0-9]+'→ exit 1, zero
matches; positive control on the same text,#186→ 9 lines. Same check on the PR
body → zero matches against 1 issue reference. #186 stays open — B4 is 1 of 9.
Approving and merging.
Brick B4 — de-globalize the generation run
Refs #186. This is one of nine bricks in that epic; the epic itself stays open.
The design chosen, and why
The B4 row allowed either per-
IndexedServicestate or an explicit mutex + documentedsingle-tenancy. This is the first one. The mutex alternative was rejected because it does not
actually make B5 possible: a mutex serializes access to shared storage, but the storage would
still be shared, so a second sync in a persistent process would still have to
reset()the firstone's ledger and root package to start clean. Serializing that is still wrong — it just makes the
wrongness orderly.
Six values that belong to one generation run were process globals that each new run OVERWROTE:
DropLedgerobject+reset()fromIndexedServiceImpl.initclass, one per runGenerationContext(intern cache,rootPackage,noRtti)object+reset(...)class, one per runBaseBindProfilerobject+reset()class, one per runcppParseIncludeDirsvar, set byKrapperServiceImpl.indexKrapperRun.includeDirscppModelDumpDirvar, set byKrapperServiceImpl.indexKrapperRun.dumpModelDircppBaseModelTuvar, written byparseHeader, never resetKrapperRun.baseModelTuBaseBindProfileris not named in the B4 row, but §1.4 lists it as the third objectIndexedServiceImpl.initreset; leaving it global while de-globalizing its two siblings wouldhave been a gap.
IndexedServiceImplbuilds aKrapperRunfrom its own(config, request)and installs it(
KrapperRun.using) for the duration of each service call. The deep, context-less read sites —WrappedType.invoke's intern cache, the drop sites inResolver/Parsing/ModelResolution/WrappedKotlinType, the parse's-Iroots — read the installed run. Threading a carrier throughevery resolve and codegen signature was rejected for the reason
GenerationContext's own KDocalready gave: those surfaces are non-suspend and called pervasively.
What this does not do. Two runs may now be alive at once without sharing state, which is
what §1.4 claims and what G3(c) gates. It is not a concurrency fix: the installed-run slot
is still process-wide, so B5 must still take §4's mutex. B4 makes serializing sufficient, not
optional. §1.4, §4, G3 and R4 now all say precisely that — R4's earlier "largely retired" has been
replaced with "reduced, NOT retired".
What round 1 got wrong: G3(b) does not gate this brick
The reviewer built the mutation I had not: not "no reset at all" (M1/M2), but shared globals
reset at construction — the literal pre-B4 shape,
IndexedServiceImpl.initcallingGenerationContext.reset(config.rootPackage, config.noRtti). Call it M4b. All four shippedtests, G3(b) included, stayed green.
The mechanism:
generateOnceconstructed its run on the line before installing it, so thesequence was
construct A → use A → construct B → use B. Two runs are never configured at thesame time, so a construction-time reset is invisible. G3(b) pins "sequential runs don't leak" —
which the old
reset()already satisfied. It does not pin whatGenerationContext's own KDocclaims and B5 needs: two runs alive at once would silently share an intern cache and a root
package.
In fairness to the spec rather than to me: G3(b) as written in
live-service.md:538-542isexactly what I delivered. The insufficiency is inherited. That is why fixing the spec is the
more important half of this round — the test protects this brick, the spec protects B5.
G3(c) — interleaved lifetimes
Both runs are constructed before either generates, then used out of construction order:
B's config here, so this leg looks right either way — it is the control, not the catch.
All three legs compare the full transcript, drop ledger included. Round 2 shipped leg 3 with a
ledger exemption justified by "a run used twice has recorded twice" — the reviewer measured that
this is false of this test (
Config.Brecords no drops, so its ledger is empty both times) andwas right. The exemption is gone rather than re-justified: comparing the ledger is strictly
stronger, it passes, and no claim is left to rot. A forward-looking note in the KDoc and the doc
records what would make an exemption necessary.
This needed construction split from use (
runFor/generateWith);generateOnceis now theircomposition, so the three existing tests are untouched.
Mutation results — including the one that matters
Each mutation was applied to production source, the suite re-run, the file restored with
git checkout -- <path>(nevercp), and residue re-grepped to zero.M4b — construction-time reset (the acceptance bar for G3(c)). Result: only G3(c) RED; the
other four GREEN. That is the finding, reproduced from my side.
Proof it went red for the stated reason, not because I broke the baseline — the trap the
reviewer hit.
Expectedis the fresh child process for config A: it carries A's module name(
/alpha.cc) and A's unrooted package (fixture_Dep.kt,rootPackage = null). That is anuncontaminated A baseline; a clobbered one would have shown
com_example_beta_*on both sides.actualis in-process A: A's module name carrying B's root package. Additionally thefreshA != freshBprecondition and leg 1 (freshBvsbFirst) both passed, so both baselineswere valid. Keeping them clean required a bypass constructor so the lazily-initialised
detachedcould not clobber the shared store during companion init.
Retained from round 1: M1 (shared
DropLedger) → 3 of 4 RED including G3(b) on the realcomparison; M2 (sticky
GenerationContextglobal) → only G3(b) RED; M3 (CHILD_TEST_NAMEpointed at a nonexistent test) → RED with "wrote no transcript", proving the baseline is a real
subprocess and an absent child fails loudly.
detachedno longer fails silentlyFlagged by the reviewer as B5-relevant. A read that escapes its
usingscope quietly got sharedstate and surfaced much later as "diverged from baseline" with nothing naming the cause.
GenerationContext.noteDetachedReadis now a one-shot latch shared by both carriers, reportingthrough an installable hook;
App.runinstalls a stderr warning under a newdiag.detachedReadsflag. It is a hook because the carriers live in
:krapper_model, which cannot see:krapper'sflag registry or logger. Off by default; the only hot-path cost is an identity compare.
Verified by a temporary probe (built, run, then deleted): silent for reads inside a scope, fires
exactly once for reads outside one, and names the site —
PROBE-RESULT site=GenerationContext.current.Verification
:krapper:nativeTest— 316 tests, 0 failures (312 before; +4).:featuregen:nativeTest— 195 tests, 0 failures (the real end-to-end surface).3d9d4f4. Not just "the sync still runs": trees captured on this branch, worktree checked outat the base commit, sync re-run, trees diffed:
diff -rclean over 149 featuregen files and 132 krapper files (the self-host tree), with identical generated-file and diagnostic counts on both sides (124/126 files; 10749/3921 diagnostics). A positive control then appended one lineto a captured file and re-ran the same
diff— exit 1, so the comparison could see a change.ktlintFormatclean; rebased onto3d9d4f4.varremains of 69.ktfiles inspected, andit is
var LocalVar.isVal, an extension accessor. Of 26 top-levelobjects in the same 69,only
ExperimentalFlags.resolvedandLog.loggerImplstill hold mutable state — processconfiguration, not run state, and not named by B4; called out so the omission reads as
deliberate.
Not done
the doc says so rather than implying B4 discharged it. G3 exercises the deserialize → resolve →
codegen half, where five of the six values are read; the three parse-config values are only
reachable through a clang parse, so their per-run wiring is covered by G1 rather than a unit
test.