Make the job the cache identity - #308
Closed
AntonOresten wants to merge 2 commits into
Closed
AntonOresten wants to merge 2 commits into
AntonOresten wants to merge 2 commits into
Conversation
AntonOresten
force-pushed
the
ao/cached-results
branch
from
September 4, 2026 15:02
2d4d9b0 to
91cda60
Compare
AntonOresten
force-pushed
the
ao/cached-results
branch
from
September 6, 2026 13:18
91cda60 to
1eca5d9
Compare
A cuTile compilation is a `TileJob`: a MethodInstance and world, and a `TileConfig` (target architecture and bytecode version, hints, const-seeded argument types, kernel name). The job keys the results cache, is what the `@device_code_*` hook reports, and is what the reflection entry points take. Inference is partitioned by a single owner, `:cuTile`. cuTile's inference depends on neither the target nor the hints — the interpreter takes only a world — so every configuration of a kernel shares one CodeInstance, with const-seeded arguments as `const_entries` on it; a Symbol is interned, so the owner is identity-stable for Julia's inference engine. Codegen results (`CuTileResults`: CUBIN and per-context kernels) are stored on that CodeInstance per config, so they survive world bumps and persist into package images with it. `compile_or_lookup(job)` runs `compile(job)` on a miss, under a lock. This replaces cuTile's own cache layer — `TileCacheKey`, the `CacheView` results plumbing, `ensure_compiled` and the cached `emit_*!` chain — and with it the per-hint re-inference that layer caused. Reflection and launches now share one inference partition, so `@device_code_typed` no longer re-infers. `get_inferred` reads its const-specialized entry directly: CompilerCaching's `get_source(ci, argtypes)` takes the first `CachedResult` on the CodeInstance whatever its results type, and `TileResults` is attached to the same one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNHajvrhhKBBUF7w6cDiKr
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This follows up on the work queued in #302: a single value,
TileJob, now identifies a compilation throughout inference, codegen, caching, launch, and reflection.Previously, “which compilation?” was represented three incompatible ways:
(f, tt)in the compilation hookCGOptsduring codegenTileCacheKeyin the cacheThese representations did not round-trip cleanly.
Design
TileJobcontains a sourceMethodInstance, world age, andTileConfig.TileConfigcontains the target (SM architecture and bytecode version), parameters (compiler hints and const-seeded argument types), and kernel name.Inference uses a single
:cuTileCompilerCaching owner. Since cuTile inference depends on neither the target nor compiler hints, configurations of the same kernel share one inferredCodeInstance. Constant arguments remain separate const-seeded entries.This removes the per-hint re-inference previously investigated in #95 and JuliaGPU/GPUCompiler.jl#917.
Codegen results are stored on the
CodeInstanceinCompilerCaching.results, keyed byTileConfig. Because the world age is represented by the selectedCodeInstancerather than the config, unrelated world-age bumps do not rerun Tile IR generation ortileiras.compile_or_lookup(job)is the cached entry point. It resolves a targetless job before lookup, ensuring the cache identity always includes the architecture of the stored CUBIN.compile(job)remains the uncached codegen path used by reflection and the compilation hook.CUBINs continue to persist through the object cache added in #307.
assemblecallsObjCache.get!using the same key fields and schema, and the cross-process cache harness has been ported to the job API.Reflection now accepts jobs directly through both IO and stdout forms:
Removed
TileCacheKeyensure_compiledemit_*!pipelineCompilerCaching workaround
get_inferredcurrently reads the typed const-seeded entry directly.CompilerCaching.get_source(ci, argtypes)selects the firstCachedResulton aCodeInstanceregardless of its result type. That is ambiguous here because inference results andTileResultsshare the sameCodeInstance.Tracked upstream as maleadt/CompilerCaching.jl#30. The workaround can be removed once CompilerCaching provides result-type-scoped source lookup.
Stacked on #302; the base is
ao/code-ptx-sass, and only the final commit is new.The suite is green against registry CompilerCaching 0.4.6. Locally, the updated branch passes 2,885 tests with one expected broken test on CompilerCaching 0.4.7.