refactor: replace process globals with an explicit transpile context - #60
Merged
frosty00 merged 3 commits intoJul 31, 2026
Merged
Conversation
The typescript SourceFile, TypeChecker and Program were stored on the node `global` object (`global.src`, `global.checker`, `global.program`) and read back from ~96 call sites across the language printers. Because that state is process-wide, two Transpiler instances cannot be used at the same time: the second `createProgram*` call overwrites the state the first one is still printing against, so the output silently belongs to the wrong source file. Hold the state in an `ITranspileContext` instead: - `Transpiler` keeps the context of the transpilation in flight and hands it to every language printer through `setContext` - `BaseTranspiler` exposes `getSrc()` / `getChecker()` / `getProgram()`, so the printers read their own instance state rather than process globals - `createProgram*AndSetGlobals` become `createProgram*AndSetContext` and return the context they created; the old names remain as deprecated forwarders - reading the context before a program exists now throws instead of dereferencing whatever the previous transpilation left behind Public API is unchanged: `transpileGoByPath`, `transpile`, and friends keep their signatures, and transpilation output is byte for byte identical. Adds tests/transpileContext.test.ts covering instance isolation, printer context propagation, and the absence of the globals.
Drop the getContext() helper and read the context field directly. getSrc/getChecker/getProgram keep their existing behaviour.
carlotestor
force-pushed
the
refactor/transpile-context-no-global
branch
from
July 31, 2026 10:25
499cbc8 to
622731a
Compare
A ts.Program cannot be shared between worker threads: each worker is a separate V8 isolate and postMessage uses structured clone, which rejects a Program, a TypeChecker, a SourceFile and even a single AST node. A SharedArrayBuffer only shares raw bytes, so it cannot hold one either. What can be shared is the work behind the program. The parsed SourceFiles (the es lib chain plus the import closure seen so far) and the last program built from them move out of the Transpiler instance into a standalone ITranspileProgramCache: - `Transpiler.createProgramCache()` builds one, and the constructor takes it as an optional second argument; omitting it keeps the previous behaviour of a private cache per instance - `getProgramCache()` / `cloneSharingProgramCache()` point further instances at an existing cache - the in-memory path now uses the cache too, so repeated transpileX(content) calls no longer re-parse the lib chain every time Each instance keeps its own printers and its own transpile context, so instances over one cache can be interleaved without clobbering each other, and output is unchanged. Verified byte for byte across all six languages, both modes, plus imports/exports/methodsTypes. For worker pools the cache is per worker: worker.ts now keeps a module-level cache so parse work happens once per worker instead of once per task. On 30 ccxt source files in that shape (fresh instance per 5-file task) this is 4528ms -> 2881ms, 1.57x. Adds tests/sharedProgramCache.test.ts covering cache sharing, context independence, lib SourceFile reuse, output parity and concurrent use.
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The typescript
SourceFile,TypeCheckerandProgramare stored on the nodeglobalobject:and read back from ~96 call sites across
baseTranspiler.tsand the six language printers (global.checker.getTypeAtLocation(...),global.src.getFullText(), ...).Because that state is process-wide, two
Transpilerinstances cannot be used at the same time. The secondcreateProgram*call overwrites the state the first one is still printing against, so the first transpilation silently resolves types against the wrong source file. That also rules out running several transpilations concurrently in one process, which is the natural way to speed up large multi-language builds.Change
The state now lives in an explicit per-transpile context instead:
Transpilerowns the context of the transpilation in flight and hands it to every language printer viasetContext.BaseTranspilerholds it inthis.contextand exposesgetSrc()/getChecker()/getProgram(). All ~96global.*reads becamethis.getChecker()/this.getSrc()/this.getProgram(), so each printer reads its own instance state.createProgramInMemoryAndSetGlobals/createProgramByPathAndSetGlobalsbecome...AndSetContextand return the context they built. The old names stay as thin deprecated forwarders so no external caller breaks.transpile(...)'ssetGlobalsparameter is renamed tocreateContext(same position, same semantics).After this change
global.src,global.checkerandglobal.programare never written or read — aTranspilerinstance is self-contained.Compatibility
Public API is unchanged.
transpileGoByPath,transpilePython,transpileDifferentLanguagesByPath,getFileImports,getFileExportsand friends keep their signatures, so ccxt and other consumers need no call-site changes.Verification
npm test— 369 passed, 9 suites (364 pre-existing + 5 new)npm run build(lint + tsup) — passes, 0 eslint errorsnpx tsc --noEmit— cleanOutput is byte-for-byte identical. I transpiled
tests/integration/source/transpilable.tsonmasterand on this branch and diffed the results: all 6 languages, bothByPathandByContentmodes, plusgetFileImports/getFileExports/methodsTypes— no differences.npm run integrationwas also run; it only fails locally at the rust step becausecargois not installed in this environment (code: 127), unrelated to this change.New
tests/transpileContext.test.tscovers what the refactor buys:global.src/global.checker/global.programstayundefinedafter a transpileTranspilerinstances that interleave program creation keep their own context and both produce correct output (this is the case that was broken before)Sharing a program between threads
The follow-up question was whether a
ts.Programcan be shared between threads. It cannot, and that is a hard limit rather than a missing feature. Each worker thread is a separate V8 isolate andpostMessageuses structured clone. Measured againsttypescriptdirectly:A
SharedArrayBufferonly shares raw bytes, so it cannot hold aProgrameither. Anything that claims to share one across isolates is really rebuilding it on the other side.What can be shared is the work behind the program, so this PR now does that.
ITranspileProgramCacheThe parsed
SourceFiles (the es lib chain plus the import closure seen so far) and the last program built from them move out of theTranspilerinstance into a standalone cache:getProgramCache()/cloneSharingProgramCache()point further instances at an existing cachetranspileX(content)calls no longer re-parse the lib chain every timeEach instance keeps its own printers and its own transpile context, so instances over one cache can be interleaved without clobbering each other — that is exactly what the context refactor above makes safe.
For worker pools
The cache is per thread, not per process: each worker keeps its own long-lived cache and the pool ships file paths as it does today.
worker.tsnow holds a module-level cache, so parse work happens once per worker instead of once per task.On 30 ccxt source files in that shape (a fresh
Transpilerper 5-file task, as a Piscina worker does):Verification
npm test— 375 passed, 10 suites (369 + 6 new intests/sharedProgramCache.test.ts)getFileImports/getFileExports/methodsTypes— byte for byte identicaloldProgramreuse does not invalidate a live program: a checker still resolves its own types after another instance built a program from the same cache (covered by a test)Caches are same-thread only, and that is documented in the README next to the worker example.