Repository navigation
scope resolution with JuliaLowering (first part of #99) - #172
Closed
singhharsh1708 wants to merge 8 commits into
Closed
singhharsh1708 wants to merge 8 commits into
singhharsh1708 wants to merge 8 commits into
Conversation
`@resumable` has to know which binding every identifier in a function body refers to, so that each distinct local gets its own field on the state machine. That currently comes from a reimplementation of Julia's scope rules in this package. This adds the same information from JuliaLowering, which is the compiler's own implementation of them, behind a package extension so nothing changes unless JuliaLowering is loaded. `resolve_bindings` runs the first three lowering passes and reports one `Binding` per distinct binding, carrying `kind`, `n_assigned`, `is_captured` and `is_always_defined`. Nothing consumes it yet. JuliaLowering needs julia 1.12 and is not registered, and pinning it in test/Project.toml makes that environment unresolvable on 1.10 and 1.11, so its tests live in their own environment under test/lowering.
JuliaLowering expands macros in its first pass, and `@yield`, `@yieldfrom` and `@nosave` throw when expanded outside a `@resumable` function, which is what makes them useful as errors elsewhere. `substitute_markers` replaces them with calls to three method-less marker functions first, so a body can be lowered and the markers found again afterwards. `@nosave var = rhs` becomes `var = nosave_marker(rhs)` rather than a call wrapping the whole assignment, which would turn the binding into a keyword argument and lose it. The macros themselves are unchanged and still throw outside `@resumable`; nothing tested that before, so test/test_markers.jl now does.
`slot_bindings` groups the bindings that need storage on the state machine by source name, so that a name bound in more than one scope reports one binding per scope. The differential test runs both scopers over the same bodies and compares how many slots each allocates per name. They agree on straight-line code, shadowing and nested lets, loops, reassignment, comprehensions, an argument shadowed by a let, and `a = a`. They differ in two places, both bindings that need no slot: the exception of a `catch`, which `get_slots` already drops, and a closure's own parameter, which belongs to the closure. Those are pinned so that a later change that filters them out has to update the test deliberately.
`slot_bindings` reported every local and argument in the body, including those belonging to a nested closure. A closure's own parameter is not stored on the state machine, so it is now filtered out by taking the bindings of the function's own lambda, which is the outermost one that binds anything. A variable the closure captures is bound by the function as well, so it still appears. That also drops the loop variable of a comprehension, which the current scoper does allocate. Nothing is lost: a comprehension is desugared into a closure, and `@yield` cannot appear inside one anyway, since julia rejects it with `"return" not allowed inside comprehension or generator`, so the variable never has to survive a yield. The differential test now agrees on nine bodies including closures, and pins the two remaining differences, the exception of a `catch` and the comprehension variable.
`resolve_scopes` leaves the body in JuliaLowering's own IR, which cannot be turned back into an `Expr`: it still holds `function_decl`, `method_defs`, `scope_block`, `_do_while` and friends, which have no `Expr` head. Nor can the scope information be applied to the original `Expr` instead, because a tree built by `expr_to_syntaxtree` carries only line provenance, so the two `a`s of `let a = a + 1` are indistinguishable by position. So the remaining two passes are run as well and the method's `Core.CodeInfo` taken from the result. That form is already linear, with a slot per local and explicit control flow, and the yield markers survive into it, which is what the state machine needs.
Member
|
Thank you for spending tokens on this, but we do not accept LLM-generated PRs from new developers. If you would like to discuss your own work on implementing this feature, please come to the QuantumSavory office hours. |
Author
|
Yeah i will join. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First part of #99, scoping only. Nothing is wired into
@resumableyet, so no behaviour changes, and this is not a claim the bounty is done.The useful finding is that two routes are closed. Scope information cannot be mapped back onto the original
Expr, sinceexpr_to_syntaxtreegives only line provenance and bothas oflet a = a + 1land on one line. The desugared tree cannot round-trip either:Expr(tree)throws onfunction_decl,scope_block,_do_while. So the state machine has to be built from the method'sCore.CodeInfo, whichlower_to_codeinforeaches and the yield markers survive into.Two other things worth knowing. JuliaLowering needs 1.12, for
Base.IncludeInto. Anda = areading the global (v1.0.2, #122) is a deliberate divergence from julia, which JuliaLowering reports as one local withis_always_definedfalse.julia --project=test/lowering test/lowering/runtests.jlruns 43 tests, including a differential test that agrees with the current scoper on nine bodies and pins two differences that need no slot. Main suite green on 1.10 and 1.11, and on 1.12 apart fromjet, which fails the same way on master here.