Skip to content

JAX 0.11 compatibility (flattree scan port) + rename CLAUDE.md→AGENTS.md#140

Merged
chaoming0625 merged 5 commits into
mainfrom
worktree-rename-agents-md
Jul 24, 2026
Merged

JAX 0.11 compatibility (flattree scan port) + rename CLAUDE.md→AGENTS.md#140
chaoming0625 merged 5 commits into
mainfrom
worktree-rename-agents-md

Conversation

@chaoming0625

@chaoming0625 chaoming0625 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Fixes the daily CI failure caused by three independent JAX 0.11.0 internal API breaks, and renames the project instruction file. Compatible with all jax ≥ 0.8.0 via capability detection (no hard-coded version checks).

JAX 0.11 fixes

  • ClosedJaxpr merged into Jaxpr — the unified constructor dropped the jaxpr= keyword; ModuleInfo.add_jaxpr_outs now passes jaxpr/consts positionally.
  • jax.core.new_jaxpr_eqn removed — annotate the older-JAX import fallback with type: ignore so mypy passes on new JAX.
  • scan primitive "flattree" refactornum_consts/num_carry/linear/_split_transpose params were removed; the input/output splits now live in ft_in/ft_out flattrees. Added two capability-detecting shims in _compatible_imports (scan_num_consts_carry, scan_params_add_ys), routed all num_consts/num_carry reads through them, and rebuild ft_out in add_scan_ys. while/cond params are unchanged in 0.11 and untouched.

CI

  • Add 0.10.0 to the jax-version matrix (now covers 0.8.0, 0.9.0, 0.10.0, latest — old encoding + flattree).

Docs / meta

  • Rename the gitignored CLAUDE.md to a tracked AGENTS.md (contents unchanged, then working-agreement rules refined: spec location → docs/specs).

Verification

  • Full suite on local jax 0.11: 2128 passed, 3 skipped.
  • mypy braintrace: clean (59 files).

🤖 Generated with Claude Code

Summary by Sourcery

Make braintrace compatible with JAX 0.11's updated jaxpr and scan flattree APIs while retaining support for older JAX versions, expand CI JAX coverage, and introduce a tracked AGENTS.md project guidance document.

Bug Fixes:

  • Adapt scan handling to JAX 0.11's flattree-based scan encoding so const/carry and output structure are derived compatibly across supported versions.
  • Update ClosedJaxpr construction to work with the unified Jaxpr constructor introduced in JAX 0.11.
  • Silence mypy issues around the deprecated new_jaxpr_eqn import path when running on newer JAX versions.

Enhancements:

  • Add compatibility shims and refactor scan analysis and rewrite code to consistently query scan metadata via helper functions rather than direct param access.
  • Strengthen tests around scan const/carry counting and scan output extension to cover both legacy and flattree encodings.

CI:

  • Extend the CI matrix to test against JAX 0.10 in addition to existing versions.

Documentation:

  • Replace the gitignored CLAUDE.md with a tracked AGENTS.md that documents durable architecture and contributor working agreements.

Track the agent instruction file as AGENTS.md (previously the
gitignored, untracked CLAUDE.md). Remove the /AGENTS.md ignore entry so
it is committed; keep /CLAUDE.md ignored so any local CLAUDE.md stays
private.
JAX 0.11 merged ClosedJaxpr into a unified Jaxpr whose first parameter is
constvars, dropping the jaxpr= keyword. Pass jaxpr/consts positionally in
ModuleInfo.add_jaxpr_outs so it works on old and new JAX, matching the
other ClosedJaxpr call sites (fixes TypeError in the compile tests).

jax.core also dropped new_jaxpr_eqn in 0.11; annotate the older-JAX
import fallback with a type: ignore so mypy passes against new JAX.

Add jax 0.10.0 to the CI test matrix.
JAX 0.11 refactored the scan primitive: num_consts/num_carry/linear/
_split_transpose params were removed and the (consts, carry, xs) input
split and (carry, ys) output split are now encoded in ft_in/ft_out
flattrees. braintrace read num_consts/num_carry directly and rebuilt one
scan eqn (add_scan_ys), which broke on 0.11 with KeyError: 'num_consts'.

Add two capability-detecting shims in _compatible_imports (detect by which
params exist, so the same code path serves every jax >=0.8.0):
- scan_num_consts_carry(eqn): read params on jax <0.11, else derive from
  ft_in.unpack().
- scan_params_add_ys(params, n_extra): identity on jax <0.11 (num_ys is
  implicit), else grow ft_out's ys component by n_extra leaves.

Route all num_consts/num_carry reads (canonicalize, hidden_group,
scan_descent) through the shim, and rebuild ft_out in add_scan_ys. while/
cond params are unchanged in 0.11 and untouched. Full suite green on jax
0.11 (2128 passed) and mypy clean; CI matrix covers 0.8/0.9/0.10 (old
encoding) and latest (flattree).
Merge the two duplicate spec/plan-location rules (working-agreement #7 and
Rules #2) into a single rule, and point it at the in-repo docs/specs
directory instead of the gitignored dev/superpowers path.
Simplify the spec rule to 'Write specs under docs/specs before
implementation' and fold the brainstate.random rule into the working
agreement list.
@sourcery-ai

sourcery-ai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Reviewer's Guide

Makes braintrace compatible with JAX 0.11’s scan flattree refactor and ClosedJaxpr merge while preserving support for older JAX versions via capability-detecting shims, extends tests and CI to cover the new behavior, and renames the contributor guidance document to AGENTS.md as a tracked file.

File-Level Changes

Change Details Files
Add capability-detecting shims for JAX scan flattree refactor and use them throughout scan handling and canonicalization.
  • Introduce scan_num_consts_carry to abstract over JAX <0.11 num_consts/num_carry params vs JAX 0.11 ft_in flattree encoding.
  • Introduce scan_params_add_ys to grow scan output flattree ft_out only when needed on JAX ≥0.11.
  • Replace direct accesses to eqn.params['num_consts'] and ['num_carry'] in scan_descent, canonicalize, hidden_group, and tests with calls to scan_num_consts_carry.
  • Update add_scan_ys to use scan_params_add_ys so added ys outputs are reflected in ft_out on JAX 0.11 without changing input structure.
braintrace/_compatible_imports.py
braintrace/_compiler/scan_descent.py
braintrace/_compiler/canonicalize.py
braintrace/_compiler/hidden_group.py
braintrace/_compatible_imports_test.py
braintrace/_compiler/scan_descent_test.py
Adjust ClosedJaxpr construction to work with JAX 0.11’s merged Jaxpr/ClosedJaxpr API.
  • Change ClosedJaxpr instantiation in ModuleInfo.add_jaxpr_outs to pass jaxpr and consts positionally instead of using the jaxpr= keyword.
  • Document the rationale in a comment explaining the JAX 0.11 merge and positional compatibility with older JAX.
braintrace/_compiler/module_info.py
Align imports and type checking with removal of jax.core.new_jaxpr_eqn in JAX 0.11.
  • Keep the existing import fallback from jax.core.new_jaxpr_eqn but annotate it with type: ignore to satisfy mypy when the attribute only exists on older JAX.
  • Preserve the primary import from jax.extend.core.new_jaxpr_eqn for newer JAX versions.
braintrace/_compatible_imports.py
Extend tests and CI matrix to cover JAX 0.10 and the new scan compatibility shims.
  • Add focused tests for scan_num_consts_carry and scan_params_add_ys covering both legacy and flattree encodings.
  • Update scan_descent tests to use scan_num_consts_carry to derive num_consts/num_carry and to assert preservation of input structure after add_scan_ys.
  • Add JAX 0.10.0 to the GitHub Actions CI matrix to exercise both pre- and post-flattree encodings.
braintrace/_compatible_imports_test.py
braintrace/_compiler/scan_descent_test.py
.github/workflows/CI.yml
Rename and track the project working-agreement document as AGENTS.md with refined rules.
  • Stop ignoring the previous CLAUDE.md and replace it with a tracked AGENTS.md file containing working agreement, architecture, principles, and docstring guidelines.
  • Update the working-agreement rules to require writing specs under docs/specs and codify testing and loop-usage practices.
.gitignore
AGENTS.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@chaoming0625
chaoming0625 merged commit 4452845 into main Jul 24, 2026
6 checks passed
@chaoming0625
chaoming0625 deleted the worktree-rename-agents-md branch July 24, 2026 08:08

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 4 issues, and left some high level feedback:

  • In _compatible_imports.scan_params_add_ys, consider caching the jax._src.flattree import at module scope instead of inside the function to avoid repeated imports on hot paths.
  • The new helpers (scan_num_consts_carry, scan_params_add_ys) currently use very loose types (Dict, JaxprEqn-like); tightening their type hints and explicitly documenting expected param keys would make misuse easier to detect and maintainers clearer on supported shapes.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `_compatible_imports.scan_params_add_ys`, consider caching the `jax._src.flattree` import at module scope instead of inside the function to avoid repeated imports on hot paths.
- The new helpers (`scan_num_consts_carry`, `scan_params_add_ys`) currently use very loose types (`Dict`, `JaxprEqn`-like); tightening their type hints and explicitly documenting expected param keys would make misuse easier to detect and maintainers clearer on supported shapes.

## Individual Comments

### Comment 1
<location path="AGENTS.md" line_range="20" />
<code_context>
+3. After writing code, list edge cases + suggest test cases.
+4. Bug? Write a test that reproduces it, then fix until the test passes.
+5. Every correction: reflect on the mistake, plan to avoid repeating it.
+6. All updates must be happened on the worktree branch, not main. 
+7. Write specs under `docs/specs` before implementation.
+8. Tests should >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines. 
</code_context>
<issue_to_address>
**issue (typo):** The phrase "must be happened" is ungrammatical; consider "must happen" or "must be made".

Consider updating this sentence to: "All updates must happen on the worktree branch, not main." or "All updates must be made on the worktree branch, not main."

```suggestion
6. All updates must happen on the worktree branch, not main. 
```
</issue_to_address>

### Comment 2
<location path="AGENTS.md" line_range="22" />
<code_context>
+5. Every correction: reflect on the mistake, plan to avoid repeating it.
+6. All updates must be happened on the worktree branch, not main. 
+7. Write specs under `docs/specs` before implementation.
+8. Tests should >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines. 
+9. Co-locate tests with the code under test: each module `foo.py` has its tests in a sibling `foo_test.py` (suffix style — never a separate `tests/` directory, never the `test_*.py` prefix). 
+10. **Never drive a model with a bare Python `for`/`while` loop when it runs repeatedly.** Python loops execute op-by-op (dispatch overhead, no fusion) and trace fresh each step; the `brainstate.transform` primitives lower the whole loop into one compiled XLA program, tracing the body only once. Pick by shape of the work:
</code_context>
<issue_to_address>
**issue (typo):** Missing verb in "Tests should >90% coverage"; consider adding "have" or similar.

Consider "Tests should have >90% coverage" or "Tests should achieve >90% coverage" to fix the grammatical omission.

```suggestion
8. Tests should have >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines. 
```
</issue_to_address>

### Comment 3
<location path="AGENTS.md" line_range="146" />
<code_context>
+- JAX as the core computation framework.
+- Additional brain-ecosystem utilities.
+
+Pin exact names/versions in `pyproject.toml` / requirements files,.
+
+## Known limitations
</code_context>
<issue_to_address>
**issue (typo):** There is an extra comma before the period at the end of this sentence.

Please remove the trailing comma so the sentence ends with just "files."

```suggestion
Pin exact names/versions in `pyproject.toml` / requirements files.
```
</issue_to_address>

### Comment 4
<location path="AGENTS.md" line_range="175" />
<code_context>
+
+#### Rules for the Examples section
+
+- Wrap example code in `.. code-block:: python` directive so Sphinx render with syntax highlighting.
+- Prefix every input line with `>>>` (continuation lines with `...`) for `doctest` compatibility.
+- Show expected output on line immediately after statement, **without** prompt prefix.
</code_context>
<issue_to_address>
**issue (typo):** Subject-verb agreement issue: "Sphinx render" should be "Sphinx renders".

Update the bullet to: "Wrap example code in `.. code-block:: python` directive so Sphinx renders with syntax highlighting."

```suggestion
- Wrap example code in `.. code-block:: python` directive so Sphinx renders with syntax highlighting.
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread AGENTS.md
3. After writing code, list edge cases + suggest test cases.
4. Bug? Write a test that reproduces it, then fix until the test passes.
5. Every correction: reflect on the mistake, plan to avoid repeating it.
6. All updates must be happened on the worktree branch, not main.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): The phrase "must be happened" is ungrammatical; consider "must happen" or "must be made".

Consider updating this sentence to: "All updates must happen on the worktree branch, not main." or "All updates must be made on the worktree branch, not main."

Suggested change
6. All updates must be happened on the worktree branch, not main.
6. All updates must happen on the worktree branch, not main.

Comment thread AGENTS.md
5. Every correction: reflect on the mistake, plan to avoid repeating it.
6. All updates must be happened on the worktree branch, not main.
7. Write specs under `docs/specs` before implementation.
8. Tests should >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): Missing verb in "Tests should >90% coverage"; consider adding "have" or similar.

Consider "Tests should have >90% coverage" or "Tests should achieve >90% coverage" to fix the grammatical omission.

Suggested change
8. Tests should >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines.
8. Tests should have >90% coverage, but focus on meaningful tests that cover edge cases and critical paths, not just trivial lines.

Comment thread AGENTS.md
- JAX as the core computation framework.
- Additional brain-ecosystem utilities.

Pin exact names/versions in `pyproject.toml` / requirements files,.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): There is an extra comma before the period at the end of this sentence.

Please remove the trailing comma so the sentence ends with just "files."

Suggested change
Pin exact names/versions in `pyproject.toml` / requirements files,.
Pin exact names/versions in `pyproject.toml` / requirements files.

Comment thread AGENTS.md

#### Rules for the Examples section

- Wrap example code in `.. code-block:: python` directive so Sphinx render with syntax highlighting.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): Subject-verb agreement issue: "Sphinx render" should be "Sphinx renders".

Update the bullet to: "Wrap example code in .. code-block:: python directive so Sphinx renders with syntax highlighting."

Suggested change
- Wrap example code in `.. code-block:: python` directive so Sphinx render with syntax highlighting.
- Wrap example code in `.. code-block:: python` directive so Sphinx renders with syntax highlighting.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
braintrace/_compatible_imports.py 93.33% 1 Missing ⚠️
braintrace/_compiler/hidden_group.py 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant