Skip to content

[Attn] Remove erroneous @torch.compile on ParallelAttentionFunction class#980

Draft
arbi-dev wants to merge 2 commits into
fla-org:mainfrom
arbi-dev:fix/parallel-attn-class-torch-compile
Draft

[Attn] Remove erroneous @torch.compile on ParallelAttentionFunction class#980
arbi-dev wants to merge 2 commits into
fla-org:mainfrom
arbi-dev:fix/parallel-attn-class-torch-compile

Conversation

@arbi-dev

Copy link
Copy Markdown

Summary

fla/ops/attn/parallel.py applies @torch.compile directly to the torch.autograd.Function class ParallelAttentionFunction. torch.compile is meant for functions / nn.Modules / callables, not for an autograd.Function subclass — the decorated object's innermost fn resolves to the class, which has no __code__.

It has been silently harmless because the decorator is inert for the .apply path that parallel_attn uses. But it hard-crashes import fla the moment torch's caching_precompile is enabled (a torch 2.9+ feature that persists the Dynamo trace cross-process), because caching_precompile eagerly resolves a precompile cache key at decoration time:

fla/ops/attn/parallel.py:719  @torch.compile
torch/_dynamo/eval_frame.py:690  result = DynamoCache.load(fn)
torch/_dynamo/package.py:914     key = CompilePackage.source_id_from_fn(fn)
torch/_dynamo/package.py:684     sha256_hash.update(str(innermost_fn_.__code__.co_firstlineno).encode())
AttributeError: type object 'ParallelAttentionFunction' has no attribute '__code__'

Since it fails at import, a downstream project that enables caching_precompile cannot import fla at all. This removes the stray decorator. The real compute lives in the parallel_attn_fwd / parallel_attn_bwd Triton kernels invoked inside forward/backward, so removing the class-level decorator leaves forward/backward numerics unchanged.

Closes #979.

Test plan

  • import fla.ops.attn.parallel with torch._dynamo.config.caching_precompile = True no longer raises AttributeError (was a hard crash before; succeeds after).
  • Syntax/AST parse verified.
  • I have not run the full GPU op test suite (tests/ops/test_parallel_attn*) for parallel_attn — I don't have that op exercised in my environment, and per the project's "match the reference numerically (fwd + bwd)" principle I'd rather flag that than assert it. Requesting a maintainer run of the parallel-attn fwd/bwd numerics + the standard CI to confirm before merge. Marked draft for that reason.

Breaking changes

None expected. The decorator was a no-op for the .apply code path (autograd handles .apply specially), so output and gradients are unchanged. If the original intent was to compile the Python glue, the correct placement would be @torch.compile on the parallel_attn function (line 771) instead — happy to switch to that if preferred.

arbi-dev and others added 2 commits June 25, 2026 12:22
…lass

`@torch.compile` must target a function / nn.Module / callable, not an
`autograd.Function` subclass. The decorator is inert for the `.apply`
path (so it has been silently harmless), but it crashes `import fla`
the moment torch's `caching_precompile` is enabled: that feature eagerly
resolves a precompile cache key at decoration time via
`DynamoCache.load -> CompilePackage.source_id_from_fn -> innermost_fn.__code__`,
and the class has no `__code__`:

    AttributeError: type object 'ParallelAttentionFunction' has no attribute '__code__'

The actual compute lives in the `parallel_attn_fwd` / `parallel_attn_bwd`
Triton kernels invoked inside `forward`/`backward`, so removing the
class-level decorator leaves forward and backward numerics unchanged.

Fixes fla-org#979.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] @torch.compile on ParallelAttentionFunction class breaks torch caching_precompile (AttributeError: no __code__)

2 participants