[Attn] Remove erroneous @torch.compile on ParallelAttentionFunction class#980
Draft
arbi-dev wants to merge 2 commits into
Draft
[Attn] Remove erroneous @torch.compile on ParallelAttentionFunction class#980arbi-dev wants to merge 2 commits into
arbi-dev wants to merge 2 commits into
Conversation
…lass
`@torch.compile` must target a function / nn.Module / callable, not an
`autograd.Function` subclass. The decorator is inert for the `.apply`
path (so it has been silently harmless), but it crashes `import fla`
the moment torch's `caching_precompile` is enabled: that feature eagerly
resolves a precompile cache key at decoration time via
`DynamoCache.load -> CompilePackage.source_id_from_fn -> innermost_fn.__code__`,
and the class has no `__code__`:
AttributeError: type object 'ParallelAttentionFunction' has no attribute '__code__'
The actual compute lives in the `parallel_attn_fwd` / `parallel_attn_bwd`
Triton kernels invoked inside `forward`/`backward`, so removing the
class-level decorator leaves forward and backward numerics unchanged.
Fixes fla-org#979.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
fla/ops/attn/parallel.pyapplies@torch.compiledirectly to thetorch.autograd.FunctionclassParallelAttentionFunction.torch.compileis meant for functions /nn.Modules / callables, not for anautograd.Functionsubclass — the decorated object's innermost fn resolves to the class, which has no__code__.It has been silently harmless because the decorator is inert for the
.applypath thatparallel_attnuses. But it hard-crashesimport flathe moment torch'scaching_precompileis enabled (a torch 2.9+ feature that persists the Dynamo trace cross-process), becausecaching_precompileeagerly resolves a precompile cache key at decoration time:Since it fails at import, a downstream project that enables
caching_precompilecannotimport flaat all. This removes the stray decorator. The real compute lives in theparallel_attn_fwd/parallel_attn_bwdTriton kernels invoked insideforward/backward, so removing the class-level decorator leaves forward/backward numerics unchanged.Closes #979.
Test plan
import fla.ops.attn.parallelwithtorch._dynamo.config.caching_precompile = Trueno longer raisesAttributeError(was a hard crash before; succeeds after).tests/ops/test_parallel_attn*) forparallel_attn— I don't have that op exercised in my environment, and per the project's "match the reference numerically (fwd + bwd)" principle I'd rather flag that than assert it. Requesting a maintainer run of the parallel-attn fwd/bwd numerics + the standard CI to confirm before merge. Marked draft for that reason.Breaking changes
None expected. The decorator was a no-op for the
.applycode path (autograd handles.applyspecially), so output and gradients are unchanged. If the original intent was to compile the Python glue, the correct placement would be@torch.compileon theparallel_attnfunction (line 771) instead — happy to switch to that if preferred.