AttributeError: 'RotaryEmbedding' object has no attribute 'pos_idx_in_fp32'

[AmmarAliHakeem]

I am currently running the following command to obtain scores:

python -m scripts.score --input-fasta examples/example_seqs.fasta --output-tsv scores.tsv --model-name 'evo-1-8k-base' --device cuda:0
It runs pretty well. However when I change the model to 131k , I get the following error(detailed error message at the end):

AttributeError: 'RotaryEmbedding' object has no attribute 'pos_idx_in_fp32'

I tried different combinations of cuda and pytorch versions but I get the same error each time (details below).

Could you please advise on what might be causing this issue or suggest potential solutions? I would greatly appreciate your guidance.

#############
OS : Ubuntu 22.04
#############

###########################################################
tested combinations (installed from https://pytorch.org/ using pip)
###########################################################
pytorch 2.71 , cuda 12.8
pytorch 2.71, cuda 12.6
pytorch 2.6, cuda 12.6
pytorch 2.6, cuda 12.4

############################
complete error msg
############################

python -m scripts.score --input-fasta examples/example_seqs.fasta --output-tsv scores.tsv --model-name 'evo-1-131k-base' --device cuda:0
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/sulclab/Documents/evo/scripts/score.py", line 66, in
main()
File "/home/sulclab/Documents/evo/scripts/score.py", line 35, in main
evo_model = Evo(args.model_name)
^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/Documents/evo/evo/models.py", line 52, in init
self.model = load_checkpoint(
^^^^^^^^^^^^^^^^
File "/home/sulclab/Documents/evo/evo/models.py", line 96, in load_checkpoint
model = AutoModelForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 593, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/transformers/modeling_utils.py", line 311, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4766, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/modeling_hyena.py", line 38, in init
self.backbone = StripedHyena(model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/model.py", line 350, in init
self.blocks = nn.ModuleList(
^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/torch/nn/modules/container.py", line 310, in init
self += modules
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/torch/nn/modules/container.py", line 359, in iadd
return self.extend(modules)
^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/torch/nn/modules/container.py", line 442, in extend
for i, module in enumerate(modules):
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/model.py", line 351, in
get_block(config, layer_idx, flash_fft=self.flash_fft) for layer_idx in range(config.num_layers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/model.py", line 323, in get_block
return AttentionBlock(config, layer_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/model.py", line 57, in init
swap_mha_rope(
File "/home/sulclab/.cache/huggingface/modules/transformers_modules/togethercomputer/evo-1-131k-base/c206aab77ae5967a069c4200ecb1858588528c9d/positional_embeddings.py", line 98, in swap_mha_rope
pos_idx_in_fp32 = mha.rotary_emb.pos_idx_in_fp32,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sulclab/miniconda3/envs/evo_torch/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1928, in getattr
raise AttributeError(
AttributeError: 'RotaryEmbedding' object has no attribute 'pos_idx_in_fp32'

[ffalfred]

I'm also having the same issue

[davidkell]

Also had this issue, pinning flash_attn as per this comment worked for me.

Also had to pin numpy version to get tokenizer working, see huggingface/transformers.js#1343

Example pyproject.toml for usage with uv that is working for me right now, including pre-built binaries for flash_attn:

[project]
name = "evo"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = "==3.12.*"
dependencies = [
    "evo-model>=0.4",
    "flash-attn",
    "numpy==2.2",
    "torch==2.8",
]

[tool.uv.sources]
flash-attn = { url = "https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.3.18/flash_attn-2.7.4+cu128torch2.8-cp312-cp312-linux_x86_64.whl" }