Skip to content

[Bug] TestGemma4MoE::test_bf16 fails deterministically in pre-merge DGX_H100-4_GPUs-AutoDeploy-1: Not registered specs (bf16/auto) #16275

Description

@chang-l

Summary

accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 fails deterministically with:

ValueError: Not registered specs: {'dtype': 'auto', 'quant_algo': None, 'kv_cache_quant_algo': None, 'spec_dec_algo': None, ...}

The accuracy reference files (mmlu/gsm8k/mmmu yaml) contain google/gemma-4-26B-A4B-it entries, but none matching the plain bf16/auto config this test requests.

Impact

Ask

Either register the missing spec for the bf16/auto config, or waive the test until it is fixed. Also reported on #12710 (test origin).

cc @NVIDIA/trt-llm-autodeploy-devs

Metadata

Metadata

Assignees

No one assigned

    Labels

    AutoDeploy<NV> AutoDeploy Backend

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions