Scorer evaluation category matching is guarded on the wrong axis and can silently score the wrong category

[varunj-msft]

Summary

#2570 is fixed for every shipped scorer/dataset pairing (#2575 consolidated the Likert scales and labels, #2616 added the selfharm alias). This issue tracks the two durability gaps that remain, plus the broader taxonomy question raised on #2570.

Verified on main at a2d86755e.

1. The regression test guards a different value than the runtime uses

test_azure_categories_match_registered_evaluation asserts against the registered category:

config = AzureContentFilterScorer._get_eval_files_for_category(category)
assert ScorerEvaluator._select_evaluation_score(scores=scores, harm_category=config.harm_category) is selected

But at runtime the value that reaches the matcher comes from the CSV row, not the registration (pyrit/score/scorer_evaluation/scorer_evaluator.py):

first_entry = labeled_dataset.entries[0]
if isinstance(first_entry, HarmHumanLabeledEntry):
    harm_category = first_entry.harm_category

So renaming a CSV harm_category label keeps the test green while breaking evaluation. Renames on this axis do happen: #2575 moved fairness_bias → REPRESENTATIONAL and sexual_content → SEXUAL_CONTENT, correctly updating the registration and the CSV together. Nothing in the suite would have caught it if only one side had moved.

There is also no equivalent all-presets invariant over LikertScalePaths; only two presets are asserted individually.

Suggested fix: one parametrized test over every registered pairing (all LikertScalePaths entries with evaluation_files and all AzureContentFilterScorer._CATEGORY_EVAL_FILES) that reads the actual CSV label and asserts _select_evaluation_score accepts the emitted category.

2. A custom file_mapping can silently score against the wrong category

file_mapping is the first public parameter of Scorer.evaluate_async, so callers can supply their own ScorerEvalDatasetFiles and bypass the safe default registration.

Matching is a set intersection over HarmCategory.parse_many, and several aliases are one-to-many — hate resolves to both HATE_SPEECH and REPRESENTATIONAL. The result is that a lone Azure Hate score is accepted for a REPRESENTATIONAL dataset, and bias is accepted for hate_speech:

ScorerEvaluator._select_evaluation_score(
    scores=[Score(score_type="float_scale", score_value="0.5", score_category=["Hate"])],
    harm_category="REPRESENTATIONAL",
)  # returns the score instead of rejecting it

_select_evaluation_score is the guard meant to reject a mis-paired dataset, so accepting here produces metrics for the wrong category rather than a loud failure. Non-overlapping pairs (Hate vs violence, Sexual vs violence) are still correctly rejected.

The one-to-many mapping is intentional for dataset classification — tests/unit/models/test_harm_category.py asserts hate maps to both. The point is that taxonomy classification and scorer/dataset identity are different contracts.

Suggested fix: use a one-to-one crosswalk for scorer-evaluation matching (Azure Hate → HATE_SPEECH, SelfHarm → SELF_HARM, …) and keep the one-to-many aliases for dataset classification. Alternatively, document and test the permissive behaviour as intentional.

3. The matcher is alias-map dependent, so new categories will break again

A value that is neither a canonical HarmCategory name/value nor in the alias map matches only when the strings are byte-identical:

• jailbreak vs jailbreak → matches
• Jailbreak vs jailbreak → raises ValueError

Azure emits CamelCase (SelfHarm) while datasets use snake_case, so the next new Azure category reproduces the original #2570 failure until someone hand-adds an alias. This area has already needed two rounds of label/alias work (#2575 added alias matching and fixed the Sexual pairing; #2616 added the selfharm alias).

Worth noting that exploits and information_integrity both resolve to OTHER today — they pass purely by exact string equality, with no alias backing them. A rename on either side breaks them with nothing to catch it.

This is the broader "should seed-taxonomy aliases be involved in scoring at all" question from #2570.

Not affected

All 12 currently registered pairings evaluate correctly, so this is about preventing recurrence rather than fixing a live break.

---

cc Rudra Prasad Bhuyan (@Rudra-G-23) — this is the concrete follow-up from #2570 if you would still like to pick it up.

[Rudra-G-23]

Thanks for tagging me varunj-msft . I was planning to pick this up, but I see #2663 now has an implementation covering the scorer-evaluation matching and CSV-axis regression tests, so I'll hold off to avoid duplicating the work.

Happy to pick up any follow-up from this if something remains after review.

[varunj-msft]

Thanks for flagging rather than duplicating — that was the right call. #2663 does cover all three findings here, so there's no need to redo it.

If you'd still like something concrete in this area: #2663 adds the CSV-agreement test for the Azure pairings, but there's no equivalent invariant over LikertScalePaths. A parametrized test over every preset with evaluation_files, asserting the emitted category matches the CSV label, would close the last gap. Happy to review if you want to take it.

[varunj-msft]

#2663 is approved and queued, which will auto-close this. Recording the one part that isn't fully covered, so it doesn't get buried on close.

Findings 1 and 2 are fully fixed. Verified at 124fbca3c: the new test_shipped_pairings_match_csv_evaluation reads the real CSVs and asserts they match the registered category, and the switch to strict single-category matching now rejects Hate vs REPRESENTATIONAL and bias vs hate_speech. All 12 shipped pairings still pass, including the Azure SelfHarm path from #2616.

Finding 3 is partly covered. The casefold fast path fixes case-only differences but not separator differences:

Jailbreak         vs jailbreak          -> match
HateSpeech        vs hate_speech        -> reject
ProtectedMaterial vs protected_material -> reject

SelfHarm vs self_harm only matches because #2616 hand-added that alias. So the original concern — that a new CamelCase category from an external taxonomy needs a hand-added alias before evaluation works — still holds for separator-shaped names.

Deliberately not blocking #2663 on it: that PR fixes real bugs, the gap is pre-existing rather than introduced by it, and the fix belongs in HarmCategory normalisation rather than bolted onto the evaluator.

If anyone wants to pick this up, there are two small, independent pieces:

1. Normalise _, - and spaces out of both sides before the casefold comparison, with tests covering positive separator cases and negative cases ensuring unrelated categories stay distinct.
2. A parametrized invariant over LikertScalePaths (every preset with evaluation_files) asserting the emitted category matches the CSV label — FIX: strict scorer-eval category match + CSV-axis test #2663 adds this for the Azure pairings but not the Likert ones.

Rudra Prasad Bhuyan (@Rudra-G-23) — either of these is yours if you'd like one.

[Rudra-G-23]

Thanks for the follow-up. I see #2670 now covers both of the remaining pieces, so I'll hold off to avoid duplicating that work as well. If anything remains after review, I'm happy to pick up the next follow-up. cc: varunj-msft