Repository navigation
fix(utils): reject non-finite numbers in LLM JSON loaders - #243
Merged
helloml0326 merged 1 commit intoNov 17, 2026
Merged
helloml0326 merged 1 commit into
helloml0326 merged 1 commit into
Conversation
Python's json module parses the NaN, Infinity and -Infinity literals
that RFC 8259 does not define, so both repair_and_load_json and
trim_and_load_json returned {'score': nan} for malformed model output
and the nan silently poisoned every aggregate computed from it.
agentscope-ai#226/agentscope-ai#232 already enforce this rule in the JSON graders; route both
loaders through the shared reject_non_finite helper so structured-output
parsing fails cleanly into its existing error handling instead.
helloml0326
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Third stop for the RFC 8259 rule from #226 and #232: the LLM-response JSON loaders still parse the
NaN/Infinity/-Infinityliterals that the JSON spec does not define.openai_chat_model(structured-output parsing) andqwen_vl_modelrely onrepair_and_load_json/trim_and_load_jsoninopenjudge/utils/utils.py. Today a complete model response such as{"score": NaN}loads as{'score': float('nan')}, and the nan silently poisons every aggregate computed from it (nan + 1.0 = nan), instead of failing into the loaders' existing error handling:Fix: route both
json.loadscalls through the sharedreject_non_finitehelper that #232 added, and widen the except clauses fromjson.JSONDecodeErrortoValueError(a subclass relationship — the loaders' public contract of raisingValueErroris unchanged). Malformed responses now fail cleanly into the existing error paths.Note: incomplete JSON that goes through
json_repairkeeps its current behavior — the library stringifies a bareNaNinto"NaN"while repairing (e.g.{"score": NaN→{'score': 'NaN'}), which never produces a float; a test locks that in.How I Tested
tests/utils/test_json_loaders.py(the loaders previously had zero test coverage): 6 rejection tests across both loaders (plain and fenced NaN/Infinity/-Infinity) plus 5 behavior locks (valid JSON, code-fence stripping, incomplete-JSON repair, the json_repair stringification path)nanon main) — real regression testspytest tests/utils→ 47 passedtests/harness/test_codex.py, also fails on cleanmain— environment-dependent, unrelated)