fix: guard against empty/filtered LLM responses in legacy LLM models#457
fix: guard against empty/filtered LLM responses in legacy LLM models#457qizwiz wants to merge 3 commits into
Conversation
Add null checks before accessing choices[0].message across 5 files in legacy/pentestgpt/: open_ai.py, deepseek.py, jina.py, chatgpt_api.py, deepseek_api.py.
There was a problem hiding this comment.
Code Review
This pull request adds validation logic to ensure LLM responses are not empty or filtered before processing them across several files. However, the implementation in chatgpt_api.py and deepseek_api.py contains critical syntax errors because if and raise statements were placed directly inside class constructor arguments. Additionally, these checks are positioned after the response data is already accessed, which would cause a crash before the validation logic is executed. These checks should be moved to the start of the methods.
| if not response.choices or response.choices[0].message is None: | ||
| raise ValueError("LLM returned empty or filtered response") |
There was a problem hiding this comment.
This code introduces a SyntaxError because an if statement and raise cannot be placed inside a class constructor call (InitialGeneration(...)).
Furthermore, this check is performed too late for the langfuse logging block. If response.choices is empty, the code will crash at line 149 when attempting to access response.choices[0].message.content before the check at line 156 is ever reached.
Recommendation: Move the validation check to the beginning of the _chat_completion method (or at least before line 140) to ensure the response is valid before any of its attributes are accessed.
| if not response.choices or response.choices[0].message is None: | ||
| raise ValueError("LLM returned empty or filtered response") |
There was a problem hiding this comment.
This code introduces a SyntaxError because an if statement and raise cannot be placed inside a class constructor call (InitialGeneration(...)).
Furthermore, this check is performed too late for the langfuse logging block. If response.choices is empty, the code will crash at line 152 when attempting to access response.choices[0].message.content before the check at line 159 is ever reached.
Recommendation: Move the validation check to the beginning of the _chat_completion method (or at least before line 143) to ensure the response is valid before any of its attributes are accessed.
The guard was mistakenly placed inside the InitialGeneration constructor argument list, which is invalid Python syntax. Move it before the langfuse logging block so it runs before the constructor call. This also removes the now-redundant second guard after the block since a single guard at the top of the response-handling section is sufficient.
|
Thanks @gemini-code-assist — you're right, the guard was incorrectly placed inside the Fixed in commit 88d021a: moved the guard before the langfuse block in both files, so it runs as a standalone statement before any constructor is called. The now-redundant second guard after the langfuse block was removed. Syntax verified with |
|
Thanks for the update, @qizwiz. Moving the guard before the |
Extend the empty-response guard to cover the case where message exists but content is None (e.g., tool-call responses or filtered content). Guards message.content before it is passed to langfuse InitialGeneration and before the return value.
|
Thanks for the review! A quick clarification on the placement concern: The guard at lines 138–142 is outside the That said, your point about |
|
The syntax issue was fixed in 88d021a (guard moved before the |
What
Adds null checks before accessing
choices[0].messagein five files underlegacy/pentestgpt/:llm_generation/models/open_ai.pyllm_generation/models/deepseek.pyllm_generation/models/jina.pyutils/APIs/chatgpt_api.pyutils/APIs/deepseek_api.pyWhy
Two silent failure modes crash at these lines:
IndexError—choicesis an empty list ([]) when the provider returns no completionsAttributeError—choices[0].messageisNonewhen content is filtered (some providers return HTTP 200 withPROHIBITED_CONTENTfinish reason)Fix
Detected by pact static analysis.