Compatible with both thinking and enable_thinking.#1260
Conversation
Add model validator to synchronize thinking and enable_thinking in chat_template_kwargs.
There was a problem hiding this comment.
Code Review
This pull request introduces a model validator to synchronize the 'thinking' and 'enable_thinking' keys within 'chat_template_kwargs', ensuring compatibility across different chat templates like Qwen and DeepSeek. Feedback suggests optimizing the implementation by modifying the dictionary in place to avoid unnecessary allocations and re-assignments.
lightllm/server/api_models.py
Outdated
| d = dict(self.chat_template_kwargs) | ||
| if "thinking" not in d and "enable_thinking" in d: | ||
| d["thinking"] = d["enable_thinking"] | ||
| elif "enable_thinking" not in d and "thinking" in d: | ||
| d["enable_thinking"] = d["thinking"] | ||
| self.chat_template_kwargs = d |
There was a problem hiding this comment.
The current implementation creates a shallow copy of the dictionary and re-assigns it to the field even when no changes are necessary. Since chat_template_kwargs is a mutable dictionary, you can modify it in place to avoid unnecessary allocations and re-assignments. This improves efficiency, especially for requests where these specific keys are not present or both are already set.
| d = dict(self.chat_template_kwargs) | |
| if "thinking" not in d and "enable_thinking" in d: | |
| d["thinking"] = d["enable_thinking"] | |
| elif "enable_thinking" not in d and "thinking" in d: | |
| d["enable_thinking"] = d["thinking"] | |
| self.chat_template_kwargs = d | |
| if "thinking" not in self.chat_template_kwargs and "enable_thinking" in self.chat_template_kwargs: | |
| self.chat_template_kwargs["thinking"] = self.chat_template_kwargs["enable_thinking"] | |
| elif "enable_thinking" not in self.chat_template_kwargs and "thinking" in self.chat_template_kwargs: | |
| self.chat_template_kwargs["enable_thinking"] = self.chat_template_kwargs["thinking"] |
No description provided.