fix(api): render Anthropic prompts per architecture - #861
Merged
JustVugg merged 1 commit intoAug 7, 2026
Merged
Conversation
Contributor
|
Heads-up from #712, in the same spirit as JustVugg's note to us when #165 landed: our branch adds a Whoever merges second: the resolution is one line — add renderer = (render_chat_kimi if ARCH == "kimi" else
render_chat_qwen if ARCH == "qwen" else
render_chat_v4 if ARCH == "deepseek_v4" else render_chat)to the helper and keep both functions whole. If this PR goes first, we fold our entry into |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/v1/messagesthrough the same architecture-aware chat renderer used by/v1/chat/completionsProblem
The OpenAI chat endpoint selected
render_chat_inkling,render_chat_kimi, orrender_chat_v4fromARCH, but the Anthropic endpoint always called the GLM renderer. A request sent to Kimi K3, for example, reached the engine as:instead of its required length-framed payload:
Kimi only invokes its native XTML builder when the payload begins with
K3CHAT1\n; otherwise it tokenizes the GLM markers as ordinary text. Inkling and DeepSeek V4 similarly received a prompt from the wrong model family. Because GLM rendering accepted tools, Anthropic tool declarations also bypassed the explicit unsupported-parameter checks in the sibling renderers.Anthropic response handling had the symmetric Inkling problem: it looked only for GLM
</think>boundaries, not Inkling<|content_thinking|>/<|content_text|>markers.Implementation
render_chat_for_archdispatch point and use it from both protocol handlerssplit_inklingfor non-streaming Anthropic responsesInklingStreamSplitfor streaming responses and close the Anthropic thinking block before opening the text blockTests
Added regressions that cover:
Validation run locally on macOS:
The portable build was single-threaded because local
libompis unavailable; the build and full check suite still passed.