System prompt forwarded with per-request `x-anthropic-billing-header` line — defeats upstream prompt cache

[ajcasagrande]

Summary

When converting an Anthropic /v1/messages request into the OpenAI-shaped messages array, the proxy concatenates the inbound system field verbatim into a single system-role message. Anthropic clients (notably Claude Code) prefix that field with a non-semantic header line whose hash changes on every request, so the upstream prompt prefix is unique per call and the upstream prompt cache never hits.

Background

Claude Code prefixes its system prompt with a line of the form:

x-anthropic-billing-header: cc_version=2.1.117.48f; cc_entrypoint=cli; cch=71fea;

The cch=<hash> portion regenerates on every request. The line carries no semantic value to the model — it's a billing/telemetry header. When forwarded into the upstream system message unchanged, every turn presents a brand-new prefix to the backend.

Root cause

server.py, around line 425:

# Add system message if present
if anthropic_request.system:
    # Handle different formats of system messages
    if isinstance(anthropic_request.system, str):
        # Simple string format
        messages.append({"role": "system", "content": anthropic_request.system})
    elif isinstance(anthropic_request.system, list):
        # List of content blocks
        system_text = ""
        for block in anthropic_request.system:
            if hasattr(block, 'type') and block.type == "text":
                system_text += block.text + "\n\n"
            elif isinstance(block, dict) and block.get("type") == "text":
                system_text += block.get("text", "") + "\n\n"

        if system_text:
            messages.append({"role": "system", "content": system_text.strip()})

The string form is forwarded verbatim; the list form concatenates each text block as-is. Neither path filters the x-anthropic-billing-header: line (nor any other non-semantic Anthropic header line), so it propagates verbatim into the upstream system message.

There's also no equivalent of OpenAI Responses' prompt_cache_key set on the outbound request, so there's no fallback cache-routing key.

Impact

• Cost. Long multi-turn Claude Code sessions re-bill the full input context (often a large CLAUDE.md + tool catalog) on every turn. Typical multiplier vs. correct cache reuse: 5–10×.
• Latency. Cache-miss prefills are slower than cache hits, so every turn after the first feels sluggish.
• Quota. Token rate limits exhaust faster than they should because effective input-token throughput is lower.

This particularly affects users routing Claude Code through this proxy to backends that publish prompt-cache pricing, since the entire pricing benefit is voided.

Suggested fix

Strip non-semantic Anthropic header lines before concatenating:

def _strip_nonsemantic_system_lines(text: str) -> str:
    return "\n".join(
        line for line in text.splitlines()
        if not line.strip().lower().startswith("x-anthropic-billing-header:")
    ).strip()


# in the request builder:
if anthropic_request.system:
    if isinstance(anthropic_request.system, str):
        cleaned = _strip_nonsemantic_system_lines(anthropic_request.system)
        if cleaned:
            messages.append({"role": "system", "content": cleaned})
    elif isinstance(anthropic_request.system, list):
        parts = []
        for block in anthropic_request.system:
            text = ""
            if hasattr(block, "type") and block.type == "text":
                text = block.text or ""
            elif isinstance(block, dict) and block.get("type") == "text":
                text = block.get("text", "") or ""
            cleaned = _strip_nonsemantic_system_lines(text)
            if cleaned:
                parts.append(cleaned)
        if parts:
            messages.append({"role": "system", "content": "\n\n".join(parts)})

Optional second hardening: when the upstream backend is OpenAI Responses (or any backend that supports an explicit cache routing key), also set prompt_cache_key to a stable session-scoped value so cache routing has an explicit anchor independent of prefix bytes.

Reproduction

1. Send any Anthropic /v1/messages request whose system is either a string or a list whose first text block begins with x-anthropic-billing-header: cc_version=...; cch=<hash>;. (Real Claude Code traffic does this automatically.)
2. Observe the outbound payload — the system-role message includes the billing-header line.
3. Send a second request with a different cch=<hash> to simulate a fresh Claude Code session. The two outbound system messages differ in their first line, so prefix-based caching at the upstream cannot hit between them.