stage_router classifier hardcodes response_format type=json_schema; incompatible with providers that only support json_object

[hallelujah-shih]

Problem

The stage_router LLM classifier (judge) hardcodes response_format: { type: "json_schema" } in every judge request. Many provider gateways (and some model backends) do not support json_schema structured output — they only support response_format: { type: "json_object" }. On those providers the judge call fails with HTTP 400, the verdict is dropped, and the router silently degrades to always picking the efficient tier (judge becomes dead weight — costs a call, never produces a verdict).

Reproduction

Config:

[routes.code]
type = "stage_router"
capable_target = "pro"
efficient_target = "fast"
picker = "efficient_first"
confidence_threshold = 0.5

[routes.code.classifier]
target = "flash"          # model that supports json_object but NOT json_schema
base_threshold = 0.5

Server log:

WARN ... libsy: judge verdict unavailable; routing without one
  judge_model="deepseek-v4-flash"
  reason="upstream_non_5xx"
  error=client call to target "deepseek-v4-flash" failed: upstream returned HTTP 400:
  {"error":{"message":"This response_format type is unavailable now","type":"invalid_request_error"}}

The same model accepts json_object without issue:

# json_schema -> 400
curl ... -d '{"response_format":{"type":"json_schema","json_schema":{...}}}'
# -> {"error":{"message":"This response_format type is unavailable now"}}

# json_object -> 200 (with "json" in prompt)
curl ... -d '{"response_format":{"type":"json_object"}}'
# -> 200 OK

Root cause (source-verified, v0.2.0)

The json_schema type is enforced at the contract layer with no config-level override:

1. algorithms/util/classifier_contract.rs — from_inner_schema() wraps every schema in a hardcoded json_schema envelope:

// line 72-83
let validator = compile_schema(&schema)?;
Self::from_response_format(
    prompt_template,
    json!({
        "type": "json_schema",          // <- hardcoded
        "json_schema": {
            "name": "switchyard_classifier_response",
            "strict": true,
            "schema": schema,
        }
    }),
    Some(validator),
)

2. from_response_format() requires the /json_schema/schema JSON pointer — a json_object response_format would fail validation:

// line 101-105
response_format
    .pointer("/json_schema/schema")
    .ok_or_else(|| LibsyError::AlgorithmError {
        message: "response schema has no json_schema.schema".to_string(),
    })?;

3. algorithms/llm_class.rs:33 — the capability-classifier schema is compiled in via include_str!:

const SCHEMA_TEMPLATE: &str = include_str!("../prompts/capability-classifier/schema.json");

The schema file itself starts with "type": "json_schema".

There is no configuration field on ClassifierContractConfig, StageRouterConfig, or the classifier sub-block in TOML to select json_object instead.

Impact

Any target model whose provider only supports response_format: { type: "json_object" } cannot serve as a stage_router classifier. In practice this excludes models on providers that expose json_object but not json_schema — a common combination on OpenAI-compatible gateways in front of open-weight models.

When the judge model lacks json_schema support, the failure is silent and costly: every ambiguous turn still fires a judge call (which 400s), the verdict is discarded, and the router falls open to the default tier. The only signal is a WARN log line.

Proposal

Allow the response_format type to be configured per classifier target, with a sensible default. For example:

[routes.code.classifier]
target = "flash"
base_threshold = 0.5
response_format_type = "json_object"   # or "json_schema" (default)

When json_object is selected:

• Send response_format: { type: "json_object" } instead of the json_schema wrapper.
• Still validate the parsed JSON against the schema client-side (the Validator already exists for from_inner_schema).
• Inject the schema description into the system prompt (or rely on the existing prompt which already instructs the judge on output shape).

This keeps json_schema as the default (strict, zero-prompt-overhead) while unblocking providers that only offer json_object.

Environment

• switchyard-server / switchyard-libsy 0.2.0 (crates.io)
• Provider: OpenAI-compatible gateway (DeepSeek backend)
• Model: deepseek-v4-flash — accepts json_object, rejects json_schema with HTTP 400

[ting-hong-shieh]

I would like to work on this. I plan to keep json_schema as the default, add an opt-in json_object response format for classifier judges, preserve client-side verdict validation, and cover the request shape and fallback behavior with offline mocked tests. I will also wire the supported configuration surfaces and update the relevant docs. No live provider call should be needed.