Repository navigation
Accept a service_tier the enum does not name - #42
Merged
Merged
Conversation
OpenAI reports the tier that served a request in `service_tier` and adds tiers on its side, so a closed enum fails the whole reply when it names a new one (`"fast"`): a client whose first call is answered on that tier cannot deserialize anything it gets back. `chat::ServiceTier` and `responses::ServiceTier` gain an untagged `Other(String)` variant that keeps an unnamed tier verbatim and serializes it back unchanged. The request types share these enums, so a caller's own unnamed tier is accepted and forwarded too. The `ToSchema` derive ignores `#[serde(untagged)]` on a variant and would describe `Other` as an object, so both enums get a hand-written schema: any string, listing the named tiers. `responses::ServiceTier` loses `Copy`, which a `String` variant cannot have.
phillipleblanc
approved these changes
Oct 9, 2026
krinart
added a commit
to spiceai/spiceai
that referenced
this pull request
Oct 9, 2026
The fork PR squash-merged to the `spiceai` branch as 2c102dc, whose tree is identical to the tested branch commit 3a2d567. Repoint the pin and the ledger at it and drop the TEMPORARY marker.
krinart
added a commit
that referenced
this pull request
Oct 9, 2026
#42 made `chat::ServiceTier` and `responses::ServiceTier` open, and the request types share them, so a request naming any tier became valid. A caller that validates incoming requests against these types widened its accepted values as a side effect of a response-decoding fix. Split the two directions. `ServiceTier` is closed again, exactly as before #42, and stays on `CreateChatCompletionRequest` and `CreateResponse`. The response fields (`CreateChatCompletionResponse`, `CreateChatCompletionStreamResponse`, `Response`) use `ServiceTierResponse`, which keeps an unnamed tier verbatim in `Other(String)`, with the hand-written any-string schema. `chat::ServiceTierResponse` was unused; it gains `Auto` so it covers every value the request enum can hold.
krinart
added a commit
to spiceai/spiceai
that referenced
this pull request
Oct 9, 2026
…openai#44 spiceai/async-openai#42 opened `ServiceTier`, which the request types share, so `/v1/chat/completions` and `/v1/responses` started accepting and forwarding any `service_tier` a client sent. That widens a published request field, which the #14916 fix did not need. #44 (merged as dfadaed) restores the closed `ServiceTier` on requests and moves only the response fields to the open `ServiceTierResponse`. The Databricks stream type follows the response field. The guard now sends a named tier and checks the server's unnamed tier comes back, and a new `a_request_naming_an_unnamed_tier_is_refused` pins both request types to the listed tiers, which fails on the previous pin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the fork half of spiceai/spiceai#14916.
OpenAI reports the serving tier in
service_tierand adds tiers on its side. Replies with"service_tier": "fast"fail to deserialize against the closed enum (unknown variant 'fast'), so a model whose load-time health check is answered on that tier never loads.Change
chat::ServiceTierandresponses::ServiceTiergain#[serde(untagged)] Other(String). An unnamed tier is kept verbatim and serializes back unchanged. The request types share the enums, so a caller's own unnamed tier is accepted and forwarded.ToSchemafor both:anyOf: [{type: string, enum: [named tiers]}, {type: string}]. The derive ignores variant-leveluntaggedand would renderOtheras{"other": string}.responses::ServiceTierdropsCopy(aStringvariant cannot beCopy). Builds clean with--all-features --all-targets.Validation
cargo test --features "chat-completion-types response-types" --test ser_de --test responses_ser_de: 11 passed. Covers unnamed tier round trip, named tier to its variant, non-string refused, and the schema shape with every listed name mapping to a named variant.--all-features. The 12 failures in the lib tests andboxed_futureare live-API tests that api.openai.com rejects withinvalid_api_key.service_tier: "fast"spiceai#14916):spicedagainst a mock endpoint answeringservice_tier: "fast"loads all four models, and/v1/readyreturns 200. On the current pin thefastmodels fail to load and/v1/readystays 503.