feat(voice): price gpt-realtime token usage and add realtime session constants - #2391
Merged
Conversation
…constants Voice bills deterministically today by audio-second (whisper) and input-character (tts). The audio-native realtime path bills TOKENS, per modality, in a shape different enough to need its own table. Additive only: the whisper/tts rates and behaviour are untouched, since on-demand TTS (Read Aloud) is not being retired. voice-pricing.ts - REALTIME_RATES: published gpt-realtime rates as USD per token, every rate env-overridable via the file's existing envFloat, defaults pinned by tests. - calculateRealtimeCostDollars(model, usage): pre-markup provider cost for one response.done frame; the credit pipeline applies MARKUP_BPS as it does for every other call. A call's total is the sum across its responses, since usage is reported per RESPONSE, not per call. Three wire asymmetries are modelled rather than smoothed over: - Caching is input-only, so the output rate type has no cached member to fill in wrongly, and no image member either. - cached_tokens is a SUBSET of input_tokens: the cached slice bills at the cache rate and only the REMAINDER at the full input rate. Each modality's cached count is clamped to its own gross count, so a malformed payload cannot drive the fresh remainder negative and subtract money from the bill. - Modality is the cost driver (one short spoken sentence measured 30 text + 86 audio output tokens, audio being ~2.7x the text rate), so each side is priced per modality rather than blended. Unknown model, absent usage, or missing/negative/NaN quantities bill 0 — never a negative or NaN charge. A usage object carrying only totals also bills 0: an unattributed token cannot be priced (audio input is 8x text input) and guessing a modality would over-charge, so the metering layer is left to notice that shape. credit-pricing.ts — realtime session constants beside the existing voice ones: - REALTIME_SESSION_HOLD_ESTIMATE_CENTS (10c). Realtime settles CONTINUOUSLY as usage arrives per response, so the hold need only cover the window between settles rather than the whole call. - REALTIME_MAX_SESSION_SECONDS (600). Derived, not picked: it must stay inside CREDIT_HOLD_TTL_SECONDS (900) or the reconcile cron reclaims a live call's own reservation mid-session. A test asserts a >=300s margin. - REALTIME_IDLE_TIMEOUT_SECONDS (120). Reaps the abandoned call, which otherwise holds a slot and streams ambient audio into a per-token-billed model. - REALTIME_MAX_INFLIGHT (2 per user). Voice is physically exclusive, so 1 is the semantically correct cap; 2 keeps a not-yet-reaped zombie session from locking a user out of reconnecting. - REALTIME_MAX_GLOBAL_SESSIONS (8). The per-user cap structurally cannot enforce the binding constraint: the OpenAI account is limited to 40,000 tokens/MINUTE across every session on the key, which a talking session approaches at roughly 4k tokens/min — putting the ceiling near 10. Pure: no route wiring, that is the metering leaf's job. Verified: voice-pricing.ts at 100% statement/branch/function/line coverage, with 7 mutations (double-count guard, cached clamp, a published rate, the negative quantity guard, the pro-rata share cap, and both session constants) each confirmed to turn the suite red. Root typecheck 17/17 and lint 15/15. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JCyj6kA4A7bwmmpwDhHrer
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third leaf of the audio-native voice epic. Realtime bills tokens per modality, not audio-seconds and characters, so it needs its own table beside the existing whisper/tts rates.
Rates measured against the live API in the W0 spike, not guessed — including the
usageshape, which is not the Chat Completions shape (input_tokens/output_tokens, with per-modality detail nested one level down).Three asymmetries built against rather than around:
cached_tokensis a SUBSET ofinput_tokens, so cached is billed at the cached rate and only the remainder at the full rate. Double-counting here would silently overcharge every call.Also adds realtime session constants (hold estimate, max duration, idle timeout, concurrency cap) — the account's measured ceiling is 40,000 tokens/min, so unbounded concurrent calls would hit it.
Verification: 52 tests pass; rates asserted against published figures. Mutation-checked the double-count guard — replacing
(gross − cached)withgrossturns 6 tests red.Additive only: the existing whisper/tts rates and behavior are untouched, because open PR #2173 (Read Aloud) depends on the tts path.