fix(dashboard): price sessions per model, not by their primary-model label - #173
Open
ollo12-prog wants to merge 1 commit into
Open
fix(dashboard): price sessions per model, not by their primary-model label#173ollo12-prog wants to merge 1 commit into
ollo12-prog wants to merge 1 commit into
Conversation
`sessions.model` holds one primary-model label (opus > sonnet > haiku), chosen by the scanner. Every session-derived cost in the dashboard priced that single label against the session's summed tokens, so any session that also ran a cheaper model — a dispatched subagent on Haiku, or a mid-session /model switch — had those tokens billed at the expensive model's rates. On a real 902-turn session (684 claude-opus-5 + 218 claude-haiku-4-5) the sessions table reported $91.5232; the correct per-model total is $83.4677, 9.6% high. The error scales with how much subagent work a session dispatches. This is the same rule v1.5.5 applied to the Daily Token Usage cost line (phuryn#151: "priced per model before the daily aggregation") — sessions were still on the old path. get_dashboard_data now attaches a per-model token breakdown to each session (`by_model`, one GROUP BY over `turns`), and `sessionCost()` prices each model's tokens at its own rate and sums. It falls back to the existing single-label path when no breakdown is present, so nothing else has to change and single-model sessions are bit-identical to before. Fixed: Recent Sessions table and its CSV export, Cost by Project, Cost by Project & Branch, and the sessions cost sort. Untouched: the by-model, daily and hourly aggregations, which already group by model, and the CLI, which computes per turn. Each of the three new tests fails against the unfixed code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: sessions are priced by their primary-model label, overcharging multi-model sessions
What
Prices each session's tokens at the rates of the model that actually produced them, instead of pricing the whole session at its single primary-model label.
Why
sessions.modelholds one primary-model label per session, picked by the scanner's opus > sonnet > haiku priority. The dashboard's session-derived costs pass that one label tocalcCost()along with the session's summed tokens — so every token in a session is billed at the primary model's rate, including turns that ran on a cheaper model.That happens on any session that dispatches subagents (Task/Agent subagents commonly run on Haiku while the main thread is on Opus) or where the user switches model mid-session with
/model.Measured on a real 902-turn session — 684
claude-opus-5turns + 218claude-haiku-4-5turns:9.6% overstated. The error grows with how much subagent work a session dispatches, and it's always in the same direction — the primary-model priority picks the most expensive model present, so the label is never cheaper than the turns it stands in for.
This is the same rule v1.5.5 already applied to the Daily Token Usage cost line (#151, "priced per model before the daily aggregation"). Sessions were still on the old path.
Changes
dashboard.py— server_session_model_breakdowns(conn): oneGROUP BY session_id, modeloverturns, returning each session's tokens split by producing model.sessions_allgains aby_modellist.dashboard.py— clientsessionCost(s): sumscalcCost()per entry ins.by_model. Falls back to the existingcalcCost(s.model, …)path whenby_modelis absent or empty, so single-model sessions are bit-identical to before and nothing breaks if the field is missing.sortSessions(cost column),applyFilter's Cost by Project and Cost by Project & Branch aggregations,renderSessionsTable, andexportSessionsCSV.Deliberately unchanged
sortModelsand the by-model / daily / hourly aggregations already group by model and were correct.cli.pycomputes per turn and was correct.isBillable(s.model)gate on the cost cell — existing behavior, out of scope here.tests/test_dashboard.pyTestMixedModelSessionCost: a session labeled opus with one opus turn and one haiku turn. Assertsby_modelsplits by producing model, that the split sums losslessly back to the session rollup, that per-model pricing comes out below the single-label price, and that the JS session sites no longer callcalcCost(s.model, …).Verification
python -m unittest discover -s tests— 150 tests, all pass.usage.db: the session above now renders$83.4677in Recent Sessions, andsessionCost()vs the old expression on the same row returns83.46766vs91.52315.by_modelis derived fromturnsat read time.Note on payload size
by_modeladds one small object per (session, model) pair to/api/data. On a DB with ~1000 sessions that's on the order of tens of KB against a payload that already ships every session row. If you'd rather keep it leaner, emittingby_modelonly when a session has more than one distinct model would cut it to near zero without touching the client — the fallback already handles absence. Happy to make that change if you prefer it.