Skip to content

fix(dashboard): price sessions per model, not by their primary-model label - #173

Open
ollo12-prog wants to merge 1 commit into
phuryn:mainfrom
ollo12-prog:upstream-pr/mixed-model-session-cost
Open

fix(dashboard): price sessions per model, not by their primary-model label#173
ollo12-prog wants to merge 1 commit into
phuryn:mainfrom
ollo12-prog:upstream-pr/mixed-model-session-cost

Conversation

@ollo12-prog

Copy link
Copy Markdown

Fix: sessions are priced by their primary-model label, overcharging multi-model sessions

What

Prices each session's tokens at the rates of the model that actually produced them, instead of pricing the whole session at its single primary-model label.

Why

sessions.model holds one primary-model label per session, picked by the scanner's opus > sonnet > haiku priority. The dashboard's session-derived costs pass that one label to calcCost() along with the session's summed tokens — so every token in a session is billed at the primary model's rate, including turns that ran on a cheaper model.

That happens on any session that dispatches subagents (Task/Agent subagents commonly run on Haiku while the main thread is on Opus) or where the user switches model mid-session with /model.

Measured on a real 902-turn session — 684 claude-opus-5 turns + 218 claude-haiku-4-5 turns:

Est. Cost
before (all 902 turns @ opus) $91.5232
after (per producing model) $83.4677

9.6% overstated. The error grows with how much subagent work a session dispatches, and it's always in the same direction — the primary-model priority picks the most expensive model present, so the label is never cheaper than the turns it stands in for.

This is the same rule v1.5.5 already applied to the Daily Token Usage cost line (#151, "priced per model before the daily aggregation"). Sessions were still on the old path.

Changes

dashboard.py — server

  • New _session_model_breakdowns(conn): one GROUP BY session_id, model over turns, returning each session's tokens split by producing model.
  • Each entry in sessions_all gains a by_model list.

dashboard.py — client

  • New sessionCost(s): sums calcCost() per entry in s.by_model. Falls back to the existing calcCost(s.model, …) path when by_model is absent or empty, so single-model sessions are bit-identical to before and nothing breaks if the field is missing.
  • Switched the five session-derived call sites to it: sortSessions (cost column), applyFilter's Cost by Project and Cost by Project & Branch aggregations, renderSessionsTable, and exportSessionsCSV.

Deliberately unchanged

  • sortModels and the by-model / daily / hourly aggregations already group by model and were correct.
  • cli.py computes per turn and was correct.
  • The isBillable(s.model) gate on the cost cell — existing behavior, out of scope here.

tests/test_dashboard.py

  • New TestMixedModelSessionCost: a session labeled opus with one opus turn and one haiku turn. Asserts by_model splits by producing model, that the split sums losslessly back to the session rollup, that per-model pricing comes out below the single-label price, and that the JS session sites no longer call calcCost(s.model, …).
  • All three fail against the unfixed code.

Verification

  • python -m unittest discover -s tests — 150 tests, all pass.
  • Ran the dashboard against a real usage.db: the session above now renders $83.4677 in Recent Sessions, and sessionCost() vs the old expression on the same row returns 83.46766 vs 91.52315.
  • No schema change and no rescan needed — by_model is derived from turns at read time.

Note on payload size

by_model adds one small object per (session, model) pair to /api/data. On a DB with ~1000 sessions that's on the order of tens of KB against a payload that already ships every session row. If you'd rather keep it leaner, emitting by_model only when a session has more than one distinct model would cut it to near zero without touching the client — the fallback already handles absence. Happy to make that change if you prefer it.

`sessions.model` holds one primary-model label (opus > sonnet > haiku), chosen
by the scanner. Every session-derived cost in the dashboard priced that single
label against the session's summed tokens, so any session that also ran a
cheaper model — a dispatched subagent on Haiku, or a mid-session /model switch
— had those tokens billed at the expensive model's rates.

On a real 902-turn session (684 claude-opus-5 + 218 claude-haiku-4-5) the
sessions table reported $91.5232; the correct per-model total is $83.4677,
9.6% high. The error scales with how much subagent work a session dispatches.

This is the same rule v1.5.5 applied to the Daily Token Usage cost line (phuryn#151:
"priced per model before the daily aggregation") — sessions were still on the
old path.

get_dashboard_data now attaches a per-model token breakdown to each session
(`by_model`, one GROUP BY over `turns`), and `sessionCost()` prices each
model's tokens at its own rate and sums. It falls back to the existing
single-label path when no breakdown is present, so nothing else has to change
and single-model sessions are bit-identical to before.

Fixed: Recent Sessions table and its CSV export, Cost by Project, Cost by
Project & Branch, and the sessions cost sort. Untouched: the by-model, daily
and hourly aggregations, which already group by model, and the CLI, which
computes per turn.

Each of the three new tests fails against the unfixed code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant