Repository navigation
feat(metrics): catalog and track metrics from every harness, and bridge omp - #109
Conversation
han could only catalog Claude Code. Bridged harnesses wrote correctly formed event files to ~/.han/<harness>/projects/** and nothing ever read them back, because the indexer only merged a *-han.jsonl into a sibling native transcript and returned Ok(None) when that sibling was absent. Bridges never produce one, so no OpenCode, Gemini CLI, Kiro, Codex, or Antigravity session has ever appeared in the database. A han events file standing alone is now indexed as a session in its own right. The transcript parser is skipped for those files: parsing han events as transcript lines produced unknown-typed rows whose ids collided with the han event rows, which then silently failed to insert, so the events were lost even once the file was reached. Sessions carry a harness, derived from the path rather than from file contents, since a session's harness cannot legitimately vary line to line. Existing rows are backfilled to claude-code so a filter on the column and a read through the API agree; leaving them NULL would have made harness = 'claude-code' silently skip every session recorded before today. The coordinator now watches and scans ~/.han/<harness>/projects, discovered from disk. Requiring a registration call would mean a bridge that never made it stayed invisible, which is the failure mode being fixed. Adds a token_usage event, the first token-shaped event han has had. The han-event insert path passed None for all four token columns, so cost and token metrics were structurally impossible for any harness without a native transcript. Those counts now land in the same columns every existing aggregate already reads.
|
🚅 Deployed to the han-pr-109 environment in han-team-platform
3 services not affected by this PR
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Review of #109 (commit
|
test_read_real_database read $HOME/.han/han.db directly, so it failed the moment a migration added a column, since a developer's database sits at whatever state their last coordinator run left it in. That says nothing about whether the entities can read real data, which is the only thing the test is for. It now copies the database to a temp dir and migrates the copy. The test keeps its value, stops reporting schema lag as a product failure, and no longer opens a database it does not own.
All five bridge installation pages claimed the coordinator indexed their sessions into SQLite and served them through the Browse UI. That was false from the day the bridges shipped, and this branch is what makes it true, so the pages now describe the real mechanism: harness roots discovered from disk, a standalone events file indexed as its own session, attributed to its harness. Adds the harness dimension to the metrics docs with the field, the filter, and worked queries, and a per-harness matrix of what each one actually reports. Only Claude Code and omp produce token and cost figures today; the other five record sessions, tool calls and file changes with empty token columns. Saying han tracks cost across harnesses would repeat the class of claim this change exists to delete. blueprints/cross-harness-metrics.md records the ingestion path, the single decision point that separates a standalone events file from one beside a transcript, and each of the four breaks repaired, so they are not reintroduced.
omp had no bridge, so none of its sessions were catalogued. This one ships as an omp extension and writes han events to ~/.han/omp/projects, which the coordinator now discovers and indexes. It is the first bridge to report real cost. omp records per-message token counts and dollar cost in its own session file, so the bridge reads them there rather than estimating: input, output, cacheRead and cacheWrite map onto the token_usage contract, cost.total onto cost_usd, and a message with no model is dropped rather than given an invented one. Emitted totals were reconciled against a real 4.6 MB omp session, 559 assistant messages, matching to the cent, and the extension was then exercised inside omp 17.2.15. token_usage uuids are a hash of the omp entry id rather than random, because omp rewrites session files in full on migration, fork and move, which forces a re-scan. messages::insert_batch conflicts do nothing on id, so a re-emitted event collapses onto the existing row instead of inflating token columns. It deliberately subscribes to tool_execution_start and tool_execution_end rather than tool_call and tool_result: omp fail-closes when a tool_call handler throws, so a telemetry bug there would block a user's tool. Telemetry must not be able to stop work.
…butable Every bridge called `han coordinator ensure --background --watch-path <dir>`. `--watch-path` was never a registered option, so Commander exited 1 on every call and the coordinator was never started by any bridge. Reproduced verbatim as `error: unknown option '--watch-path'`. The flag is gone; the coordinator now discovers harness roots from disk. The event envelope field is `harness` rather than `provider`, and every bridge carries the same seven-id union instead of five partial unions where each bridge knew only its own name plus opencode and claude-code. Three session-id defects that broke correlation: - kiro minted a random UUID in agent-spawn instead of resolving from the payload, so a session's first event landed under a different id than every later one. - opencode generated a UUID at plugin load while OpenCode's real session id was available on every event and used elsewhere in the same file. Loggers are now built lazily per host session. - gemini-cli re-derived its id independently at five call sites, so one process could fragment a session five ways. Consolidated to one resolver. The cross-process case is unsolved and stays unsolved: nothing documents GEMINI_SESSION_ID as a variable Gemini CLI actually sets, and inventing a replacement would be worse than a known gap. Also fixes an unguarded optional dereference in kiro's discovery that gemini-cli already guarded, and corrects five READMEs that documented the old field name and the indexing that never happened.
Reading token counts off omp's session JSONL looks like avoidable on-disk coupling until you check the alternative: after_provider_response fires before the response stream body is consumed, so final output token counts do not exist yet, and the only usage consumer on that path handles provider rate-limit headers rather than per-message accounting. Recorded in both the blueprint and the omp install page so nobody simplifies it into silently incomplete numbers. Also documents session_switch, which the bridge wires because omp retargets the session file mid-process; without it every tool call and token after a switch would be billed to the session you just left.
Verification Review of #109 (commit
|
This branch was successfully deployed
1 participant
Problem
han could only catalog Claude Code. The audit found this is not a labeling gap. Every bridge already stamped a correct harness id on its events; the ingestion path was broken in four independent places, so no bridged harness session has ever reached the database.
han coordinator ensure --background --watch-path <dir>.--watch-pathwas never a registered option, so Commander exited 1 on every call. Reproduced:error: unknown option '--watch-path'.~/.claude/projectsplus DB-registeredconfig_dirsonly. Bridge dirs were never watched.registerConfigDirthrowscoordinator-internal operation, sohan coordinator registercould not register them either.*-han.jsonlonly into a sibling native transcript and returnedOk(None)otherwise. Bridges never produce a sibling, and the full scan explicitly skipped standalone han-event files.The payoff is measurable. Running the new discovery against this machine found four harness roots quietly accumulating unindexed sessions:
All five bridge installation docs claimed these sessions appear in the Browse UI. That has been false since the bridges shipped.
What changed
Ingestion. A han events file standing alone is now indexed as a session in its own right. The transcript parser is skipped for those files: parsing han events as transcript lines produced
unknown-typed rows whose ids collided with the han event rows, which then silently failed to insert, so the events were lost even once the file was reached.native_sibling()is the single place that decides.Attribution.
sessions.harness, derived from the path rather than file contents, since a session's harness cannot legitimately vary line to line. Existing rows are backfilled toclaude-codeso a filter on the column and a read through the API agree. Exposed asSession.harness: String!, filterable and orderable.Discovery. The coordinator watches and scans
~/.han/<harness>/projects, enumerated from disk. Requiring registration would mean a bridge that never called it stayed invisible, which is the failure mode being fixed.Cost, for the first time. The han-event insert path passed
Nonefor all four token columns, so cost metrics were structurally impossible for any harness without a native transcript. A newtoken_usageevent lands those counts in the columns every existing aggregate already reads.An omp bridge. omp records per-message tokens and dollar cost in its own session file, so the bridge reads them rather than estimating. It subscribes to
tool_execution_start/endrather thantool_call/tool_resulton purpose: omp fail-closes when atool_callhandler throws, and telemetry must not be able to block a user's tool.Five bridges repaired. Broken coordinator call removed,
providerrenamed toharnesswith one shared seven-id union, and three session-id correlation bugs fixed.Honest scope
Only Claude Code and omp report tokens and cost. The other five record sessions, tool calls, and file changes with empty token columns. Closing that is a per-bridge change and needs nothing further from the indexer. The docs say so rather than implying coverage.
One caveat: harness roots are enumerated when the watcher starts, so the first session from a newly installed bridge appears at the next coordinator start or scan rather than live. Every session after that is live.
Verification
cargo test --workspacecargo check --workspace --all-targetsbun test(packages/han)tsc --noEmit,biome lintplugin generate-hooks --all --checkFour new integration tests in
han-indexer/tests/bridged_harness_test.rscover standalone ingestion, token-column population, per-harness attribution, and the negative case that a han events file beside a transcript must not become a second session.Beyond tests, the omp bridge was reconciled against a real 4.6 MB omp session, 559 assistant messages, matching omp's own totals to the cent, then exercised inside omp 17.2.15. A live check wrote a bridge events file to the real
~/.han/omp/projectsand read the session back withharness = "omp"and populated token columns.blueprints/cross-harness-metrics.mdrecords the ingestion path and each of the four breaks so they are not reintroduced.