Skip to content

fix(browse): serve the dashboard from loopback and create the tables it reads - #112

Merged
jwaldrip merged 1 commit into
mainfrom
fix/browse-loopback-default
Aug 12, 2026
Merged

jwaldrip merged 1 commit into
mainfrom
fix/browse-loopback-default

Conversation

@jwaldrip

Copy link
Copy Markdown
Collaborator

Problem

han browse still hung after #111. That PR fixed three coordinator defects but left the reported symptom in place, because the default surface is the one that cannot work.

Three layers, each hidden behind the previous one.

1. The default origin cannot reach the coordinator

han browse opened https://dashboard.local.han.guru. Chrome 151 gates a public-origin page reaching a loopback port behind the local-network-access permission. No response header grants it: Access-Control-Allow-Private-Network is the superseded PNA mechanism.

Same browser, same daemon, two origins:

Page origin Result
http://127.0.0.1:4900 RESOLVED 200 in 22ms
https://dashboard.local.han.guru request sent, never completes, no error
navigator.permissions.query({ name: 'local-network-access' }) -> "prompt"

--local was not a fallback. It runs Vite from a source checkout, and the published package contains only bin/, schemas/ and README.md:

files: ["bin/","schemas/","README.md"]
browse-client entries in tarball: 0

So the same bundle is now served from http://127.0.0.1, putting the page and the coordinator in one address space. This process fetches upstream itself, which is unrestricted. No assets ship and the release pipeline is untouched.

2. Four panels then loaded forever

With the connection working, Activity, Code Changes, Model Usage and Time of Day spun indefinitely:

Query Error: error returned from database: (code: 1) no such table: daily_aggregates

The indexer writes daily_aggregates, hourly_aggregates and global_aggregates; the API reads them for every unscoped dashboard query. No migration ever created them. Every write was discarded:

let _ = db.execute(Statement::from_sql_and_values(

which is why this shipped: the writes failed silently and only a read ever complained. Those three writes now log their failures, and a migration creates the tables.

Backfill is deliberately not in the migration. A full rebuild measured 22s for the daily pass alone on ~900k messages, and inline that would blow the coordinator's startup health budget, reintroducing the failure #111 just fixed. han_indexer::backfill_aggregates_if_empty runs in the background once the server is listening, beside the existing deferred scan, and is idempotent.

3. Backfill then failed on a missing column

Aggregate backfill failed: backfilling daily_aggregates: no such column: lines_added

messages had no lines_added, lines_removed or files_changed, though the entity declares all three and every activity query sums them. m20260215_000001_initial creates them, and it is recorded as applied. But the table carries -- comments SeaORM never emits:

CREATE TABLE messages (
    id TEXT PRIMARY KEY,  -- This IS the message UUID from JSONL

It was created by hand-written SQL from the TypeScript era and the migration was stamped without running. The migration now adds those columns when absent. Historical rows stay NULL, which reads as 0; the indexer fills them going forward.

Verification

Against the real 8.8GB database, not a fixture:

daily=52  hourly=24  global=1  (1036 sessions, 885,304 messages)
messages change columns: 3/3 present
INFO han_indexer::processor: Backfilling dashboard aggregates from message history
INFO han_coordinator: Dashboard aggregates ready

Dashboard, driven in Chrome 151 at http://127.0.0.1:41956:

stuckPanels: 0
connecting:  false
Activity -> Active Days 52, contribution heatmap
Model Usage -> Opus 4.6 35.7B, Opus 4.5 52.1B, Sonnet 4.6 5.1B

WebSocket subscriptions confirmed over the same path:

WS created wss://coordinator.local.han.guru:41957/graphql
WS handshake status=101
WS sent {"type":"connection_init"} -> WS recv {"type":"connection_ack"}
  • Rust workspace: 519 passed, 0 failed
  • TypeScript: 2168 pass, 0 fail; tsc clean; biome clean; cargo fmt clean; no clippy findings in the new files
  • 5 proxy tests (path forwarding, content-encoding not double-encoded, upstream status preserved, unreachable upstream reports instead of hanging)
  • 4 migration tests. Falsified by unregistering the migration, which reproduces the production error verbatim: no such table: daily_aggregates

Notes

han browse is now a long-running foreground process, as --local already was, since it serves the page. It prints "Press Ctrl+C to stop" and shuts the listener down on SIGINT/SIGTERM.

Code Changes will read zero until new sessions are indexed. Those columns never existed on old databases, so that history is genuinely unrecoverable rather than merely unaggregated.

Unchanged from #111 and still worth fixing separately: install.sh never downloads han-coordinator, and the release uploads all four Unix coordinators under one asset name. On this machine no coordinator binary existed at all, which is why indexed history stops at 2026-02-08.

…it reads

`han browse` opened https://dashboard.local.han.guru directly. Chrome 151 gates
a public-origin page reaching a loopback port behind the local-network-access
permission, which no response header can grant, so every request to the
coordinator stalled with no error and the dashboard sat on "Connecting to Han
Coordinator..." forever. `--local` was no fallback: it needs a source checkout,
and the published package ships only bin/, schemas/ and README.md.

The same bundle is now served from http://127.0.0.1, which puts the page and
the coordinator in one address space so no permission is involved. This process
fetches upstream itself, which is unrestricted.

That exposed the next layer. Four dashboard panels then loaded forever on
"no such table: daily_aggregates". The indexer writes daily_aggregates,
hourly_aggregates and global_aggregates and the API reads them, but no migration
ever created them, and every write was discarded with `let _` so nothing
surfaced until a read failed. Those writes are now logged, and a migration
creates the three tables.

Backfill lives in han_indexer::backfill_aggregates_if_empty and runs in the
background after the server is listening, alongside the existing deferred scan.
A full rebuild costs about 45 seconds per 900k messages, which is long enough to
blow the startup health budget if it ran inline.

Backfilling then failed on "no such column: lines_added". Databases old enough
to predate the SeaORM migrations were created by hand-written SQL and had
m20260215_000001_initial recorded as applied without ever running, so their
messages table lacks three columns the entity declares and every activity query
sums. The migration adds them when absent.

Verified against a real 8.8GB database: 52 daily rows, 24 hourly rows, 1036
sessions and 885,304 messages backfilled, all three columns added, and the
dashboard renders with zero panels left loading. Guard tests were falsified by
unregistering the migration, which reproduces the exact production error.
@railway-app

railway-app Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the han-pr-112 environment in han-team-platform

Service Status Web Updated (UTC)
api ✅ Success (View Logs) Aug 12, 2026 at 4:21 pm
3 services not affected by this PR
  • postgres
  • redis
  • website

@railway-app
railway-app Bot temporarily deployed to han-team-platform / han-pr-112 August 12, 2026 16:18 Destroyed
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

@claude

claude Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review

Read through the coordinator/migration changes, the loopback proxy, and the new tests. Solid diagnosis and the layered writeup in the PR description is genuinely helpful for anyone hitting this later. One correctness issue in the backfill path that I think undermines the fix for the exact scenario it's meant to solve; a couple of smaller things below it.

CRITICAL

Backfill-vs-incremental-update race can silently skip the backfill, leaving history permanently incomplete.

backfill_aggregates_if_empty (packages/han-rs/crates/han-indexer/src/processor.rs:400-414) decides whether to run based on SELECT COUNT(*) FROM global_aggregates. But main.rs spawns three things concurrently right after the server starts listening: the file watcher (watcher_bridge::start_watcher_bridge, started even earlier), the --scan-on-start full scan (main.rs:283-298, and --scan-on-start is always passed by the TS CLI per coordinator-service.ts:212), and the backfill task (main.rs:301-311) — with no ordering between them.

index_session_file calls update_aggregates(db, &session_id) for any session with total_indexed > 0 (processor.rs:2068), and update_aggregates does an INSERT OR REPLACE into global_aggregates (single row, id = 1) every time it runs. So if a single session gets indexed — a live/actively-writing session picked up by the watcher, or a session with a few unindexed trailing lines found by the full scan — before backfill_aggregates_if_empty's SELECT COUNT(*) executes, that check sees count > 0, treats the tables as "already filled", and returns Ok(false) without ever running the backfill queries. daily_aggregates/hourly_aggregates then only contain the dates/hours that one incremental update touched, not the full history — which is precisely the failure mode the function's own doc comment calls out ("the dashboard would otherwise show only the days you happened to work after upgrading"). It's silent: no error, no log line distinguishing "genuinely already backfilled" from "raced and skipped", and no mechanism to retrigger short of manually clearing global_aggregates.

This is exactly the upgrade scenario the PR targets (large pre-existing DB, restarting the coordinator), and it's plausible in normal use — e.g. upgrading while a Claude Code session is actively running and still appending to its JSONL file, so the watcher indexes new lines within the first second of startup. The 8.8GB verification run in the PR description apparently didn't hit it (probably because that DB had nothing left to index incrementally at the time), but that's scheduling luck, not a guarantee, and there's no test exercising the concurrent path — aggregate_tables_test.rs only calls backfill_aggregates_if_empty in isolation.

Suggested directions: make backfill's "has this run" signal independent of global_aggregates row presence (e.g. a dedicated sentinel/flag row, or a marker set only by the backfill path itself), or run the backfill-emptiness-check/full-scan/watcher-start in a defined order (e.g. gate watcher/full-scan start on backfill's initial check completing) so the race can't happen.

MODERATE

Default han browse now binds a TCP port and blocks in the foreground; a second concurrent invocation will fail. Before this PR, han browse (non---local) just opened a browser tab and exited — you could run it as many times as you liked. Now it calls startHostedDashboardProxy({ port }) and blocks on SIGINT/SIGTERM (index.ts:238-260). Running han browse a second time while one is already active on the same port will hit EADDRINUSE and exit 1 (via the catch in main.ts:277-286) instead of just opening another tab. This is a reasonable tradeoff for fixing the actual bug, but worth confirming it's intended — e.g. someone with a shell alias or script that calls han browse opportunistically will now get an error instead of a no-op.

MINOR

  • hosted-proxy.ts's upstream fetch (packages/han/lib/commands/browse/hosted-proxy.ts:613-620) never forwards req.method — it always issues a GET to upstream regardless of the incoming method. Harmless for the current GET-only asset-serving use, but silently wrong if anything ever sends the proxy a HEAD/OPTIONS request.
  • The proxy also doesn't forward conditional-request headers (If-None-Match, If-Modified-Since) or Range, so every asset reload fetches the full body from upstream instead of getting a 304/partial response. Not a correctness issue, just a bit more bandwidth/latency than necessary for a page you might reload often during a session.

Nit

The 45s backfill estimate in the processor.rs/migration doc comments doesn't match the "22s for the daily pass alone" figure in the PR description — not a code issue, just worth reconciling so the comment doesn't mislead the next reader.

Everything else — the migration itself (idempotent CREATE TABLE IF NOT EXISTS, guarded ALTER TABLE ADD COLUMN via pragma_table_info), the let _ = → logged-warning fix, the loopback-proxy design and its header-stripping/error-page tests — looks correct and well-tested for the paths that were tested.

@jwaldrip
jwaldrip merged commit b13c116 into main Aug 12, 2026
10 checks passed
@jwaldrip
jwaldrip deleted the fix/browse-loopback-default branch August 12, 2026 16:28

This branch was successfully deployed

No deployments
han-team-platform / han-pr-112 — cfb46a92 Deployed Aug 12, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant