feat(observer): Data Observer Phase 2 — Catalog/Explorer + 4V Health dashboard

[oldhero5]

Summary

Phase 2 of the internal Data Observer — the core "explore what I have." Builds a real catalog/explorer and a 4-V (Volume/Velocity/Variety/Veracity) Health dashboard on top of the Phase-1 auth/RBAC spine, sourced entirely from ArcticDB via energex.core (no Dagster). Follows #27.

What's included

Backend (observer-api, src/energex/observer/) — all endpoints require_role(viewer):

• Cheap catalog — /catalog now uses lib.get_description() (row count + latest valid_time, no full-data read) + vintage sidecars, replacing the Phase-1 per-symbol full-scan.
• Per-symbol detail — /symbol/{lib}/{sym}/series (point-in-time via storage.read_as_of with an explicit mode), /schema, /vintages.
• Veracity — /symbol/{lib}/{sym}/quality re-runs the platform's own pandera gate (quality.validate) against stored data, so "broken" means exactly what the pipeline means (the ercot_load freshness failure surfaces honestly); plus valid_time gap counting and an OHLCV anomaly summary.
• 4V metrics — /metrics/overview + /metrics/health, backed by a per-symbol freshness heuristic (TTL-cached) whose tolerances mirror schemas.py (importable constants imported; a test cross-checks against drift). The full gate stays on-demand per symbol, never a full scan at request time.
• A library → core.schemas routing map (schema_map.py) mirroring orchestration/checks.py.

Frontend (observer-web/, Next.js 16 + Tailwind v4 + ECharts):

• Overview / Health — 4V headline tiles, a freshness heatmap, and a broken-data alert rail.
• Catalog / Explorer — a sources→libraries→symbols tree + a detail panel with Overview / Series / Schema / Vintages / Quality tabs. The Series tab is the signature bitemporal as_of point-in-time slider over an ECharts chart ("what we knew then").
• A session-gated, GET-only same-origin route-handler proxy (/api/observer/[...path]) — the browser never reaches observer-api directly (in-network URL, CORS-scoped); interactive fetches go through the proxy, which attaches the bearer server-side.

Security & correctness

Every data endpoint is role-enforced server-side; the browser holds only the anon key + JWT. The proxy is not an open proxy (fixed observer-api base, no host/scheme injection) and never returns the bearer. A five-lens whole-branch review (spec / security / backend / frontend / cross-cutting) returned security PASS, backend PASS, and caught two cross-task contract mismatches (the /series response shape vs the chart; a tri-state quality.passed rendering a false "fail" badge) — both fixed and now locked by contract tests on each side.

Testing

• Backend: 262 pytest passed (offline, ArcticDB on LMDB — including a deliberately gate-failing symbol proving Veracity surfaces it, point-in-time reconstruction, the freshness mirror cross-check, and the new /series contract assertions); ruff check + format clean.
• Frontend: Vitest 18 passed (incl. new series-chart + quality-panel contract tests) + npm run build green; a Playwright @smoke explorer flow (deferred run, needs the composed stack).

Notes

• No Dagster / map / graph / admin this phase (later phases); no new Supabase migrations.
• Follow-up minors (non-blocking) are tracked for the finish-cleanup pass: an early-401 in the proxy for defense-in-depth, overview() scanning the catalog twice, and totalizing the freshness-mirror drift guard.