Phase 3: playground — grounded extraction demo (Hono API + Vite/React)

CLAUDE.md

@@ -4,7 +4,7 @@ extractkit is an open-source TypeScript document-extraction engine: Zod schema +
 
 ## Current Phase
 
-**Phase 2 (evals) harness shipped; live runs and the DocILE half are pending.** `packages/core` and the `packages/evals` harness are implemented and tested against mock models. The 25 CORD-v2 receipts are pinned in `packages/evals/data/manifest.json`; the 25 DocILE invoices are blocked on the dataset token (ROADMAP Phase 0, pending human action) — loader and curation script are ready. The eval lineup spans Anthropic, OpenAI, and Google Gemini (`packages/evals/src/models.ts`); a run includes every provider whose API key is set, or the subset named in `EVAL_PROVIDERS`. The first live eval run (needs at least one provider key — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or `GOOGLE_GENERATIVE_AI_API_KEY`) doubles as core's first live-provider validation and fills the benchmark page + README table via `pnpm report`. `apps/playground` does not exist yet. Keep README/ROADMAP/docs in sync with what actually ships.
+**Phase 3 (playground) shipped against mock models; the first live provider run is the remaining gate across evals + playground.** `packages/core`, the `packages/evals` harness, and `apps/playground` are all implemented and tested against mock models. The 25 CORD-v2 receipts are pinned in `packages/evals/data/manifest.json`; the 25 DocILE invoices are blocked on the dataset token (ROADMAP Phase 0, pending human action) — loader and curation script are ready. The eval lineup spans Anthropic, OpenAI, and Google Gemini (`packages/evals/src/models.ts`); a run includes every provider whose API key is set, or the subset named in `EVAL_PROVIDERS`. The playground (`apps/playground`) is a Hono API + Vite/React client: `GET /api/config`, `POST /api/extract` streamed as SSE via core's `streamExtract`, preset invoice/receipt schemas defined server-side, and a model registry resolved from whichever provider keys are present. It needs the same provider keys to run live, and its model registry is injectable so the routes test against a mock model. The first live run (any one of `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_GENERATIVE_AI_API_KEY`) doubles as core's first live-provider validation, fills the benchmark page + README table via `pnpm report`, and enables capturing the playground demo GIF. Keep README/ROADMAP/docs in sync with what actually ships.
 
 ## Planned Architecture
 

README.md

@@ -2,7 +2,7 @@
 
 **Extraction you can audit.** Define a Zod schema, feed it a PDF or image, get back schema-validated JSON where every field carries provenance — the page and bounding box it came from — plus a confidence score.
 
-> **Status: in development.** The core library ([`packages/core`](./packages/core)) is implemented and tested against mock models; the eval benchmark and playground are next. See [ROADMAP.md](./ROADMAP.md).
+> **Status: in development.** The core library ([`packages/core`](./packages/core)), the eval harness ([`packages/evals`](./packages/evals)), and the playground ([`apps/playground`](./apps/playground)) are implemented and tested against mock models. What's left is the first live provider run, which publishes the benchmark table and the demo GIF. See [ROADMAP.md](./ROADMAP.md).
 
 ## Why
 
@@ -12,7 +12,7 @@ TypeScript has structured-output libraries (instructor-js, AI SDK `generateObjec
 
 - **Core library** (`packages/core`, shipped) — Zod schema + PDF/image → validated JSON with per-field `{ value, confidence, page, bbox }`. Provider-agnostic via the Vercel AI SDK. Document validation, typed failure handling, repair retries, streaming, and cost tracking built in. [Usage docs →](./packages/core/README.md)
 - **Eval harness** (`packages/evals`, harness shipped) — public benchmark on ~50 pinned real documents (CORD-v2 receipts + DocILE invoices): field accuracy per model, grounding accuracy, cost per 1k docs. Fully reproducible — documents pinned by checksum, reports generated only from recorded runs. [Reproduce it →](./packages/evals/README.md)
-- **Playground** (planned) — drag-drop a document, watch fields extract; hover a JSON field to highlight its source region on the page.
+- **Playground** (`apps/playground`, built) — drag-drop a document, watch fields stream in, hover a field to highlight its source region on the page. Hono API + Vite/React client, running `extractkit` against a live model. [Run it →](./apps/playground/README.md)
 
 See [ROADMAP.md](./ROADMAP.md) for the build plan.
 

ROADMAP.md

@@ -24,9 +24,9 @@ Tested against mock models only so far; first live-provider validation happens w
 
 ## Phase 3 — Playground (`apps/playground`)
 
-- [ ] Hono API + Vite/React client
-- [ ] Drag-drop extraction with hover-field → highlight-source-bbox interaction
-- [ ] Demo GIF for the README
+- [x] Hono API + Vite/React client — `GET /api/config`, `POST /api/extract` streamed as SSE; routes tested against a mock model
+- [x] Drag-drop extraction with hover-field → highlight-source-bbox interaction (images + PDFs via pdfjs)
+- [ ] Demo GIF for the README (needs one live run — same provider key as the Phase 2 first run)
 
 ## Phase 4 — Release
 

apps/playground/.env.example

@@ -0,0 +1,8 @@
+# Set at least one provider key; the playground offers the models whose key is present.
+# The server reads these from the environment (or a local .env file, git-ignored).
+ANTHROPIC_API_KEY=
+OPENAI_API_KEY=
+GOOGLE_GENERATIVE_AI_API_KEY=
+
+# Port the Hono API server listens on (default 8787). The Vite dev server proxies /api here.
+PORT=8787

apps/playground/README.md

@@ -0,0 +1,62 @@
+# extractkit playground
+
+The hosted demo for [extractkit](../../packages/core): drop in a PDF or image, watch fields
+stream in as they extract, and **hover any field to highlight the exact region it came from** on the
+page. A Vite + React client talks to a small Hono API that runs `extractkit` against a live model.
+
+## Run it
+
+From the repo root (the core library must be built once so the server can import it):
+
+```sh
+pnpm install
+pnpm --filter extractkit build
+```
+
+Set at least one provider key — copy `.env.example` to `.env` and fill one in, or export it:
+
+```sh
+# .env  (git-ignored)
+ANTHROPIC_API_KEY=sk-ant-...
+# or OPENAI_API_KEY=... / GOOGLE_GENERATIVE_AI_API_KEY=...
+```
+
+Then, from `apps/playground`:
+
+```sh
+pnpm dev      # Hono API on :8787 + Vite client on :5173 (open this)
+```
+
+The client offers whichever models have a key set. With no key, the UI loads but extraction is
+disabled and tells you which variable to set.
+
+### Production preview
+
+```sh
+pnpm build    # bundles the client to dist/client
+pnpm start    # Hono serves the API and the built client on :8787
+```
+
+## How it works
+
+- **`src/server`** — a Hono app (`app.ts`) with two routes:
+  - `GET /api/config` — the preset schemas and the models available in this environment.
+  - `POST /api/extract` — a multipart upload streamed back as Server-Sent Events. It calls
+    `streamExtract` from core and forwards each field as it completes, then a final `result` or a
+    typed `error`. The model registry is injected, so tests exercise the routes against a mock model
+    with no API key.
+- **`src/client`** — the React app. `DocumentViewer` renders images directly and PDFs via `pdfjs`,
+  overlaying each field's bounding box (normalized 0–1 coordinates from core) on the page.
+  `ResultPanel` shows fields streaming in, then the validated tree with per-field confidence, token
+  usage, and cost.
+- **`src/shared`** — the DTOs both sides share.
+
+Schemas are fixed presets (invoice, receipt) defined server-side — the playground does not accept
+arbitrary schemas from the browser.
+
+## Test & typecheck
+
+```sh
+pnpm test        # server routes (mock model) + client logic (SSE, geometry, fields, upload)
+pnpm typecheck   # client (DOM) and server (node) tsconfigs
+```

apps/playground/index.html

@@ -0,0 +1,12 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>extractkit · playground</title>
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/client/main.tsx"></script>
+  </body>
+</html>

apps/playground/package.json

@@ -0,0 +1,43 @@
+{
+  "name": "@extractkit/playground",
+  "private": true,
+  "version": "0.0.0",
+  "description": "Hosted demo for extractkit: drag-drop a document, watch fields extract, hover a field to highlight its source region on the page.",
+  "type": "module",
+  "engines": {
+    "node": ">=20.19"
+  },
+  "scripts": {
+    "dev": "concurrently -k -n server,client -c blue,green \"pnpm dev:server\" \"pnpm dev:client\"",
+    "dev:server": "tsx watch src/server/index.ts",
+    "dev:client": "vite",
+    "build": "vite build",
+    "start": "NODE_ENV=production tsx src/server/index.ts",
+    "test": "vitest run",
+    "typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.node.json --noEmit"
+  },
+  "dependencies": {
+    "@ai-sdk/anthropic": "^4.0.10",
+    "@ai-sdk/google": "^4.0.12",
+    "@ai-sdk/openai": "^4.0.11",
+    "@hono/node-server": "^2.0.8",
+    "ai": "^7.0.16",
+    "extractkit": "workspace:*",
+    "hono": "^4.12.29",
+    "pdfjs-dist": "^6.1.200",
+    "react": "^19.2.7",
+    "react-dom": "^19.2.7",
+    "zod": "^4.4.3"
+  },
+  "devDependencies": {
+    "@types/node": "^26.1.0",
+    "@types/react": "^19.2.17",
+    "@types/react-dom": "^19.2.3",
+    "@vitejs/plugin-react": "^6.0.3",
+    "concurrently": "^10.0.3",
+    "tsx": "^4.23.0",
+    "typescript": "^6.0.3",
+    "vite": "^8.1.4",
+    "vitest": "^4.1.10"
+  }
+}

apps/playground/src/client/App.tsx

@@ -0,0 +1,174 @@
+import { lazy, Suspense, useCallback, useEffect, useMemo, useRef, useState } from 'react';
+import type { ApiError, ConfigResponse, SerializedResult } from '../shared/api';
+import { fetchConfig, runExtract } from './lib/api';
+import type { FieldEntry } from './lib/fields';
+import { fieldEntries, pathKey } from './lib/fields';
+import { Dropzone } from './components/Dropzone';
+import { Toolbar } from './components/Toolbar';
+import { ResultPanel } from './components/ResultPanel';
+
+// The viewer pulls in pdfjs; load it only once a document is opened.
+const DocumentViewer = lazy(() =>
+  import('./components/DocumentViewer').then((m) => ({ default: m.DocumentViewer })),
+);
+
+export type Phase = 'idle' | 'running' | 'done' | 'error';
+
+interface RunState {
+  phase: Phase;
+  live: FieldEntry[];
+  liveKeys: Set<string>;
+  result: SerializedResult | null;
+  error: ApiError | null;
+}
+
+const IDLE_RUN: RunState = { phase: 'idle', live: [], liveKeys: new Set(), result: null, error: null };
+
+export function App() {
+  const [config, setConfig] = useState<ConfigResponse | null>(null);
+  const [configError, setConfigError] = useState<string | null>(null);
+  const [schemaId, setSchemaId] = useState('');
+  const [modelId, setModelId] = useState('');
+  const [file, setFile] = useState<File | null>(null);
+  const [docUrl, setDocUrl] = useState<string | null>(null);
+  const [run, setRun] = useState<RunState>(IDLE_RUN);
+  const [activeKey, setActiveKey] = useState<string | null>(null);
+  const abortRef = useRef<AbortController | null>(null);
+
+  useEffect(() => {
+    const controller = new AbortController();
+    fetchConfig(controller.signal)
+      .then((cfg) => {
+        setConfig(cfg);
+        setSchemaId(cfg.defaults.schema);
+        setModelId(cfg.defaults.model ?? '');
+      })
+      .catch((err: unknown) => {
+        if (!controller.signal.aborted) {
+          setConfigError(err instanceof Error ? err.message : 'Failed to load configuration.');
+        }
+      });
+    return () => controller.abort();
+  }, []);
+
+  useEffect(() => {
+    if (docUrl === null) return;
+    return () => URL.revokeObjectURL(docUrl);
+  }, [docUrl]);
+
+  const selectFile = useCallback((next: File) => {
+    setFile(next);
+    setDocUrl(URL.createObjectURL(next));
+    setActiveKey(null);
+    setRun(IDLE_RUN);
+  }, []);
+
+  const cancel = useCallback(() => {
+    abortRef.current?.abort();
+  }, []);
+
+  const extract = useCallback(async () => {
+    if (file === null || modelId === '') return;
+    abortRef.current?.abort();
+    const controller = new AbortController();
+    abortRef.current = controller;
+    setActiveKey(null);
+    setRun({ phase: 'running', live: [], liveKeys: new Set(), result: null, error: null });
+
+    try {
+      for await (const ev of runExtract({ file, schema: schemaId, model: modelId, signal: controller.signal })) {
+        if (ev.type === 'field') {
+          const key = pathKey(ev.path);
+          setRun((prev) => {
+            if (prev.liveKeys.has(key)) return prev;
+            const liveKeys = new Set(prev.liveKeys);
+            liveKeys.add(key);
+            return { ...prev, live: [...prev.live, { key, path: ev.path, field: ev.field }], liveKeys };
+          });
+        } else if (ev.type === 'result') {
+          setRun((prev) => ({ ...prev, phase: 'done', result: ev.result }));
+        } else {
+          setRun((prev) => ({ ...prev, phase: 'error', error: ev.error }));
+        }
+      }
+    } catch (err) {
+      if (controller.signal.aborted) {
+        setRun(IDLE_RUN);
+      } else {
+        setRun((prev) => ({
+          ...prev,
+          phase: 'error',
+          error: { name: 'NetworkError', code: null, message: err instanceof Error ? err.message : 'Extraction failed.' },
+        }));
+      }
+    }
+  }, [file, modelId, schemaId]);
+
+  const entries = useMemo<FieldEntry[]>(
+    () => (run.result !== null ? fieldEntries(run.result.fields) : run.live),
+    [run.result, run.live],
+  );
+  const boxes = useMemo(
+    () => entries.filter((e) => e.field.bbox !== null && e.field.page !== null),
+    [entries],
+  );
+
+  return (
+    <div className="app">
+      <header className="app-header">
+        <div className="brand">
+          <span className="brand-mark">extractkit</span>
+          <span className="brand-sub">playground</span>
+        </div>
+        <p className="tagline">Drag in a document, watch fields extract, hover a field to see where it came from.</p>
+        <a className="repo-link" href="https://github.com/RATCHAW/extractkit" target="_blank" rel="noreferrer">
+          GitHub ↗
+        </a>
+      </header>
+
+      <Toolbar
+        config={config}
+        configError={configError}
+        schemaId={schemaId}
+        modelId={modelId}
+        hasFile={file !== null}
+        phase={run.phase}
+        onSchema={setSchemaId}
+        onModel={setModelId}
+        onFile={selectFile}
+        onExtract={() => void extract()}
+        onCancel={cancel}
+      />
+
+      <main className="workspace">
+        <section className="viewer-pane">
+          {file !== null && docUrl !== null ? (
+            <Suspense fallback={<div className="viewer-message">Loading viewer…</div>}>
+              <DocumentViewer
+                file={file}
+                docUrl={docUrl}
+                boxes={boxes}
+                activeKey={activeKey}
+                onActivate={setActiveKey}
+                onFile={selectFile}
+              />
+            </Suspense>
+          ) : (
+            <Dropzone onFile={selectFile} />
+          )}
+        </section>
+        <section className="result-pane">
+          <ResultPanel
+            phase={run.phase}
+            live={run.live}
+            result={run.result}
+            error={run.error}
+            hasFile={file !== null}
+            activeKey={activeKey}
+            onActivate={setActiveKey}
+          />
+        </section>
+      </main>
+    </div>
+  );
+}