parser: capture sessionKind and promptSource from Claude Code transcripts by TweedBeetle · Pull Request #1253 · kenn-io/agentsview

TweedBeetle · 2026-07-24T08:25:35Z

What

Capture two Claude Code launch/prompt-provenance fields that the indexer currently reads past and discards:

sessionKind — session-level (e.g. "bg"). Present on background/headless sessions, absent on interactive ones. Lands in a new session_kind column on sessions.
promptSource — per user turn (e.g. "typed", "queued", "system", "sdk"). Marks whether a turn was a human-typed prompt or a queued/system/SDK-injected one. Lands in a new prompt_source column on messages.

Why

Both signals exist on essentially every current Claude Code transcript but are thrown away at parse time, so the archive can't answer two questions it otherwise has the data for: was a session interactive or headless, and was a given prompt typed by a human or injected. This mirrors the recently-added entrypoint capture — same shape of information, same reason to keep it.

How

The change follows the existing entrypoint / agent_label and source_type / is_sidechain handling rather than inventing a new pattern:

sessionKind is treated as a session-identity field alongside entrypoint: first-non-empty-wins across the file, wired into the incremental identity-escalation check, and blanked by the Cowork and Qoder parsers that reuse the Claude parse body (exactly as they blank entrypoint).
promptSource is treated as a per-message source column alongside source_type/is_sidechain, including the retained-field whitelist in compactClaudeEntry and the message token fingerprints the PG/DuckDB push fast-paths compare against.

Both columns are NOT NULL DEFAULT '' and back-fill lazily on the next parse of each session, so transcripts that predate the fields and other agents that never emit them read empty and are otherwise untouched. Capture is mirrored across the SQLite, Postgres, and DuckDB stores and the artifact manifest.

Scope / limitations

Pure field capture — no counting, aggregation, health-signal, or automation-classification logic is touched.
session_kind/prompt_source are populated going forward; existing rows stay empty until their session is next re-parsed (no forced resync, matching how additive columns are handled elsewhere).
No new API/CLI surface or UI is added; the columns are available for consumers that read the archive directly.

Where to look

internal/parser/claude.go — the capture site (sessionKind next to entrypoint; promptSource next to isSidechain) and the compactClaudeEntry retained-field whitelist.
internal/parser/claude_provider.go, provider.go, internal/sync/engine.go — incremental identity threading for sessionKind.
internal/db/{schema.sql,db.go,sessions.go,messages.go} — SQLite columns, migrations, and read/write plumbing.
internal/postgres/* and internal/duckdb/* — the mirrored columns and fingerprints.

The source of truth for both fields is the Claude Code transcript format, where sessionKind and promptSource appear as top-level keys on session JSONL entries.

roborev-ci · 2026-07-24T08:31:20Z

roborev: Combined Review (`e35f71d`)

The change is incomplete: three Medium-severity issues could leave Claude metadata stale, break fingerprint comparisons, or lose prompt_source across persistence paths.

Medium

internal/db/db.go:324 — The parser’s persisted output changed, but dataVersion remains 71. Existing unchanged Claude sessions will be considered current and retain empty session_kind and prompt_source values indefinitely. Bump the data version to 72, document the reparse reason, and update the regression test.
internal/db/fingerprints_batch.go:125 — tokenFingerprintRow.appendTo hashes promptSource, but the batched query and scan never populate it. PostgreSQL batch comparison may skip backfilling a changed prompt_source or unnecessarily rewrite messages. Select and scan prompt_source, then test that batched and per-session fingerprints match for non-empty values.
internal/db/orphaned.go:975, internal/artifact/wire.go:175, internal/server/upload.go:208 — Several persistence paths omit PromptSource: full-resync orphan copying loses it, artifact segments do not export it, and uploaded Claude transcripts discard it during database conversion. Add prompt_source to the orphan-copy columns, artifact DTO and mapping, and upload conversion, with round-trip coverage for each path.

Reviewers: 2 done | Synthesis: codex, 9s | Total: 5m43s

mjacobs · 2026-07-24T16:49:06Z

looking

roborev-ci · 2026-07-24T20:27:51Z

roborev: Combined Review (`0db83fc`)

The changes are generally sound, but uploaded Claude sessions lose prompt provenance.

Medium

internal/server/upload.go:208 — The Claude transcript upload converter omits m.PromptSource, so uploaded sessions silently lose per-message prompt provenance. Because uploaded files live outside normal Claude discovery roots, the value will not be backfilled automatically. Map PromptSource: m.PromptSource into each db.Message and add an upload-conversion test covering both SessionKind and PromptSource.

Reviewers: 2 done | Synthesis: codex, 7s | Total: 8m28s

roborev-ci · 2026-07-24T22:14:29Z

roborev: Combined Review (`0db83fc`)

One medium-severity issue found: the Claude upload path drops per-message prompt provenance.

Medium

internal/server/upload.go:208 — sessionBatchWriteFromParsed does not copy ParsedMessage.PromptSource into the corresponding db.Message. Claude transcripts imported through the upload API therefore permanently lose prompt provenance even though the parser captured it. Populate PromptSource during conversion and add an upload-conversion round-trip test.

Reviewers: 2 done | Synthesis: codex, 5s | Total: 4m3s

roborev-ci · 2026-07-25T00:29:23Z

roborev: Combined Review (`7219443`)

No issues found.

Reviewers: 2 done | Synthesis: codex | Total: 5m1s

roborev-ci · 2026-07-25T00:29:38Z

roborev: Combined Review (`7219443`)

Frontend provenance fields are not fully propagated to the response/domain types.

Medium

Locations: frontend/src/lib/api/types/core.ts:22, frontend/src/lib/api/types/core.ts:90, frontend/src/lib/api/types/core.ts:143, frontend/src/lib/api/generated/models/ServiceSessionDetail.ts:6
Finding: The frontend response/domain types omit session_kind and prompt_source. As a result, sidebarIndexRowToSession discards session_kind, and session detail/message consumers cannot access the new fields type-safely. Updating only DbSession, DbSidebarSessionIndexRow, and DbMessage does not cover endpoints consumed through ServiceSessionDetail and the manual core interfaces.
Recommended fix: Add the fields to Session, SidebarSessionIndexRow, and Message; preserve them in the sidebar mapping; regenerate ServiceSessionDetail; and add frontend coverage verifying the fields survive API-to-store conversion.

Reviewers: 2 done | Synthesis: codex, 11s | Total: 7m0s

…ipts Claude Code JSONL entries carry two launch/prompt-provenance fields that the indexer currently discards: - `sessionKind` (session-level, e.g. "bg"): distinguishes headless / background sessions from interactive ones. - `promptSource` (per user turn, e.g. "typed", "queued", "system", "sdk"): distinguishes a human-typed prompt from a queued, system, or SDK-injected one. Neither is currently recoverable from the archive, so downstream consumers cannot tell a headless run from an interactive one, nor a typed prompt from an injected one. Both are captured by mirroring the existing `entrypoint`/`agent_label` handling exactly: - `sessionKind` becomes a session-identity field alongside `entrypoint` (first-non-empty-wins, incremental identity escalation, sessions column). It is blanked for the Cowork and Qoder parsers that reuse the Claude parse body, matching how they blank `entrypoint`. - `promptSource` becomes a per-message column alongside `source_type` and `is_sidechain`, including the retained-field whitelist in compactClaudeEntry and the message token fingerprints that the PG/DuckDB push fast paths compare against. Both columns default to empty, so historical transcripts that predate the fields and other agents that never emit them are unaffected. Capture is mirrored across the SQLite, PostgreSQL, and DuckDB stores and the artifact manifest. No counting or aggregation logic changes; this is field capture only. The source of truth for both fields is the Claude Code transcript format itself, where `sessionKind` and `promptSource` appear as top-level keys on session JSONL entries.

The batched MessageTokenFingerprints query omitted prompt_source while tokenFingerprintRow.appendTo already hashed it, so batched fingerprints disagreed with the per-session MessageTokenFingerprint for any non-empty value, causing perpetual push mismatches. The fixture now carries a non-empty prompt_source so the batched/per-session identity test pins the field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Rows now bind 26 parameters, so 39 rows per statement overflowed the historic 999-variable limit (1014 binds). Reduce to 38 rows (988). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…cile The full-resync orphan copy's message column allowlist omitted prompt_source, silently dropping it for sessions whose source files no longer exist. The transcript-revision reconcile comparison now treats prompt_source as a peer of source_subtype; as with that column's own introduction, sources predating the column skip the reconcile pass. Adds a round-trip test covering session_kind and prompt_source survival. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Parser output changed but the data version stayed at 71, so already archived Claude sessions would never re-parse and their session_kind and prompt_source columns would stay empty indefinitely. Bumping forces the standard full resync, matching how agent_label/entrypoint landed in v66. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

recomputeHeapBytes sums per-message string field sizes to bound heap release; the new prompt_source field was missing from the sum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Per the provider format provenance rule, note the 2026-07-24 local reverification that observed the two top-level keys the parser now captures; evidence class stays no-public-source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

segmentMessage mirrored source_subtype but omitted the new prompt_source field, so artifact exports dropped it. The canonical golden is unchanged because the field is omitempty and absent there. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Follows the dataVersion bump for session_kind/prompt_source backfill; the pin test's name and reason now reference the new version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

SidebarSessionIndexRow carried agent_label and entrypoint but not the new session_kind, so the sidebar API could surface one identity field and not the other. Adds the column to the SQLite, PostgreSQL, and DuckDB sidebar queries and scans, keeping all backends in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

npm run generate:api picked up the new fields on DbMessage, DbSession, and DbSidebarSessionIndexRow. Unrelated generator drift from earlier merges (duckdb attach_timeout, vector push flags, source-layout stats) is left out to keep this change scoped; it needs its own regen chore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The mirror gained sessions.session_kind and messages.prompt_source; per the mirror invariant, a schema change bumps SchemaVersion and forces a full rebuild. The column shape gate already rebuilds old mirrors, so this makes the version metadata agree with the shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

extractQueuedCommand dropped the top-level promptSource on type=attachment queued_command entries, so queued prompts stored an empty messages.prompt_source while standard user turns captured it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

roborev-ci · 2026-07-27T02:20:04Z

roborev: Combined Review (`386d033`)

Code review passed: no Medium, High, or Critical findings.

Reviewers: 2 done | Synthesis: codex, 8s | Total: 11m20s

mjacobs force-pushed the capture-sessionkind-promptsource branch from e35f71d to 0db83fc Compare July 24, 2026 20:19

TweedBeetle and others added 16 commits July 26, 2026 22:03

fix(db): keep message inserts under the 999-variable SQLite limit

1fc3860

Rows now bind 26 parameters, so 39 rows per statement overflowed the historic 999-variable limit (1014 binds). Reduce to 38 rows (988). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(sync): count prompt_source in recompute heap accounting

7a9c434

recomputeHeapBytes sums per-message string field sizes to bound heap release; the new prompt_source field was missing from the sum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

test(sync): cover session_kind and prompt_source parse-diff drift

4daf958

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

test(db): pin current data version at 72

bf24661

Follows the dataVersion bump for session_kind/prompt_source backfill; the pin test's name and reason now reference the new version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

test(storage): cover Claude provenance round trips

34e3df2

fix(server): preserve prompt source on upload

386d033

mariusvniekerk force-pushed the capture-sessionkind-promptsource branch from 7219443 to 386d033 Compare July 27, 2026 02:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

parser: capture sessionKind and promptSource from Claude Code transcripts#1253

parser: capture sessionKind and promptSource from Claude Code transcripts#1253
TweedBeetle wants to merge 16 commits into
kenn-io:mainfrom
TweedBeetle:capture-sessionkind-promptsource

TweedBeetle commented Jul 24, 2026

Uh oh!

roborev-ci Bot commented Jul 24, 2026

Uh oh!

mjacobs commented Jul 24, 2026

Uh oh!

roborev-ci Bot commented Jul 24, 2026

Uh oh!

roborev-ci Bot commented Jul 24, 2026

Uh oh!

roborev-ci Bot commented Jul 25, 2026

Uh oh!

roborev-ci Bot commented Jul 25, 2026

Uh oh!

roborev-ci Bot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

TweedBeetle commented Jul 24, 2026

What

Why

How

Scope / limitations

Where to look

Uh oh!

roborev-ci Bot commented Jul 24, 2026

roborev: Combined Review (e35f71d)

Medium

Uh oh!

mjacobs commented Jul 24, 2026

Uh oh!

roborev-ci Bot commented Jul 24, 2026

roborev: Combined Review (0db83fc)

Medium

Uh oh!

roborev-ci Bot commented Jul 24, 2026

roborev: Combined Review (0db83fc)

Medium

Uh oh!

roborev-ci Bot commented Jul 25, 2026

roborev: Combined Review (7219443)

Uh oh!

roborev-ci Bot commented Jul 25, 2026

roborev: Combined Review (7219443)

Medium

Uh oh!

roborev-ci Bot commented Jul 27, 2026

roborev: Combined Review (386d033)

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

roborev: Combined Review (`e35f71d`)

roborev: Combined Review (`0db83fc`)

roborev: Combined Review (`0db83fc`)

roborev: Combined Review (`7219443`)

roborev: Combined Review (`7219443`)

roborev: Combined Review (`386d033`)