Skip to content

Preserve episode identity, caller lineage, and unified session metadata - #67518

Merged
pelikhan merged 3 commits into
mainfrom
pelikhan-episode-computation-status
Oct 10, 2026
Merged

pelikhan merged 3 commits into
mainfrom
pelikhan-episode-computation-status

Conversation

@pelikhan

Copy link
Copy Markdown
Collaborator

Why

Episode reporting dropped canonical lineage fields, split partial selections of the same episode, and recorded a grandparent instead of the immediate caller across multiple workflow hops. Unified agent sessions also omitted the available episode metadata.

Approach

  • Preserve canonical episode and root metadata through CLI log decoding, caching, and JSON output. Group selected runs by canonical episode identity while rejecting conflicting identities, unrelated inferred parents, and same-run self-edges.
  • Share parent-hop resolution between runtime context propagation and OpenTelemetry emission. Preserve immediate caller lineage across multi-hop workflows, replayed context, and reusable workflows sharing a run ID.
  • Persist an optional, allowlisted workflow.info.data.episode object in usage/aw_session.jsonl, with type declarations, generated schema, text rendering, and specification updates.

Compatibility and boundaries

Legacy contexts retain their existing behavior. CLI rollups remain run-level and cover only selected runs; this does not introduce per-hop graph nodes or full-lifetime totals. Unified-session metadata preserves the recorded caller's identifiers rather than synthesizing a current hop. Legacy call IDs alone do not create canonical episode metadata. The session file-format version remains 1, and unrelated caller context is excluded.

These changes add no GitHub API requests.

Validation

  • Focused Go episode, JSON, and cache regressions passed.
  • Runtime context and OpenTelemetry JavaScript suites passed (600 tests).
  • Unified-session payload, persistence, rendering, and schema suites passed (224 tests).
  • npm run typecheck, formatting, generated-schema freshness, and workflow drift checks passed.
  • Final unified-session increment passed BASE_REF=HEAD PATH="/opt/homebrew/bin:$PATH" make agent-report-progress.

The earlier combined validation encountered a shared-host golangci-lint lock; its non-test checks passed on retry with an isolated temporary directory.

pelikhan and others added 3 commits October 10, 2026 13:24
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review October 10, 2026 22:30
Copilot AI balanced review requested due to automatic review settings October 10, 2026 22:30
@pelikhan
pelikhan merged commit 91ccb55 into main Oct 10, 2026
26 of 45 checks passed
@pelikhan
pelikhan deleted the pelikhan-episode-computation-status branch October 10, 2026 22:30
Copilot stopped reviewing on behalf of pelikhan due to an error October 10, 2026 22:30
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

Copy link
Copy Markdown
Contributor

🔎 PR Code Quality Reviewer is reviewing code quality for this pull request...

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Generated by Ponytail Reviewer for #67518

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Copilot review overview

4 open findings
What changed in this PR

Adds canonical episode lineage fields end-to-end (schemas, Go rollups, JS session normalization/rendering, and OTLP attribute behavior) so logs and unified sessions can preserve runner-owned episode identity across workflow hops.

Changes:

  • Extend log/audit schemas and Go AwContext to include canonical episode lineage fields (episode_id, hop_id, parent_hop_id, etc.).
  • Rework Go episode aggregation to prefer canonical lineage, avoid conflicting merges, and centralize rollup logic in an accumulator.
  • Project/render episode lineage into unified session payloads and update OTLP attribute handling + tests/docs.
File Description
schemas/​logs.schema.json Adds canonical episode lineage fields to logs schema.
schemas/​logs-jsonl.schema.json Adds canonical episode lineage fields across JSONL schema sections.
schemas/​audit.schema.json Adds canonical episode lineage fields to audit schema.
pkg/​cli/​logs_models.go Extends AwContext with canonical episode lineage fields.
pkg/​cli/​logs_episode.go Reworks episode grouping/edge building to respect canonical lineage; refactors rollup.
pkg/​cli/​logs_episode_rollup.go New accumulator-based episode rollup implementation.
pkg/​cli/​*.go tests Adds tests asserting canonical context preservation and conflict handling.
actions/​setup/​js/​unified_session_payload.cjs Projects episode lineage from trusted workflow context into unified payload.
actions/​setup/​js/​unified_session_render.cjs Renders episode lineage in session summaries.
actions/​setup/​js/​send_otlp_span.cjs Uses resolveParentHopId for hop parent attribute emission.
docs/​* Documents episode lineage behavior and updated CLI examples/schema.

🧠 Review effort: Lite


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +178 to +182
return (
fields(data, ["engineId", "model", "requestedModel", "modelRouting", "triggerType", "cliVersion", "awfVersion", "mcpgVersion", "agentVersion", "workflow", "repository", "runId"]) +
" " +
fields(data.episode, ["episodeId", "hopId", "parentHopId", "originEvent", "rootRepo", "rootWorkflowId", "rootRunId"])
);
Comment on lines +92 to +99
if run.StartedAt.IsZero() && run.UpdatedAt.IsZero() {
return
}
if !run.StartedAt.IsZero() && !run.UpdatedAt.IsZero() && run.UpdatedAt.After(run.StartedAt) {
acc.duration += run.UpdatedAt.Sub(run.StartedAt)
} else if processedRun.Run.Duration > 0 {
acc.duration += processedRun.Run.Duration
}
Comment thread pkg/cli/logs_episode.go
Comment on lines +537 to +549
for _, parent := range candidates {
return EpisodeEdge{
SourceRunID: parent.RunID,
TargetRunID: run.RunID,
EdgeType: edgeType,
Confidence: confidence,
Reasons: append([]string(nil), reasons...),
SourceRepo: parent.Repository,
SourceRef: parent.WorkflowPath,
EventType: parent.Event,
}, true
}
return EpisodeEdge{}, false
Comment thread pkg/cli/logs_models.go
Comment on lines +429 to +441
EpisodeID string `json:"episode_id,omitempty"`
HopID string `json:"hop_id,omitempty"`
ParentHopID string `json:"parent_hop_id,omitempty"`
OriginEvent string `json:"origin_event,omitempty"`
RootRepo string `json:"root_repo,omitempty"`
RootWorkflowID string `json:"root_workflow_id,omitempty"`
RootRunID string `json:"root_run_id,omitempty"`
Time string `json:"time,omitempty"` // ISO 8601 timestamp of the dispatch
Actor string `json:"actor,omitempty"` // GitHub actor that triggered the calling workflow
EventType string `json:"event_type,omitempty"` // GitHub event name of the calling workflow
ItemType string `json:"item_type,omitempty"` // Kind of triggering item: "issue", "pull_request", "discussion", "check_run", "check_suite", or ""
ItemNumber string `json:"item_number,omitempty"` // Number (issue/PR/discussion) or database id (check_run/check_suite) of the triggering item
CommentID string `json:"comment_id,omitempty"` // ID of the triggering comment or review; empty when not a comment/review event
@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (426 new lines across pkg/cli/ and actions/setup/js/) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR: intended path docs/adr/67518-preserve-canonical-episode-identity-and-caller-lineage.md — the automatic commit to this branch failed (the PR head commit is not available in the gate's checkout, which is pinned to the merge ref), so the full draft is inlined below. Copy it into your branch at that path.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📄 Draft ADR-67518 (copy to docs/adr/67518-preserve-canonical-episode-identity-and-caller-lineage.md)

ADR-67518: Preserve Canonical Episode Identity and Immediate Caller Lineage

Date: 2026-10-10
Status: Draft
Deciders: pelikhan


Context

gh aw reconstructs multi-hop automation "episodes" from workflow run logs, and the agentic runtime propagates episode/lineage context between workflow hops (actions/setup/js/aw_context.cjs) and into OpenTelemetry spans (send_otlp_span.cjs). Episode reporting was dropping canonical episode and root identity fields during CLI log decoding and caching, which caused partial selections of the same episode to be split into separate episodes in rollups. Lineage resolution also walked past the immediate caller and recorded a grandparent run when a workflow chain was more than two hops deep, or when context was replayed, or when reusable workflows shared a single run ID. Separately, the unified agent session file (usage/aw_session.jsonl) carried no episode metadata at all even though the information was available at write time. The constraint is that these are observability surfaces with existing published contracts: the session file-format version must stay 1, legacy contexts without canonical identity must keep their current behaviour, and no new GitHub API requests may be introduced.

Decision

We will treat the canonical episode identifier as the primary grouping key for episode reconstruction and preserve it end-to-end through CLI log decoding, the log cache, and JSON output. Runs are grouped by canonical episode identity (canonicalEpisodeID), and edges whose source and target resolve to different canonical identities are rejected and logged rather than merged; unrelated inferred parents and same-run self-edges are likewise excluded. Rollup accumulation is extracted into a dedicated pkg/cli/logs_episode_rollup.go accumulator so identity resolution and metric aggregation are separate concerns. Parent-hop resolution is shared by runtime context propagation and OTEL span emission so both record the immediate caller rather than an ancestor. Finally, we will persist an optional, allowlisted workflow.info.data.episode object in the unified session payload, with matching TypeScript declarations, generated JSON schema, text rendering, and specification updates, keeping the session file-format version at 1.

The primary driver is correctness of episode attribution: identity must be carried, not re-derived, and lineage must be recorded at the hop that actually made the call.

Alternatives Considered

Alternative 1: Keep inferring episode grouping from run parentage only

Continue grouping runs purely by walking the inferred parent/root relationships, without persisting or trusting a canonical episode ID. This requires no schema or cache changes and keeps the decoding path simple. It was rejected because inference is lossy for partial selections — when only some runs of an episode are selected, parent walking yields multiple disconnected roots and reports them as separate episodes — and because inference cannot distinguish a genuine caller from an unrelated run that happens to share a run ID (reusable workflows) or from a replayed context.

Alternative 2: Bump the unified session file-format version for the new episode metadata

Introduce the episode object as a version 2 change to usage/aw_session.jsonl, giving a clean break for consumers. This would allow a less constrained payload shape. It was rejected because the addition is purely additive and optional: existing readers of version 1 ignore unknown fields, and a version bump would force every downstream consumer to update for a field they may not need.

Alternative 3: Synthesize the current hop's identity when writing session metadata

Rather than echoing the recorded caller's identifiers, compute a fresh hop identity at session-write time. This would guarantee a value is always present. It was rejected because it fabricates lineage that was never observed — the session record would claim a caller relationship that the runtime did not actually propagate — undermining the audit value of the file.

Consequences

Positive

  • Partial selections of a single episode now roll up as one episode, so token, AIC, and risk aggregates in gh aw logs are no longer silently split across duplicate rows.
  • Immediate-caller lineage is correct across multi-hop chains, replayed context, and reusable workflows that share a run ID, and runtime context and OTEL spans agree because they share one resolution path.
  • Unified agent sessions now carry episode metadata, so a session file alone is enough to correlate a run back to its episode without re-reading workflow logs.
  • Conflicting identities, unrelated inferred parents, and self-edges are explicitly rejected and logged, making bad lineage data visible instead of silently merged.

Negative

  • Episode identity is now carried in the log cache and JSON output, so cached entries written before this change lack the canonical fields and fall back to the older inference behaviour until re-fetched.
  • The episode code path grows a second file and an accumulator abstraction (logs_episode_rollup.go), adding indirection a reader must follow to understand how a rollup is produced.
  • The unified session schema, TypeScript declarations, renderer, and specification must now be kept in sync for any future episode-metadata change — four surfaces instead of one.
  • Legacy call IDs alone will not produce canonical episode metadata, so older runs remain on the inference path and will not benefit from the stricter grouping.

Neutral

  • CLI rollups stay run-level and cover only the selected runs; this change does not introduce per-hop graph nodes or full-lifetime episode totals.
  • The session file-format version remains 1 and unrelated caller context is excluded, so no consumer migration is required.
  • No additional GitHub API requests are introduced; all new data is derived from already-fetched logs and already-propagated runtime context.

ADR created by [adr-writer agent]. Review and finalize before changing status from Draft to Accepted.

📋 What to do next
  1. Review the draft ADR above — it was generated from the PR diff and body
  2. Complete the missing sections — add context the AI could not infer, refine the decision rationale, and list the real alternatives you considered
  3. Commit the finalized ADR to docs/adr/67518-preserve-canonical-episode-identity-and-caller-lineage.md on your branch
  4. Reference the ADR in this PR body:

    ADR: ADR-67518: Preserve Canonical Episode Identity and Immediate Caller Lineage

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · opus50 · 64.3 AIC · ⌖ 41.9 AIC · ⊞ 1.8K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design and /tdd to the episode-lineage core (pkg/cli/logs_episode.go, logs_episode_rollup.go) and the JS lineage projection (aw_context.cjs, unified_session_payload.cjs). The refactor into episodeAccumulator methods and the canonical-episode union-find logic are clean and well-covered; test coverage for conflicting lineage, missing hops, and shared-run self-edges is strong.

📋 Key Themes & Highlights

Key Themes

  • Inconsistent trimming in classifyEpisode's standalone detection vs. the trimmed canonicalEpisodeID helper used two lines above it — flagged inline.
  • Four pre-existing Copilot review comments (trailing-space rendering bug, duration-fallback early return, unreachable-looking for loop, missing field comments) remain valid and unaddressed; recommend resolving those alongside this round.

Positive Highlights

  • ✅ episodeAccumulator extraction into logs_episode_rollup.go cleanly separates aggregation concerns per /codebase-design.
  • ✅ Canonical-identity union-find (groupCanonicalEpisodes/buildEpisodeEdges) is well tested against conflicting lineage, reversed/partial selections, and shared-run self-edges (TestBuildEpisodeDataPreservesCanonicalIdentityAcrossSelections, TestBuildEpisodeDataRejectsConflictingCanonicalLineage).
  • ✅ resolveParentHopId consolidation in aw_context.cjs/send_otlp_span.cjs removes duplicated parent-resolution logic and is backed by table-driven tests.

Note: JS test suite (vitest) could not be run locally in this environment due to a registry/cert restriction; findings are based on static review of the diff and surrounding code.

@copilot please address the review comment above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 325.7 AIC · ⌖ 14.5 AIC · ⊞ 10.3K
Comment /matt to run again

Comment thread pkg/cli/logs_episode.go
kind := "dispatch_workflow"
if run.Event == "workflow_call" {
kind = "workflow_call"
} else if run.AwContext.RunID == strconv.FormatInt(run.RunID, 10) && run.AwContext.ParentHopID == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] classifyEpisode's standalone detection compares run.AwContext.ParentHopID and run.AwContext.RunID without trimming, while canonicalEpisodeID (used just above) trims the same context via strings.TrimSpace. A whitespace-only parent_hop_id (e.g. " ") would be treated as non-empty here, misclassifying a root hop as dispatch_workflow instead of standalone.

💡 Suggested fix

Use strings.TrimSpace(run.AwContext.ParentHopID) == "" (and trim RunID similarly) so whitespace-only lineage fields can't flip the classification, matching the trimming already applied by canonicalEpisodeID.

@copilot please address this.

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.91.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants