Include LLM-generated domain rationale in distilled knowledge drafts

@Hareet

Background

Suggested by @Hareet in #109 (review comment, Jun 17 2026), based on output seen when running the pipeline against cht-core PR #10804.

During distillation the LLM already produces a domain placement rationale (fit strength + explanation of why the PR belongs to a domain). This reasoning is currently discarded — only the structured frontmatter and body reach _pending/. Preserving it would help with:

Deterministic re-organisation of domains in the future
Auditing why a PR was placed in a given domain
Surfacing sub-issue groupings without re-running the LLM

Example from PR #10804 (contacts domain):

## Domain Rationale

**Fit:** strong

The change operates entirely on the contact profile page, the contact view-model
generator, and the child-contact hierarchy display — squarely contact lookup and
presentation. No sync, permission, or config concerns are involved.

Goal

Persist the LLM's domain rationale in the draft frontmatter or body so it is available to the reviewer and to downstream tooling.

Acceptance criteria

Distiller extracts the domain rationale from the LLM response
Rationale (fit strength + explanation) is written into the draft — either as a frontmatter field or a dedicated ## Domain Rationale section
Schema updated to reflect the new field (optional, non-breaking)
Existing tests updated; new test covers rationale extraction

References

feat(#108): memory distillation pipeline — scraper, filter, distiller, CLI, and CI workflow #109 — memory pipeline implementation (distiller.ts)
Memory scaffolding: domain-agnostic pipeline and wiki conventions #108 — parent epic

Merge pull request #109 from medic/feat/108-memory-pipeline-scaffolding

Closing as delivered — this shipped in #119 (feat(#108): seeding pipeline - CLI provider, domain-rationale, infrastructure domain, concurrency).

Against the acceptance criteria:

Distiller extracts the rationale — distiller.ts produces domainReasoning from the LLM response.
Written into the draft — both forms, actually: a domainFit frontmatter field and a dedicated ## Domain Rationale body section, in exactly the **Fit:** strong + explanation shape sketched above.
Schema updated — domainFit is in agent-memory/schema.json (optional, non-breaking).
Tests — test/scripts/distiller.spec.ts covers the rationale/fit fields.

One follow-up worth recording: the promote-PR reviews (#120-123, #130-132) surfaced the flip side of this feature — a handful of drafts' ## Domain Rationale leaks the classifier's own scaffolding into stored memory ("Per the classification seeds", "…so the configuration pitfall does not apply", "per seed example #5"), which is prompt-internal reasoning rather than durable knowledge. That cleanup is tracked under #136/#138, not here.

Navigation Menu

Repository navigation

Include LLM-generated domain rationale in distilled knowledge drafts #125

Background

Goal

Acceptance criteria

References

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Include LLM-generated domain rationale in distilled knowledge drafts #125

Description

Background

Goal

Acceptance criteria

References

Activity

sugat009 commented on Jul 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions