Week 4 extension: COW/radix cache plan (design only)

docs/week4-cow-radix-extension-plan.md

@@ -0,0 +1,88 @@
+# COW / Radix Cache Extension Plan (design only)
+
+Status: **design only — not implemented.** This document preserves the deferred
+copy-on-write / radix-cache plan so it can be built as a Week 5 module or a
+Week 4 extension after the core Week 4 refsol is reviewed. It intentionally
+implements nothing; the core Week 4 stack uses sequential rewind/do-again
+(Day 4) and covers every learner scenario without concurrent forks.
+
+## Why this is deferred
+
+The core Week 4 thesis is that four planes must agree: durable events/evidence,
+model-visible context, derived KV state, and world/approval authority. The
+sequential model (checkpoint -> act -> rewind -> do-again) teaches the derived
+KV-state plane honestly: reuse the unchanged prefix, recompute only the
+divergent suffix. Concurrent COW forks and cross-session prefix sharing are a
+*scaling* mechanism, not a correctness one — they answer "how does a production
+server serve many sessions/subagents cheaply?" and fit the Week-5-style
+optimization arc (like quantization, flash attention, and paged attention did
+for earlier weeks).
+
+## What COW would add (capabilities the sequential model cannot provide)
+
+1. **Concurrent divergence** — two live branches sharing immutable KV pages
+   while both continue decoding (parallel what-if exploration, concurrent
+   subagents). Sequential rewind covers "try A, reverse, try B"; COW covers
+   "run A and B at the same time."
+2. **Cross-session prefix sharing** — a radix/prefix registry that shares one
+   physical copy of a common prompt prefix across many sessions, with
+   refcounted immutable pages and copy-on-write tails.
+
+## Design sketch (for a future implementer)
+
+### Page model
+
+- Immutable full pages, content-addressed by exact token block.
+- Per-page reference counts; a page is freed only when its last reference
+  releases it.
+- Copy-on-write partial tail: divergence allocates a private page; the shared
+  prefix pages are never mutated.
+
+### Registry API (proposed)
+
+```text
+PrefixRegistry(model_hash, tokenizer_hash, block_size, page_budget)
+  publish(token_ids) -> page_ids            # idempotent, content-addressed
+  acquire(token_ids) -> ForkedPrefix        # longest published block-aligned prefix
+  release(page_ids)                         # refcounted, fails on double-free
+  stats() -> PrefixStats                    # live/shared/private/refs/budget
+ForkedPrefix
+  fork(boundary) -> ForkedPrefix            # block-aligned COW child
+  append(token_ids) -> ForkedPrefix         # private divergent tail
+  close()                                   # releases shared refs exactly once
+```
+
+### Safety invariants
+
+- A cache hit requires exact model, tokenizer, and token identity; registry is
+  model-scoped and fails closed on identity mismatch.
+- Pages are immutable after publish; closing a child can never alter another
+  fork's cache.
+- Reference counts cannot leak or double-free; a memory budget evicts only
+  unreferenced pages.
+- Child policies can only narrow parent authority; private child material is
+  never placed in a shared prefix.
+- Branch summaries stay event-tree-level (Day 3) and need no COW.
+
+### Honest accounting
+
+Report cold vs optimized wall time separately, exact reused/rewound/prefilled/
+generated/discarded token counts, metadata-copy and KV-page-copy bytes,
+live/shared/private page counts and peak KV bytes, and branch-discard costs.
+Never call reduced visible bytes or more cache hits a speedup unless end-to-end
+latency improves without changing the accepted action or losing evidence.
+
+### Evaluation
+
+A read-only fake workspace forks two action plans from one prefix, scores a
+deterministic expected result, discards one, and proves no external mutation.
+Negative tests attempt a write, change the workspace version, and exhaust the
+page budget. Real-model runs compare aggregate time-to-first-token and peak KV
+bytes after a warm-up publisher.
+
+## Source influence
+
+The design direction is informed by the Week 4 research lanes (Oracle #196,
+Tuner #197) and by production radix-cache systems; the core Week 4 stack keeps
+the four-plane thesis without this mechanism. No COW behavior is implied in any
+core Week 4 feature.

docs/week4-day-split.md

@@ -1,29 +1,63 @@
 # Week 4 Day Split (reference for reviewers)
 
-Status: decided by Forge per Chi's instruction ("7/10/x days your call"); the
-refsol stack implements features one PR per day so reviewers can see exactly
-what belongs to each day. Curriculum prose (book) is not part of this stack.
+Status: decided by Forge per Chi's instruction ("7/10/x days your call"). The
+refsol stack is feature-based (one PR per feature so reviewers see exactly
+what belongs to each review boundary); the 7-day split groups adjacent
+features into teaching days. Curriculum prose (book) is not part of this
+stack.
 
 ## 7-day structure
 
 | Day | Theme | Features (PRs) | Modules |
 |---|---|---|---|
-| 1 | Validated agent loop + tool protocol | feat1 | `protocol.py`, `loop.py` |
-| 2 | Effect receipts (durable evidence) | feat2 | `receipts.py`, `workspace.py` wiring |
+| 1 | Validated agent loop + tool protocol | feat1 | `protocol.py`, `loop.py`, `generation.py` (minimal) |
+| 2 | Authorize effects + durable receipts | feat2 | `workspace.py` (simplified), `receipts.py` |
 | 3 | Session tree (event-level id/parentId) | feat3 | `session.py` |
-| 4 | KV checkpoint/resume + sequential rewind | feat4+5 | `checkpoint.py`, `branch.py`, `generation.py` |
-| 5 | Receipt-backed compaction | feat6 | `compaction.py`, `context.py` |
-| 6 | Steering + public status + exactly-once reconcile | feat7+8 | `control.py`, `status.py`, `reconcile.py` |
-| 7 | Equivalence harness | feat9 | `harness.py`, `evaluation.py` |
+| 4 | Derived KV state: checkpoint/resume + sequential rewind | feat4 + feat5 (two PRs) | `checkpoint.py`, `branch.py`, `generation.py` |
+| 5 | Receipt-backed compaction | feat6 | `compaction.py` |
+| 6 | Control/inspect/recover: steering + status, then exactly-once reconcile | feat7 + feat8 (two PRs) | `control.py`, `status.py`, `reconcile.py` |
+| 7 | Equivalence harness | feat9 | `harness.py` |
 
 Extension (not a day): COW/radix cache — `docs/week4-cow-radix-extension-plan.md`.
 
-## Stack shape
+The old static-held-out grader (`evaluation.py` with `TaskPackage`/`StagedTask`)
+is not part of the new course: Day 7's `harness.py` measures the integrated
+system (warm/cold, fork/cold, compact/full, crash/resume equivalence) over
+the three planes, which replaces the old standalone grader as the evaluation
+story.
 
-- PR 1: remove the old 7-day refsol; create the starter skeleton mapped to the
-  new refsol (this is the map reviewers read).
-- PRs 2-8: implement each day's feature(s) in the refsol + focused tests.
-- PR 9: COW/radix plan (design only).
+## Design note: simplified workspace (Day 2)
+
+The old 7-day workspace carried a write-ahead mutation journal and undo
+machinery (old Day 6 content). The new design drops that machinery: the
+workspace keeps the authorization core (bounds, protected paths, observed
+digests, approvals, atomic writes) plus effect receipts; crash/effect
+recovery is taught by Day 6's exactly-once reconcile instead. This keeps each
+day's surface small and matches the "start simple, extend" arc.
+
+The old summarizer-based `ContextManager` (whole-history compaction with a
+model summary) is not part of the new course: Day 5's receipt-backed
+compaction replaces it, keeping the durable trace untouched and re-expanding
+verified ranges on demand. This removes a large control-coupled module and
+keeps Day 5 self-contained.
+
+## Stack shape (11 PRs)
+
+1. reset: remove the old 7-day refsol; create the starter skeleton mapped to
+   the new refsol (this is the map reviewers read).
+2. loop + tool protocol (Day 1)
+3. effect receipts (Day 2)
+4. session tree (Day 3)
+5. KV checkpoint (Day 4a)
+6. sequential rewind (Day 4b)
+7. receipt-backed compaction (Day 5)
+8. steering/status (Day 6a)
+9. exactly-once reconcile (Day 6b)
+10. equivalence harness (Day 7)
+11. COW/radix plan (design only, non-day extension)
+
+Each feature PR is independently reviewable; the day mapping above shows how
+adjacent features group into teaching days.
 
 ## Why 7 days
 

src/tiny_llm_ref/agent/__init__.py

@@ -1,6 +1,34 @@
 # WARNING: Under review - generated by LLM.
 
-from .generation import GenerationStats, generate_response, initial_messages
+from .branch import BranchStats, RewindError, SequentialBranch
+from .checkpoint import (
+    CacheManifest,
+    ManifestError,
+    export_cache_manifest,
+    validate_resume,
+)
+from .compaction import (
+    CompactionError,
+    CompactionResult,
+    compact_tool_results,
+    expand_receipt_range,
+    reexpand_receipt_message,
+)
+from .control import AgentInterrupted, CancellationToken, SteeringHandle
+from .harness import (
+    EquivalenceReport,
+    PlaneResult,
+    RunSnapshot,
+    compare_runs,
+    snapshot_run,
+)
+from .generation import (
+    GenerationSession,
+    GenerationStats,
+    generate_response,
+    initial_messages,
+)
+from .session import SessionEvent, SessionLog, SessionStore, memory_session
 from .loop import AgentEvent, AgentLimits, AgentRun, run_agent
 from .protocol import (
     AgentError,
@@ -11,21 +39,70 @@
     parse_action,
     tool_catalog_hash,
 )
+from .receipts import EffectReceipt, ReceiptStore
+from .reconcile import (
+    ReconciliationResult,
+    SafeCheckpoint,
+    largest_safe_checkpoint,
+    reconcile_effect,
+    reconcile_interrupted_effects,
+)
+from .status import AgentStateCard, StatusQuery, StatusQueryResult, build_state_card
+from .workspace import ToolPolicy, Workspace
 
 
 __all__ = [
     "AgentError",
     "AgentEvent",
     "AgentLimits",
     "AgentRun",
+    "AgentStateCard",
+    "AgentInterrupted",
+    "CancellationToken",
+    "BranchStats",
+    "EffectReceipt",
+    "EquivalenceReport",
+    "CacheManifest",
+    "CompactionError",
+    "CompactionResult",
     "FinalAction",
+    "GenerationSession",
     "GenerationStats",
+    "ManifestError",
+    "PlaneResult",
+    "ReceiptStore",
+    "ReconciliationResult",
+    "RewindError",
+    "RunSnapshot",
+    "SafeCheckpoint",
+    "SequentialBranch",
+    "StatusQuery",
+    "StatusQueryResult",
+    "SteeringHandle",
+    "StatusQueryResult",
+    "SessionEvent",
+    "SessionLog",
+    "SessionStore",
     "TOOL_CATALOG_HASH",
     "ToolAction",
+    "ToolPolicy",
+    "Workspace",
+    "build_state_card",
+    "compare_runs",
     "build_system_prompt",
     "generate_response",
+    "export_cache_manifest",
+    "compact_tool_results",
+    "expand_receipt_range",
     "initial_messages",
+    "largest_safe_checkpoint",
+    "memory_session",
     "parse_action",
+    "reconcile_effect",
+    "reconcile_interrupted_effects",
+    "reexpand_receipt_message",
     "run_agent",
+    "snapshot_run",
+    "validate_resume",
     "tool_catalog_hash",
 ]