Week 4 Day 9: Bound Tool Evidence

README.md

@@ -37,12 +37,13 @@ The course follows a four-week learning path:
   the scheduler does not rebuild dense history on every step.
 - **Week 4: Build a Coding Agent.** Start with a bounded, validated agent loop,
   then connect it to a small workspace. The course is publishing one reviewed
-  checkpoint at a time; Days 1 through 8 now cover inspection, approved edits,
+  checkpoint at a time; Days 1 through 9 now cover inspection, approved edits,
   one validation command, simple effect receipts, and one visible
   checkpoint-and-resume boundary, receipt-backed context compaction, and one
   visible inspect-and-steer pause, and deterministic evaluation of observable
   outcomes, then tokenizer/KV-prefix reuse for two isolated steered branches
-  and one explicit evidence-backed selection.
+  and one explicit evidence-backed selection, followed by bounded,
+  range-retrievable evidence for oversized tool results.
 
 ## Why MLX and Qwen3?
 
@@ -75,7 +76,7 @@ implementation, test, and publication readiness is tracked below.
 
 ## Roadmap
 
-The table tracks implementation (`Code`), tests (`Test`), rendered chapters (`Doc`), and Chi's review of learner-facing material (`Audit`). Week 4 is publishing one reviewed day at a time; Days 1 through 8 are currently available to learners. The Audit column reflects Chi's personal editorial pass on the published course content and is independent of code/test/doc readiness.
+The table tracks implementation (`Code`), tests (`Test`), rendered chapters (`Doc`), and Chi's review of learner-facing material (`Audit`). Week 4 is publishing one reviewed day at a time; Days 1 through 9 are currently available to learners. The Audit column reflects Chi's personal editorial pass on the published course content and is independent of code/test/doc readiness.
 
 Day 3 can send file contents to the model, modify files after approval, and run
 one exact configured command. Use a disposable workspace without secrets and
@@ -93,6 +94,9 @@ Day 8 reuses one real tokenizer/KV checkpoint for two differently steered,
 effect-isolated continuations, evaluates both with Day 7's harness, and makes
 one explicit passing selection without pretending completed effects were
 rewound.
+Day 9 stores exact oversized tool-result bytes outside the model prompt, shows
+a bounded identity/digest/head-tail observation, and lets the model retrieve
+one explicit byte range through the existing loop.
 
 | Week + Chapter | Topic | Code | Test | Doc | Audit |
 |---|---|---|---|---|---|
@@ -125,6 +129,7 @@ rewound.
 | 4.6 | Inspect and Steer a Paused Agent | ✅ | ✅ | ✅ | 🚧 |
 | 4.7 | Evaluate Observable Outcomes | ✅ | ✅ | ✅ | 🚧 |
 | 4.8 | Fork, Steer, and Select | ✅ | ✅ | ✅ | 🚧 |
+| 4.9 | Bound Tool Evidence | ✅ | ✅ | ✅ | 🚧 |
 
 Other topics not covered include quantized or compressed KV caches,
 cross-request prefix caching, fine-tuning, and long-context techniques.

book/src/SUMMARY.md

@@ -39,6 +39,7 @@
     - [🚧 Day 6: Inspect and Steer a Paused Agent](./week4-06-steering.md)
     - [🚧 Day 7: Evaluate Observable Outcomes](./week4-07-evaluation.md)
     - [🚧 Day 8: Fork, Steer, and Select](./week4-08-fork-steer-select.md)
+    - [🚧 Day 9: Bound Tool Evidence](./week4-09-bound-tool-evidence.md)
 - [🚧 Appendix: Performance Evidence Ledger](./appendix-performance.md)
 - [Sponsored by Raft.build](./sponsor.md)
 

book/src/week4-08-fork-steer-select.md

@@ -199,4 +199,8 @@ concurrently, share a mutable workspace, merge receipts, or provide a session
 server/tree. Day 8 teaches the boundary visibly before adding any serving-scale
 machinery.
 
+Continue with [Day 9: Bound Tool Evidence](week4-09-bound-tool-evidence.md) to
+keep oversized results verifiable without placing their complete bytes in each
+later model prompt.
+
 {{#include copyright.md}}

book/src/week4-09-bound-tool-evidence.md

@@ -0,0 +1,236 @@
+# Day 9: Bound Tool Evidence
+
+An agent can read a log that is much larger than the useful part. Appending the
+whole result to every later model prompt wastes context, but silently slicing
+it loses the evidence needed to verify what happened.
+
+Day 9 keeps those two concerns separate:
+
+- preserve the exact UTF-8 tool-result bytes outside the prompt;
+- give the model a bounded observation with identity, size, digest, and
+  head/tail previews;
+- let the model request one explicit byte range and continue in the unchanged
+  agent loop.
+
+This is byte selection, not semantic summarization. The model decides which
+range to inspect from visible facts.
+
+## Files You Implement
+
+| File | Public names | Responsibility |
+| --- | --- | --- |
+| `src/tiny_llm/agent/evidence.py` | `ArtifactRef`, `ArtifactStore`, `BoundedEvidenceWorkspace` | Store exact results, render bounded observations, and serve explicit ranges. |
+| `src/tiny_llm/agent/__init__.py` | the names above | Export the cumulative Day 9 API. |
+
+The protocol, loop, workspace, receipts, and Days 1–8 modules do not change.
+`BoundedEvidenceWorkspace` is a small adapter around the existing `Workspace`.
+
+Copy the Day 9 test into the learner workspace:
+
+```bash
+pdm run copy-test --week 4 --day 9
+pdm run test --week 4 --day 9
+```
+
+Before you implement the TODOs, the implementation-dependent cases across six
+tasks are expected to fail; the shared constructor-validation cases already pass.
+
+## Task 1: Give Exact Bytes an Identity
+
+`ArtifactStore.put(result)` encodes the complete result as UTF-8, writes those
+bytes under its explicit artifact root, and returns:
+
+```python
+ArtifactRef(
+    artifact_id="artifact-<lowercase SHA-256>",
+    byte_count=...,
+    sha256="<lowercase SHA-256>",
+)
+```
+
+The content-addressed ID and full digest deliberately repeat the same hash in
+different roles: one is the handle used by the range request; the other is a
+separately labeled model-visible verification field. The store registers the
+ID in memory. A different store cannot retrieve it merely because the caller
+guessed the filename.
+
+Before every range read, verify the stored byte count and digest again. The
+course store is local and single-process. It does not promise retention,
+garbage collection, encryption, access control, or a network blob service. It
+preserves the exact bytes returned by the wrapped tool; earlier tool-level
+limits, such as Day 3's command-output cap, still apply before this adapter.
+
+## Task 2: Replace Only Oversized Successful Results
+
+Wrap an existing workspace:
+
+```python
+from tiny_llm.agent import ArtifactStore, BoundedEvidenceWorkspace
+
+bounded = BoundedEvidenceWorkspace(
+    workspace,
+    ArtifactStore(artifact_root),
+    max_inline_bytes=512,
+    preview_bytes=64,
+    max_range_bytes=512,
+)
+```
+
+Short results and every `error:` observation remain byte-for-byte unchanged.
+For a successful result larger than `max_inline_bytes`, persist the full bytes
+and return a compact JSON observation containing:
+
+- `artifact_id`, `byte_count`, and `sha256`;
+- valid UTF-8 head and tail previews with their byte ranges;
+- the omitted half-open byte interval;
+- one exact `read_file` range-request example.
+
+The entire compact observation, including metadata and previews, must fit
+`max_inline_bytes`. Reduce previews at UTF-8 boundaries when the metadata needs
+more space. Require `max_range_bytes >= 4` so the default range can always hold
+one maximum-width UTF-8 code point. Never split a code point or silently replace
+one.
+
+## Task 3: Reuse the Existing Tool Protocol
+
+Day 9 does not add a new action schema. It reserves one virtual relative-path
+namespace for the existing `read_file` action:
+
+```text
+.tool-artifacts/<artifact-id>/bytes/<start>-<end>
+```
+
+`[start,end)` is an exact half-open byte range. The adapter intercepts the
+reserved prefix before the real workspace sees it. A successful reply names
+the same artifact, total size, digest, start, end, returned byte count, and the
+strictly decoded UTF-8 data.
+
+The reply is not sent back through externalization. Its selected data is
+already limited by `max_range_bytes`.
+
+## Task 4: Fail Closed Without Leaking
+
+Every path beginning with `.tool-artifacts/` belongs to the virtual namespace.
+Malformed paths must not fall through to a learner file of the same name.
+
+Return short ordinary `error:` observations for:
+
+- an invalid or unknown artifact ID;
+- negative, reversed, out-of-bounds, or oversized ranges;
+- stored bytes whose size or digest changed;
+- a range that cuts through a UTF-8 code point.
+
+Do not print the host artifact-root path, enumerate known IDs, or reveal bytes
+from another store while reporting an error.
+
+## Task 5: Continue Through the Same Loop
+
+The deterministic test creates a large ASCII build log whose diagnostic is
+outside both previews. A scripted model performs three normal steps:
+
+```text
+read_file build.log
+        |
+        v
+bounded identity + previews
+        |
+        v
+read_file .tool-artifacts/<id>/bytes/<start>-<end>
+        |
+        v
+exact diagnostic range -> final answer
+```
+
+`run_agent` is unchanged. Its first event contains only the bounded
+observation; the second contains only the selected range; the artifact file
+still matches the complete original result.
+
+## Task 6: Preserve the Workspace Contract
+
+Delegate `policy`, `available_tools`, and `modified_files` to the wrapped
+workspace. This lets `build_system_prompt`, action validation, and the existing
+event loop operate without knowing about the storage adapter.
+
+The virtual range path is still a normal JSON `read_file` request, so the
+learner does not need a second parser or a replacement generation interface.
+
+## Manual Cached-Qwen Walkthrough
+
+Complete the Day 9 TODOs first. Create separate disposable workspace and
+artifact directories, put a large UTF-8 `build.log` in the workspace, and use
+the same local-model adapter as the exploratory Week 4 exercise:
+
+```python
+import hashlib
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+from mlx_lm import generate as mlx_generate, load
+from tiny_llm.agent import (
+    ArtifactStore,
+    BoundedEvidenceWorkspace,
+    ToolPolicy,
+    Workspace,
+    run_agent,
+)
+
+workspace_directory = TemporaryDirectory(prefix="tiny-llm-day9-workspace-")
+artifact_directory = TemporaryDirectory(prefix="tiny-llm-day9-artifacts-")
+workspace_root = Path(workspace_directory.name)
+artifact_root = Path(artifact_directory.name)
+(workspace_root / "build.log").write_text(
+    "build started α\n"
+    + "x" * 256
+    + "\nERROR code=E42 dependency mismatch\n"
+    + "y" * 3_000,
+    encoding="utf-8",
+)
+
+mlx_model, tokenizer = load("Qwen/Qwen3-0.6B-MLX-4bit")
+artifacts = ArtifactStore(artifact_root)
+workspace = BoundedEvidenceWorkspace(
+    Workspace(ToolPolicy(workspace_root, max_file_bytes=64_000)),
+    artifacts,
+)
+
+def generate(messages):
+    prompt = tokenizer.apply_chat_template(
+        messages,
+        tokenize=False,
+        add_generation_prompt=True,
+        enable_thinking=False,
+    )
+    return mlx_generate(
+        mlx_model, tokenizer, prompt, max_tokens=256, verbose=False
+    )
+
+run = run_agent(
+    "Read build.log. If it is externalized, retrieve one useful byte range.",
+    generate,
+    workspace,
+)
+
+for event in run.events:
+    print(event.result)
+print(run.final)
+for artifact_path in artifact_root.iterdir():
+    data = artifact_path.read_bytes()
+    print(artifact_path.name, len(data), hashlib.sha256(data).hexdigest())
+```
+
+Model choices vary. Inspect the actual first observation, requested artifact
+ID and range, returned bytes, final answer, and on-disk artifact digest. Do not
+use a workspace or artifact root containing secrets. After inspection, call
+`workspace_directory.cleanup()` and `artifact_directory.cleanup()`.
+
+## Checkpoint
+
+You can now keep a complete large tool result available for verification while
+placing only bounded facts in the model context. The model can retrieve an
+explicit range by identity and continue through the same tokenizer and agent
+loop.
+
+Day 9 does not summarize the result, stream concurrent chunks, retain artifacts
+for production, or add a network service.
+
+{{#include copyright.md}}

book/src/week4-overview.md

@@ -1,7 +1,7 @@
 # 🚧 Week 4: Build a Coding Agent
 
 > **Course status:** Week 4 is being published one checkpoint at a time. Days 1
-> through 8 are ready to learn and review. Additional capabilities will appear
+> through 9 are ready to learn and review. Additional capabilities will appear
 > only after their implementation, starter, and reviews are ready.
 
 Weeks 1 through 3 turn tokens into text and make serving that text efficient.
@@ -21,6 +21,9 @@ transcript shape.
 Day 8 reconnects the agent to the inference system from Weeks 1–3: it saves one
 real tokenizer/KV prefix, forks two isolated steered continuations without
 rewinding completed effects, evaluates both, and makes one explicit selection.
+Day 9 keeps oversized tool-result bytes in a local artifact store while the
+model sees a bounded identity, digest, preview, and explicit range-retrieval
+path through the unchanged agent loop.
 
 ## What Day 1 Builds
 
@@ -125,8 +128,15 @@ Select](week4-08-fork-steer-select.md). Its cumulative command is:
 pdm run test --week 4 --day 8
 ```
 
-Only the Day 1 through Day 8 starter modules are published. Do not add session
-trees, effect rewind, reconciliation, an LLM judge, radix serving, or other
-later public APIs to your solution.
+After Day 8 passes, continue with [Day 9: Bound Tool
+Evidence](week4-09-bound-tool-evidence.md). Its cumulative command is:
+
+```bash
+pdm run test --week 4 --day 9
+```
+
+Only the Day 1 through Day 9 starter modules are published. Do not add session
+trees, effect rewind, reconciliation, an LLM judge, semantic summarization,
+radix serving, or other later public APIs to your solution.
 
 {{#include copyright.md}}

docs/week4-day-split.md

@@ -1,9 +1,9 @@
 # Week 4 Day Split (reference for reviewers)
 
-Status: Days 1--8 are published checkpoints. Each day ships as one cumulative
+Status: Days 1--9 are published checkpoints. Each day ships as one cumulative
 learner PR so reviewers can see exactly what belongs to that checkpoint.
 
-## 8-day structure
+## 9-day structure
 
 | Day | Theme | Features (PRs) | Modules |
 |---|---|---|---|
@@ -15,6 +15,7 @@ learner PR so reviewers can see exactly what belongs to that checkpoint.
 | 6 | Inspect and steer | safe-boundary status and one visible steering message | `steering.py` |
 | 7 | Evaluate outcomes | declared final/file/result/receipt facts | `evaluation.py` |
 | 8 | Fork, steer, and select | dense tokenizer/KV prefix reuse, isolated branches, explicit selection | `branching.py`, `workspace.py` |
+| 9 | Bound tool evidence | exact external bytes, bounded observation, explicit range retrieval | `evidence.py` |
 
 Extension (not a day): COW/radix cache — `docs/week4-cow-radix-extension-plan.md`.
 
@@ -26,10 +27,11 @@ Extension (not a day): COW/radix cache — `docs/week4-cow-radix-extension-plan.
 - Days are implemented sequentially. A later day does not leak API or prose
   into the current starter.
 
-## Why 8 days
+## Why 9 days
 
 The first seven days establish the agent loop and its observable evidence. Day
 8 reconnects that control path to the tokenizer and KV cache built in Weeks
-1--3. Each day adds one visible concept; scaling and production-hardening
-machinery stay outside the core course unless a later checkpoint explicitly
-teaches it.
+1--3. Day 9 keeps large observable evidence available without filling every
+later model prompt. Each day adds one visible concept; scaling and
+production-hardening machinery stay outside the core course unless a later
+checkpoint explicitly teaches it.

src/tiny_llm/agent/__init__.py

@@ -18,6 +18,7 @@
     ResultExpectation,
     evaluate_run,
 )
+from .evidence import ArtifactRef, ArtifactStore, BoundedEvidenceWorkspace
 from .generation import generate_response, initial_messages
 from .loop import (
     AgentEvent,
@@ -47,6 +48,9 @@
     "AgentRun",
     "AgentStatus",
     "ApprovalDecision",
+    "ArtifactRef",
+    "ArtifactStore",
+    "BoundedEvidenceWorkspace",
     "BranchOutcome",
     "CompactionResult",
     "EvaluationCase",