feat(mcp): TOON compact tool output — ~10x fewer response tokens across the query surface#1013
Merged
Conversation
… tokens
Tool responses land in an LLM agent's context window, where the legacy
per-node JSON objects were dominated by data no agent can use: the
similarity/semantic pipeline intermediates fp (~450B minhash hex), bt
(body-token bag) and sp (structural profile) plus 14 complexity metrics
were grafted onto every result by enrich_node_properties — ~64% of every
node emission, ~370 tokens per search hit.
search_graph now defaults to TOON (Token-Oriented Object Notation,
toonformat.dev): scalars as 'key: value' lines and results as a
'results[N]{qn,label,file,lines,in,out}:' header + one row per hit.
Published benchmarks show equal-or-better LLM retrieval accuracy at
40-60% fewer tokens than JSON; rows now also carry line ranges, which
the legacy output lacked.
- fields:[...] opts into per-node property columns (complexity,
signature, docstring, ...); fp/sp/bt are blocklisted even there
- format:"json" restores the legacy verbose objects byte-identically;
include_connected forces it (nested neighbor lists)
- BM25 and semantic modes emit the same table shape (rank/score column)
- semantic-only calls no longer run the unfiltered regex search that
silently prepended up to 200 enriched nodes (102.8KB -> 4.2KB)
- default limit 200 -> 50 (BM25 100 -> 50): cheap first page, page via
has_more/offset as before
- new emitter module src/mcp/compact_out.{c,h} (string builder + TOON
scalar/table emission with spec quoting rules)
Measured on a real index (14-hit regex search): 20,793B -> 1,327B
(-94%); BM25 19,667B -> 5,081B; semantic 102,837B -> 4,181B. A typical
discovery session drops ~50.9K -> ~4.7K tokens combined with the
follow-up slices.
Guards: tool_search_graph_toon_never_leaks_internal_fields (sentinel
fp/sp/bt never emitted, even when requested), rewritten
tool_search_graph_includes_node_properties pins default-lean/fields-
opt-in/json-escape-hatch; suite 5983 passed, smoke green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…text-bomb defaults
Completes the compact-output pass across the remaining query tools
(consumers are LLM agents; every response byte is context tokens):
- trace_path: callees[N]{qn,hop}/callers[N]{qn,hop} tables (risk/test/
args columns per flags); the per-hop JSON key envelope was 84% of the
payload. 3,589B -> 1,634B on a representative trace.
- query_graph: rows[N]{cols}: table (columns double as the header);
format:"json" restores columns/rows arrays.
- search_code compact mode: results/raw/dirs tables; drops the
field (duplicate of the qn tail). 2,488B -> 1,135B for 17 hits.
- search_code full mode: per-hit source capped at a 60-line window
anchored on the first match, with source_start/source_truncated
markers — uncapped whole-symbol dumps ran to 142KB per response; the
complete symbol stays one get_code_snippet call away.
- get_architecture: default (no aspects) is now a summary (languages,
packages, entry_points + always-on totals/label/type counts) with an
aspects_hint — the old default rendered EVERYTHING including the full
file_tree (94KB -> 4.5KB -> 2.8KB with TOON tables); all sections
emit as TOON tables, aspects:[...] and ["all"] keep full access
(52KB vs 94KB even for the explicit full dump).
- get_code_snippet: drops the property-blob enrichment (41% of the
response; signature/docstring are literally in the source; fp/sp/bt
never belonged there). Metrics remain reachable via search_graph
fields=[...].
- list_projects: per-project 12-field git block (mostly null) replaced
by the branch name; name/root/nodes/edges/size kept.
- search_graph description: in/out documented as TOTAL all-edge degree,
NOT caller/callee counts (agent evals mis-read it as callers).
- format:"json" escape hatch on every converted tool.
Guards: tool_output_byte_budgets pins absolute byte ceilings on default
search/trace outputs (a property blob sneaking back into row emission
blows them immediately); snippet_enriched_properties inverted to pin
the no-spill contract. Suite 5,984 green; smoke updated to the TOON
contract end-to-end.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ph calls query_graph now defaults to TOON text; the guard's definition-count probe parsed it as JSON, swallowed the parse failure, and reported the ASCII baseline as having zero definitions (SETUP FAIL on the test-windows-guards leg). Scripted consumers use the format="json" escape hatch. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…call
search_graph now defaults to TOON text, but ha_format_context parses
the inner payload as JSON ("results"), so the Grep/Glob/Read hook
augmenter silently produced no hookSpecificOutput — caught by the
test_hook_augment.py Windows guard going red. Both internal call sites
route through ha_build_args; request the JSON encoding there.
Verified end-to-end: indexed fixture + PreToolUse payload via stdin
emits the symbol context again.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tool responses are consumed by LLM agents, where every byte is context tokens. This lands the compact-output pass designed in-session: TOON (header-once + rows) as the default encoding for homogeneous result sets, context-bomb defaults fixed, and internal fields permanently blocked — with
format:"json"escape hatches everywhere and zero capability loss (fields/aspects/get_code_snippetrestore anything on request).Measured (real index, identical calls)
fullsource_truncated)["all"]A typical discovery session (1 arch + 4 searches + 2 traces + 3 snippets + 2 search_code) drops ~51K → ~4.7K tokens. TOON text is not valid JSON, so the duplicate
structuredContentcopy disappears at the protocol layer too.Key changes
fp/sp/bt) never emitted — they were 45% of every node payload (450 B minhash hex per hit) with zero agent value; blocklisted even when explicitly requested viafields.fields:["complexity","signature",...]) instead of an unconditional 28-field dump.aspects_hint, semantic-only calls no longer run the unfiltered base search (that alone was 102 KB).src/mcp/compact_out.{c,h}.Quality gates
tool_search_graph_toon_never_leaks_internal_fields(sentinel fp/sp/bt can never appear, even when requested),tool_output_byte_budgets(absolute byte ceilings on default outputs — re-bloat goes red immediately),snippet_enriched_propertiesinverted to pin the no-spill contract.Not armed for auto-merge — worth a look before it lands given the surface area. Follow-ups tracked separately: #997 (CLI strict flags, PR #1012), #998 (semantic determinism), #999 (junk infra routes), #1000 (registry regression, PR #1011).