Skip to content

The DSH preamble is now the biggest line in every planner call (~13,400 tok, 36%) #232

Description

@hugolytics

Where a planner call's tokens go, after today's fixes

One call, decomposed from a real harness log and from the MCP servers directly.

part tokens share how known
DSH system preamble + residual core tools ~13,400 36% by difference
MCP tool schemas (12 tools, 4 servers) 6,034 16% measured off the servers
our brief (after 061eca2, 8787cc7) 4,768 13% measured
persona / agent instructions 4,144 11% measured
accumulated tool results, skills, plugins ~remainder

The whole thing is re-sent on every tool round-trip — nine of them in a session — so every token
here is paid nine times.

Why this is now the target

Two fixes landed today inside our own brief:

  • 061eca2 — the same 40 constraints were sent twice, as applicable_constraints and again as
    an active_constraints fact. Identical uid sets, identical bytes. The facts exist to satisfy
    readiness requirements and satisfied_by is a presence test, so nothing ever read their value.
  • 8787cc7 — a third of the constraint block was [], {} and null.

Together the brief went 10,914 → 4,768 tokens, 56%. Which means the comparison has flipped:
the harness preamble is now nearly three times our whole brief, and it is the one part
FateForger does not control.

The tool schemas are not the problem

Measured, so this can stop being guessed at:

tmbx              5 tools  2,791   (plan_apply alone is 1,117)
planning_result   1 tool   1,227
memory            4 tools  1,459
progress          2 tools    557
                          ------
                           6,034

Twelve plugins are already disabled on planning turns via FF_PLANNING_TURN, and that work is
done. Trimming further here buys at most a few hundred tokens and costs capability.

What to do

  1. Size the ~13,400. It is preamble plus residual core tools together and nothing separates
    them; the DSH session log does not record the system block. This needs the harness, not us —
    most likely a debug flag that dumps the assembled system prompt.
  2. Then decide whether a planning turn needs the full persona. A turn that is handed an
    authoritative brief and must answer with one typed tool call may not need the same preamble as
    an interactive /dsh session. That is the same argument as FF_PLANNING_TURN, applied one
    level up.

Also worth knowing: the cache miss

call  fresh in  cache read    prompt
   2    31,035           0    31,035   <- 31k at the full rate
   3     1,748      30,976    32,724

Fresh is 10× cached, so one miss costs more than three hits. Misses cluster at turn start, where
the session-specific brief makes a prefix no previous session sent — which is the second reason
to shrink the brief rather than only shorten the loop.

Blocked on measurement: the OpenRouter key hit its monthly cap mid-investigation (403, "Key limit
exceeded"), so the post-fix figures above are arithmetic on a pre-fix log, not fresh observations.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions