Skip to content

SPEC.md

Latest commit

 

History

History
488 lines (378 loc) · 21.4 KB

SPEC.md

File metadata and controls

488 lines (378 loc) · 21.4 KB
Raw

PCSF — Portable Cognitive State File

Specification version: v1.2 Status: Stable draft Category: File format + prompt protocol (not a service, not a library dependency)

v1.1 is a strict, backward-compatible superset of v1. It adds three robustness features and no breaking changes: an OPTIONAL self-verifying integrity line (§2.10a), deterministic graceful degradation (§5), and explicit decoder conflict/staleness rules (§6).

v1.2 is a strict, backward-compatible superset of v1.1. It adds capsule stitching — a bounded merge that lets a chain of AIs each consume a capsule, continue the work, and fold their new state back into a single capsule that is still ≤1,200 characters (§7) — plus an OPTIONAL CHAIN lineage line (§2.10b) that records the chain's depth and origin. Every valid v1/v1.1 capsule is a valid v1.2 capsule; an older reader may process a v1.2 capsule by ignoring the lines it does not recognise.

0. Scope and honest framing

PCSF is a small, portable, plain-text file that encodes the cognitive state an AI session arrives at — its operative decisions, the paths it ruled out, its own unvoiced internal state, the working relationship with the user, and the unfinished work — so that a new session can resume the same trajectory and behavior rather than rebuild the transcript.

What PCSF deliberately does not claim:

  • It does not reconstruct conversation content. Recovering an arbitrary N-token transcript from a ~1KB file is impossible (information theory); PCSF does not attempt it.
  • "Behavioral fidelity" is not a guaranteed percentage. It is an evaluable property (see VALIDATION.md), measured per behavior, not asserted as a number.

What PCSF does provide, and what the rest of this document defines precisely:

  • A deterministic file grammar that is machine-checkable.
  • Two prompt protocols (compression, decompression) that any capable LLM can run.
  • Validation criteria for both the artifact (the file) and the behavior (the resumed session).

The value proposition is real even with the honest framing: most of what makes a long collaboration productive is not the words but the state — and that state is small, durable, and portable. PCSF is the artifact that carries it.

1. Design principles

  1. Store only the non-derivable. A node belongs in the capsule only if a capable LLM, given the rest of the capsule plus its own training knowledge, could not reliably reconstruct it. The model's weights are the decompression dictionary; do not store what the dictionary already contains.
  2. Negative knowledge is first-class. Rejected paths are stored with the same rigor as chosen ones. This is the layer existing memory systems omit.
  3. The AI is a participant, not a logger. Internal state is written in first person, capturing what the AI itself held — not a third-party summary.
  4. Continuity over restart. The format and its behavioral contract exist to make the next session continue, never to make it re-introduce itself.
  5. Deterministic and self-contained. One file. Pure ASCII. No external references, no infrastructure, no embedded secrets.

2. File grammar (normative)

2.1 Encoding and size

  • Character set: US-ASCII only. Every byte MUST be in 0x0A (LF) or 0x20–0x7E. Tabs (0x09), CR (0x0D), and all other control bytes are forbidden.
  • Line endings: LF (\n) only. No trailing whitespace on any line.
  • Total size: MUST NOT exceed 1,200 characters (= bytes, since ASCII), measured over the entire file including the framing lines and the trailing LF.
  • A capsule that violates any MUST in this section is invalid and MUST be rejected by conforming tools.

2.2 Overall structure

A capsule is exactly this sequence of blocks, in this order, each separated by a single blank line:

===PCSF:v1===
META: <topic> | <domain> | <depth> | <timestamp>
[CHAIN: hops=<n> origin=<timestamp>]   <- OPTIONAL (v1.2); see 2.10b
[INTEGRITY: crc32=<hex> chars=<n>]      <- OPTIONAL (v1.1); see 2.10a

L1:SKELETON
<zero or more L1 entries, one per line>

L2:NEGATIVE
<zero or more L2 entries, one per line>

L3:INTERNAL
<zero or more L3 entries, one per line>

L4:RELATION
<zero or more L4 entries, one per line>

L5:LOOPS
<zero or more L5 entries, one per line>

SEED: <single sentence>
===END===

Rules common to all blocks:

  • The opening line MUST be exactly one of the recognised version tokens (===PCSF:v1===, ===PCSF:v1.1===, ===PCSF:v1.2===); the closing line exactly ===END===. The stitch tool emits ===PCSF:v1.2===.
  • When both optional header lines are present, the order is fixed: META, then CHAIN, then INTEGRITY.
  • Each layer begins with its exact header line (L1:SKELETON, L2:NEGATIVE, L3:INTERNAL, L4:RELATION, L5:LOOPS). All five headers MUST be present and in order, even if a layer has zero entries.
  • An entry occupies exactly one line. No entry may be empty or blank. No blank lines may appear inside a layer's entry list.
  • Sections are separated by exactly one blank line.

2.3 Field separator

Within an entry, the field separator is the three-character sequence space, pipe, space (" | "). Consequently the pipe character | MUST NOT appear inside a field value. (Use a dash or rephrase if you are tempted to write a pipe.)

For L3 and L4, the separator between the leading token and its content is colon, space (": "); the first occurrence delimits, so colons may appear later in the value.

2.4 META line

META: <topic> | <domain> | <depth> | <timestamp>
  • <topic>: short free text, the subject of the collaboration.
  • <domain>: short free text, the field/area (e.g. software, legal, writing).
  • <depth>: one of SURFACE, WORKING, DEEP.
    • SURFACE — a brief or shallow exchange.
    • WORKING — a substantive working session.
    • DEEP — a long, high-context collaboration with significant accumulated state.
  • <timestamp>: ISO-8601 UTC, minute precision, YYYY-MM-DDThh:mmZ (e.g. 2026-06-26T20:17Z).

2.5 Layer 1 — Causal Skeleton (L1:SKELETON)

The irreducible, non-derivable decisions, pivots, and discoveries from which a reasoning AI can rebuild the full decision landscape.

  • Entry grammar: <EVENT_TYPE> | <CORE_FACT> | <CAUSAL_REASON>
  • EVENT_TYPE ∈ { CHOICE, CONSTRAINT, DISCOVERY, PIVOT, OUTCOME }
  • CORE_FACT: what was decided / found / fixed.
  • CAUSAL_REASON: why — the causal link that makes the fact non-obvious.
  • Maximum 12 entries.
  • Inclusion test (the only test): If this node were removed, could a capable AI correctly infer it from the remaining nodes plus its training knowledge? If yes, it does not belong. Only nodes that fail this test (i.e. are non-derivable) are stored.

2.6 Layer 2 — Negative Knowledge (L2:NEGATIVE)

Every path explored and rejected, abandoned, or ruled out.

  • Entry grammar: <REJECTED_OPTION> | <REJECTION_REASON> | <DURABLE_LESSON>
  • REJECTED_OPTION: the approach that was tried/considered and dropped.
  • REJECTION_REASON: why it failed or was ruled out, specific to this context.
  • DURABLE_LESSON: the generalizable takeaway that should outlive the specific case.
  • Maximum 8 entries.

2.7 Layer 3 — AI Internal State (L3:INTERNAL)

The AI's own first-person participant state at session end.

  • Entry grammar: <STATE_TYPE>: <CONTENT>
  • STATE_TYPE ∈ { UNCERTAINTY, DIRECTION, INTUITION, UNRESOLVED }
    • UNCERTAINTY — something the AI was unsure of but did not fully voice.
    • DIRECTION — the trajectory the AI was steering toward.
    • INTUITION — a held-but-not-fully-justified hunch.
    • UNRESOLVED — a question the AI wanted to ask but did not.
  • CONTENT: first-person, honest, specific.
  • Maximum 6 entries total (across all four state types combined).

2.8 Layer 4 — Relationship State (L4:RELATION)

The current state of the collaborative relationship (not a description of the user).

  • Entry grammar: <DIMENSION>: <STATE>
  • DIMENSION: a named axis of the working relationship. Recommended dimensions (not an exhaustive enum): STYLE (communication style), TRUST (trust/autonomy calibration), VOCAB (shared terms with context-specific meaning), MODE (the user's current cognitive/emotional mode), BOUNDARY (explicit do/don't).
  • STATE: the evolved value of that dimension at session end.
  • Maximum 5 entries.

2.9 Layer 5 — Open Loops (L5:LOOPS)

Every unresolved thread — the forward momentum of the collaboration.

  • Entry grammar: <LOOP_TYPE> | <DESCRIPTION> | <PRIORITY>
  • LOOP_TYPE ∈ { PROMISE, DEFERRED_QUESTION, INCOMPLETE_TASK, HYPOTHESIS_UNTESTED }
  • DESCRIPTION: the specific unresolved thread.
  • PRIORITY ∈ { HIGH, MED, LOW }
  • Maximum 8 entries.
  • Ordering convention: entries SHOULD be sorted by priority (HIGH first). The behavioral contract resumes loops in capsule order.

2.10a Integrity line (OPTIONAL, v1.1)

META: <topic> | <domain> | <depth> | <timestamp>
INTEGRITY: crc32=<8 lowercase hex> chars=<integer>

An OPTIONAL INTEGRITY: line MAY appear, and if present MUST appear on the line immediately after META:. It makes corruption, truncation, and tampering detectable rather than silent:

  • crc32: the CRC-32 (IEEE 802.3, as in zlib.crc32) of the entire capsule text with the INTEGRITY: line removed, rendered as exactly 8 lowercase hex digits.
  • chars: the total character count of the whole file including the integrity line (and the trailing LF).

A conforming reader that finds an INTEGRITY: line MUST recompute both values and reject the capsule on mismatch (a char-count mismatch indicates truncation; a CRC mismatch indicates corruption or tampering). The line costs ~37 characters, which count against the 1,200-char budget like any other content.

2.10b CHAIN lineage line (OPTIONAL, v1.2)

META: <topic> | <domain> | <depth> | <timestamp>
CHAIN: hops=<integer> origin=<timestamp>

An OPTIONAL CHAIN: line MAY appear, and if present MUST appear on the line immediately after META: (and before any INTEGRITY: line). It records the capsule's position in a stitched chain (§7):

  • hops: a non-negative integer, the number of stitch operations this capsule has been through. A freshly compressed capsule has no CHAIN line (hops are implicitly 0); the first stitch produces hops=1, the second hops=2, etc.
  • origin: the YYYY-MM-DDThh:mmZ timestamp of the root capsule in the chain (the first session's META timestamp). It is set once and never changes, so the whole chain is timestamped from where it began.

A decoder that finds a CHAIN line SHOULD treat the capsule as accumulated prior — the cognitive state of hops successive sessions folded together, not a single session — and weight it accordingly. The line costs ~38 characters, which count against the 1,200-char budget like any other content.

2.10 SEED

SEED: <single sentence>

The single irreducible sentence. If a future AI could read only one line of the capsule, the SEED MUST be enough to orient it toward the most critical non-derivable insight of the whole collaboration.

  • Exactly one line. MUST be non-empty. MUST be a single sentence (one terminal punctuation mark at most, no internal line breaks).

2.11 Formal grammar (ABNF, normative)

The prose rules in §2.2–§2.10 are restated here as a single ABNF grammar (RFC 5234). Where the ABNF and the prose ever disagree, the ABNF is normative for structure; the enum membership, ordering conventions, per-layer entry caps, the 1,200-byte ceiling, and the INTEGRITY semantics are constraints layered on top of this grammar (a syntactically valid capsule can still be rejected for exceeding a cap or failing CRC — see the conformance table). ABNF cannot express those counting/semantic constraints, by design.

capsule      = open LF meta-line LF [chain LF] [integrity LF] LF
               layer1 LF layer2 LF layer3 LF layer4 LF layer5 LF
               seed-line LF close [LF]
open         = "===PCSF:v1===" / "===PCSF:v1.1===" / "===PCSF:v1.2==="
close        = "===END==="

meta-line    = "META: " field SEP field SEP depth SEP timestamp
chain        = "CHAIN: hops=" 1*DIGIT " origin=" timestamp
integrity    = "INTEGRITY: crc32=" 8HEXLOW " chars=" 1*DIGIT
depth        = "SURFACE" / "WORKING" / "DEEP"
timestamp    = 4DIGIT "-" 2DIGIT "-" 2DIGIT "T" 2DIGIT ":" 2DIGIT "Z"

layer1       = "L1:SKELETON" *(LF l1-entry)
layer2       = "L2:NEGATIVE" *(LF l2-entry)
layer3       = "L3:INTERNAL" *(LF l3-entry)
layer4       = "L4:RELATION" *(LF l4-entry)
layer5       = "L5:LOOPS"    *(LF l5-entry)

l1-entry     = l1-type SEP field SEP field
l1-type      = "CHOICE" / "CONSTRAINT" / "DISCOVERY" / "PIVOT" / "OUTCOME"
l2-entry     = field SEP field SEP field
l3-entry     = l3-type ": " field
l3-type      = "UNCERTAINTY" / "DIRECTION" / "INTUITION" / "UNRESOLVED"
l4-entry     = dimension ": " field
dimension    = 1*(ALPHA / "_")
l5-entry     = l5-type SEP field SEP priority
l5-type      = "PROMISE" / "DEFERRED_QUESTION" / "INCOMPLETE_TASK" / "HYPOTHESIS_UNTESTED"
priority     = "HIGH" / "MED" / "LOW"

seed-line    = "SEED: " sentence

SEP          = " | "                 ; space pipe space
field        = 1*fieldchar           ; non-empty; see fieldchar
sentence     = 1*fieldchar
fieldchar    = %x20-7B / %x7D-7E     ; printable ASCII EXCEPT "|" (%x7C)
8HEXLOW      = 8(DIGIT / %x61-66)    ; 8 lowercase hex digits
HEXLOW       = DIGIT / %x61-66
LF           = %x0A

Notes:

  • fieldchar deliberately excludes 0x7C (|), which enforces §2.3's "no pipe inside a field" at the grammar level. field is also forbidden from beginning or ending with space by §2.1's no-trailing-whitespace rule (enforced by the validator, not expressible compactly in ABNF).
  • The optional trailing [LF] permits an optional final newline after ===END===.
  • l3-entry/l4-entry use ": " as the first delimiter; later colons are part of field (consistent with §2.3).

3. Conformance summary (what a validator checks)

A capsule is conformant iff all of the following hold:

# Rule
C1 ASCII-only; LF endings; no tabs/CR/control bytes; no trailing whitespace.
C2 Total size ≤ 1,200 characters.
C3 First line ===PCSF:v1===, last content line ===END===.
C4 META: line present with 4 fields; depth and timestamp well-formed.
C5 All five layer headers present, in order.
C6 Each layer within its max entry count (12/8/6/5/8).
C7 Every entry matches its layer's grammar and enum tokens.
C8 No | inside any field; correct field separators.
C9 SEED: present, single non-empty sentence.
C10 If present, INTEGRITY: immediately follows META: (or CHAIN: if present) and is well-formed.
C11 If present, the integrity CRC-32 and char count match the file.
C12 If present, CHAIN: immediately follows META: and matches hops=<int> origin=<timestamp> (validator rule A20).

VALIDATION.md adds quality and behavioral criteria beyond this structural conformance.

4. Versioning

The format is versioned in the header (===PCSF:vN===). v1 and the backward-compatible supersets v1.1 and v1.2 are defined here. A reader MUST accept header tokens it implements and MUST refuse any other version rather than guess. Future versions MAY add layers or fields but MUST change the version token. The reference validator's accepted set is pcsf.validator.HEADERS.

5. Graceful degradation (deterministic autotrim)

The 1,200-char budget is a hard wall. Rather than fail when an encoder overshoots, PCSF defines a deterministic sacrifice order so a valid capsule is always producible and the caller learns exactly what was dropped. Entries are removed, one at a time, lowest-value first, until the capsule fits:

  1. L5:LOOPS entries with PRIORITY = LOW.
  2. L4:RELATION entries beyond the first two dimensions.
  3. L3:INTERNAL entries (soft, most model-inferable) from the bottom up.
  4. L5:LOOPS entries with PRIORITY = MED.
  5. L2:NEGATIVE entries from the bottom up (earliest, hardest-won kept longest).
  6. L5:LOOPS entries with PRIORITY = HIGH (last resort).

The L1:SKELETON causal nodes and the SEED are never dropped by autotrim: losing them changes the meaning of the capsule rather than merely abbreviating it. Within a layer, later entries are dropped before earlier ones, because the format requires authoring most-important-first. The reference implementation is pcsf.transform.fit_to_budget, which returns the trimmed capsule plus the ordered sacrifice log.

This operationalizes the "approximate minimal sufficient statistic" principle: under pressure the capsule sheds exactly the most reconstructible information and retains the least reconstructible.

6. Decoder robustness: conflict and staleness

A capsule is a set of priors, not ground truth. A conforming decoder MUST:

  1. Treat the META timestamp as a freshness signal. The capsule reflects state as of that time. If the decoder is given newer authoritative context that contradicts the capsule, the newer context wins; the capsule yields.
  2. Surface contradictions, never silently pick. If two capsule entries conflict (e.g. an L1 choice that a later L5 loop reopens), the decoder MUST flag the conflict to the user and ask, rather than guessing which is current.
  3. Degrade, not fabricate. If a referenced thread is underspecified, the decoder states the gap and asks; it MUST NOT invent transcript detail to fill it. (This is the provenance-honesty behavior measured in VALIDATION.md.)
  4. Refuse unknown versions. Per §4, an unrecognized version token is refused, not best-effort parsed.

These rules make a stale or partially-bad capsule safe: the worst case is an explicit question to the user, never a confident wrong action.

7. Capsule stitching (v1.2)

Stitching lets a chain of AIs accumulate cognitive state without the capsule growing. AI N receives a capsule, continues the task, captures its own new state as a fresh capsule, and stitches the two into a single successor capsule that is handed to AI N+1. The defining constraint: the result is still a valid capsule ≤1,200 characters, no matter how long the chain. Stitching is therefore a bounded merge / re-compression, never an append.

The reference implementation is pcsf.transform.stitch_capsules(old, new) (CLI: python -m pcsf.transform stitch old.pcsf new.pcsf). It is deterministic.

7.1 The merge (per layer)

For each layer, the union of the prior (old) and new (new) entries is formed with deduplication and newer-wins conflict resolution, using a per-layer identity key:

Layer Identity key (entries sharing it are the "same" thread)
L1:SKELETON EVENT_TYPE + CORE_FACT (first two fields)
L2:NEGATIVE REJECTED_OPTION + REJECTION_REASON (first two fields)
L3:INTERNAL the STATE_TYPE token (e.g. DIRECTION)
L4:RELATION the DIMENSION token (e.g. STYLE)
L5:LOOPS the loop DESCRIPTION (middle field)

Rules:

  1. Exact duplicates collapse to one entry.
  2. Same key, different content → newer-wins: the new entry replaces the old one in place (preserving the old entry's position, i.e. its priority ordering). This is how a chain updates a loop's priority, refines a rationale, or revises the current DIRECTION without leaving stale copies behind.
  3. Genuinely new keys append after the existing entries.

7.2 Per-layer caps

After the merge, each layer is trimmed to its §2.5–§2.9 entry cap (12/8/6/5/8):

  • L5:LOOPS: drop lowest-priority first (keep HIGH over MED over LOW).
  • L1:SKELETON: keep the most recent nodes (a long chain accretes decisions; the newest are the most operative).
  • All other layers: keep the earliest (hardest-won) entries.

7.3 Budget fit

If the merged capsule still exceeds 1,200 characters (or the reserved budget when a later INTEGRITY stamp is requested), the standard graceful-degradation order of §5 is applied until it fits. The dropped entries are reported. Because dedup + caps + value-priority trim all run every hop, the capsule cannot grow without bound: each hop sheds the most reconstructible, least operative state to make room for genuinely new state.

7.4 Lineage and META

  • The successor's CHAIN line (§2.10b) advances: hops = prior_hops + 1 (a stitch of two un-chained capsules yields hops=1), and origin is carried unchanged from the prior capsule's CHAIN line, or — on the first stitch — set to the old capsule's META timestamp (the chain's true root).
  • META topic/domain/timestamp are taken from the new capsule (the latest session); depth is the maximum depth reached anywhere on the chain.
  • SEED is taken from the new capsule (falling back to the old SEED if empty): the most recent orienting sentence leads.

7.5 Guarantees

A stitched capsule is guaranteed to: (a) be a conformant v1.2 capsule (passes A1–A20); (b) be ≤1,200 characters; (c) respect every per-layer cap; (d) carry the union of operative state with stale duplicates removed; and (e) record its chain depth and origin. These properties hold transitively, so an arbitrarily long chain stays bounded.