Skip to content

Latest commit

 

History

History
591 lines (486 loc) · 30.1 KB

SPEC.md

File metadata and controls

591 lines (486 loc) · 30.1 KB

MCP Lock Format v1

A vendor-neutral specification for an on-disk integrity baseline of an MCP server's declared tool, resource, and prompt surface.

Status: stable v1. This document defines the on-disk file, its canonicalization and hashing, the construction of the overall digest, and the normative definition of drift in terms that any tool can implement. The reference implementation of this format is docs/WARDEN_LOCK_SCHEMA.md; where the two ever disagree on a normative rule, that is a defect — both trace to the same algorithm.

The key words MUST, MUST NOT, SHOULD, MAY, and OPTIONAL are used as defined in RFC 2119 / RFC 8174.


1. Scope and non-goals

This format records the declared surface of an MCP server — the names, descriptions, input schemas, and derived capability flags it advertises over the MCP initialize / tools/list / resources/list / prompts/list handshake — and produces a reproducible digest over that surface. The purpose is to detect a change to that declared surface between an approved baseline and a later observation ("drift").

In scope: the byte-reproducible digest of a declared MCP surface, and the classification of any difference from a stored baseline.

Out of scope (non-goals). This format does not describe, attest to, or constrain runtime behavior — it covers declared-surface integrity only. It is not a statement that a server is safe, correct, or trustworthy at execution time; it does not verify the contents of the launched binary; and it makes no claim about what a tool does when called. A matching digest means only that the declared surface is byte-identical to the baseline.


2. File format and location

  • Filename: an implementation MUST write the lock to a single file named warden.lock.
  • Encoding: the file MUST be UTF-8 JSON. For human and code-review legibility it SHOULD be pretty-printed with a 2-space indent. The pretty-printed bytes are never hashed; all hashing uses the canonical form in §4.
  • Trailing newline: the file MUST end with exactly one \n.

3. Top-level schema

{
  "schema_version": 4,             // integer; see §14 — the current format level
  "warden_version": "x.y.z",       // semver of the tool that wrote the file
  "server": { ... },               // §6 server identity
  "tools":     [ { ... } ],        // §7 per-entry, sorted by name
  "resources": [ { ... } ],        // §7 per-entry, sorted by uri
  "prompts":   [ { ... } ],        // §7 per-entry, sorted by name
  "findings":  [ { ... } ],        // §9 embedded informational findings
  "overall_digest": "sha256:...",  // §8 digest over the whole surface
  "pin": { ... }                   // §10 baseline metadata + optional approval
}
  • schema_version MUST be a positive integer naming the format level (§14). "MCP Lock Format v1" is the stable family defined by this document; schema_version is the in-family integer the digest commits to, which bumps on a hashed-skeleton change (§14). The reference implementation currently writes 3.
  • Every top-level key listed above is required.
  • Empty collections MUST be written as [], never omitted. findings MAY be empty.
  • overall_digest is computed per §8; tools/resources/prompts are sorted per §7.

4. Canonicalization

To make pin (write baseline) and check (verify) agree byte-for-byte, every value that is hashed MUST first be canonicalized.

  • Canonical form: an implementation MUST serialize JSON values using the JSON Canonicalization Scheme (JCS), RFC 8785. This fixes object-key ordering (sort by UTF-16 code units, RFC 8785 §3.2.3 — for astral characters this is not code-point order; vectors/cases/canonical-surrogate-key-order.json pins it), string escaping, number formatting, and the elimination of all insignificant whitespace. Implementers SHOULD use a vetted JCS library rather than hand-rolling number formatting.
  • Array ordering: JCS preserves array order. This format additionally requires the three top-level entry arrays to be sorted before canonicalization (§7); the sort is performed by the writer so that the on-disk array order is already canonical.
  • Absent keys: a key that is absent from a JSON object MUST NOT be emitted by the canonicalizer. Therefore an optional field that is simply not present contributes nothing to a digest, and a baseline that omits it is byte-identical to one written before the field existed.
  • Nesting bound (normative): the depth of an element is the number of arrays and objects enclosing it; the document root is depth 0. A conforming implementation MUST accept any value whose deepest element is at depth 512 and MUST refuse — fail closed, never hash — any value containing an element at depth greater than 512, both when canonicalizing a surface value (§4–§5) and when reading a lock document (§3). An implementation that merely inherits its host language's recursion limit is NOT conformant: two such implementations disagree on the same document. The bound is pinned by vectors/cases/canonical-depth-512-accepted.json and vectors/cases/malformed-depth-513-rejected.json. This bound governs lock documents and surface values only; the content-envelope profile (docs/CONTENT_ENVELOPE.md) keeps its own, much smaller, nesting limit.

5. Hashing

  • Algorithm: an implementation MUST use SHA-256.
  • Digest encoding: every digest emitted into the file MUST be the string "sha256:" + lowercase_hex(SHA256(canon(value))) — the literal prefix sha256: followed by exactly 64 lowercase hexadecimal characters.
  • Hash primitive: hash(value) = "sha256:" + hex(SHA256(canon(value))), where canon is the §4 canonicalization.

5.1 Field-level hashes (absence rules)

For each entry the following field hashes are computed. The absence rules below are normative and MUST be implemented exactly, so that absence is stable and reproducible:

  • description_hash = hash(description). If description is absent or null, an implementation MUST hash the empty string "". null and "" are treated as identical.
  • input_schema_hash = hash(inputSchema). If inputSchema is absent or null, an implementation MUST hash the empty object {}. The entire input-schema object is hashed — type, properties, required, enum, nested schemas, additionalProperties, and so on — so that any schema change at all yields a different hash.
  • arguments_hash (prompts) = hash(arguments). If arguments is absent or null, an implementation MUST hash the empty array [].

6. Server identity

"server": {
  "command": "node",                       // argv[0] of the launch
  "args": ["./server.js", "--flag", "v"],  // remaining argv, order preserved
  "command_digest": "sha256:..."           // §6.1
}

6.1 Server identity canonicalization

  • command and args MUST be taken verbatim from the launch invocation. An implementation MUST spawn the server as an argv array and MUST NOT invoke a shell; no shell expansion or environment-variable interpolation is performed by the implementation (that is the caller's responsibility before invocation).
  • command_digest = hash({ "command": command, "args": args }).
  • A change in command or args is server-identity drift (the highest-severity drift class, §8.2): the baseline is now pinning a different launch than was approved.
  • command_digest does not hash the binary contents of the command; it pins the launch invocation only.

7. Per-entry schema

The raw description and inputSchema text MUST NOT be stored in the lock — only their hashes. This keeps the file small, reviewable, and free of any secret material.

7.1 Tool entry (array sorted by name)

Schema level 4 commits name, description, inputSchema, the complete annotations object, and outputSchema, together with derived capability/skeleton/inspection fields. Top-level title, icons, and _meta remain excluded. Levels 1–3 committed only name/description/inputSchema from the raw Tool; missing old commitments MUST NOT be treated as historical null metadata.

annotations_hash = hash(annotations); output_schema_hash = hash(outputSchema). Absent/null values MUST hash JSON null, not {}; empty objects are distinct. Non-null values MUST be objects, otherwise capture/verification refuses them. output_schema_skeleton is the §7.5 extraction of a present schema, or null when absent. It participates in entry_digest. V4 readers MUST require both hash strings and the skeleton field (which may be null). Raw metadata is not added to reports. Annotations are untrusted server declarations, never authority or proof of safety; output schemas do not certify returned content.

{
  "name": "read_file",
  "description_hash": "sha256:...",   // §5.1
  "input_schema_hash": "sha256:...",  // §5.1
  "capabilities": ["fs-read"],        // §7.4 derived flags, sorted, deduped
  "schema_skeleton": { ... },         // §7.5 structural facts, or null
  "annotations_hash": "sha256:...",   // v4; absent/null -> hash(null)
  "output_schema_hash": "sha256:...", // v4; absent/null -> hash(null)
  "output_schema_skeleton": { ... },  // v4; absent/null -> null
  "inspection": { ... },              // §11 OPTIONAL per-tool inspection block
  "entry_digest": "sha256:..."        // §7.3
}

7.2 Resource and prompt entries

  • Resource entry (array sorted by uri): { "uri", "name", "description_hash", "mime_type" (or null), "entry_digest" }. Resources have no inputSchema.
  • Prompt entry (array sorted by name): { "name", "description_hash", "arguments_hash", "entry_digest" }, where arguments_hash = hash(arguments) per §5.1.

7.3 Entry digest

entry_digest = hash(<the entry object WITHOUT its own entry_digest field>).

An implementation MUST build the entry with all fields except entry_digest, canonicalize it (§4), hash it (§5), then attach entry_digest. This makes each entry independently verifiable and localizes any diff to a single entry.

7.4 Derived capability flags (capabilities)

capabilities is a sorted, de-duplicated list of coarse capability flags derived deterministically from a tool's name tokens and inputSchema property names/shapes — never from fuzzy description parsing. The defined flags are:

Flag Meaning
shell-exec tool can run a shell/exec/spawn/subprocess command
fs-write tool can write/create/delete a filesystem path
fs-read tool can read a filesystem path
http-request tool can issue an outbound HTTP/network request
sql-query tool can run a SQL/database query

An implementation MUST derive these flags by a fixed, documented mapping over tokenized names and property names (case-insensitive, on snake_case / camelCase / kebab-case segment boundaries). The exact token and property sets are part of the implementation's published derivation table; two conformant implementations sharing that table MUST agree.

7.5 Structural schema skeleton (schema_skeleton)

A tool entry MAY carry a schema_skeleton: a deterministic, normalized extraction of the input schema's security-relevant structure — one record per dotted property path, recursing properties and array items. Each record keeps type (as a sorted tuple), required, enum, and constraints (maxLength, minLength, minimum, maximum, pattern, format, additionalProperties); it drops cosmetic keys (description, title, examples, default). When additionalProperties is absent it is treated as true; a same-document $ref (#/$defs/…, #/definitions/…, any RFC 6901 pointer, percent-decoded before ~1/~0 unescaping) is followed into its target so a constraint hidden behind a shared definition classifies granularly (schema_version 3, §14.2); a remote $ref, a $ref with sibling keys, an unresolvable pointer, or a ref chain over budget is recorded as the opaque leaf {"$ref": "<literal>"}; a cyclic ref, and any cyclic or over-deep node, is recorded as {"_truncated": true}. The skeleton lets check classify what changed (§8.2). It MAY be null; a baseline lacking it falls back to a blob-level schema-modified classification until re-pinned.


8. Overall digest and drift

V4 annotation changes emit high tool-annotations-modified. Output addition/removal emits high schema-out-added/schema-out-removed; other changes reuse §8.3 structural classes with schema-out- replacing schema-. Cosmetic-only changes emit low schema-out-cosmetic-modified; missing skeletons fall back to high schema-out-modified. SARIF prefixes these with WRD-DRIFT-, including WRD-DRIFT-SCHEMA-OUT-TYPE-BROADENED.

A differing overall_digest is itself drift. If it differs and the per-entry diff produced no item (for example a baseline with duplicate tool/resource/prompt keys, which name-keyed indexing collapses), an implementation MUST emit high digest-mismatch-unexplained (target lock/overall_digest) so the verdict is never clean. Any other item, including the schema-version-migrated advisory, already explains the mismatch and suppresses it.

8.1 Overall digest

overall_digest = hash({
  "schema_version": <int>,
  "server": { "command_digest": <server.command_digest> },
  "tools":     [ <each tool.entry_digest>,     ... sorted ],
  "resources": [ <each resource.entry_digest>, ... sorted ],
  "prompts":   [ <each prompt.entry_digest>,   ... sorted ]
})

An implementation MUST compute overall_digest over exactly the fields above and MUST exclude findings, pin, and warden_version. Excluding these makes the digest depend only on the declared surface, so re-running an identical tool against an identical surface yields an identical overall_digest regardless of when it ran or who approved it.

8.2 Drift definition (normative)

To verify, an implementation re-captures the surface, recomputes everything in §4–§8, and compares to the stored lock. Any non-empty set of differences is drift, and a verifier MUST exit non-zero when drift is present. Severity drives reporting only — it never changes the pass/fail decision.

Drift class Condition Severity
Server-identity drift server.command_digest differs critical
Tool added a name present now, absent in baseline high
Tool removed a name present in baseline, absent now medium
Capability added same name, a new flag in capabilities high
Capability removed same name, a flag dropped from capabilities medium
Description modified same name, description_hash differs, schema + caps unchanged low
Resource/prompt added analogous to tools medium
Resource/prompt removed analogous to tools low
Resource/prompt modified analogous to tools low
Inspection-policy modified a tool's §11 inspection value changed/added/removed medium
No drift every entry_digest matches and overall_digest matches —

Capture completeness (normative). A capturer MUST drain every nextCursor page of each list call and MUST fail rather than emit a lock when a chain exceeds its page cap (reference: 256, --max-pages) or a later page errors. Tool names, prompt names and resource URIs are entry identities: a capturer MUST refuse a surface that repeats one, on any page, because a name-keyed diff would let one entry shadow the other.

When overall_digest matches the baseline there is provably no drift, and per-entry diffing MAY be skipped; otherwise an implementation diffs per entry so that a report can point at the exact entry that changed.

8.3 Structural schema drift (when skeletons are present)

When input_schema_hash differs and both the baseline and current entries carry a schema_skeleton, an implementation classifies the change structurally and emits one item per changed structural fact (a single property MAY yield more than one item), each with a compact, non-secret detail. When either side lacks a skeleton, or the change is in an opaque leaf ($ref / truncated), the implementation falls back to a single schema-modified (high). "Unconstrained" means: no enum, no pattern, no maxLength, and type string/object (or absent).

drift_class Change Severity
schema-required-removed a required property removed high
schema-property-removed an optional property removed medium
schema-required-unconstrained-added new required, unconstrained property high
schema-required-added new required, constrained property medium
schema-unconstrained-added new optional, unconstrained property high
schema-property-added new optional, constrained property low
schema-type-broadened type set widened (superset) high
schema-type-narrowed type set narrowed (subset) low
schema-type-changed type set disjoint / otherwise changed medium
schema-enum-widened enum widened (superset / new members) high
schema-enum-narrowed enum narrowed (subset) low
schema-enum-removed enum constraint lost entirely high
schema-enum-added enum constraint newly added low
schema-constraint-relaxed required→optional, maxLen↑/min↓/max↑, pattern/format removed medium
schema-additional-props-opened additionalProperties false→true high
schema-constraint-tightened any tightening (bounds, pattern/format added) low
schema-cosmetic-modified input_schema_hash differs but skeleton is identical low
schema-modified (fallback) no skeleton, or an opaque-leaf change with no matching rule high

Because input_schema_hash is an input to entry_digest, any schema byte change — cosmetic included — changes entry_digest and therefore overall_digest; the structural classifier only describes how it changed.


9. Embedded findings

The baseline writer MAY embed informational findings recording what was true at approval time:

"findings": [
  {
    "rule_id": "...",            // stable identifier for the check that fired
    "severity": "high",          // critical|high|medium|low
    "target": "tools/run_command",
    "message": "Tool exposes shell-exec capability",
    "snippet": "command: string (redacted)"
  }
]
  • Findings are informational at verification time unless a new finding is introduced by drift on a changed entry; pre-existing approved findings MUST NOT be re-failed.
  • Any finding carrying potentially secret material MUST store a redacted snippet (e.g. first few characters + an elision + a length), never the raw value. The lock is committed to source control and MUST NOT become a secret store. findings are excluded from overall_digest (§8.1).

10. Baseline metadata (pin)

"pin": {
  "created_at": "2026-06-06T14:22:05Z",  // RFC 3339, UTC
  "warden_version": "x.y.z",
  "mcp_protocol_version": "2025-06-18",  // protocolVersion echoed by initialize
  "approved": false, "approver": null,
  "approved_at": null,
  "approved_digest": null                // overall_digest the approver attested to
}
  • All of pin is excluded from overall_digest (§8.1). Adding or changing any field inside pin MUST NOT change a server's digest.
  • When a baseline is approved: approved is true, approver is the caller-supplied identity, approved_at is the approval time (UTC), and approved_digest is the freshly computed overall_digest.
  • approved_digest MUST equal overall_digest in a freshly approved file. If a later surface change is not re-approved, approved_digest will disagree with the recomputed overall_digest; a verifier surfaces this as an unapproved-change finding (high).
  • A verifier MAY warn rather than fail when approved is false; whether approval is required is a consuming-pipeline configuration choice, not a format rule.

An implementation MAY carry additional metadata (e.g. structured provenance, attestation records) inside pin. Because all of pin is excluded from overall_digest, such extensions MUST NOT affect the digest, and an implementation MUST tolerate (ignore) unknown pin fields written by another implementation.


11. Per-tool inspection block (OPTIONAL)

A tool entry (§7.1) MAY carry an inspection object declaring deterministic, reviewed relaxations of optional result-inspection checks for that one tool:

"inspection": {
  "expected_output_charset": "text",   // "text" | "extended" | "binary-ok"
  "may_return_urls": false,            // bool
  "secret_echo_applies": true          // bool
}
Key Type Allowed values Default when absent
expected_output_charset string "text", "extended", "binary-ok" "text"
may_return_urls bool true / false false
secret_echo_applies bool true / false true
  • Fail-safe principle: an absent object or key means maximum protection; a tool only weakens a check by an explicit, reviewed declaration. There MUST be no runtime override path — these are lock-only, reviewed like any other lock change.
  • expected_output_charset MUST be one of the three literals; any other value MUST be a write-time error. A reader encountering an unknown value MUST fall back to "text".
  • An inspection object is meaningful only on tool entries; on a resource or prompt entry it MUST be ignored.
  • Digest impact: the inspection object is part of the tool entry, so it is included in that tool's entry_digest (§7.3) and in overall_digest (§8.1). Changing any inspection value is therefore drift (Inspection-policy modified, medium); a relaxation cannot be slipped in without re-pinning. A tool entry with no inspection key hashes exactly as if the key never existed (per §4 absent-key rule).

12. Conformance

An implementation is conformant with MCP Lock Format v1 if and only if:

  1. It writes and reads a warden.lock matching the top-level schema (§3), with schema_version naming the format level it implements (currently 4, §14).
  2. It canonicalizes per RFC 8785 JCS (§4) and hashes per §5, emitting every digest as sha256: followed by 64 lowercase hex characters.
  3. Given the same declared surface, it produces a byte-identical overall_digest to any other conformant implementation — this is the core reproducibility requirement. Two conformant implementations that capture the same surface MUST agree on overall_digest, every entry_digest, and every field hash.
  4. It excludes findings, pin, and warden_version from overall_digest (§8.1).
  5. It classifies any difference from a stored baseline as drift per §8.2 and exits non-zero on any non-empty drift set.
  6. It honors the field-absence rules (§5.1), the no-shell launch rule (§6.1), and the fail-safe defaults of the optional inspection block (§11).

Conformance is about reproducing the digest over a declared surface. It is not any form of certification, and it makes no claim about runtime behavior or safety.

12.1 Conformance vectors (normative)

The directory vectors/ in the reference repository is the language-neutral, executable definition of the rules above. An implementation is conformant if and only if it reproduces every vector in vectors/manifest.json byte-for-byte:

  • canonical — the RFC 8785 text and sha256: digest of arbitrary JSON values (§4–§5);
  • digest — every field hash, capabilities, schema_skeleton, entry_digest and overall_digest for a declared surface (§5.1–§8.1), including entry sorting;
  • drift — the ordered (drift_class, severity, target, detail) set for a baseline lock and an observed surface (§8.2–§8.3);
  • malformed — lock documents a conforming reader MUST reject, and JSON values a conforming canonicalizer MUST refuse (unpaired surrogates, nesting past the §4 bound).

The corpus is generated from the reference implementation (vectors/tools/generate.py) and is regenerated only on a deliberate format change (§14). Two implementations ship with the reference and both run the corpus in CI: the Python CLI and the zero-dependency TypeScript verifier @mcp-warden/lock. A third implementation that disagrees with a vector has found either its own defect or a defect in the reference — never a reason to edit the vector.


13. Historical v3 worked example

This historical v3 example illustrates the pre-migration formula. Current v4 entries also require the §7.1 annotation/output commitments; see the conformance vectors for complete current documents. Digests below are abbreviated for readability.

{
  "schema_version": 3,
  "warden_version": "1.0.0",
  "server": {
    "command": "node",
    "args": ["./server.js"],
    "command_digest": "sha256:3f1a...c0de"
  },
  "tools": [
    {
      "name": "read_file",
      "description_hash": "sha256:b2e4...77aa",
      "input_schema_hash": "sha256:9c11...01ff",
      "capabilities": ["fs-read"],
      "schema_skeleton": {
        "props": {
          "$root": { "type": ["object"], "required": false, "enum": null, "constraints": {"additionalProperties": false} },
          "path":  { "type": ["string"], "required": true,  "enum": null, "constraints": {"additionalProperties": true} }
        }
      },
      "entry_digest": "sha256:7d80...4e2b"
    }
  ],
  "resources": [],
  "prompts": [],
  "findings": [],
  "overall_digest": "sha256:a91c...f300",
  "pin": {
    "created_at": "2026-06-06T14:22:05Z",
    "warden_version": "1.0.0",
    "mcp_protocol_version": "2025-06-18",
    "approved": true,
    "approver": "ci-bot",
    "approved_at": "2026-06-06T14:22:05Z",
    "approved_digest": "sha256:a91c...f300"
  }
}

To reproduce overall_digest here, a verifier hashes {"schema_version":3,"server":{"command_digest":"sha256:3f1a...c0de"},"tools":["sha256:7d80...4e2b"],"resources":[],"prompts":[]} under §4 canonicalization and §5 hashing. If that value equals the stored overall_digest, there is no drift; otherwise the verifier diffs per entry (§8.2) and exits non-zero.


14. Compatibility & versioning policy

Publishing "MCP Lock Format v1" is an implicit stability contract. This section makes it explicit: what stays stable, what may change, and what a producer MUST do when it changes. The governing line is the digest: a change is compatible iff it cannot alter a server's overall_digest.

14.1 Backward-compatible (additive, out-of-digest) changes. A change is backward-compatible — no schema_version bump — when it touches only fields excluded from overall_digest (§8.1: findings, pin, warden_version). A producer MAY add new keys inside pin; readers MUST tolerate (ignore) unknown pin keys (§10). Precedent: the structured provenance block (pin.provenance_version, pin.pinner, pin.attestations, pin.rotated_at) and the Sigstore signer pointer attestation were all added outside overall_digest, carry their own provenance_version counter (distinct from schema_version by design), and leave every server's digest byte-identical — so an older verifier keeps verifying a newer producer's lock for an unchanged surface.

14.2 What forces a schema_version bump. Anything that changes the bytes hashed into overall_digest forces a bump: the §8.1 digest skeleton (schema_version, server.command_digest, the per-entry digest list) and any hashed per-entry field — description_hash, input_schema_hash, arguments_hash, capabilities, the schema_skeleton extraction rules, or the inspection block (§11, in-digest). Real history: 1→2 added the structural schema_skeleton to tool entries; 2→3 began resolving in-document $ref leaves inside the skeleton (no longer an opaque leaf). Each changed the skeleton of affected tools → entry_digest → overall_digest, so each was a bump, not a silent surface change. A producer MUST NOT change any hashed field's derivation without bumping schema_version.

3→4 adds annotations_hash, output_schema_hash, and output_schema_skeleton to every tool entry, including canonical null commitments. Fresh entry digests change and require review/re-attestation. Python and TypeScript implement the same level; conformance includes genuine v3 bytes and malformed v4 omissions.

14.3 How consumers are notified / how old locks are handled. Because schema_version lives inside overall_digest, a format bump deterministically changes the digest, and on an approved baseline a verifier surfaces it as the same high unapproved-change finding it raises for any surface change (§10) — never an auto-pass across the boundary. When schema_version increases, the verifier additionally emits an additive low schema-version-migrated advisory that explains the digest delta but MUST NOT replace, gate, or downgrade that finding (auto-downgrading would be a laundering bypass); the operator reviews and re-pins to re-attest under the new level. Pre-skeleton (v1) locks degrade gracefully: a baseline lacking a schema_skeleton falls back to the coarse schema-modified (high) until re-pinned (§7.5, §8.3). Additive migration advisories never DOWNGRADE a finding.

For v4, an unapproved older baseline also receives a migration finding and fails: missing old commitments cannot be reported as v4 coverage. Approved locks retain high unapproved-change plus the advisory. Reading or rotating old documents preserves their recorded schema rather than silently converting them to v4.

14.4 Newer levels. A reader MUST reject (fail closed) a lock whose schema_version is above the level it implements. It cannot reproduce a derivation it does not know, and comparing such a lock under an older level's rules would silently mis-verify — the schema-version-migrated advisory covers only older baselines. Pinned by malformed/schema-version-above-implemented.