feat(verify): verify the Level 3 TRACE claim signature and evidence pack (#204) - #301
Conversation
Implements the outstanding half of agentrust-io#204. Phase 1 of that issue, the attestation chain, landed in 2073576 and has been built out since by the TDX DCAP work and the SNP report union. The second clause was never picked up: The SDK does not verify the attestation chain (quote/VCEK/expected measurement) or the Level 3 TRACE claim signature, so an unverified report or claim currently passes. This is that clause. Nothing in the attestation path is touched. THE PROBLEM The SDK stored trace_id and evidence_pack and never checked either signature. Any field inside a TRACE record could be edited after the fact and still be accepted as evidence: the Cedar decision, the policy and catalog hashes actually in force, or the manifest verification result recorded at the time of the call. TRACE records are the forensic layer, and a record whose signature is never verified is not evidence of anything. TWO PRE-IMAGES, NOT SYMMETRIC This is the part the issue did not specify and the specification had to settle. A TRACE envelope signs every field except signature, as RFC 8785 canonical JSON. Spec 6.3.2 types signature as a bare string, so the envelope carries no algorithm and no key id at all; the caller supplies both. Hybrid is not expressible in that shape and is rejected rather than guessed at. An evidence pack signs every field except pack_signature, which is the full detached signature object of spec 3.6 with algorithm and key id inside, so hybrid works there. compute_pack_hash returns the SHA-256 of exactly those bytes, which is what spec 5.2.1 defines pack_hash to be. That asymmetry is why verification goes through verify_bytes on the verifier classes rather than the manifest pre-image helpers. AUTHENTIC AND ADMISSIBLE ARE DIFFERENT QUESTIONS admissible is the question a relying party actually asks: may this record be used as evidence of a valid tool call? That needs a verified signature and a manifest_verification_result outside INADMISSIBLE_RESULTS. A perfectly signed envelope reporting MISMATCH is authentic and inadmissible at the same time, since the signature proves the runtime honestly recorded that the policy hash did not match. Collapsing the two would either discard honest failure records or admit them as proof of success. THE HASH CONFLICT RULE Spec 6.3.2, SCHEMA F-21. The manifest is authoritative for approved artifact hashes and the TRACE reports what was actually in force, so when they disagree the producer MUST have written manifest_verification_result: MISMATCH. An envelope reporting a conflicting policy_hash while still claiming VALID is a specification violation, and a self-serving one, so it is a failure rather than a warning. FAIL CLOSED A missing key, an unknown algorithm, or a build without an ML-DSA backend yields UNVERIFIABLE, never VERIFIED. MALFORMED covers a missing required field or an illegal enum value, because the shape cannot be vouched for and the absent field may be the one a relying party is about to read. UNVERIFIABLE is deliberately distinct from FAILED: a capability gap is not a bad record. VERIFICATION 63 new tests. Full suite 923 passed, 6 skipped. _trace.py at 92 percent. mypy strict, bandit, and ruff under the rule set CI applies are all clean. Coverage includes valid envelopes, tampered fields, wrong keys, missing signatures, malformed shapes, the hash conflict rule, hybrid rejection on TRACE envelopes, hybrid acceptance on evidence packs, and pack_hash agreeing with spec 5.2.1. Refs agentrust-io#204, agentrust-io#201 Signed-off-by: Mohammed Zoheb Shaik <zoheb.shaik7@gmail.com>
|
🟡 Contributor Check: MEDIUM
Automated check by AgenTrust Contributor Check. |
imran-siddique
left a comment
There was a problem hiding this comment.
The signature primitives and admissibility split are promising, but the evidence-pack verifier currently accepts a signed structurally empty pack as VERIFIED. �erify_evidence_pack computes a pre-image for any dict, verifies pack_signature, defaults missing race_envelopes to [], and never requires the four normative section 5.2.1 members (manifest, �erification_result, race_envelopes, �ttestation_report). A valid signature over {pack_signature: ...} therefore becomes a VERIFIED evidence pack. Please validate all four required members and their container types before signature appraisal, add a regression test proving a correctly signed pack missing each member is MALFORMED, and ensure an empty/malformed manifest cannot silently disable manifest-id/policy binding. The same review should add structural type/format validation for required TRACE envelope fields beyond hitl_required; presence plus signature is not enough to call a malformed evidence record VERIFIED.
Addresses the review on agentrust-io#301. A pack carrying nothing but its own pack_signature verified as VERIFIED. The pre-image was computed over whatever was present, the signature covered it honestly, and the result said VERIFIED about a document with no manifest, no verification result, no envelopes and no attestation report. Reproduced before changing anything: pack contents : ['pack_signature'] status : VERIFIED signature_verified : True failures : [] A signature proves who assembled a document, not that the document is the thing it claims to be. WHAT CHANGED The four members spec 5.2.1 defines are now required, and their container types checked, before pack_signature is appraised at all. Structure first means a malformed pack is MALFORMED even when the signature would have verified, so the two failures cannot mask one another. An empty manifest is rejected separately: it is well-typed and still useless, because every binding check downstream reads from it. THE HALF THE FIRST FIX MISSED Rejecting an empty manifest was not enough. _check_manifest_binding skips the manifest-id comparison when the manifest carries no manifest_id, so a manifest of {"note": "..."} is non-empty, well-typed, passes the emptiness check, and still disables the binding completely. An envelope naming an entirely different manifest came back VERIFIED with no failures. manifest_id is now required to be a non-empty string, since the binding is the reason the manifest is in the pack at all. ENVELOPE FIELDS Presence is not enough there either. A field of the wrong type passes an `in envelope` check and then flows into a comparison that silently does nothing: policy_hash of [] never equals the manifest's hash, so the section 6.3.2 conflict rule would never fire against it. Fourteen required fields are now type-checked. hitl_required is deliberately excluded from the generic check. It already has a dedicated one naming SCHEMA F-11 and explaining why a string is dangerous rather than merely wrong, and a generic message would have replaced a better one. TESTS Twenty-one added. Each of the four members missing individually from a correctly signed pack, each of the four carrying the wrong container type, the empty manifest, the manifest without a manifest_id, structure being checked ahead of the signature, seven wrong-type envelope fields, and two asserting the happy path and the working binding are unaffected. VERIFICATION 84 TRACE tests, 944 passed, 6 skipped overall. _trace.py at 91 percent. mypy strict, bandit, and ruff under the rule set CI applies are all clean. Refs agentrust-io#204 Signed-off-by: Mohammed Zoheb Shaik <zoheb.shaik7@gmail.com>
|
Fixed in the commit above. Structure before signature. The four members of 5.2.1 are now required, and their container types checked, before Manifest binding. An empty Envelope fields. Fourteen required fields are now type-checked. Tests, twenty-one added. Each member missing individually from a correctly signed pack, each with the wrong container type, the empty manifest, the manifest without a 84 TRACE tests, 944 passed and 6 skipped overall, |
imran-siddique
left a comment
There was a problem hiding this comment.
The previous empty-pack bypass is narrowed but not fully closed. Two blocking fail-closed gaps remain:
-
verify_evidence_packvalidates only the top-level container types. A correctly signed pack withverification_result: {}andattestation_report: ""is still returned as VERIFIED, even though neither value is the required section 5.2 verification-result object / raw base64url attestation report. A signature authenticates malformed content; it does not make it a conforming evidence pack. Please structurally validate the required verification-result fields (including legal result enum) and require a non-empty, valid base64url attestation report before signature appraisal, with signed regression cases. -
TRACE validation added types but not the requested formats/enums. A correctly signed envelope can still use
decision: "root", invalidpayload_classification, malformed UUID v7 IDs, non-SPIFFEagent_id, non-sha256 hashes, or a non-UTC/non-ISO timestamp and receiveadmissible=True. These fields have normative formats/enums in section 6.3.2 and the general UUID rule. Please reject invalid enum/format values as MALFORMED and add adversarial signed tests for each class.
CI is green, but these are correctness/security blockers because the new API labels structurally non-conforming forensic records VERIFIED/admissible.
Addresses the second review on agentrust-io#301. Both gaps reproduced before changing anything: verification_result: {}, attestation_report: "" -> VERIFIED decision: "root" -> VERIFIED, admissible trace_id: "not-a-uuid" -> VERIFIED, admissible agent_id: "just-a-name" -> VERIFIED, admissible policy_hash: "deadbeef" -> VERIFIED, admissible timestamp: "yesterday" -> VERIFIED, admissible Type-correct is not the same as spec-conforming, and a signature authenticates malformed content rather than making it conforming. EVIDENCE PACK CONTENTS verification_result must be non-empty, must carry a result, and that result must be one of the section 5.2 values. attestation_report must be non-empty and decodable as base64url, since spec 5.2.1 carries the raw platform report in that encoding and a value that cannot be decoded is not a report the pack could have produced. Both checked before signature appraisal, alongside the existing member and type checks. TRACE ENVELOPE FORMATS AND ENUMS Spec 6.3.2 fixes the shapes, and each is now enforced: decision is one of allow, deny, require-approval; payload_classification is one of public, internal, confidential, restricted; trace_id and agent_manifest_id are UUID v7, as is hitl_approval_id when it is not null; agent_id is a SPIFFE URI; policy_hash and catalog_hash are sha256:<64-hex>; timestamp is ISO 8601 UTC. TWO PLACES THIS DELIBERATELY DOES NOT FOLLOW THE REVIEW LITERALLY The review asks to reject non-sha256 hashes. Spec 6.3.2 constrains policy_hash and catalog_hash to sha256:<64-hex> and types tee_measurement as "<platform-specific measurement>", so a hash shape is enforced on the first two only. Enforcing one on tee_measurement would reject records the specification permits, and a test asserts that a value such as mrenclave:abc123 still verifies. Timestamps reject offset-naive strings, not merely unparseable ones. 2026-08-03T12:00:00 parses, but its time zone has to be guessed, and a record whose instant cannot be ordered is of little use in an audit trail. A real offset that is not UTC is rejected for the same reason. Both Z and +00:00 are accepted. egress_destination is "<FQDN | IP | none>" and tool_id is a reverse-domain identifier with no stated grammar, so both are checked for emptiness only. TESTS Twenty-nine added, one class per violation and each over a correctly signed record: illegal enums, malformed UUID v7 in all three id fields, a v4 where a v7 is required, a present-but-broken hitl_approval_id against a null one, non-SPIFFE agent ids, non-sha256 hashes, four timestamp shapes against both legal UTC spellings, empty required strings, and every evidence pack content case. Plus assertions that the happy path and a platform-specific tee_measurement are unaffected. VERIFICATION 113 TRACE tests, 973 passed and 6 skipped overall. _trace.py at 93 percent. mypy strict, bandit, and ruff under the rule set CI applies are all clean. Refs agentrust-io#204 Signed-off-by: Mohammed Zoheb Shaik <zoheb.shaik7@gmail.com>
|
Both fixed in the commit above. Reproduced each case first, and every one behaved as you described. Evidence pack contents. Envelope formats and enums. Two notes on scope, in case you want them tightened. You mentioned non-sha256 hashes. Section 6.3.2 constrains On timestamps I have gone slightly further than non-ISO and also rejected offset-naive strings.
Tests, twenty-nine added, one class per violation and each over a correctly signed record, including a v4 where a v7 is required, a present-but-broken 113 TRACE tests, 973 passed and 6 skipped overall, |
imran-siddique
left a comment
There was a problem hiding this comment.
Reviewed the follow-up commit. The previously reported empty-pack and malformed verification-result bypasses are covered, and the required TRACE enum/format checks are now fail-closed. Focused verification: 113 tests passed; exact CI lint rule passed. Approving.
Summary
Implements verification of the Level 3 TRACE claim signature, the outstanding half of #204.
Phase 1 of that issue, the attestation chain, landed in 2073576
feat(verify): attestation-chain verifier scaffold (#204 phase 1) (#210)and has been built out since by the TDX DCAP work and the SNP report union in #260. The second clause of the issue was never picked up:This PR is that clause. It touches nothing in the attestation path.
The problem
The SDK stored
trace_idandevidence_packand never checked either signature. Any field inside a TRACE record could be edited after the fact and still be accepted as evidence: the Cedar decision, the policy and catalog hashes actually in force, or the manifest verification result recorded at the time of the call.TRACE records are the forensic layer. A record whose signature is never verified is not evidence of anything.
What is verified
A TRACE envelope is the per tool call record cMCP emits, carrying the agent's manifest id, the manifest verification result at the time of the call, the policy and catalog hashes in force, and the Cedar decision, signed by a TEE sealed key.
An evidence pack bundles the manifest, a verification result, the session's TRACE envelopes and the raw attestation report under one detached
pack_signature.The two pre-images are not symmetric
This is the part the issue did not specify and the spec had to settle.
TRACE envelope: every field except
signature, as RFC 8785 canonical JSON. Spec 6.3.2 typessignatureas a bare string, "Ed25519 | ML-DSA-65 by TEE-sealed key", so the envelope carries no algorithm and no key id at all. The caller supplies both. Hybrid is not expressible in that shape and is rejected rather than guessed at.Evidence pack: every field except
pack_signature, which is the full detached signature object of spec 3.6 with algorithm and key id inside, so hybrid works there.compute_pack_hashreturns the SHA-256 of exactly those bytes, which is what spec 5.2.1 definespack_hashto be.That asymmetry is why verification goes through
verify_byteson the verifier classes rather than the manifest pre-image helpers: the manifest pre-image is fixed and normative, and these two cover different field sets.Authentic and inadmissible are different questions
admissibleis the question a relying party actually asks: may this record be used as evidence of a valid tool call? That needs a verified signature and amanifest_verification_resultoutsideINADMISSIBLE_RESULTS.A perfectly signed envelope reporting
MISMATCHis authentic and inadmissible at the same time. The signature proves the runtime honestly recorded that the policy hash did not match, which is exactly the case spec 6.3.2 excludes from reporting. Collapsing the two into one boolean would either discard honest failure records or admit them as proof of success.The hash conflict rule
Spec 6.3.2, SCHEMA F-21. The manifest is authoritative for approved artifact hashes; the TRACE reports what was actually in force. When they disagree, the producer MUST have written
manifest_verification_result: MISMATCH.An envelope reporting a conflicting
policy_hashwhile still claimingVALIDis a spec violation, and a self serving one, so it is treated as a failure rather than a warning.Fail closed
A missing key, an unknown algorithm, or a build without the post quantum extra yields
UNVERIFIABLE, neverVERIFIED.MALFORMEDcovers a missing required field or an illegal enum value: the shape cannot be vouched for, and the absent field may be the one a relying party is about to read.UNVERIFIABLEis deliberately distinct fromFAILED. A capability gap is not a bad record.Verification
_trace.pycoverage--select E,F,W --ignore E501as CI runs itCoverage includes valid envelopes, tampered fields, wrong keys, missing signatures, malformed shapes, the hash conflict rule, hybrid rejection on TRACE envelopes, hybrid acceptance on evidence packs, and
pack_hashagreeing with spec 5.2.1.Scope
Only the TRACE claim signature half of #204. The attestation chain half is already on
mainand is untouched here: no changes to_attestation.py,_snp_verify.py,_tdx_verify.py,_tpm_verify.pyor_cert_chain.py.Leaving it to a maintainer whether #204 closes on this or stays open until both halves are confirmed complete.
Refs #204, #201