feat(canonicalization): use RFC 8785 (JCS) for sign/verify

[imran-siddique]

What changed

_canonical_bytes in src/agentrust_trace/sign.py previously used json.dumps(sort_keys=True, separators=(",",":"), ensure_ascii=True), which is not RFC 8785 (JCS) conformant. The spec (spec/trace-v0.1.md §3.2.2) normatively mandates JCS as the signature pre-image. _canonical_bytes is now backed by the rfc8785 library.

sign_record and verify_record already share _canonical_bytes (confirmed: it is the only producer of the signing pre-image, called once in each), so this single change applies the same canonicalization to both signing and verification.

Why JCS, not json.dumps

json.dumps(sort_keys=True, ensure_ascii=True) diverges from RFC 8785 in three load-bearing ways:

• Non-ASCII strings: json.dumps escapes them as \uXXXX; JCS emits raw UTF-8 (e.g. ü → \xc3\xbc, not ü).
• Numbers: json.dumps zero-pads exponents (1e-07); JCS uses the ECMAScript shortest round-trip form (1e-7), per RFC 8785 §3.2.2.3.
• Key ordering: json.dumps sorts by Unicode code point; JCS sorts by UTF-16 code unit (these differ for astral-plane code points).

Each divergence breaks cross-implementation verification and opens a signature-preserving mutation surface.

Library vs hand-rolled

Used the rfc8785 PyPI package (added to dependencies in pyproject.toml, pinned >=0.1.2) rather than hand-rolling JCS. Justification: a vetted library is the spec's own recommendation ("Implementations MUST use an RFC 8785-conformant library"), avoids re-implementing the subtle ECMAScript Number-to-String and UTF-16 ordering rules, and installs cleanly. I cross-checked it against the RFC 8785 Appendix B number vectors (1e-7, 1e+21, 9.999999999999997e+22, 5e-324, etc.) and it matches.

Tests (in tests/test_sign.py)

• test_round_trip_with_non_ascii_payload — sign then verify a record carrying non-ASCII data (modèle français 🤖); round-trip still passes.
• test_canonical_bytes_known_answer_non_ascii — literal expected-bytes assertion {"id":"caf\xc3\xa9","msg":"h\xc3\xbcllo"}, plus an explicit assertion that the json.dumps form (é/ü) differs. A regression to json.dumps fails this.
• test_canonical_bytes_known_answer_numbers — literal {"n":1e-7}, plus assert json.dumps(1e-7) == "1e-07" to show the form we moved away from.
• test_canonical_bytes_matches_reference_library — full record cross-checked against rfc8785.dumps.
• test_jcs_distinguishes_unicode_key_order_from_json_dumps — object with a z key and a U+1F600 key; JCS (UTF-16 code unit) puts z first and the two schemes produce different bytes.
• test_canonical_bytes_sorts_keys_no_whitespace — basic JCS shape.

All 71 tests pass (including the #73 freshness / trusted-key tests). ruff check src/ tests/ --select E,F,W --ignore E501 is clean.

Scope

This PR is canonicalization only. Two related items remain separate Wave 2 work:

• Revocation enforcement (the spec's "verifiers MUST consult current revocation status" requirement is not yet enforced in code).
• The cnf-in-preimage scope decision (whether cnf should be inside or outside the signed pre-image — currently cnf is signed and only signature is excluded).

🤖 Generated with Claude Code