Formal JSON Schema for EQL v2.2 baseline and v2.3 target payload format

[coderdan]

Note

ORE fields have not been consolidated in the 2.3 schema version (upcoming release) but may be before release finalisation

Summary

• Adds two JSON Schema 2020-12 files under docs/reference/schema/ describing the on-the-wire EQL payload format: eql-payload-v2.2.schema.json (current baseline) and eql-payload-v2.3.schema.json (target for the in-flight v2.3 release).
• Adds 20 Rust unit tests in tests/sqlx/tests/payload_schema_tests.rs that validate hand-crafted positive/negative samples against both schemas. Database-free.
• Wires the tests into a new fast schema CI job that gates the 4× Postgres matrix.

What the schemas capture

Two mutually exclusive top-level forms, discriminated by k:

• EncryptedPayload — scalar ciphertext form. Required v, c, i.{t,c}; any subset of the index-term fields (hm, bf, ob, ocf, ocv, …) depending on configured indexes. Must not carry sv.
• SteVecPayload — jsonb / containment form. Required v, k=\"sv\", i, sv. No top-level c or scalar index terms — those live on individual sv elements.

A shared IndexTerms $defs block describes each index-term field in exactly one place; root and sv-element shapes $ref into it.

v2.2 → v2.3 delta (encoded in the schema and in the changelog)

• The root-level b3 (Blake3) term is removed. STE vector element equality now uses hm directly (which is now permitted at the sv-element level — it was previously root-only).
• opf (fixed-width / numeric OPE) and opv (variable-width / text OPE) collapse into a single op field. Width information that was previously implied by the field name is now carried on the value.
• OPE (op) and ORE (ob, ocf, ocv) are mutually exclusive within a payload or sv element. Encoded as a shared OreOpeExclusion \$def referenced from both shapes.

Tests

mise run test:schema (or cargo test --test payload_schema_tests from tests/sqlx/) runs the 20 tests in ~0.02s, no database required. Coverage:

• v2.2 (10 tests): minimal & fully-populated valid forms; missing required fields; wrong version; EncryptedPayload ⊥ sv and SteVecPayload ⊥ c; hm / i / v / nested sv rejected on elements; additionalProperties: false enforced.
• v2.3 (10 tests): minimal valid; OPE-only valid; ORE-only valid; hm accepted on elements; b3 rejected at root and in elements; legacy opf/opv rejected at root and in elements; op × {ob, ocf, ocv} mutual exclusion at both levels; required-field enforcement across both variants.

Locked to jsonschema = \"0.17\" for API stability. The crate falls back to Draft 7 because it doesn't recognise the 2020-12 \$schema URL — every construct we use (\$defs, \$ref, oneOf, allOf, not, const, pattern, …) is valid in Draft 7, so semantics are unaffected, but we're not strictly enforcing 2020-12 features. Worth a note if we later reach for 2020-12-only constructs.

CI

New schema job on plain ubuntu-latest (no Postgres, no SQL build). The test matrix job now needs: schema, so a broken JSON Schema fails the build in seconds instead of after spinning up 4× Postgres containers. The job shares the Swatinem/rust-cache key sqlx-tests with the matrix so it primes (or reuses) the same Rust dep cache.

Follow-ups (not in this PR)

The v2.3 schema is documentation only — the SQL changes that match it (drop b3 from sv elements, accept hm there, replace opf/opv with op, enforce OPE/ORE exclusion in the SQL layer) land in subsequent PRs.

Summary by CodeRabbit

• Documentation

  • Added formal EQL JSON Schemas: v2.2 (baseline) and v2.3 (target), and documented v2.3 payload changes (remove root-level b3, promote hm in vector elements, collapse opf/opv into op, and enforce mutual exclusivity between op and ORE fields).

• Tests

  • Added automated schema validation tests to assert payloads against v2.2/v2.3.

• CI

  • Schema validation runs as a dedicated CI job and is required before the main test suite.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ed9f72c6-1d5b-4a7a-8a3a-d89053d6861f

📥 Commits

Reviewing files that changed from the base of the PR and between de75d26 and beb64ed.

⛔ Files ignored due to path filters (1)

• tests/sqlx/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (3)

• CHANGELOG.md
• tests/sqlx/Cargo.toml
• tests/sqlx/tests/payload_schema_tests.rs

✅ Files skipped from review due to trivial changes (1)

• CHANGELOG.md

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/sqlx/Cargo.toml
• tests/sqlx/tests/payload_schema_tests.rs

---

📝 Walkthrough

Walkthrough

Adds formal JSON Schema files for EQL payloads (v2.2 and v2.3), a Rust test suite that validates sample payloads against both schemas, a mise task to run those tests, and a CI job that runs schema validation before the main test matrix.

Changes

EQL Payload Schema Validation

Layer / File(s)	Summary
EQL v2.2 Payload Schema docs/reference/schema/eql-payload-v2.2.schema.json	Baseline JSON Schema defining the EQL v2.2 payload format with mutually exclusive EncryptedPayload and SteVecPayload shapes, SteVecElement constraints, and an IndexTerms catalog.
EQL v2.3 Payload Schema docs/reference/schema/eql-payload-v2.3.schema.json	Evolved JSON Schema for v2.3: removes b3, collapses opf/opv into op, promotes element-level equality to hm, and enforces OreOpeExclusion for mutual exclusivity between op and ORE terms.
Schema Documentation CHANGELOG.md	Changelog entry describing the newly added v2.2 and v2.3 JSON Schema files and summarizing v2.3 breaking payload-format changes.
Schema Validation Test Infrastructure tests/sqlx/Cargo.toml, mise.toml, tests/sqlx/tests/payload_schema_tests.rs (helpers/setup)	Adds jsonschema dependency, defines mise task test:schema to run payload schema validation tests, and establishes test helpers that load and cache compiled v2.2/v2.3 schemas with assert_valid/assert_invalid utilities.
Payload Schema Validation Tests tests/sqlx/tests/payload_schema_tests.rs (v2.2 & v2.3 test cases)	Comprehensive test suite validating EQL payloads: v2.2 tests enforce required fields, mutual exclusivity between payload kinds, and SteVecElement constraints; v2.3 tests assert removal of b3 and legacy opf/opv, acceptance of hm-bearing sv elements, and op/ORE mutual exclusivity at root and element levels.
CI Schema Validation Job .github/workflows/test-eql.yml	Adds a dedicated schema GitHub Actions job running mise run test:schema with checkout, mise setup (pinned to 2026.4.0), and Rust cache for tests/sqlx; updates the test job to depend on it via needs: schema.

Sequence Diagram

sequenceDiagram
  participant CI as GitHub Actions
  participant SchemaJob as schema job
  participant Mise as mise
  participant CargoTest as cargo test
  participant Validator as jsonschema validator
  participant Schema22 as v2.2 Schema
  participant Schema23 as v2.3 Schema

  CI->>SchemaJob: trigger schema job
  SchemaJob->>Mise: mise run test:schema
  Mise->>CargoTest: cargo test --test payload_schema_tests
  CargoTest->>Validator: load and compile schemas
  Validator->>Schema22: compile v2.2 schema
  Validator->>Schema23: compile v2.3 schema
  Validator->>Validator: validate sample payloads
  Validator-->>CargoTest: assertion results
  CargoTest-->>Mise: test exit code
  Mise-->>SchemaJob: task completion
  SchemaJob-->>CI: schema job result
  CI->>CI: test job runs only if schema job passes

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• cipherstash/encrypt-query-language#196: Implements and validates the same payload-format discipline (removing b3, promoting hm, and op/ORE changes); this PR adds schema + validation/tests while #196 updates functions/tests/fixtures accordingly.

Suggested reviewers

• freshtonic

🐰 The schemas now stand so tall and bright,
I hopped through tests with candlelight,
op and hm in tidy rows,
No b3 left to twitch its nose,
Payloads validated — hop, delight!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding formal JSON schemas for EQL v2.2 baseline and v2.3 target payload formats, which is the core focus of this PR.
Docstring Coverage	✅ Passed	Docstring coverage is 81.48% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch dan/v2.3-payload-schema

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

tests/sqlx/tests/payload_schema_tests.rs (1)

28-34: 💤 Low value

Draft 7 fallback is documented but consider future compatibility.

The comment explains that the jsonschema crate falls back to Draft 7 despite the schema declaring draft 2020-12, and notes that the used constructs are compatible. This is acceptable for the current implementation, but be aware that future schema enhancements using 2020-12-specific features may require a crate upgrade or workaround.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/payload_schema_tests.rs` around lines 28 - 34, The comment
in compile() notes jsonschema::JSONSchema falls back to Draft 7 for schemas
declaring draft 2020-12; update the compile function to explicitly handle
future-compatibility by detecting a 2020-12 $schema and emitting a clear warning
or TODO that the crate may need upgrading or a workaround if 2020-12-specific
features are later used. In short: inside compile (where JSONSchema::compile is
called), inspect schema.get("$schema") for "2020-12" and log a warning message
(or add a TODO comment) indicating the fallback to Draft 7 and recommending
upgrading jsonschema or adding a compatibility path if new draft features are
required.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 27: Update the CHANGELOG.md paragraph about the Formal JSON Schema for
the EQL payload format to include the PR link in parentheses at the end of the
paragraph; append the suggested PR link text
`([`#208`](https://github.com/cipherstash/encrypt-query-language/pull/208))` to
the existing sentence so the entry follows the repository's CHANGELOG guideline.

In `@tests/sqlx/Cargo.toml`:
- Line 13: Update the jsonschema crate dependency in Cargo.toml by changing the
version specifier for "jsonschema" (currently jsonschema = { version = "0.17",
default-features = false }) to a more recent stable release (e.g., "0.46.4" or a
current patch), keeping or adjusting default-features as needed; run cargo
update and cargo test to ensure compatibility and fix any API changes in code
that references the jsonschema types/functions if compilation fails.

---

Nitpick comments:
In `@tests/sqlx/tests/payload_schema_tests.rs`:
- Around line 28-34: The comment in compile() notes jsonschema::JSONSchema falls
back to Draft 7 for schemas declaring draft 2020-12; update the compile function
to explicitly handle future-compatibility by detecting a 2020-12 $schema and
emitting a clear warning or TODO that the crate may need upgrading or a
workaround if 2020-12-specific features are later used. In short: inside compile
(where JSONSchema::compile is called), inspect schema.get("$schema") for
"2020-12" and log a warning message (or add a TODO comment) indicating the
fallback to Draft 7 and recommending upgrading jsonschema or adding a
compatibility path if new draft features are required.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: be5d45f8-da6e-4f98-8d39-aa55c0ddae22

📥 Commits

Reviewing files that changed from the base of the PR and between 7941200 and de75d26.

⛔ Files ignored due to path filters (1)

• tests/sqlx/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (7)

• .github/workflows/test-eql.yml
• CHANGELOG.md
• docs/reference/schema/eql-payload-v2.2.schema.json
• docs/reference/schema/eql-payload-v2.3.schema.json
• mise.toml
• tests/sqlx/Cargo.toml
• tests/sqlx/tests/payload_schema_tests.rs

[freshtonic]

Nice - it's good to have proper schemas to compare.

I left a comment about upgrading the jsonschema version.