Skip to content

perf: flip eql_v2_encrypted infix operator implementations to inlinable SQL (#193)#196

Merged
coderdan merged 7 commits into
mainfrom
dan/phase-1-operator-inlining
May 11, 2026
Merged

perf: flip eql_v2_encrypted infix operator implementations to inlinable SQL (#193)#196
coderdan merged 7 commits into
mainfrom
dan/phase-1-operator-inlining

Conversation

@coderdan
Copy link
Copy Markdown
Contributor

@coderdan coderdan commented May 6, 2026

Summary

Resolves #193. Implements the v2 payload scheme discipline described in docs/plans/eql-payload-scheme-discipline-rfc.md (forthcoming).

Makes the =, <>, ~~, ~~*, @>, and <@ operator implementations on eql_v2_encrypted eligible for planner inlining. Once inlined, bare queries like WHERE col = val from PostgREST and ORMs that don't wrap columns themselves engage the documented functional indexes (bench_text_hmac_idx, bench_text_bloom_idx, bench_jsonb_stevec_idx) instead of falling back to seq scan.

This fixes cipherstash/stack#420 — encryptedSupabase silent seq-scan — at the EQL layer. No changes are needed in encryptedSupabase itself.

What changed

Operator inlining (commit 1)

src/operators/=.sql, <>.sql, ~~.sql: wrapper functions rewritten from LANGUAGE plpgsql (with SET search_path) to LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE with single-statement bodies of the form extractor(a) op extractor(b).

src/operators/@>.sql, <@.sql: existing LANGUAGE SQL wrappers gain explicit IMMUTABLE STRICT PARALLEL SAFE (previously default-VOLATILE which blocks inlining).

tests/sqlx/tests/lint_tests.rs: tightens the lint test added in #195 with a Phase 1 regression assertion that the targeted operators report zero violations.

v2 payload scheme discipline (commit 2)

The operator inlining surfaced that the v2 payload supports multiple parallel SEM terms for the same query purpose at the root, which is structurally incompatible with index matching. Tightening that contract is what makes the inlining actually engage end-to-end:

  • tasks/pin_search_path.sql: extend the inline-critical allowlist to cover all six Phase 1 operators in all three argument-type permutations ((enc, enc), (enc, jsonb), (jsonb, enc)). Critical: without this, the post-install SET search_path pin re-disables inlining on the wrappers commit 1 made inlinable. This was the missing piece.
  • eql_v2.compare: drop the Blake3 branch from the equality fallback. Equality at the root is hmac-only; ordering branches (ORE, OPE) unchanged. Blake3 still lives inside ste_vec_contains for selector-level element comparison.
  • eql_v2.hash_encrypted: hmac-only, with a clearer error when hm is absent. The previous Blake3-first priority existed to satisfy a hash/equality contract that has no production analogue (root payloads never carry b3 — Blake3 only appears inside sv[] ste_vec elements).
  • eql_v2.ste_vec_contains: switch the inner element comparison to eql_v2.eq, which routes through compare's literal fallback for ste_vec elements lacking hm/ORE/OPE. Documents why neither bare = nor compare_blake3 is appropriate for this internal call site.

Test cleanup (commit 2)

tests/sqlx/migrations/004_install_test_helpers.sql: drop the fictional root-level b3 field from create_encrypted_json. ste_vec inner elements still carry b3 via get_numeric_ste_vec_*.

22 tests removed across 9 files because they exercised payload shapes that have no production analogue:

  • equality_tests: 6 × equality_operator_*_blake3 + 2 × eq_function_*_blake3.
  • inequality_tests: 2 × *_finds_non_matching_records_blake3.
  • index_compare_tests: 3 × blake3_compare_{equal,less_than,greater_than}.
  • hash_operator_tests: 3 × mixed_index_* (the fictional hm+b3 vs b3-only contract) + hash_function_uses_blake3_first, hash_consistency_full_index_matches_blake3_only, hmac_and_blake3_produce_different_hashes, ste_vec_wrapped_hashes_same_as_unwrapped.
  • ope_tests: 4 × OPE-only =/<> (OPE is for ordering; equality requires hmac).
  • ore_equality_tests: 6 × ORE-only =/<> (same reason).
  • ore_text_operator_tests: 6 × ORE-text =/<> variants.
  • operator_compare_tests: 1 × compare_blake3_index (root-level Blake3 compare no longer exists).

Two tests in operator_class_tests.rs rewritten to use hmac literals instead of ob/ore-only literals. add_encrypted_constraint_prevents_invalid_data switched to compare '{}' via the underlying jsonb (since the new = requires hmac, and '{}' has none — intentionally).

Mechanism

For each operator, inlining produces:

WHERE col = val      →  eql_v2.hmac_256(col)      = eql_v2.hmac_256(val)
WHERE col <> val     →  eql_v2.hmac_256(col)     <> eql_v2.hmac_256(val)
WHERE col ~~ val     →  eql_v2.bloom_filter(col) @> eql_v2.bloom_filter(val)
WHERE col @> val     →  eql_v2.ste_vec_contains(col, val)

Functional indexes built on the matching expression engage automatically.

Verification

EXPLAIN SELECT id FROM bench WHERE enc = '...';

 Bitmap Heap Scan on bench
   Recheck Cond: ((eql_v2.hmac_256(enc))::text = 'abc'::text)
   ->  Bitmap Index Scan on bench_hmac_idx
         Index Cond: ((eql_v2.hmac_256(enc))::text = 'abc'::text)

Where the same query previously produced Seq Scan. The planner inlined = through to the wrapped form, matched the functional hash index, picked Bitmap Index Scan.

lint_phase_1_operators_are_clean asserts zero inlinability_* violations on the targeted operators against the installed EQL surface.

Behavioural change

Per the RFC and the issue, = / <> previously dispatched through eql_v2.compare, which fell back to ORE / Blake3 / literal comparison when the column's HMAC variant wasn't present. The new implementations require both sides to carry hm. Calling = on a payload missing hm now raises (Expected a hmac_256 index (hm) value in json: ...) — surfacing the config error rather than silently degrading to seq scan or returning false.

For customers configured correctly (the common case — a unique index gives you hm), this change is purely a perf improvement. The RFC documents the migration-window guidance: during a add_search_config('unique', ...) rollout against a column that already has rows, eql_v2.eq() (which still walks compare) remains as the explicit predicate for callers that need to span the un-backfilled window.

What's NOT in this PR

Downstream effect

Summary by CodeRabbit

  • New Features

    • Equality/inequality and hashing now require an HMAC-256 index term on encrypted values; missing HMAC raises an explicit error.
  • Bug Fixes

    • Restored selector-scoped containment behavior and removed prior Blake3 fallback at the root level for consistent comparisons.
  • Performance

    • Core operator implementations annotated/inlinable to improve planner/index matching and query performance.
  • Documentation

    • Added changelog and an upgrade guide with migration and verification steps.
  • Tests

    • Updated tests to reflect HMAC-only root semantics and Blake3 relocation to selector-level.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 6, 2026

Warning

Rate limit exceeded

@coderdan has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 40 minutes and 55 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b36de6a0-dcf3-4a52-b12c-81848090d2bf

📥 Commits

Reviewing files that changed from the base of the PR and between 38bccc5 and a8d73bf.

📒 Files selected for processing (1)
  • tests/sqlx/tests/containment_tests.rs
📝 Walkthrough

Walkthrough

This PR centralizes root-level equality/hashing on HMAC‑256, rewrites many operator wrappers as inlinable SQL (IMMUTABLE/STRICT/PARALLEL SAFE), updates ste_vec element matching to use selector-aware Blake3 comparisons, and adjusts pinning/CI/test fixtures and docs for the HMAC-only root contract.

Changes

Phase 1 HMAC-256 Root Constraint

Layer / File(s) Summary
Hash and Comparison Contract Documentation
src/encrypted/hash.sql, src/operators/compare.sql, CHANGELOG.md, docs/upgrading/v2.3.md
eql_v2.hash_encrypted() and eql_v2.compare() now use HMAC‑256 at the root; Blake3 is removed from root priority and described as ste_vec-internal only.
Equality and Inequality Operator Implementations
src/operators/=.sql, src/operators/<>.sql
= and <> overloads converted from PL/pgSQL dispatchers to single-statement SQL functions comparing eql_v2.hmac_256(...); jsonb cross-type overloads cast to eql_v2_encrypted.
Pattern and Containment Operator Attributes
src/operators/~~.sql, src/operators/@>.sql, src/operators/<@.sql
~~, @>, <@ functions simplified to SQL wrappers delegating to eql_v2.like/ste_vec_contains and explicitly marked IMMUTABLE STRICT PARALLEL SAFE to preserve inlinability.
Hash Function & ste_vec Element Matching
src/encrypted/hash.sql, src/ste_vec/functions.sql
hash_encrypted() now requires hmac_256 index term and returns hashtext(hmac); ste_vec_contains() enforces selector equality and uses compare_blake3 for element-level Blake3, otherwise falls back to eql_v2.eq.
Operator Inlining Preservation Configuration
tasks/pin_search_path.sql, tasks/test/splinter.sh
Post-install pin_search_path allowlist expanded to skip SET search_path for core operator wrappers; Splinter CI allowlist extended to permit these findings.
Test Helpers & Fixture Updates
tests/sqlx/migrations/004_install_test_helpers.sql, tests/sqlx/tests/constraint_tests.rs
create_encrypted_json() no longer emits root-level b3; constraint cleanup now deletes invalid rows using e::jsonb = '{}'::jsonb.
Containment Regression Tests
tests/sqlx/tests/containment_tests.rs
Added deterministic ste_vec payload builder and two tests validating selector-scoped Blake3 containment matching and non-matching behavior.
Test Removals & Updates
multiple tests/sqlx/tests/*
Removed root-level Blake3 equality/inequality/hash/compare and mixed-index regression tests; updated comments and tightened hash_encrypted error message expectations to require hmac_256.
Operator-Class & Index Plan Tests
tests/sqlx/tests/operator_class_tests.rs
Updated tests to use HMAC-shaped literals for equality EXPLAIN checks; removed ORE/bloom equality assertions while retaining plan-shape assertions.
Lint Test Enforcement (Phase 1)
tests/sqlx/tests/lint_tests.rs
Replaced prior lint smoke-test with a strict zero-violation test filtering Phase 1 operator-name prefixes (=, <>, ~~, ~~*, @>, <@).
Docs & Changelog
CHANGELOG.md, CLAUDE.md, docs/upgrading/v2.3.md
Added release notes, changelog, release discipline, and upgrade guidance documenting the HMAC-only root contract and functional-index guidance.

Sequence Diagram

sequenceDiagram
  participant Client
  participant OperatorFunc as "Operator SQL Wrapper"
  participant HMAC as "eql_v2.hmac_256 / hash_encrypted"
  participant SteVec as "ste_vec_contains / compare_blake3"
  Client->>OperatorFunc: evaluates expression (e.g., a = b / hash)
  OperatorFunc->>HMAC: extract/compare hmac_256(a), hmac_256(b)
  alt both hmac present
    HMAC-->>OperatorFunc: boolean / hashtext
  else
    OperatorFunc->>SteVec: selector-aware element checks (Blake3 when present)
    SteVec-->>OperatorFunc: match/no-match
  end
  OperatorFunc-->>Client: result or error
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers

  • tobyhede
  • freshtonic

Poem

🐰 Hop through hashes, a careful code ballet,
HMAC leads the root — Blake3 shelved away.
SQL inlines neatly for planner delight,
Tests and docs updated to match the light,
A rabbit cheers: "One path, clear as day!"

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately and concisely summarizes the main change: converting eql_v2_encrypted infix operator implementations to inlinable SQL for performance optimization.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dan/phase-1-operator-inlining

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderdan
Copy link
Copy Markdown
Contributor Author

coderdan commented May 7, 2026

Pushed 1ab0b00 (force-push because I rebased onto main first to pull in the splinter allowlist mechanism that landed in #190).

The 11 operator overloads this PR makes inlinable — 3× =, 3× <>, 3× ~~, plus eql_v2.like / eql_v2.ilike — would have been re-pinned at install time by the post-install search_path enforcer from #190, killing inlining and quietly reverting your hmac_256/bloom_filter index matching to seq scans. The new commit:

  • Extends tasks/pin_search_path.sql so the pinner skips these 11 OIDs (matched by proname + proargtypes against pre-resolved eql_v2_encrypted and jsonb oids).
  • Adds 5 new entries to the tasks/test/splinter.sh allowlist with per-rule justifications referencing which index each one needs to match (hmac_256 for =/<>, bloom_filter for ~~/like/ilike).

Verified the 11 functions retain proconfig=NULL after install, and splinter is clean against a fresh install (raw=22, allowlisted=22, unallowlisted=0).

I haven't touched the existing test failures (add_encrypted_constraint_prevents_invalid_data and the 6 containment_tests) — those reproduce on the PR branch without my commit and look like a separate semantic question about how the new operators handle encrypted values that lack the relevant index field (eql_v2.hmac_256({}) raises). Worth flagging for the PR description's "Behaviour change" note — leaving that call to you.

Filed #198 to track the broader story of why downstream Supabase users will still see these flagged in their Security Advisor until EQL can ship as a real CREATE EXTENSION package.

coderdan added 2 commits May 7, 2026 17:14
…le SQL (#193)

Resolves cipherstash/stack#420 (encryptedSupabase silent seq-scan) by
making the `=`, `<>`, `~~`, `~~*`, `@>`, and `<@` operator
implementations on `eql_v2_encrypted` eligible for planner inlining.

The Postgres planner inlines a custom operator's implementation function
during index matching, provided the function is `LANGUAGE sql IMMUTABLE`
with a single-statement body and no `SET` clause. Previously every
operator wrapper was either `LANGUAGE plpgsql` (which can never be
inlined) or had `SET search_path = pg_catalog, extensions, public`
(which blocks inlining even on sql functions). As a result, bare
queries like `WHERE col = val` from PostgREST and ORMs that don't wrap
columns themselves silently fell back to seq scan on every install
where the catch-all `eql_v2.encrypted_operator_class` btree wasn't
available — most prominently Supabase.

This change rewrites the wrappers as inlinable SQL, so the planner
reduces them during planning and matches functional indexes built on
the underlying extractors:

  WHERE col = val            inlines to    eql_v2.hmac_256(col)         = eql_v2.hmac_256(val)
                                            └── matches functional hash idx
  WHERE col ~~ val           inlines to    eql_v2.bloom_filter(col)    @> eql_v2.bloom_filter(val)
                                            └── matches functional GIN idx
  WHERE col @> val           inlines to    eql_v2.ste_vec_contains(col, val) (preserves)

Verified empirically: bare `WHERE enc = ...` produces a Bitmap Index
Scan on the hmac functional index, where it previously seq-scanned.

The lint introduced in the parent commit (#194) goes from 12+
inlinability errors on these operators to zero. Comparison operators
(<, <=, >, >=), JSONB extractors (->, ->>), and ORE block operators
remain flagged — those are out of scope for Phase 1 and tracked
separately in the predicate/extractor RFC.

Behaviour change: `=` and `<>` previously dispatched through
`eql_v2.compare`, which fell back to ORE / Blake3 / literal comparison
when the column's HMAC variant wasn't present. The new implementations
require the column to have `equality` configured (i.e. carry an `hm`
field). Calling `=` on an ORE-only encrypted column now returns NULL
instead of a Boolean — surfacing the config error rather than hiding
it. This is intentional per the RFC (`docs/plans/uniform-predicate-extractor-pairs-rfc.md`).
…pose

Implements the v2 payload scheme discipline that PR #193 began. Tightens
operator-class scaffolding so functional indexes engage uniformly across
self-hosted and Supabase, removes root-level Blake3 dead code, and drops
test coverage of fictional payload shapes.

SQL changes
-----------

- `eql_v2.compare`: drop the Blake3 branch from the equality fallback.
  Equality at the root is hmac-only; ordering branches (ORE, OPE)
  unchanged. Blake3 is preserved for ste_vec internal element comparison.
- `eql_v2.hash_encrypted`: hmac-only. The previous Blake3-first priority
  existed to satisfy a hash/equality contract that has no production
  analogue (root payloads never carry b3). Hashing now requires a `unique`
  index and raises clearly if absent.
- `eql_v2.ste_vec_contains`: switch the inner element comparison to
  `eql_v2.eq` (which routes through compare's literal fallback for
  ste_vec elements lacking hm/ORE/OPE). Documents why neither bare `=`
  nor `compare_blake3` is appropriate here.
- `tasks/pin_search_path.sql`: extend the inline-critical allowlist to
  cover the operators rewritten in #193 — `=`, `<>`, `~~`, `~~*` for
  same-type and both cross-type (encrypted, jsonb) directions. Without
  this, the post-install search_path pin re-disables inlining on the
  very wrappers #193 made inlinable.

Test changes
------------

- `tests/sqlx/migrations/004_install_test_helpers.sql`: drop the
  fictional root-level `b3` field from `create_encrypted_json`. ste_vec
  inner elements still carry b3 via `get_numeric_ste_vec_*`.
- `equality_tests`: remove `equality_operator_*_blake3` and
  `eq_function_*_blake3` (6 + 2 tests). Tested root-level Blake3
  fallthrough that no longer exists.
- `inequality_tests`: remove `*_finds_non_matching_records_blake3`.
- `index_compare_tests`: remove `blake3_compare_{equal,less,greater}`.
- `hash_operator_tests`: remove `mixed_index_*` (3 tests, fictional
  hm+b3 vs b3-only contract), `hash_function_uses_blake3_first`,
  `hash_consistency_full_index_matches_blake3_only`,
  `hmac_and_blake3_produce_different_hashes`,
  `ste_vec_wrapped_hashes_same_as_unwrapped` (Blake3-first hash priority
  contract). Update remaining error-message assertions to reference
  `hmac_256` only.
- `ope_tests`: remove `encrypted_{eq,neq}_operator_uses_op{f,v}` —
  OPE-only payloads no longer support `=`/`<>` (no hm at root). Range
  operators on OPE remain.
- `ore_equality_tests`: remove `ore64`/`ore_cllw_*` `eq`/`neq` tests
  for the same reason.
- `ore_text_operator_tests`: remove ORE-text `=`/`<>` variants
  (same-type and both cross-type directions).
- `operator_class_tests`: rewrite the affected EXPLAIN-shape assertions
  to use hmac literals instead of ob/ore-only literals.
- `operator_compare_tests`: remove `compare_blake3_index` (root-level
  compare via Blake3 no longer exists).
- `constraint_tests`: switch `add_encrypted_constraint_prevents_invalid_data`
  to compare via the underlying jsonb (the `'{}'` literal has no `hm`).

See `docs/plans/eql-payload-scheme-discipline-rfc.md` for the full
rationale and v3 forward path.
@coderdan coderdan force-pushed the dan/phase-1-operator-inlining branch from a5229a6 to 8e80a44 Compare May 8, 2026 03:10
The post-install pin_search_path step intentionally skips =, <>, ~~,
@>, <@ and the jsonb_contains/_by wrappers — they must inline for
the planner to match the documented functional indexes. Splinter's
function_search_path_mutable lint flags those same functions because
SET search_path is precisely the thing inlining forbids. Both lists
must agree, or splinter blocks CI on the operators we deliberately
left unpinned.

Adds eql_v2.=, eql_v2.<>, eql_v2.~~ to the splinter allowlist with
the same Phase 1 inlining justification used by the inline-critical
allowlist in tasks/pin_search_path.sql. Note that the eql_v2.~~*
operator routes to the eql_v2.~~ function (case-insensitivity is
meaningless on encrypted ciphertexts) so a single allowlist entry
covers both LIKE variants.

Verified locally: raw=20 findings, allowlisted=20, unallowlisted=0.
@coderdan coderdan marked this pull request as ready for review May 9, 2026 09:32
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
tests/sqlx/tests/ore_equality_tests.rs (1)

1-5: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Module docstring is stale — file no longer exercises equality.

The //! header still describes this as "ORE equality/inequality operator tests" and "Tests equality with different ORE encryption schemes", but every = / <> test is gone (line 10–16 explains why) and only < / <= / > / >= cases remain. The filename is also now misleading; at minimum the doc block should be updated, and you may want to either rename the file (e.g., ore_comparison_tests.rs) or fold the remaining tests into an existing comparison module to avoid future confusion.

📝 Suggested header update
-//! ORE equality/inequality operator tests
+//! ORE comparison operator tests
 //!
-//! Tests equality with different ORE encryption schemes (ORE64, CLLW_U64_8, CLLW_VAR_8)
+//! Tests `<`, `<=`, `>`, `>=` against different ORE encryption schemes
+//! (ORE64, CLLW_U64_8, CLLW_VAR_8). Equality / inequality (`=`, `<>`) require
+//! an HMAC root term and are not applicable to ORE-only fixtures.
 //! Uses ore table from migrations/002_install_ore_data.sql (ids 1-1000)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/ore_equality_tests.rs` around lines 1 - 5, Update the stale
module docstring in ore_equality_tests.rs to describe that the file contains ORE
comparison (ordering) tests for <, <=, >, >= across the ORE schemes (ORE64,
CLLW_U64_8, CLLW_VAR_8) and uses the ore table from
migrations/002_install_ore_data.sql; either rename the file to
ore_comparison_tests.rs or move these tests into an existing comparison module
to match the new behavior and avoid future confusion.
tests/sqlx/tests/index_compare_tests.rs (1)

19-32: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Macro doc references the now-stale 'b3' example.

The assert_compare! doc block uses create_encrypted_json(1, 'b3') to motivate the format!-over-bind decision, but b3 is no longer emitted by create_encrypted_json, and the tests that exercised that shape were removed (lines 38–43). Recommend updating the example to a still-valid filter such as 'hm' so future readers don't follow the stale pattern.

📝 Suggested doc tweak
 // Helper macro to reduce repetition for compare tests
 //
 // Note: Uses format! for SQL construction because test data expressions
-// (like "create_encrypted_json(1, 'b3')") must be evaluated by PostgreSQL,
+// (like "create_encrypted_json(1, 'hm')") must be evaluated by PostgreSQL,
 // not passed as parameters. SQLx cannot pass PostgreSQL function calls as
 // query parameters - they must be part of the SQL string.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/index_compare_tests.rs` around lines 19 - 32, The macro doc
for assert_compare! references a stale example create_encrypted_json(1, 'b3');
update that example to use a currently emitted filter value (e.g., replace 'b3'
with 'hm') so the rationale for using format! over parameter binding stays
correct; locate the macro named assert_compare! and change the quoted literal in
its comment from 'b3' to 'hm' (or another valid current example) and run the
tests to ensure the comment-only change compiles cleanly.
tests/sqlx/tests/inequality_tests.rs (1)

173-189: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Stale _blake3 jsonb inequality test now duplicates the _hmac variant.

inequality_operator_encrypted_not_equals_jsonb_blake3 builds the operand with create_encrypted_json(1)::jsonb - 'ob' and asserts count(2) — identical setup and assertion to inequality_operator_encrypted_not_equals_jsonb_hmac at lines 112–128. Since create_encrypted_json no longer emits a root b3 term, this case is no longer Blake3-specific and the name is misleading. Either drop it (consistent with the encrypted/encrypted Blake3 removals immediately above) or rename it to make the distinct intent explicit.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/inequality_tests.rs` around lines 173 - 189, The test
inequality_operator_encrypted_not_equals_jsonb_blake3 duplicates the hmac
variant and is no longer Blake3-specific because create_encrypted_json no longer
emits a root b3 term; either remove this redundant test or rename it to reflect
its generic intent (e.g., inequality_operator_encrypted_not_equals_jsonb) and
update any references; locate the test function
inequality_operator_encrypted_not_equals_jsonb_blake3 and either delete the
entire function or rename it and adjust its test attributes/fixture names
accordingly so it no longer claims Blake3-specific behavior.
🧹 Nitpick comments (2)
tests/sqlx/tests/ore_text_operator_tests.rs (1)

13-20: 💤 Low value

Empty section banners now wrap only removal notes.

The "Equality and inequality operators", "JSONB variants: e = jsonb, e <> jsonb", and "JSONB variants: jsonb = e, jsonb <> e" banners (lines 13–15, 238–240, 246–248) no longer have any tests under them — just the removal comments. Consider either deleting the banners (folding the rationale into the file header alongside the line-3 docstring, which still mentions "equality") or collapsing the three removal notes into a single explanation at the top, so the section structure of the file matches what it actually exercises.

Also applies to: 238-252

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/ore_text_operator_tests.rs` around lines 13 - 20, The test
file contains empty section headers ("Equality and inequality operators", "JSONB
variants: e = jsonb, e <> jsonb", "JSONB variants: jsonb = e, jsonb <> e") that
only hold removal notes for tests like ore_text_equality_operator_finds_match
and related JSONB variants; remove these empty banners or consolidate their
removal notes into a single explanatory paragraph near the file header/docstring
(which currently mentions "equality") so the file’s section structure matches
the actual tests; update or delete the three banner comments and move any
remaining rationale into one top-level comment adjacent to the existing
docstring to keep the file organized.
tests/sqlx/tests/equality_tests.rs (1)

85-91: ⚡ Quick win

Add one regression test for the new “missing hm raises” contract.

The comment explains the behavior change, but the file still doesn’t lock it in. Please add a direct assertion that bare = fails when either side lacks hm, so a future fallback-to-scan regression can’t slip back in unnoticed.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/sqlx/tests/equality_tests.rs` around lines 85 - 91, Add a regression
test in tests/sqlx/tests/equality_tests.rs (e.g., fn
bare_eq_requires_hm_regression_test) that issues the bare `=` equality query
cases and asserts they fail when either operand lacks an `hm` index term: 1) a
query where left side has no `hm` and right side does, and 2) a query where
right side has no `hm` and left side does; for each case call the same execution
path used by the existing equality tests (so it exercises the inlinable operator
code path) and assert the result is an Err (or matches the expected error
kind/message) rather than succeeding, preventing fallback-to-scan regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/operators/compare.sql`:
- Around line 46-49: Update the Doxygen note about Blake3 in the comment so it
reflects the new call path: replace the statement that "ste_vec_contains calls
eql_v2.compare_blake3 directly" with wording that ste_vec_contains now invokes
eql_v2.eq(...) which may in turn call eql_v2.compare_blake3 as a fallback, and
clarify that Blake3 remains scoped to ste_vec elements and root-level equality
is still HMAC; update references to the functions ste_vec_contains, eql_v2.eq,
and eql_v2.compare_blake3 accordingly.

In `@src/ste_vec/functions.sql`:
- Around line 492-501: The current OR clause uses eql_v2.eq(_a, b) directly,
which lets selector-matched STEs that only carry Blake3 drop to literal
comparison; restore explicit Blake3 matching before the eq fallback by changing
the clause so that when eql_v2.selector(_a) = eql_v2.selector(b) you first check
if both sides carry a b3 (e.g., eql_v2.has_b3(_a) AND eql_v2.has_b3(b)) and, if
so, use eql_v2.compare_blake3(_a, b) = 0 as the match; otherwise fall through to
eql_v2.eq(_a, b). Update the expression around result := result OR (...) to
perform this conditional: selector-equals → (both-have-b3 AND
compare_blake3(...) = 0) OR eql_v2.eq(...).

---

Outside diff comments:
In `@tests/sqlx/tests/index_compare_tests.rs`:
- Around line 19-32: The macro doc for assert_compare! references a stale
example create_encrypted_json(1, 'b3'); update that example to use a currently
emitted filter value (e.g., replace 'b3' with 'hm') so the rationale for using
format! over parameter binding stays correct; locate the macro named
assert_compare! and change the quoted literal in its comment from 'b3' to 'hm'
(or another valid current example) and run the tests to ensure the comment-only
change compiles cleanly.

In `@tests/sqlx/tests/inequality_tests.rs`:
- Around line 173-189: The test
inequality_operator_encrypted_not_equals_jsonb_blake3 duplicates the hmac
variant and is no longer Blake3-specific because create_encrypted_json no longer
emits a root b3 term; either remove this redundant test or rename it to reflect
its generic intent (e.g., inequality_operator_encrypted_not_equals_jsonb) and
update any references; locate the test function
inequality_operator_encrypted_not_equals_jsonb_blake3 and either delete the
entire function or rename it and adjust its test attributes/fixture names
accordingly so it no longer claims Blake3-specific behavior.

In `@tests/sqlx/tests/ore_equality_tests.rs`:
- Around line 1-5: Update the stale module docstring in ore_equality_tests.rs to
describe that the file contains ORE comparison (ordering) tests for <, <=, >, >=
across the ORE schemes (ORE64, CLLW_U64_8, CLLW_VAR_8) and uses the ore table
from migrations/002_install_ore_data.sql; either rename the file to
ore_comparison_tests.rs or move these tests into an existing comparison module
to match the new behavior and avoid future confusion.

---

Nitpick comments:
In `@tests/sqlx/tests/equality_tests.rs`:
- Around line 85-91: Add a regression test in tests/sqlx/tests/equality_tests.rs
(e.g., fn bare_eq_requires_hm_regression_test) that issues the bare `=` equality
query cases and asserts they fail when either operand lacks an `hm` index term:
1) a query where left side has no `hm` and right side does, and 2) a query where
right side has no `hm` and left side does; for each case call the same execution
path used by the existing equality tests (so it exercises the inlinable operator
code path) and assert the result is an Err (or matches the expected error
kind/message) rather than succeeding, preventing fallback-to-scan regressions.

In `@tests/sqlx/tests/ore_text_operator_tests.rs`:
- Around line 13-20: The test file contains empty section headers ("Equality and
inequality operators", "JSONB variants: e = jsonb, e <> jsonb", "JSONB variants:
jsonb = e, jsonb <> e") that only hold removal notes for tests like
ore_text_equality_operator_finds_match and related JSONB variants; remove these
empty banners or consolidate their removal notes into a single explanatory
paragraph near the file header/docstring (which currently mentions "equality")
so the file’s section structure matches the actual tests; update or delete the
three banner comments and move any remaining rationale into one top-level
comment adjacent to the existing docstring to keep the file organized.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: de2b952e-c56a-4cb2-bae1-7372cae8c504

📥 Commits

Reviewing files that changed from the base of the PR and between 998a015 and 27e81b8.

📒 Files selected for processing (22)
  • src/encrypted/hash.sql
  • src/operators/<>.sql
  • src/operators/<@.sql
  • src/operators/=.sql
  • src/operators/@>.sql
  • src/operators/compare.sql
  • src/operators/~~.sql
  • src/ste_vec/functions.sql
  • tasks/pin_search_path.sql
  • tasks/test/splinter.sh
  • tests/sqlx/migrations/004_install_test_helpers.sql
  • tests/sqlx/tests/constraint_tests.rs
  • tests/sqlx/tests/equality_tests.rs
  • tests/sqlx/tests/hash_operator_tests.rs
  • tests/sqlx/tests/index_compare_tests.rs
  • tests/sqlx/tests/inequality_tests.rs
  • tests/sqlx/tests/lint_tests.rs
  • tests/sqlx/tests/ope_tests.rs
  • tests/sqlx/tests/operator_class_tests.rs
  • tests/sqlx/tests/operator_compare_tests.rs
  • tests/sqlx/tests/ore_equality_tests.rs
  • tests/sqlx/tests/ore_text_operator_tests.rs

Comment thread src/operators/compare.sql Outdated
Comment thread src/ste_vec/functions.sql Outdated
coderdan added 2 commits May 9, 2026 19:42
Introduce a Keep-a-Changelog-style CHANGELOG.md and a Postgres-style
per-version upgrade guide under docs/upgrading/.

The 2.3.0 entry covers the changes from PR #193 (operator inlining) and
PR #196 (payload scheme discipline): the indexing recipe shift from
operator-class to functional indexes, the hmac requirement for equality
and hashing, and the formalisation of Blake3 as ste_vec-internal.

This is a minor release: no public API change. The eql_v2 schema name,
function signatures, operators, and payload format are all unchanged.
Add a "Release & changelog discipline" section to CLAUDE.md so the
conventions introduced in 41e2925 (CHANGELOG.md + docs/upgrading/v2.3.md)
are picked up by AI-assisted contributions without needing a separate
CI gate.

Covers: when to add a [Unreleased] entry, when to add a numbered upgrade
note (U-NNN), how to think about semver in light of the eql_v2 schema
name being public API, and the steps for cutting a release.
Comment thread src/operators/compare.sql
@freshtonic freshtonic self-requested a review May 11, 2026 02:35
Copy link
Copy Markdown
Contributor

@freshtonic freshtonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderdan gave me a detailed walk-though. My approval is on the condition that the findings in the comments are addressed.

After the root-level Blake3 removal in this PR, `ste_vec_contains`
delegated element matching to `eql_v2.eq`, which routes through
`compare`'s ORE → OPE → hmac chain and falls through to
`compare_literal` (raw JSONB byte comparison) for elements that only
carry `b3`. This broke `@>` / `<@` for the realistic case where a
freshly-built query payload has the same plaintext as a stored row but
a different `c` (ciphertext) field — same `b3`, different bytes, no
match.

Restored explicit Blake3 matching with a `has_blake3(_a) AND
has_blake3(b)` guard around `compare_blake3` (its NULL-safe
"both-NULL ⇒ 0" return would otherwise conflate distinct OPE-only
elements). `eql_v2.eq` is retained as the fallback for the OPE-only /
future-index case.

The existing containment tests didn't catch this — they extract query
terms straight from the database, so byte equality is automatic.
The new tests in `containment_tests.rs` build the query payload by
hand:
- `contains_matches_b3_with_different_ciphertext`: positive case,
  asserts `@>` matches when `b3` matches but `c` differs.
- `contains_does_not_match_different_b3`: negative case, asserts the
  guard does not produce a false positive when `b3` differs.

Verified by patching a fresh DB to the buggy form (returns 0) and the
fixed form (returns 1) for the same query.

Also updated the `eql_v2.compare` Doxygen header per Dan's review note
to flag that the function is `LANGUAGE plpgsql` and intentionally not
inlinable, kept as the canonical multi-branch ordering helper for
back-compat callers (ORE btree opclass, `eql_v2.eq`); hot-path equality
goes through `eql_v2."="` directly post-#193. Refreshed the Blake3 note
in the same header to reflect the new `ste_vec_contains` call path.

Addresses CodeRabbit review on PR #196.
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/sqlx/tests/containment_tests.rs`:
- Around line 194-195: Update the misleading comment in containment_tests.rs to
distinguish selector-scoped b3 values from the removed “root b3” concept:
mention that the seed fixture (changed by 004_install_test_helpers.sql) no
longer sets a root-level b3, and that the B3_HELLO value lives inside elements
of the ste_vec array at the $.hello selector; explain that other_b3 (all-zero
bytes) cannot collide with those selector-scoped Blake3 hashes, which is why
count(0) remains true. Refer to the variables/identifiers B3_HELLO, other_b3,
ste_vec and the $.hello selector in the updated comment so readers can locate
the relevant test assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 35d86616-92e1-4e1a-a141-4e70c3c9a488

📥 Commits

Reviewing files that changed from the base of the PR and between 1e912a0 and 38bccc5.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • src/operators/compare.sql
  • src/ste_vec/functions.sql
  • tests/sqlx/tests/containment_tests.rs
✅ Files skipped from review due to trivial changes (1)
  • CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/operators/compare.sql

Comment thread tests/sqlx/tests/containment_tests.rs
@coderdan coderdan merged commit 4064c60 into main May 11, 2026
6 checks passed
@coderdan coderdan deleted the dan/phase-1-operator-inlining branch May 11, 2026 03:16
coderdan added a commit that referenced this pull request May 11, 2026
…ypted columns

Adds bench coverage for the three hash-strategy access patterns the Phase 1
hash operator class (#196) enabled but the Phase 2 chain inlining (#202) is
yet to make perf-competitive.

Plan assertions in bench_plan_tests.rs (gated on the bench feature, pass
today):

- group_by_encrypted_uses_hash_aggregate — confirms `GROUP BY encrypted_col`
  picks HashAggregate via the hash op class rather than degenerating to
  GroupAggregate-after-Sort or a Nested-Loop self-comparison.
- join_on_encrypted_uses_hmac_index — confirms self-join on encrypted
  equality engages bench_text_hmac_idx (Hash Join or Nested Loop + Memoize +
  Index Scan, both acceptable).
- distinct_encrypted_uses_hash_aggregate — confirms unbounded DISTINCT picks
  HashAggregate (the bounded-LIMIT variant biases toward IndexOnlyScan over
  the ORE btree opclass; that path is fine on full installs but unavailable
  on Supabase).

Regression timing assertions in bench_regression_tests.rs (#[ignore]'d
pending #202; remove the markers when it merges):

- group_by_encrypted_under_threshold — 150ms (current ~309ms via plpgsql
  hash chain, ~70ms with chain inlined; threshold ~2x the inlined target).
- self_join_encrypted_under_threshold — 350ms (current ~308ms, ~182ms with
  chain inlined; cardinality dominates so threshold is generous).
- distinct_encrypted_under_threshold — 200ms (current ~515ms unbounded via
  ORE btree path, expected to drop into HashAggregate-driven territory
  after chain inlining).

Each timing test panic message states the expected post-#202 number and
the current observed number, so the diagnostic remains useful when the
gate flips after the fix lands.
@coderdan coderdan mentioned this pull request May 13, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants