Skip to content

fix: rich-API (findAll/replaceAll/split) on annotation-processor matchers - #136

Merged
jbachorik merged 2 commits into
mainfrom
fix/apt-findlongestmatchend-missing
Sep 29, 2026
Merged

jbachorik merged 2 commits into
mainfrom
fix/apt-findlongestmatchend-missing

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

What does this PR do?

Two fixes to the rich match-result API (findAll/replaceAll/split) on generated matchers:

  1. APT dispatcher emits the findLongestMatchEnd helper where it is called. The annotation-processor dispatcher emitted findMatchFrom/findBoundsFrom for the DFA-unrolled and generic-NFA strategies whose bodies call the findLongestMatchEnd helper — without also emitting that helper. Every rich-API use on such matchers threw NoSuchMethodError at runtime. The dispatcher now mirrors RuntimeCompiler, which already emits the helper for the same strategies (dual-path rule).

  2. Shared NFA findBoundsFrom no longer uses findLongestMatchEnd for lookaround patterns. The greedy helper never evaluates lookaround assertions, so for assertion-bearing patterns it never reached an accepting state and returned -1 — replaceAll/split silently returned the input unchanged (a wrong-result bug, not a crash, and it affected the runtime path too, not just APT). For assertion-bearing NFAs the bounds now derive from findMatchFrom, which evaluates assertions correctly. Assertion-free patterns keep the zero-allocation O(n) greedy path unchanged.

Motivation

Found by the differential JDK-parity gate of the profiling-backend regex adoption wave 2 (capture/replace cohort): 8 of 21 pilot sites crashed or silently no-op'd on replaceAll/split when compiled through the annotation processor.

Related Issue(s)

None filed yet (regression present in 0.5.0; discovered during backend adoption).

Change Type

  • Bug fix
  • Test improvement

Checklist

  • I have read the CONTRIBUTING.md guidelines
  • All existing tests pass (./gradlew build)
  • I have added tests for my changes
  • My commits are signed

Performance Impact

Assertion-free patterns: unchanged (identical bytecode paths; the new emitter only engages when the NFA contains lookaround assertions). Lookaround-bearing patterns on the rich API: the bounds path now pays a per-occurrence findMatchFrom scan (O(n) matchBounded attempts, one MatchResult each) instead of the broken greedy helper — correctness over speed for a rare pattern class; noted in code comments.

Note: the helper emission was deliberately removed from the hybrid and OPTIMIZED_NFA_WITH_LOOKAHEAD arms (assertion-bearing by definition — the helper would be unreferenced dead bytecode); it is kept in the OPTIMIZED_NFA arm as defensive runtime parity (unreachable through APT refusal guards today, documented inline).

Additional Notes

Differential test (RichApiDifferentialTest): 8 patterns x 11 inputs, JDK parity for replaceAll/split/findAll across the reachable APT strategies. ^(.+?)\.([^/]+) is intentionally excluded — it routes to the PIKEVM_CAPTURE fallback whose lazy-group divergence is fixed separately on fix/pikevm-lazy-group-priority; it is added back there.

An external model review (gpt-5.6-terra) of this diff returned approve-with-comments; all medium findings were addressed (test oracle, dead emissions, cost documentation, allocator discipline).

The APT dispatcher emitted findMatchFrom/findBoundsFrom for the DFA-unrolled
and generic-NFA strategies whose bodies call the findLongestMatchEnd helper
without emitting it - every rich-API use threw NoSuchMethodError (RuntimeCompiler
already emits the helper for the same strategies).

Also: shared NFA findBoundsFrom no longer uses findLongestMatchEnd for
lookaround patterns - the greedy helper never evaluates assertions and
reported no-match, silently making replaceAll/split no-ops on both paths;
bounds now derive from the assertion-correct findMatchFrom instead.
@jbachorik jbachorik added the AI Generated or assisted by AI label Sep 29, 2026
@jbachorik
jbachorik marked this pull request as ready for review September 29, 2026 14:42
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T14:48:31.837159Z ee3ea39 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codecov-commenter

codecov-commenter commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.91304% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 84.9%. Comparing base (a1c0581) to head (e41a05b).

Files with missing lines Patch % Lines
...ggie/processor/ReggieMatcherBytecodeGenerator.java 50.0% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##              main    #136     +/-   ##
=========================================
- Coverage     85.4%   84.9%   -0.6%     
  Complexity       1       1             
=========================================
  Files          163     163             
  Lines        48759   48850     +91     
  Branches      6970    6972      +2     
=========================================
- Hits         41645   41476    -169     
- Misses        5105    5368    +263     
+ Partials      2009    2006      -3     
Files with missing lines Coverage Δ
.../codegen/codegen/DFAUnrolledBytecodeGenerator.java 90.3% <100.0%> (-5.4%) ⬇️
...q/reggie/codegen/codegen/NFABytecodeGenerator.java 74.0% <100.0%> (-2.7%) ⬇️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java 85.9% <ø> (-0.1%) ⬇️
...ggie/processor/ReggieMatcherBytecodeGenerator.java 74.2% <50.0%> (+2.3%) ⬆️

... and 3 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1c0581...e41a05b. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee3ea3979f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

Lookaround bounds now work, but competing match lengths still violate Java's leftmost-first semantics because the new path reuses a longest-end search.

Open Bits AI session

🤖 Bits Code Review · Commit ee3ea39 · @DataDog review to ask questions

The DFA-unrolled findBoundsFrom inline greedy scan aborts the whole scan on
the first failed lookahead instead of skipping the accepting-position record
and continuing (a+(?=b) on "aab": replaceAll("_") returned the input
unchanged while findAll was correct), and it ignores acceptance anchor
conditions. Route assertion-bearing DFAs to findLongestMatchEnd, whose greedy
state code gates the record on lookaheads (skipRecord) and anchors - the same
end search findMatchFrom uses - so replaceAll/split bounds match findAll.

Also document that the NFA bounds delegation inherits findMatchFrom's
longest-end search semantics: leftmost-first alternation patterns diverge on
the rich API exactly like findAll does (pre-existing, needs priority-aware
matching or an analyzer-level refusal).
@jbachorik
jbachorik merged commit 7962a07 into main Sep 29, 2026
9 checks passed
@jbachorik
jbachorik deleted the fix/apt-findlongestmatchend-missing branch September 29, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated or assisted by AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants