Skip to content

fix(#41,#42,#74): atomic groups, possessive quantifiers, first-byte skip - #92

Merged
jbachorik merged 41 commits into
mainfrom
fix/atomic-groups-possessive-quant-skip
Jul 1, 2026
Merged

jbachorik merged 41 commits into
mainfrom
fix/atomic-groups-possessive-quant-skip

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

What does this PR do?

Implements three features from the open-issues design doc:

  1. Parse-and-reject for atomic groups and possessive quantifiers (closes [feature] Atomic groups not supported ((?>...)) #41, [feature] Possessive quantifiers not supported (*+, ++, ?+, {n,m}+) #42): (?>...) and possessive quantifiers (X*+, X++, X?+, X{n,m}+) previously silently returned wrong results. They now throw UnsupportedPatternException immediately at parse time.

  2. Native atomic group semantics (closes [feature] Atomic groups not supported ((?>...)) #41, [feature] Possessive quantifiers not supported (*+, ++, ?+, {n,m}+) #42): Full implementation of atomic groups via PikeVM thread pruning — AST field GroupNode.atomic, NFA atomicEntry/atomicExit state markers in ThompsonBuilder, and sibling-thread kill logic in PikeVMMatcher when a thread crosses an atomicExit boundary. Possessive quantifiers are desugared to atomic groups at parse time. Patterns with atomic groups are routed to PIKEVM_CAPTURE via a new PatternAnalyzer gate.

  3. First-byte skip optimisation for startsAnywhere patterns (closes perf: first-byte skip / literal-suffix acceleration for BITPARALLEL_GLUSHKOV find() #74): GlushkovAutomaton.findLastRequiredChar() identifies a single ASCII character that must appear at the last position of every match. BitParallelGlushkovRuntime.findFromWithSkip() uses String.indexOf to jump past non-candidate regions, then scans from max(start, reqPos - MAX_POSITIONS).

Motivation

Issues #41 and #42 were correctness violations: Reggie was silently accepting (?>...) and *+ syntax and returning wrong results (false positives), violating the correctness guarantee. Issue #74 is a performance enhancement that avoids O(n) scanning over regions that cannot contain a match.

Related Issue(s)

Fixes #41, Fixes #42, Implements #74

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactoring (no functional change)
  • Documentation
  • Test improvement
  • Build/CI change

Checklist

  • I have read the CONTRIBUTING.md guidelines
  • All existing tests pass (./gradlew build)
  • I have added tests for my changes
  • I have updated documentation (if applicable)
  • My commits are signed

Performance Impact

The findFromWithSkip optimisation applies only to BITPARALLEL_GLUSHKOV patterns where startsAnywhere=true and a single required last character exists. For such patterns, find() skips over regions guaranteed to contain no match endpoint, reducing average-case scan cost. No regression on patterns where the optimisation does not apply (falls back to findFrom).

Additional Notes

  • scratchAtomicPos was simplified from a depth-indexed 2-D array to a single shared int[] during validation — this is correct because addThread is called sequentially (single-threaded DFS), so the scratch array is never accessed concurrently.
  • The StructuralHash was updated to mix in hasAtomicGroups to prevent structural cache collisions between atomic and non-atomic variants of the same pattern.
  • Fuzz gate: 34 divergences maintained (no new divergences introduced). 2,578 tests pass.

jbachorik and others added 30 commits July 1, 2026 00:54
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add BitParallelGlushkovBytecodeGeneratorTest verifying that findFromWithSkip
is emitted for .*a (single required last char) and findFrom for .*[abc]
(multi-char class, no required char). Remove the startsAnywhere guard from
the constructor so lastRequiredChar is always computed unconditionally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik added the AI Generated or assisted by AI label Jul 1, 2026
@jbachorik
jbachorik marked this pull request as ready for review July 1, 2026 12:05
@datadog-prod-us1-4

This comment has been minimized.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7bae647e0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

jbachorik and others added 3 commits July 1, 2026 14:38
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sessives

- Parser: desugar possessives X*+/X++/X?+ to atomic groups; parse (?>...)
- PikeVM: encode exit as -(entryPos+2) to preserve group identity post-exit
- PikeVM: replace wrong inline pruning with post-step lookahead pruning:
  kill exit threads only when inside thread can consume the next char
- Tests: update parser tests from throwsUnsupported to assertDoesNotThrow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ytecodeGenerator

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.19328% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.5%. Comparing base (6dd5dc7) to head (d606bee).

Files with missing lines Patch % Lines
...va/com/datadoghq/reggie/runtime/PikeVMMatcher.java 86.7% 9 Missing and 7 partials ⚠️
...ghq/reggie/runtime/BitParallelGlushkovRuntime.java 18.7% 10 Missing and 3 partials ⚠️
.../datadoghq/reggie/codegen/parsing/RegexParser.java 84.0% 4 Missing ⚠️
...doghq/reggie/codegen/analysis/PatternAnalyzer.java 85.0% 0 Missing and 3 partials ⚠️
...adoghq/reggie/codegen/analysis/StructuralHash.java 50.0% 0 Missing and 1 partial ⚠️
...va/com/datadoghq/reggie/codegen/ast/GroupNode.java 80.0% 0 Missing and 1 partial ⚠️
...m/datadoghq/reggie/codegen/ast/QuantifierNode.java 80.0% 0 Missing and 1 partial ⚠️
...hq/reggie/codegen/automaton/GlushkovAutomaton.java 88.8% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##              main     #92     +/-   ##
=========================================
- Coverage     82.5%   82.5%   -0.1%     
  Complexity       1       1             
=========================================
  Files          128     128             
  Lines        39226   39439    +213     
  Branches      5094    5150     +56     
=========================================
+ Hits         32400   32574    +174     
- Misses        5208    5230     +22     
- Partials      1618    1635     +17     
Files with missing lines Coverage Δ
...va/com/datadoghq/reggie/codegen/automaton/NFA.java 90.1% <100.0%> (+0.5%) ⬆️
...oghq/reggie/codegen/automaton/ThompsonBuilder.java 96.8% <100.0%> (+0.2%) ⬆️
.../codegen/BitParallelGlushkovBytecodeGenerator.java 99.1% <100.0%> (+<0.1%) ⬆️
...codegen/OptionalGroupBackrefBytecodeGenerator.java 80.7% <ø> (ø)
...adoghq/reggie/codegen/analysis/StructuralHash.java 91.1% <50.0%> (-1.1%) ⬇️
...va/com/datadoghq/reggie/codegen/ast/GroupNode.java 93.7% <80.0%> (-6.3%) ⬇️
...m/datadoghq/reggie/codegen/ast/QuantifierNode.java 58.8% <80.0%> (+1.6%) ⬆️
...hq/reggie/codegen/automaton/GlushkovAutomaton.java 91.3% <88.8%> (-0.2%) ⬇️
...doghq/reggie/codegen/analysis/PatternAnalyzer.java 74.1% <85.0%> (+<0.1%) ⬆️
.../datadoghq/reggie/codegen/parsing/RegexParser.java 77.6% <84.0%> (+<0.1%) ⬆️
... and 2 more

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6dd5dc7...d606bee. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jbachorik
jbachorik merged commit ae96462 into main Jul 1, 2026
9 checks passed
@jbachorik
jbachorik deleted the fix/atomic-groups-possessive-quant-skip branch July 1, 2026 20:48
@jbachorik jbachorik added this to the 0.4.0 milestone Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated or assisted by AI

Projects

None yet

2 participants