Skip to content

Fix DFA anchor bugs; native split for over-JIT patterns; Laurikari anchor caching - #153

Merged
jbachorik merged 4 commits into
mainfrom
fix/dfa-anchor-bugs
Oct 22, 2026
Merged

jbachorik merged 4 commits into
mainfrom
fix/dfa-anchor-bugs

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

What does this PR do?

Fixes anchor bugs in the DFA bytecode generators. Patterns with ALLOW_JDK_FALLBACK whose generated method exceeds the JIT size limit now route to a native search engine instead of the plain JDK fallback. Also speeds up the Laurikari (tagged DFA) engine for patterns with end anchors and \b.

DFA anchor fixes (codegen)

  • DFASwitchBytecodeGenerator ignored DFATransition.entryGuard (mid-pattern ^, $, \Z) at every transition site. Its matches() also had a \Z special case that accepted before a trailing \n. Examples that diverged from the JDK: x{1,25}\Z on "x\n", x{1,25}\n^b, x{1,25}$\nb.
  • Bounded anchor checks use the region [start, end).
  • Find side: findFrom empty-at-end skipped start-state assertions, and findMatchEnd had no assertion or accept checks.
  • PatternAnalyzer / FallbackPatternDetector routing fixes for shapes the DFA cannot represent. For example, (?:$b?|\r) on "\r\n" gave [0,1) on DFA_UNROLLED; the JDK gives [0,0).
  • SubsetConstructor end-guard fix, the DFA_SWITCH_WITH_GROUPS accept check, and DFAUnrolled group-action ordering.

Over-JIT-limit split (runtime)

  • With ALLOW_JDK_FALLBACK, a pattern whose generated method is over the JIT limit used to fall back to the JDK. RuntimeCompiler.overJitNative now tries, in order:
    1. OverJitSplitMatcher: the find family goes to a fresh LaurikariDfaMatcher, and the whole-input family goes to a shared JDK half.
    2. BitState.
    3. JDK.
  • The split requires Laurikari eligibility plus the BitState gates. It is declined for patterns that may enable (?i), because the JDK half folds Unicode case and Laurikari folds ASCII only.
  • When the Laurikari cache saturates, that matcher's find family switches to the JDK half.
  • The split's JDK half uses a new quiet JavaRegexFallbackMatcher(pattern) constructor, so it does not log the "Falling back" warning.
  • Without ALLOW_JDK_FALLBACK, nothing changes: the generated (interpreted) matcher is kept.

Laurikari performance

  • LaurikariDFACache caches anchor-dependent transitions under a lookahead class (10 classes). The class encodes the next character and the distance to the region end. Before this change, transitions into anchor-sensitive states were never cached.
  • LaurikariDfaMatcher caches the start state of a find from position 0 per lookahead class. Before, it rebuilt the start closure on every call.

Motivation

A differential fuzz against the JDK found the DFA anchor bugs. On 6k fuzzed patterns (seed 0xBEEF), the JDK divergences fell from 53 to 0 on DFA_SWITCH, from 225 to 0 on DFA_UNROLLED and from 11 to 0 on DFA_SWITCH_WITH_GROUPS. The review of these fixes found patterns whose generated method is over the JIT limit and that fell back to the JDK, for example [^a]{3,30}b+(?:\n{2}\n{1,25}\n$)\d{1,25}. The over-JIT split gives the find family of such patterns a native engine.

Related Issue(s)

Independent of #150. Rebased on #152 (structural cache keys); the split cache is keyed by CompilationKey like the other caches.

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactoring (no functional change)
  • Documentation
  • Test improvement
  • Build/CI change

Checklist

  • I have read the CONTRIBUTING.md guidelines
  • All existing tests pass (./gradlew build)
  • I have added tests for my changes
  • I have updated documentation (if applicable)
  • My commits are signed

Performance Impact

JMH on x86, -f 3, ops/ms, new benchmark OverJitLimitRerouteBenchmark. Its five patterns are realistic over-JIT-limit shapes.

  • Laurikari anchor tail cache, before → after (find family):
    • FindShort: +240% to +1474%;
    • FindNoMatch: +43% to +352%;
    • FindLong: +5% to +31%;
    • FindDense: hexDigitsEndZ 16.3 → 257, the other patterns flat.
  • Start-state cache, FindShort: +20% to +53%. hexDigitsEndZ goes from 1165 to 1697 ops/ms; the JDK measured 1063–1300, depending on the run.
  • Split vs JDK:
    • The find family is 1x to 754x the JDK.
    • The whole-input family is 0.87x to 1.05x, because it runs on the JDK.
  • Existing benchmarks: an A/B of the DFA anchor fixes over 99 rows showed no regression.
  • hostPortEnd: FindLong and FindNoMatch showed −15% after the start-state cache. With -gc true the difference goes away (FindLong 15.71 → 16.53), so it comes from heap layout, not from the code.

Additional Notes

  • Suite on the rebased branch (./gradlew test): reggie-runtime 3699 (31 skipped), reggie-codegen 529, reggie-processor 259; 0 failures. ./gradlew build was not run as a separate step.
  • (?!(a))[ab]{1,25} is not changed. The JDK's find() keeps a stale group from a failed attempt; Reggie gives -1, the same as JDK find(1) and the PCRE answer.

🤖 Generated with Claude Code

jbachorik and others added 3 commits October 22, 2026 10:12
…tails

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jbachorik jbachorik added the AI Generated or assisted by AI label Oct 22, 2026
@jbachorik
jbachorik marked this pull request as ready for review October 22, 2026 08:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T08:31:44.376778Z 96e865d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codecov-commenter

codecov-commenter commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.11189% with 85 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.1%. Comparing base (23a61fd) to head (db7805f).

Files with missing lines Patch % Lines
...doghq/reggie/codegen/analysis/PatternAnalyzer.java 79.5% 33 Missing and 20 partials ⚠️
...ie/codegen/codegen/DFASwitchBytecodeGenerator.java 94.7% 10 Missing and 3 partials ⚠️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java 81.0% 3 Missing and 8 partials ⚠️
...om/datadoghq/reggie/runtime/LaurikariDFACache.java 82.7% 2 Missing and 3 partials ⚠️
.../codegen/codegen/DFAUnrolledBytecodeGenerator.java 95.7% 0 Missing and 2 partials ⚠️
.../datadoghq/reggie/runtime/LaurikariDfaMatcher.java 91.6% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##              main    #153     +/-   ##
=========================================
+ Coverage     85.0%   85.1%   +0.1%     
  Complexity       1       1             
=========================================
  Files          168     169      +1     
  Lines        49882   50037    +155     
  Branches      7592    7712    +120     
=========================================
+ Hits         42417   42616    +199     
+ Misses        5464    5406     -58     
- Partials      2001    2015     +14     
Files with missing lines Coverage Δ
...ggie/codegen/analysis/FallbackPatternDetector.java 74.6% <ø> (+0.5%) ⬆️
...hq/reggie/codegen/automaton/SubsetConstructor.java 85.4% <100.0%> (-0.1%) ⬇️
...datadoghq/reggie/codegen/codegen/BytecodeUtil.java 95.8% <100.0%> (+0.5%) ⬆️
...n/codegen/DeterministicChainBytecodeGenerator.java 96.1% <100.0%> (+<0.1%) ⬆️
...ggie/codegen/codegen/OnePassBytecodeGenerator.java 99.2% <100.0%> (-0.1%) ⬇️
...ggie/processor/ReggieMatcherBytecodeGenerator.java 85.5% <ø> (ø)
...doghq/reggie/runtime/JavaRegexFallbackMatcher.java 58.4% <100.0%> (+2.4%) ⬆️
.../datadoghq/reggie/runtime/OverJitSplitMatcher.java 100.0% <100.0%> (ø)
.../datadoghq/reggie/runtime/LaurikariDfaMatcher.java 89.8% <91.6%> (+<0.1%) ⬆️
.../codegen/codegen/DFAUnrolledBytecodeGenerator.java 94.2% <95.7%> (+0.1%) ⬆️
... and 4 more

... and 4 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23a61fd...db7805f. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96e865d725

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

A long greedy search can return a truncated match on the call that saturates the Laurikari cache. Repeating the same call returns the full span because subsequent searches switch to the JDK half.

Open Bits AI session

🤖 Bits Code Review · Commit 96e865d

@jbachorik
jbachorik merged commit 57a288f into main Oct 22, 2026
9 checks passed
@jbachorik
jbachorik deleted the fix/dfa-anchor-bugs branch October 22, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated or assisted by AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants