perf(fsst): push down '%suffix' LIKE without decompression - #9728
Open
jackylee-ch wants to merge 1 commit into
Open
perf(fsst): push down '%suffix' LIKE without decompression#9728jackylee-ch wants to merge 1 commit into
jackylee-ch wants to merge 1 commit into
Conversation
LikeKind::parse handled prefix% and %needle% but not %suffix, so those patterns fell through FsstMatcher::try_new and the kernel decompressed the whole column. vortex-duckdb already lowers suffix(col, 'x') to LIKE '%x', so on the same column prefix() and contains() match on the compressed code stream while suffix() does not. Adds FlatSuffixDfa, taking the Forward DFA route the module TODO sketched: the KMP table contains uses, but with a non-sticky accept state, so the walk continues past a match and the result is whether the final state is accept. build_symbol_transitions could not be reused because it short-circuits once a symbol's bytes reach accept. Signed-off-by: jackylee <qcsd2011@gmail.com>
Contributor
|
Do you have any benchmarks using real world or synthetic data? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LikeKind::parsehandledprefix%and%needle%but not%suffix, so those patterns fell throughFsstMatcher::try_newand the kernel decompressed the whole column.vortex-duckdbalready lowerssuffix(col, 'x')to LIKE%x, so on one columnprefix()andcontains()match on the codestream while
suffix()does not.Takes the Forward DFA route from the module TODO: the KMP table contains uses, but with a non-sticky
accept state.
Tests
cargo test -p vortex-fsstgoes 91 → 106. Two assertions that pinned the old gap are replaced.Correctness cases alone would pass via the fallback, so
test_like_kernel_pushes_down_suffixassertsthe kernel returns
Some; unhookingparse_suffixfails 4 tests including that one.AI assistance
Written with agentic AI assistance; I derived the non-sticky accept states by hand and checked the
mutation fails.