MDEV-40672: Pluggable Aggregate Function - #5573
Open
drrtuy wants to merge 2 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements MDEV-40672 by extending the function plugin framework so plugins can provide aggregate functions that participate in the standard Item_sum lifecycle (grouped aggregation, DISTINCT handling, and window-function execution), with accompanying func_test plugin implementations and MTR coverage.
Changes:
- Extend the SQL grammar to recognize generic-function aggregates (including plugin aggregates) and allow
DISTINCTin the generic function-call path. - Add
PLUGIN_SUM_FUNC/Item_sum_pluginplus distinct-argument replay support via a newAggregator::arg_item()API. - Add aggregate plugin test functions (
test_plugin_first,test_plugin_count) and MTR tests covering grouping, distinct, windows, and type preservation.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| sql/sql_yacc.yy | Parser updates to detect aggregate-capable generic functions, support DISTINCT, and allow plugin aggregates in window-function contexts. |
| sql/sql_window.cc | Reject plugin aggregates with DISTINCT when used as window functions (explicit ER_NOT_SUPPORTED_YET). |
| sql/item_sum.h | Introduces PLUGIN_SUM_FUNC, Item_sum_plugin, and adds Aggregator::arg_item() to support DISTINCT replay for plugin aggregates. |
| sql/item_sum.cc | Implements Item_sum_plugin::fix_fields() and extends distinct-aggregator logic to replay distinct values for plugin aggregates. |
| sql/item_create.h | Adds Create_aggregate_func marker type to distinguish scalar vs aggregate native/plugin builders. |
| plugin/func_test/plugin.cc | Adds two aggregate function plugins used by tests (test_plugin_first, test_plugin_count). |
| plugin/func_test/mysql-test/func_test/function_plugin.test | New MTR test covering plugin aggregates (grouping, DISTINCT, windows, UUID preservation, etc.). |
| plugin/func_test/mysql-test/func_test/function_plugin.result | Expected results for function_plugin.test. |
| plugin/func_test/mysql-test/func_test/function_plugin_scalar_unload.test | New test intended to demonstrate scalar plugin unload race (currently lacks .result and is described as crashing on unfixed servers). |
| plugin/func_test/mysql-test/func_test/function_plugin_negative.test | Negative-coverage MTR test for invalid contexts/usages and error paths. |
| plugin/func_test/mysql-test/func_test/function_plugin_negative.result | Expected results for function_plugin_negative.test. |
| plugin/func_test/mysql-test/func_test/function_plugin_extra.test | Additional MTR coverage for distinct, spill-to-disk, pluggable types, and reprepares. |
| plugin/func_test/mysql-test/func_test/function_plugin_extra.result | Expected results for function_plugin_extra.test. |
| include/mysql/plugin_function.h.pp | Forward-declares Create_func to keep generated header consistent with updated API. |
| include/mysql/plugin_function.h | Documents aggregate plugin requirements and forward-declares Create_func. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (using_udf_functions) | ||
| if (!native_builder && using_udf_functions) | ||
| { | ||
| // find_udf expectes a 0-terminated string |
Comment on lines
+628
to
+631
| Item_sum_plugin(THD *thd, Item *item): Item_sum(thd, item) | ||
| { quick_group= false; } | ||
| Item_sum_plugin(THD *thd, Item_sum_plugin *item): Item_sum(thd, item) | ||
| { quick_group= false; } |
Comment on lines
+3
to
+20
| # | ||
| # Demonstrates the HIGH severity issue found in the review of | ||
| # MDEV-40672: for a *scalar* function plugin the plugin reference is | ||
| # released already at parse time, so nothing keeps the shared object | ||
| # loaded while the Item is executed. A concurrent UNINSTALL therefore | ||
| # unloads func_test.so immediately (ref_count == 0) and the still | ||
| # running query calls a virtual method (Item_func_strnxfrm::val_str) | ||
| # whose vtable lives in the now unmapped .so -> server crash / UAF. | ||
| # | ||
| # For comparison, an aggregate plugin function (Item_sum_plugin) keeps | ||
| # the plugin loaded for the lifetime of the Item tree, so the same | ||
| # scenario there reports "Plugin is busy and will be uninstalled on | ||
| # shutdown" and the query completes (see function_plugin.test). | ||
| # | ||
| # Expected behaviour on a FIXED server: the scalar case behaves like | ||
| # the aggregate one - UNINSTALL is deferred with a "Plugin is busy" | ||
| # warning and the blocked query returns its result. | ||
| # On the current server this test crashes instead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add pluggable aggregate function support
What
Implement MDEV-40672 by extending function plugins to provide aggregate functions through the standard
Item_sumlifecycle. Support grouped aggregation,DISTINCT, window functions, native and pluggable data types, and safe plugin lifetime management.Key changes
Plugin_functiondescriptors to distinguish scalar and aggregate functions.Item_sum_pluginas the base class for plugin-provided aggregates.DISTINCTargument replay and pluggable result types such as UUID.How to test
Run:
/git/BuildOf_mdb-13/mysql-test/mtr function_plugin function_plugin_extraBoth tests pass.