Initial version of DuckDB engine for MariaDB based on DuckDB 1.5.2.

[drrtuy]

DuckDB Storage Engine for MariaDB

Overview

This PR introduces DuckDB 1.5.2 as an embedded analytical (OLAP) storage engine plugin for MariaDB 11.4. The engine is ported from AliSQL's DuckDB integration and fully adapted to MariaDB's handler API, plugin system, build infrastructure, and packaging conventions. It ships as a loadable module (ha_duckdb.so) for x86_64 and ARM64 architectures.

Architecture

The engine is structured as a git submodule at storage/duckdb/duckdb (repo: drrtuy/duckdb-engine), which itself contains a nested submodule (third_parties/duckdb) pointing to a DuckDB 1.3.2 fork with MariaDB-specific patches (e.g. octet_length VARCHAR overload). DuckDB is built statically with all builtin extensions (ICU, JSON, Parquet, jemalloc, etc.) and linked into a single ha_duckdb.so plugin. A debug-STL ABI mismatch guard (-U_GLIBCXX_DEBUG -U_GLIBCXX_ASSERTIONS) is applied in debug builds to prevent SIGSEGV from sizeof(std::vector) divergence between the plugin and the server.

Supported Operations

• DDL: CREATE TABLE, DROP TABLE, ALTER TABLE, RENAME TABLE — SQL is translated from MariaDB's internal structures to DuckDB-dialect DDL via ddl_convertor.cc, including expression defaults, column types, and engine conversions (ALTER TABLE ... ENGINE=DuckDB).
• DML — INSERT: Batch ingestion through DuckDB's Appender API with delta temp tables (delta_appender.cc), supporting all major MariaDB column types.
• DML — SELECT: Full query pushdown to DuckDB via MariaDB's select_handler interface (ha_duckdb_pushdown.cc), with result-set conversion back to MariaDB row format (duckdb_select.cc).
• DML — UPDATE/DELETE: Direct UPDATE and DELETE translated to DuckDB SQL (dml_convertor.cc).
• Configuration: DuckDB-specific server variables — memory limit, thread count, operating mode, etc. (duckdb_config.cc).
• Timezone & Charset: Mapping between MariaDB and DuckDB timezone names and charset/collation conventions.
• UDFs: Registration of DuckDB-side user-defined functions accessible from MariaDB.
• Per-thread context: DuckdbThdContext managed via thd_get_ha_data/thd_set_ha_data (replacing AliSQL's THD::get_duckdb_context()).

Packaging

• New Debian package mariadb-plugin-duckdb for amd64 and arm64.
• postinst / prerm hooks automatically run install.sql / uninstall.sql to register or unregister the plugin when the server is running.
• PLUGIN_DUCKDB=NO by default in native Debian builds (debian/rules); enabled automatically by autobake-deb.sh for MariaDB.org release builds when the submodule is present.
• RPM packaging scaffolding included in the submodule.

Testing

An MTR test suite is included under mysql-test/plugin/duckdb/ covering DDL, DML, type conversions, ALTER operations, and basic query pushdown scenarios. A disable.def file documents tests that are not yet passing or are intentionally skipped.

Key Differences from AliSQL Port

• Removed MySQL 8 Data Dictionary (dd::Table, dd::Schema) dependency — table metadata comes from TABLE* and .frm / TABLE_SHARE.
• Replaced mysql_declare_plugin with maria_declare_plugin.
• Adapted all handler method signatures (no dd::Table* parameters).
• Removed partition support, replication batch mode (GTID batch, multi-trx), and Relay_log_info extensions (not applicable to MariaDB).
• Uses DB_TYPE_AUTOASSIGN instead of the AliSQL-specific DB_TYPE_DUCKDB enum.

[Copilot]

Pull request overview

This PR lays groundwork for integrating the DuckDB storage engine into MariaDB by adding build-system scaffolding for storage/duckdb, wiring Debian/source-dist packaging hooks, and exposing condition-extraction helpers from the optimizer for external consumers (e.g., engine pushdown).

Changes:

• Add storage/duckdb/ CMake integration that fetches DuckDB engine sources at configure time and installs its MTR tests.
• Update Debian autobake script and source distribution generation to include DuckDB plugin packaging inputs.
• Export make_cond_for_table*() declarations (and make their definitions non-static) for use outside sql_select.cc.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
storage/duckdb/CMakeLists.txt	Adds FetchContent-based integration and packaging-related CPack propagation for DuckDB engine plugin
storage/duckdb/.gitignore	Ignores fetched DuckDB sources directory
sql/sql_select.h	Exposes make_cond_for_table*() declarations in a public header
sql/sql_select.cc	Makes make_cond_for_table*() non-static so they can be linked from other compilation units
debian/autobake-deb.sh	Attempts to enable DuckDB plugin Debian packaging during autobake
cmake/make_dist.cmake.in	Adds extra git export step intended to include DuckDB submodule content in source tarballs

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ drrtuy
❌ vuvova
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

lots of changes, all comments addressed