Skip to content

feat: add DUCKDB dialect and bidirectional DuckDB converter#229

Open
michellepellon wants to merge 1 commit into
apache:mainfrom
michellepellon:duckdb-dialect
Open

feat: add DUCKDB dialect and bidirectional DuckDB converter#229
michellepellon wants to merge 1 commit into
apache:mainfrom
michellepellon:duckdb-dialect

Conversation

@michellepellon

Copy link
Copy Markdown

Add DUCKDB dialect and a bidirectional DuckDB converter

What

This PR adds DuckDB as a supported expression dialect and contributes a reference converter for it, in three parts:

  1. Dialect registration — adds DUCKDB to the Dialect enum in core-spec/osi-schema.json, core-spec/spec.yaml, the dialect table in core-spec/spec.md, and the OSIDialect / OSIVendor enums in python/src/ossie/models.py.
  2. Validation support — maps DUCKDB to sqlglot's duckdb dialect in validation/validate.py, so DuckDB expressions get full syntax validation (sqlglot's DuckDB support is first-class; no skip-list entry needed).
  3. Converterconverters/duckdb/, a bidirectional Ossie ↔ DuckDB converter following the hub-and-spoke pattern in converters/README.md:
    • Export: Ossie model → DuckDB SQL (one view per dataset with field expressions, COMMENT ON from descriptions, one view per metric with joins derived from declared relationships). Dialect selection prefers DUCKDB, falls back to ANSI_SQL with a warning.
    • Import: DuckDB database → Ossie YAML via information_schema and duckdb_constraints() (primary keys, unique keys, and foreign keys → relationships; table/column comments → descriptions).
    • CLI (ossie-duckdb export|import), pytest suite with round-trip tests executing generated SQL against in-memory DuckDB, and schema validation of imported models against osi-schema.json.

Why

DuckDB (and MotherDuck, its managed cloud service) is widely used as an analytics engine in lakehouse and embedded-analytics stacks, and is a common substrate for the AI-agent workloads that ai_context targets. Because DuckDB connections are passed through, the converter works against MotherDuck databases (md: connection strings) unchanged.

Notes

  • No changes to existing converters; unknown-dialect handling elsewhere is unaffected (existing fallback chains ignore dialects they don't target).
  • The example models in examples/ are untouched and still validate.
  • Aware of Only allow one dialect per OSI document #52 (single dialect per document); the enum addition is orthogonal — whichever way that lands, DuckDB needs to be a valid dialect value.

Adds DUCKDB to the dialect enum (osi-schema.json, spec.yaml, spec.md,
Python models), maps it to sqlglot's duckdb dialect for expression
validation, and contributes converters/duckdb: a bidirectional
Ossie <-> DuckDB converter with CLI, following the hub-and-spoke
converter pattern. Works with MotherDuck via md: connection strings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant