BE-306: HashQL: PostgreSQL translation#8526
BE-306: HashQL: PostgreSQL translation#8526indietyp wants to merge 9 commits intobm/be-457-hashql-mir-execution-pipeline-extensions-for-postgresfrom
Conversation
feat: checkpoint (II) feat: checkpoint (III) feat: snapshot vec feat: add dedicated filter feat: checkpoint feat: filter implementation feat: filter implementation (mostly) done chore: environment capture note chore: always postgres bigint feat: target clone feat: simplify lookup feat: move storage up feat: eval entity path chore: checkpoint chore: checkpoint chore: find entrypoint feat: eval context feat: eval cleanup chore: cleanup feat: track index feat: wire up filter feat: add error reporting chore: checkpoint feat: add traverse, and first postgres compiler outline feat: traverse bitmap feat: move traversal out feat: projections feat: projections fix: clippy feat: subquery projection for lateral feat: checkpoint feat: test plan feat: checkpoint feat: checkpoint – failing tests ;-; feat: checkpoint – failing tests ;-; feat: checkpoint — passing tests fix: import fix: entity type feat: checkpoint feat: attribute a cost to terminator placement switches fix: import feat: checkpoint feat: checkpoint chore: lint
PR SummaryMedium Risk Overview Adds supporting infrastructure in Reviewed by Cursor Bugbot for commit b399240. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
🤖 Augment PR SummarySummary: Adds a PostgreSQL compilation backend for HashQL by lowering MIR execution islands into SQL Key changes:
Technical notes: Compiled filter islands are materialized with 🤖 Was this summary useful? React with 👍 or 👎 |
33a9dcc to
8c07a05
Compare
5d95ba7 to
1aa0f1c
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## bm/be-457-hashql-mir-execution-pipeline-extensions-for-postgres #8526 +/- ##
====================================================================================================
- Coverage 85.26% 63.09% -22.18%
====================================================================================================
Files 349 1250 +901
Lines 55110 132490 +77380
Branches 1339 5409 +4070
====================================================================================================
+ Hits 46991 83594 +36603
- Misses 7672 48030 +40358
- Partials 447 866 +419 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
0bd3b25 to
a675701
Compare
1e966bd to
02e5f27
Compare
There was a problem hiding this comment.
clippy found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
02e5f27 to
8b510e5
Compare
a675701 to
3050d4a
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8b510e5. Configure here.
| ColumnName::from(Identifier::from("b")), | ||
| ColumnName::from(Identifier::from("v")), | ||
| ], | ||
| }); |
There was a problem hiding this comment.
UNNEST lateral flag contradicts required LATERAL semantics
Medium Severity
The FromItem::Function for the UNNEST call sets lateral: false, but the function body references columns from the preceding "eit" table alias (eit.base_urls, eit.versions). In PostgreSQL, explicit CROSS JOIN requires the LATERAL keyword for the right-hand side to reference columns from the left. The doc comment on line 185 correctly states CROSS JOIN LATERAL UNNEST(...), but the code passes lateral: false, which would produce invalid SQL when the entity type IDs path is accessed.
Reviewed by Cursor Bugbot for commit 8b510e5. Configure here.
8b510e5 to
c484cc9
Compare
3050d4a to
5552cbb
Compare
c484cc9 to
b399240
Compare
5552cbb to
60da845
Compare



🌟 What is the purpose of this PR?
Implements the postgres compilation backend for HashQL. Takes the MIR control flow graph (after execution analysis has assigned basic blocks to backends and partitioned them into islands) and compiles the Postgres-assigned islands into SQL
SELECTstatements.🔍 What does this change?
Postgres compiler (
eval/src/postgres/mod.rs):Top-level entry point. Compiles a
GraphReadbody island-by-island into aPreparedQuery(aSelectStatement+ deduplicatedParameterslist). Each Postgres island becomes aCROSS JOIN LATERALsubquery returning acontinuationcomposite value. The continuation carriesfilter(keep/reject/passthrough),block(next basic block), andlocals/values(live-out data for the interpreter to resume from).Filter compiler (
eval/src/postgres/filter/):Walks the MIR basic blocks within an island and compiles each statement into SQL expressions. Uses an explicit frame stack (not recursion) to handle
SwitchIntterminators: each branch becomes aCASE WHENarm, with the discriminant cast to::intto avoid boolean/integer type mismatches in PostgreSQL. Out-of-island branches produce continuation values that encode which block to resume and what locals to carry.Projections (
eval/src/postgres/projections.rs):Maps
EntityPathvariants to SQL column references or JSONB extraction expressions. Tracks which table joins are needed and only requests them when a path is actually referenced. Handles the split between "column-backed" paths (entity_uuid, web_id, etc.) and "JSONB-backed" paths (properties, type IDs).Parameters (
eval/src/postgres/parameters.rs):Builds the
$1, $2, ...parameter list for the prepared statement. Deduplicates by identity. EachParametervariant represents a different source:Input(user-provided values),Symbol/Primitive/Int(query literals),Env(closure captures),TemporalAxis(execution context). TheCompiledQueryreturn type exposes which indices correspond to which sources so the interpreter can bind them.Continuation (
eval/src/postgres/continuation.rs):Builds the
ROW(filter, block, locals, values)::continuationcomposite values that encode island exit state. Handles the three exit cases: passthrough (NULL continuation), filter-only (just a boolean), and full exit (block + live-out locals serialized as parallel int[]/jsonb[] arrays).Traverse (
eval/src/postgres/traverse.rs):Compiles graph traversal requirements into SQL joins. Reads the island's
providesset to determine which entity paths need table joins, then requests them from the database context layer.Error infrastructure (
eval/src/postgres/error.rs):Diagnostic types for compilation errors (unsupported operations, type mismatches, missing paths) with span-accurate source locations.
Context (
eval/src/context.rs):DatabaseContexttrait and implementation that the compiler uses to request table aliases, register joins, and access the schema. Bridges between the HashQL type system and the graph-store query builder.Compiletest suite (
compiletest/src/suite/eval_postgres.rs):New compiletest suite that runs the full pipeline (parse, type-check, lower to MIR, run execution analysis, compile to SQL) and compares the output against blessed
.stdoutfiles. Also emits.aux.mirsecondary outputs showing the MIR after execution analysis for debugging.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
OFFSET 0on lateral subqueries is a workaround for PostgreSQL inlining composites; seecontinuation.rsdoc comments for details.🛡 What tests cover this?
filter/tests.rs, ~1000 lines) using insta snapshots covering: straight-line blocks, branching CFGs, diamond merges, island exits, projections, property access, parameter deduplication, lateral subquery generationeval/tests/ui/postgres/covering end-to-end compilation: comparison operators, entity field access, input parameters, let bindings, if-expressions, nested branching, environment captures, list/dict/struct/tuple construction, multiple filters, mixed-source filters❓ How to test this?