Skip to content

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675

Open
indietyp wants to merge 7 commits intobm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and
Open

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675
indietyp wants to merge 7 commits intobm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and

Conversation

@indietyp
Copy link
Copy Markdown
Member

@indietyp indietyp commented Apr 29, 2026

🌟 What is the purpose of this PR?

Rework the MIR interpreter to support suspend/resume execution, richer value representations, and a proper input system. Together these let the interpreter pause when it needs external data (e.g. a database query), hand back a structured description of what it needs, and resume once the caller provides the result.

🔍 What does this change?

Suspension model (interpret/suspension/)

  • Introduce a suspend/resume protocol: when the interpreter hits a GraphRead terminator it yields a Suspension instead of blocking. Callers drive a start/resume loop until they get a Return.
  • GraphReadSuspension extracts temporal axes, filter parameters, and entity type from the suspended state.
  • Continuation carries the response value that resumes execution.
  • TemporalAxesInterval / Timestamp give structured representations of the temporal query window.
  • Runtime::run convenience wrapper for callers that can fulfill suspensions synchronously.

Input system (interpret/inputs.rs)

  • Add Inputs<'heap, A>: a typed map from interned symbols to Values, consulted for InputOp::Load and InputOp::Exists. Replaces the unstructured FastHashMap previously threaded through the runtime. Supports both global and custom allocators.

Value representation (interpret/value/)

  • Int: carry bit-width (1-bit boolean vs 128-bit integer) so serialization can distinguish true/false from 0/1 without external type info. Arithmetic promotes to 128-bit; bitwise boolean ops preserve 1-bit.
  • Struct: add StructBuilder<N> for stack-allocated fixed-field construction. Enforce sorted field invariant. Add merge and project_by_name.
  • Minor improvements to Opaque, List allocator parameterization, Str display.

SSA repair fix (pass/transform/ssa_repair)

  • Fix block parameters not being renamed during SSA repair.
  • Extend use-before-def detection to terminators, not just statements.

Entity path type resolution (pass/execution/traversal)

  • Add field_path and resolve_type to EntityPath for navigating from an entity path (e.g. WebId, Properties) to its concrete stdlib type.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

None

🐾 Next steps

  • The orchestrator in hashql-eval consumes the suspension model and input system introduced here.

🛡 What tests cover this?

  • Interpreter unit tests (interpret/tests.rs): cover the suspend/resume loop, input resolution, and value operations.
  • SSA repair UI tests (tests/ui/pass/ssa_repair): cover the block parameter and terminator fixes.
  • Existing pass UI tests re-blessed for snapshot changes from the Int representation change.

❓ How to test this?

  1. cargo nextest run --package hashql-mir
  2. cargo test --package hashql-mir --doc

feat: make Int size aware

feat: postgres serialization

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: errors

feat: psql codec

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: move bridge -> orchestrator

feat: move bridge -> orchestrator

feat: organize the runtime a bit more

feat: organize the runtime a bit more

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: organize the runtime a bit more

feat: finish orchestrator first version

feat: finish orchestrator first version

feat: feat: move out suspension requests to own module

chore: clippy

feat: split out into more manageable functions

feat: split out into more manageable functions

chore: orchestrator docs

feat: organize the runtime a bit more

feat: test orchestrator

feat: better symbol repr

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: test orchestrator

feat: pipeline

feat: bless snapshots

chore: tests

chore: masking

chore: tests

fix: SSA repair

chiore: ssa repair snapshots

feat: orchestrator test harness

chore: remove spec

fix: library features

chore: update snapshots

chore: update docs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:14 Inactive
@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 29, 2026

PR Summary

High Risk
High risk because it refactors the MIR interpreter execution model (new suspend/resume protocol, new Runtime::run signature, callstack/frame semantics) and changes core runtime value invariants (Int boolean/int width), which can affect correctness across backends and serialization.

Overview
Adds a suspend/resume execution model to the MIR interpreter: GraphRead now yields Yield::Suspension with a typed Suspension payload and resumes via Continuation, with new Runtime::start, Runtime::resume, run_until_suspension, and run_until_transition APIs.

Introduces an Inputs type to replace the ad-hoc inputs FastHashMap in Runtime, and updates benches/tests and Postgres eval constant compilation accordingly.

Reworks core interpreter data structures and values: RuntimeError becomes generic over suspension errors and gains new type-invariant variants; Value projections now traverse Opaque wrappers; Int now tracks bit-width to distinguish booleans vs integers (updating MIR/UI snapshots); and Struct adds a StructBuilder plus a sorted-field invariant. Also adds traversal path type resolution helpers and small diagnostics/graph scheduling tweaks.

Reviewed by Cursor Bugbot for commit 4cc8b7a. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment May 4, 2026 11:47am
petrinaut Ready Ready Preview May 4, 2026 11:47am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview May 4, 2026 11:47am

@github-actions github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Apr 29, 2026
@indietyp indietyp changed the title HashQL: Suspendable interpreter with structured values and input system BE-523: HashQL: Suspendable interpreter with structured values and input system Apr 29, 2026
Copy link
Copy Markdown
Member Author

indietyp commented Apr 29, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 78.51750% with 313 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.31%. Comparing base (7d9f781) to head (4cc8b7a).

Files with missing lines Patch % Lines
.../hashql/mir/src/pass/execution/traversal/entity.rs 0.00% 66 Missing ⚠️
libs/@local/hashql/mir/src/interpret/error.rs 11.42% 62 Missing ⚠️
.../hashql/mir/src/interpret/suspension/graph_read.rs 47.19% 40 Missing and 7 partials ⚠️
libs/@local/hashql/mir/src/interpret/runtime.rs 79.25% 38 Missing and 1 partial ⚠️
libs/@local/hashql/mir/src/interpret/value/int.rs 91.96% 14 Missing and 6 partials ⚠️
libs/@local/hashql/mir/src/interpret/value/mod.rs 67.27% 18 Missing ⚠️
libs/@local/hashql/mir/src/interpret/inputs.rs 63.04% 17 Missing ⚠️
libs/@local/hashql/mir/src/interpret/tests.rs 96.08% 7 Missing and 7 partials ⚠️
...bs/@local/hashql/mir/src/interpret/value/struct.rs 96.15% 5 Missing and 2 partials ⚠️
...bs/@local/hashql/mir/src/interpret/value/opaque.rs 0.00% 5 Missing ⚠️
... and 7 more
Additional details and impacted files
@@                                 Coverage Diff                                 @@
##           bm/be-522-hashql-expand-stdlib-type-definitions    #8675      +/-   ##
===================================================================================
+ Coverage                                            68.05%   68.31%   +0.26%     
===================================================================================
  Files                                                  909      925      +16     
  Lines                                                85798    88683    +2885     
  Branches                                              4539     4581      +42     
===================================================================================
+ Hits                                                 58386    60584    +2198     
- Misses                                               26828    27495     +667     
- Partials                                               584      604      +20     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.41% <ø> (ø)
apps.hash-api 0.00% <ø> (ø)
local.hash-backend-utils 0.00% <ø> (ø)
local.hash-graph-sdk 9.63% <ø> (ø)
local.hash-isomorphic-utils 0.00% <ø> (ø)
rust.hash-graph-api 2.52% <ø> (ø)
rust.hashql-ast 87.23% <ø> (ø)
rust.hashql-compiletest 29.63% <ø> (ø)
rust.hashql-core 82.17% <ø> (ø)
rust.hashql-diagnostics 72.53% <100.00%> (?)
rust.hashql-eval 81.08% <0.00%> (ø)
rust.hashql-hir 89.06% <ø> (ø)
rust.hashql-mir 91.58% <78.48%> (-0.51%) ⬇️
rust.hashql-syntax-jexpr 94.05% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:18 Inactive
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:41 Inactive
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 15:09 Inactive
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0a89a10. Configure here.

Comment thread libs/@local/hashql/mir/src/interpret/runtime.rs
@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Apr 29, 2026

🤖 Augment PR Summary

Summary: This PR upgrades the HashQL MIR interpreter to support structured runtime values and a suspend/resume execution model for external data access.

Changes:

  • Introduced a new Inputs map type and updated Runtime to borrow inputs rather than owning a hash map
  • Added interpreter yield points via Yield, plus Suspension/Continuation types to support suspend/resume around GraphRead
  • Extended runtime diagnostics/errors to model suspension-related failures and new invariant violations (unexpected value type / invalid constructor)
  • Reworked Int to track bit-width (bool vs 128-bit int) so boolean provenance round-trips cleanly through serialization
  • Improved value projection to transparently project into Opaque wrappers and added a StructBuilder helper for building sorted-field structs safely
  • Adjusted tests/benchmarks and various MIR passes (SSA repair, inst simplification, scheduling, traversal typing) to match the new runtime/value semantics

Technical Notes: Interpreter execution can now be driven synchronously via Runtime::run (with a suspension handler) or manually via Runtime::start/Runtime::resume, enabling backends like graph/DB to fulfill requests out-of-band.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

BinOp::BitAnd => return Some(lhs & rhs),
BinOp::BitOr => return Some(lhs | rhs),
// Comparisons produce booleans
BinOp::Eq => lhs.as_int() == rhs.as_int(),
Copy link
Copy Markdown

@augmentcode augmentcode Bot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libs/@local/hashql/mir/src/pass/transform/inst_simplify/mod.rs:262: eval_bin_op uses lhs.as_int()/rhs.as_int() for comparisons, which drops the new Int size/provenance (bool vs int) and makes const-folding potentially disagree with runtime comparisons. Consider using the Int comparison impls directly so folding matches interpreter behavior.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread libs/@local/hashql/mir/src/interpret/suspension/graph_read.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants