feat: add virtqueue transport foundations - #1793
Open
andreiltd wants to merge 19 commits into
Open
Conversation
andreiltd
changed the title
andreiltd
changed the title
andreiltd
added
kind/enhancement
andreiltd
force-pushed
the
virtq-foundations
branch
from
September 5, 2026 09:02
7d903d0 to
1464929
Compare
Read received payloads directly into caller owned storage and require paired completion of readable/writable chains. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Make pool restoration transactional. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Split and rename the pool implementations, add explicit lower/upper SlotPool regions. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Define deterministic producer and consumer reset behavior. Validate canonical events, descriptor chains, IDs, buffer policy, and unused descriptors. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Represent scratch bookkeeping with one repr(C) layout. Derive offsets and assert the host/guest ABI at compile time. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Use the first GPA of the reserved pages as an exclusive limit. Accept allocations ending at the limit and reject address overflow. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Define directional queue depths, buffer sizes, and pool page counts. Account for guest allocated rings and pools in minimum scratch calculations. Publish the transport contract through scratch-top metadata. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
This patch adds guest owned G2H and H2G rings and pools during guest initialization, and prefill H2G receive capacity. The guest then publishes their gpas through scratch metadata. The patch is also validates allocation order, scratch ownership, and canonical ring images before installing either host consumer. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Persist validated G2H and H2G ring images in running snapshots. Restore fixed transport allocations and install fresh host consumers before sandbox execution. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Allocate logical regions atomically Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Use queue_size instead of queue_depth Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
andreiltd
force-pushed
the
virtq-foundations
branch
from
September 8, 2026 14:41
4b50e87 to
b06bf40
Compare
andreiltd
marked this pull request as ready for review
September 8, 2026 14:41
andreiltd
requested review from
danbugs,
dblnz,
devigned,
jprendes,
jsturtevant,
ludfjig,
simongdavies,
squillace and
syntactically
as code owners
September 8, 2026 14:41
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
TransportArena::new currently sets h2g_pool_addr to the G2H pool offset, causing pool overlap and an incorrect transport arena layout.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds the foundations needed to support a virtqueue-based transport across host/guest, and wires that transport geometry into scratch layout and snapshot versioning so snapshots can faithfully restore or reinitialize transport state.
Changes:
- Extend sandbox configuration, scratch layout, and snapshot config schema to include per-direction virtqueue geometry (queue sizes, buffer sizes, pool pages) and bump snapshot ABI/config versions.
- Add host/guest virtqueue memory access primitives and guest transport context initialization during
generic_init. - Introduce canonical ring image validation (incl. fuzzing) and refactor buffer/pool plumbing toward
SlotPool+BufferLease/BufferMap.
File summaries
| File | Description |
|---|---|
| src/hyperlight_host/tests/snapshot_goldens/goldens_version.rs | Bump goldens version to v3.0 |
| src/hyperlight_host/tests/integration_test.rs | Adjust OOM/abort expectations for transport |
| src/hyperlight_host/src/sandbox/uninitialized.rs | Update tests for new scratch sizing |
| src/hyperlight_host/src/sandbox/uninitialized_evolve.rs | Attach virtq on initialize path |
| src/hyperlight_host/src/sandbox/snapshot/tripwires.rs | Bump ABI and config media type expectations |
| src/hyperlight_host/src/sandbox/snapshot/mod.rs | Carry optional in-memory virtq snapshot |
| src/hyperlight_host/src/sandbox/snapshot/file/mod.rs | Persist/load transport layout fields; config v2 gating |
| src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs | Add MT_CONFIG_V2; bump ABI to 3 |
| src/hyperlight_host/src/sandbox/snapshot/file/config.rs | Add transport fields + normalization validation |
| src/hyperlight_host/src/sandbox/snapshot/file_tests.rs | Tests for config v2 + transport round-trip |
| src/hyperlight_host/src/sandbox/initialized_multi_use.rs | Restore/attach virtq depending on NextAction |
| src/hyperlight_host/src/sandbox/config.rs | Add transport geometry to configuration |
| src/hyperlight_host/src/mem/virtq_mem.rs | Host MemOps for scratch + ring images |
| src/hyperlight_host/src/mem/shared_mem.rs | Add checked atomic load/store helpers |
| src/hyperlight_host/src/mem/mod.rs | Export virtq modules |
| src/hyperlight_host/src/mem/mgr.rs | Attach/restore/snapshot virtq in memory manager |
| src/hyperlight_host/src/mem/layout.rs | Place transport arena in fixed scratch prefix |
| src/hyperlight_guest/src/transport/mod.rs | Global guest transport context plumbing |
| src/hyperlight_guest/src/transport/mem.rs | Guest MemOps for scratch GVAs |
| src/hyperlight_guest/src/transport/context.rs | Guest virtq producers + pool construction |
| src/hyperlight_guest/src/prim_alloc.rs | Shared allocation limit helper + tests |
| src/hyperlight_guest/src/lib.rs | Export guest transport module |
| src/hyperlight_guest/src/layout.rs | Add scratch-top accessors for transport metadata |
| src/hyperlight_guest/src/error.rs | Convert VirtqError into guest error |
| src/hyperlight_guest/src/arch/amd64/prim_alloc.rs | Use scratch allocator limit helper |
| src/hyperlight_guest/src/arch/aarch64/prim_alloc.rs | Use scratch allocator limit helper |
| src/hyperlight_guest_bin/src/transport.rs | Guest transport initialization at boot |
| src/hyperlight_guest_bin/src/lib.rs | Call transport init during generic init |
| src/hyperlight_common/src/virtq/ring/fuzz.rs | QuickCheck fuzz for ring ops interleavings |
| src/hyperlight_common/src/virtq/ring/canonical.rs | Canonical packed ring image validation |
| src/hyperlight_common/src/virtq/pool/tests.rs | SlotPool unit tests |
| src/hyperlight_common/src/virtq/pool/slot.rs | SlotPool implementation |
| src/hyperlight_common/src/virtq/pool/fuzz.rs | QuickCheck fuzz for SlotPool invariants |
| src/hyperlight_common/src/virtq/msg.rs | Remove old virtq message header module |
| src/hyperlight_common/src/virtq/event.rs | Add canonical clear helper |
| src/hyperlight_common/src/virtq/desc.rs | Add descriptor table clear + base accessor |
| src/hyperlight_common/src/virtq/concurrency.rs | Update loom tests to new pool/buffer APIs |
| src/hyperlight_common/src/virtq/buffer.rs | Add Segments split/chunk APIs; BufferLease |
| src/hyperlight_common/src/virtq/access.rs | Add BufferMap interface for completed buffers |
| src/hyperlight_common/src/transport.rs | Shared message framing + external values |
| src/hyperlight_common/src/lib.rs | Export shared transport module |
| src/hyperlight_common/src/layout.rs | Scratch-top metadata struct + transport arena |
| src/hyperlight_common/src/arch/amd64/layout.rs | Make min_scratch_size checked/Option |
| src/hyperlight_common/src/arch/aarch64/layout.rs | Make min_scratch_size checked/Option |
| src/hyperlight_common/benches/virtq_api.rs | Update benchmarks to SlotPool naming |
| src/hyperlight_common/benches/common/mod.rs | Update bench harness to BufferMap/Lease |
| src/hyperlight_common/benches/buffer_pool.rs | Replace old pool benches with SlotPool |
| fuzz/README.md | Document added canonical image fuzzing |
| fuzz/fuzz_targets/virtq_packed_ring.rs | Fuzz canonical image validation paths |
| docs/snapshot-versioning.md | Update snapshot versioning docs for transport |
| docs/snapshot-oci-format.md | Update OCI snapshot format docs for v2 config |
| CHANGELOG.md | Note transport and ABI/schema changes |
Review details
- Files reviewed: 58/58 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+227
to
+233
| Some(Self { | ||
| g2h_ring_addr: base_addr, | ||
| h2g_ring_addr: addr(h2g_ring_offset)?, | ||
| g2h_pool_addr: addr(g2h_pool_offset)?, | ||
| h2g_pool_addr: addr(h2g_pool_offset)?, | ||
| ring_span_len: g2h_pool_offset, | ||
| len, |
Member
Author
There was a problem hiding this comment.
It looks to me like offsets are correct, also changeset suggestion is nonsensical.
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The patch implements shared layout and ownership model for virtq based host and guest communication. This is a preparation work for replacing stack based IO with virtq, in the meantime function calls and logging use the legacy transport. Actual takeover, buffer mappings, guest allocated pools and retained buffer snapshots are separate stages.
SlotPoolallocation,