[multicast] Host-side plumbing for externally sourced multicast

[zeeshanlakhani]

This work gets externally sourced multicast into a running rack (and relies on the latest st.louis branch of https://github.com/oxidecomputer/helios).

voxel can now provide plumbing for:

• a route for each group pointed at ce
• a mirror on the transit router carrying each group's flood toward every scrimlet
• a membership so that the router's NIC accepts the group's frames to begin with

This work adds all of these steps via voxel network multicast {up,down,check} commands.

The mirror rests on two main properties:

1. external multicast ingresses at whichever switch holds the group's external NAT entry, elected by the rack and not visible to the host (so that every scrimlet has to receive a copy).
2. flower classification ends at the first matching filter, so the targets are chained action mirred clauses inside one filter per group rather than one filter per scrimlet.

The mirror also only sees frames the NIC accepts, and a NIC accepts a multicast group only after a group join (subscription). Nothing on the router joins these groups (voxel's FRR runs only unicast BGP) on its own, so the up command pins a static link-layer membership for each group's Ethernet address (the RFC 1112 section 6.4 mapping) on the host-facing NIC.

This mapping folds 28 group bits into 23 (RFC 7042 section 2.1.1), meaning that 32 groups alias each Ethernet address and teardown drops a membership only when no remaining group still maps to it. Each membership up adds is recorded under /run on the router, and down removes only recorded ones, so a membership the router already held is never voxel's to delete. The record dies with the router, exactly as the memberships do.

Host routes are the one piece that outlives voxel destroy, and the host's route table is shared with every other Falcon environment. So up records what it installs in .falcon/multicast-<hex environment name>.json, and down and check read that record instead of scanning. The record, not the nexthop, is what makes a route owned by this specific environment: in isolated mode ce's address derives from [network] rather than from the environment name, so two environments would share a nexthop and it proves nothing. That record is written ahead of the route, so a crash between the two leaves a record with no route rather than a route nothing can prove is ours, and both directions fail closed: an unrecorded route is neither swept by up nor removed by down.

Teardown also has to work against a rack that has already gone away. ssh_try_capture separates an unreachable node (ssh exit 255) from a command that ran and failed, so down clears the host routes and the record when cr1 is unreachable while still reporting a genuine failure on a live router.

The check command also prints each group's control-plane mapping onto the underlay, read from swadm multicast list in every switch zone, naming the NAT target an external group replicates onto.

This work also repins sidecar-lite and omicron to the leaves of their respective multicast branches, zl/multicast and zl/mcast-build.

Most importantly, we add docs/multicast.md, covering the API side (pools, groups, members, probes), this plumbing, and the static production equivalents of each piece.

Also here:

• voxel commtest --traffic multicast refuses to start when the setup is missing, with --setup-mcast to plumb it in place. It also defaults --icmp-loss-tolerance to 500, the value omicron's a4x2 CI passes, and rejects an inverted --ip-pool-begin/--ip-pool-end pair up front.
• [falcon].ssh_pubkey stages a public key into every node, so ssh root@<node> authenticates by key rather than by the images' empty root password.
• voxel status reports each sled's rpool usage and zpool status -x health.
• Format arguments are inlined across the workspace, and commtest's mod test is renamed to mod tests.

Dependencies

The multicast stack spans multiple repositories, but voxel itself pins only two of them: the Omicron commit handed to voxel image create (OMICRON_REPO selects the clone source) and the sidecar-lite artifact rev the build fetches (SIDECAR_LITE_REV). Everything else rides the chosen Omicron commit's own pins, so pointing the image at the right Omicron commit pulls the whole set.

Repository	Branch / rev	PR	Carried by	Trajectory
omicron	zl/mcast-build, 521e1b39	omicron#11128 open, top of the PR stack below	workspace rack-init-config pin, via voxel image create	stack lands bottom-up into main, starting at omicron#9912
dendrite	multicast-e2e, 3d49b131	dendrite#224 open	Omicron tools/dendrite_* pins	dendrite#224 to main
maghemite	zl/ddm-mcast, 96a2f153	maghemite#696 open, stacked on zl/mrib; related maghemite#402 (zl/mgd-ddm-meta)	Omicron tools/maghemite_* pins	maghemite#696 to main after zl/mrib
opte	master, 0525f2f95 (0.41.506)	opte#1012 merged 2026-07-25	Omicron Cargo.toml / tools/opte_version	landed; Omicron pins 0.41.506 intentionally (master's later opte#1040 is a comment-only xde change)
propolis	zl/multicast, 3c07d60a	propolis#1093 open	Omicron package-manifest.toml (guest propolis-server)	propolis#1093 to master; host side also needs the viona V7 tables in docs/multicast.md
thundermuffin	zl/multicast-joiner, 486559bc	thundermuffin#14 open	Omicron package-manifest.toml (probe zone, prebuilt)	thundermuffin#14 to main
sidecar-lite	zl/multicast, 461cbe19	sidecar-lite#152 open	voxel image create (SIDECAR_LITE_REV)	sidecar-lite#152 to main
softnpu	zl/multicast, 284c6830	softnpu#183 open	Omicron tools/softnpu_version (xtask SOFTNPU_COMMIT)	softnpu#183 to main
p4	zl/multicast (p4rs)	p4#240 open	transitive, via softnpu and sidecar-lite lockfiles	p4#240 to main, then softnpu#183/sidecar-lite#152 repoint

Commtest Run (example)

The Setup

After launching voxel, we ran:

voxel network multicast up --group 239.100.0.1 --group 239.100.0.2 \
      --group 232.100.0.1 --group 239.100.0.9

voxel commtest --source /oxide/workspace/omicron --traffic multi -- run \
      --test-duration 200s --warmup 10s --packet-rate 10 \
      --mcast-group 239.100.0.1 \
      --mcast-group 239.100.0.2@172.30.199.199 \
      --mcast-group 232.100.0.1@172.30.199.199 \
      --mcast-deny-group 239.100.0.9@172.30.199.198

Group verification

Group	Type	Source filter	Members	Result
239.100.0.1	ASM	any source	4	✅ data-plane delivery verified
239.100.0.2	ASM	172.30.199.199 (host)	4	✅ data-plane delivery verified
232.100.0.1	SSM	172.30.199.199 (host)	4	✅ data-plane delivery verified
239.100.0.9	ASM, deny case	172.30.199.198 (excludes host)	4	✅ source-filter denial verified, zero delivery

Per-member connectivity

Group	Member	Sled	Tx	Rx	Lost	Loss %	RTT low (ms)	RTT high (ms)
239.100.0.1	198.51.100.30	g2	1998	1997	1	0.05	0.48	13.09
239.100.0.1	198.51.100.31	g0 (scrimlet)	1998	1998	0	0.00	0.46	7.02
239.100.0.1	198.51.100.32	g3 (scrimlet)	1998	1994	4	0.20	0.50	7.39
239.100.0.1	198.51.100.33	g1	1998	1997	1	0.05	0.51	14.79
239.100.0.2	198.51.100.30	g2	1998	1998	0	0.00	0.54	8.84
239.100.0.2	198.51.100.31	g0 (scrimlet)	1998	1998	0	0.00	0.50	9.44
239.100.0.2	198.51.100.32	g3 (scrimlet)	1998	1992	6	0.30	0.49	13.04
239.100.0.2	198.51.100.33	g1	1998	1998	0	0.00	0.52	8.26
232.100.0.1	198.51.100.30	g2	1997	1997	0	0.00	0.49	7.75
232.100.0.1	198.51.100.31	g0 (scrimlet)	1997	1997	0	0.00	0.51	12.71
232.100.0.1	198.51.100.32	g3 (scrimlet)	1997	1996	1	0.05	0.53	8.72
232.100.0.1	198.51.100.33	g1	1997	1997	0	0.00	0.54	13.76
239.100.0.9 (deny)	198.51.100.30-33	all	1997	0	1997	100 (expected)	—	—