Add emscripten_epoll_add_listener for epoll readiness callbacks - #27547
Open
guybedford wants to merge 5 commits into
Open
guybedford wants to merge 5 commits into
guybedford wants to merge 5 commits into
Conversation
guybedford
force-pushed
the
epoll-callback
branch
from
August 15, 2026 05:14
3b9ee9a to
1feb245
Compare
Collaborator
|
Can you rebase/merge? |
guybedford
force-pushed
the
epoll-callback
branch
from
August 18, 2026 00:06
aa06ebb to
f124226
Compare
Collaborator
Author
|
This is now rebased, with the local PR change isolated to commit 9a7b5d3. |
guybedford
force-pushed
the
epoll-callback
branch
4 times, most recently
from
August 18, 2026 22:21
d07dbaf to
ee21de3
Compare
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
A non-blocking readiness delivery mechanism for epoll: instead of blocking in epoll_wait, the runtime invokes registered listener callbacks on the event loop whenever the epoll set has ready events waiting to be collected (new experimental <emscripten/epoll.h>), working without ASYNCIFY/JSPI. A callback takes only its userdata and collects events itself via a zero-timeout epoll_wait(epfd, ..., 0). Firing is gated on the shared readiness derivation ($epollWouldBlock) also used by the epoll fd's own poll handler, so a stale ready-list entry never spuriously fires. Per-fd trigger modes apply exactly as in epoll_wait: a level fd left undrained re-fires every tick, an edge fd once per edge, a fired EPOLLONESHOT not until re-armed. Any number of listeners may be added, keyed by (callback, registering thread); re-adding the same identity updates userdata. Every listener is signalled while uncollected ready events remain (broadcast) and collectors race over the single shared ready list, so EPOLLET/EPOLLONESHOT items are collected by exactly one listener - the same load balancing as between multiple blocking epoll_wait callers on one epoll. Listeners hold a runtime keepalive while the set can still fire, keyed on the armed-registration count (a fired EPOLLONESHOT no longer counts): registered I/O interest holds the event loop open, following the Node.js model, and a terminal set (every watched fd closed) releases the runtime with no explicit disposal needed. Listeners are instance state shared across dup'd fds; the last close removes them all. Under pthreads the registration body runs sync-proxied on the main thread, so the registering thread is captured and each delivery is back-proxied to it via emscripten_proxy_callback (new system/lib/pthread/emscripten_epoll_callback.c), one delivery in flight at a time, paced by its completion (_emscripten_epoll_delivery_done) with a monotonic token dropping stale completions. While armed, each listener also holds its owner thread's keepalive so it survives to receive deliveries.
…xit guard Only armed registrations on host-backed fds (sockets; nested epolls counted conservatively) hold the runtime alive. A pipe can only be written by wasm, which is already running and held when it does, so a net-enabled runtime whose waker pipe stayed armed would otherwise never exit under EXIT_RUNTIME. A scheduled or in-flight delivery holds the runtime separately until it runs (as safeSetTimeout does), so a pipe write from live work still delivers; the post-callback re-wake moves inside the callUserCallback wrapper so that hold precedes maybeExit. emscripten_force_exit forfeits every hold before exitRuntime, whose FS.quit then closes the epoll fd and released the listener's hold, underflowing the counter. All epoll holds now go through one helper that treats a release on a zero counter as forfeited.
guybedford
force-pushed
the
epoll-callback
branch
from
September 14, 2026 17:37
ee21de3 to
acde4a3
Compare
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
A scheduled delivery holds the runtime until it runs, but a wake raised by teardown can never deliver, and a hold taken there outlives the exit: exitRuntime's FS.quit closes every open fd in fd order, and each close wakes the listener - the fd's own POLLNVAL, and for a pipe the peer end's close reporting POLLHUP on the still-armed registration. The hold then leaves keepRuntimeAlive() set when _proc_exit runs, so Module.onExit is skipped and the exit is left to the host loop draining. A registration now forwards the cause of its wake to the epoll node (POLLNVAL for a closing fd, POLLIN otherwise), and a listener wake holds only for a readiness wake while FS.initialized, which FS.quit clears before its first close. Under pthreads the FS.quit wakes were also what pushed a keepalive to an owner thread that had already exited. Test: a pipe registered and quiet at main's return, created before its epoll so its ends close first; the runtime exits (atexit) and the exit completes (onExit).
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
guybedford
added a commit
to guybedford/tokio
that referenced
this pull request
Sep 14, 2026
Follow-on to tokio-rs#8285 and tokio-rs#8438. A `LocalRuntime` driven by the host JavaScript event loop instead of by a park, so it needs neither JSPI nor pthreads and never blocks or suspends the host. The scheduler drives to a fixed point, then waits. On this target a wait is either a JSPI suspension (`block_on`) or a return to the host to be called back; the event-loop runtime is the latter. The same `current_thread` scheduler and drivers run one `event_interval` batch per drive followed by the zero-duration driver turn a park would do (due timers, I/O readiness, deferred wakers), then the wait is lowered to arming host callbacks: an immediate drive if runnable work remains, so the host loop gets a turn between batches as under a JSPI `block_on`, and one `emscripten_set_timeout` for the soonest timer deadline. Timers are fire-only with an epoch (`emscripten_clear_timeout` leaks its keepalive) and kept across drives while the deadline is unchanged. Socket readiness re-enters the drive through a persistent `emscripten_epoll_add_listener` on the reactor's epoll fd, which also carries the mio waker, so an external unpark of an I/O-enabled runtime needs no extra arming; the I/O-less parker arms an immediate drive. A wake during a drive is absorbed by that drive's own turn and re-arm. Host callbacks run on an empty stack, so a drive re-enters inline; armed callbacks hold `Weak` refs, so dropping the `EventLoopRuntime` has native `Runtime::drop` semantics and a late callback upgrades to nothing. `block_on` on an event-loop runtime is rejected eagerly like a nested runtime: its wait is the host loop, so no stack can hold the result. `EventLoopRuntime::schedule` takes a root and a completion callback (`Err(JoinError)` on a root panic). The API is `tokio_unstable` and gated to non-pthread builds; a drive from a host callback while another runtime's `block_on` is suspended under JSPI still panics as a nested runtime. Tests: `rt_emscripten_event_loop` pins the on-stack contracts (schedule queues only, one batch per drive, host-context wakes never drive inline, cross-runtime wakes, `block_on` and nested drive rejected, drop cancels). `rt_emscripten_event_loop_main` (`harness = false`) schedules roots on two runtimes and returns into the host loop, completing them through timer re-arms, a greedy yielding sibling, a cross-runtime oneshot and a TCP round trip between the runtimes, with and without `-sJSPI`; the pre-js fails the run if the loop drains before the runtime exits. CI adds an unstable lane. TEMPORARY: it overlays the emscripten `epoll-callback` branch (emscripten-core/emscripten#27547) for `emscripten_epoll_add_listener` until released.
A delivery was queued as a microtask. Hosts may drain the microtask queue synchronously inside unrelated calls (workerd does on a Node builtin load, which its connect() path performs), so a listener ran re-entrantly under the frames of the wasm call that had just made the set ready. Schedule deliveries with emSetImmediate instead; the two tests that ordered a check after deliveries by timeout now queue it as a later immediate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-on to #27207 implementing the non-blocking readiness callback model for epoll, split out of that PR per review feedback.
emscripten_epoll_add_listener(new experimental<emscripten/epoll.h>) registers a persistent readiness listener on an epoll fd: the runtime invokes the callback on the event loop whenever the set has uncollected ready events, and the callback collects them itself via a zero-timeoutepoll_wait(epfd, ..., 0). It never suspends the stack, so event-loop reactors can drive epoll both with and without JSPI.emscripten_epoll_remove_listenerremoves by identity.EPOLLET/EPOLLONESHOTitems are collected by exactly one listener - the same distribution semantics as between multiple blockingepoll_waitcallers on one epoll.Tests cover delivery and MOD re-arm with no producer event, edge and level modes on the callback path, multi-listener broadcast/load-balancing, registration identity and errors, drain-across-ticks with maxevents=1, nesting, dup sharing, terminal-set runtime release (including nested), a suspended blocking epoll_wait and a listener sharing one ready list (ASYNCIFY and JSPI), and real-socket delivery including PROXY_TO_PTHREAD cross-thread dispatch.
Made with AI assistance under my review