-
Notifications
You must be signed in to change notification settings - Fork 12
Description
Summary
native_scheduler exists solely to cache a timer_service* pointer, bypassing the service registry on hot paths. This was necessary because capy's find_service/use_service acquires a mutex and does an O(n) linked list scan on every call. As the number of services grows (tcp, udp, acceptor, timer, signal, resolver), this caching pattern doesn't scale — we'd keep adding *_svc_ pointers to a base class that shouldn't know about its dependents.
Why the cache exists
Two hot paths depend on it:
-
Scheduler poll loop — every reactor cycle calls
timer_svc_->nearest_expiry()andtimer_svc_->process_expired(). A mutex + linear scan per poll is unacceptable. -
Timer construction — every
timer(ctx)callstimer_service_direct()which readsnative_scheduler::timer_svc_to avoid a registry lookup. Servers creating thousands of per-request timers would contend on the registry mutex.
Why Asio doesn't have this problem
Asio's registry has the same design (mutex + linked list), but its access pattern is different:
- Services call
use_service<T>()once in their constructor and store the result as a member reference. The registry is never hit at runtime. - Timers are templates parameterized on the service type — the service lookup happens once per
io_contextwhen the service is first created. Each timer object holds a direct reference to its service. No per-timer-construction lookup.
Corosio's compilation firewall makes timer a concrete class, so timer.cpp must find its service at runtime without templates. That's the fundamental tension.
Current state
| Service | Created | Accessed via |
|---|---|---|
timer_service |
Eagerly by scheduler ctor | native_scheduler::timer_svc_ (cached) |
tcp_service |
make_service during backend init |
find_service/use_service from .cpp |
udp_service |
make_service during backend init |
find_service/use_service from .cpp |
tcp_acceptor_service |
make_service during backend init |
find_service/use_service from .cpp |
resolver_service |
Eager (IOCP) / lazy (POSIX) | use_service |
signal_service |
Lazy | use_service |
Only timer_service gets the fast path. The others pay the mutex cost but aren't yet on hot enough paths to notice. As usage grows, more services will need the same treatment.
Proposed fix
Make service lookup O(1) and lock-free so the caching layer becomes unnecessary:
-
Add an indexed service slot mechanism to capy's
execution_context— each service type gets a small integer key (similar to Asio'sservice_id). The context stores a fixed-size array (or growable vector) of service pointers indexed by key. Lookup becomes an array index — no mutex, no scan. -
Remove
native_scheduler— concrete schedulers store their owntimer_service&as a local member (grabbed once at construction viause_service), same as Asio's pattern. The base class layer disappears. -
Timer construction uses O(1) lookup —
timer_service_direct()and itsnative_schedulerdependency are eliminated.timer(ctx)does a fast indexed lookup instead.
Alternative (smaller scope)
If changing capy's registry is too invasive:
- Add a thread-local cache in
find_service— first lookup per service per thread is slow, all subsequent lookups are a pointer read. This would makenative_schedulerunnecessary without changing the registry's data structure.