Skip to content

Remove native_scheduler caching layer; fix service lookup for compilation firewall #212

@sgerbino

Description

@sgerbino

Summary

native_scheduler exists solely to cache a timer_service* pointer, bypassing the service registry on hot paths. This was necessary because capy's find_service/use_service acquires a mutex and does an O(n) linked list scan on every call. As the number of services grows (tcp, udp, acceptor, timer, signal, resolver), this caching pattern doesn't scale — we'd keep adding *_svc_ pointers to a base class that shouldn't know about its dependents.

Why the cache exists

Two hot paths depend on it:

  1. Scheduler poll loop — every reactor cycle calls timer_svc_->nearest_expiry() and timer_svc_->process_expired(). A mutex + linear scan per poll is unacceptable.

  2. Timer construction — every timer(ctx) calls timer_service_direct() which reads native_scheduler::timer_svc_ to avoid a registry lookup. Servers creating thousands of per-request timers would contend on the registry mutex.

Why Asio doesn't have this problem

Asio's registry has the same design (mutex + linked list), but its access pattern is different:

  • Services call use_service<T>() once in their constructor and store the result as a member reference. The registry is never hit at runtime.
  • Timers are templates parameterized on the service type — the service lookup happens once per io_context when the service is first created. Each timer object holds a direct reference to its service. No per-timer-construction lookup.

Corosio's compilation firewall makes timer a concrete class, so timer.cpp must find its service at runtime without templates. That's the fundamental tension.

Current state

Service Created Accessed via
timer_service Eagerly by scheduler ctor native_scheduler::timer_svc_ (cached)
tcp_service make_service during backend init find_service/use_service from .cpp
udp_service make_service during backend init find_service/use_service from .cpp
tcp_acceptor_service make_service during backend init find_service/use_service from .cpp
resolver_service Eager (IOCP) / lazy (POSIX) use_service
signal_service Lazy use_service

Only timer_service gets the fast path. The others pay the mutex cost but aren't yet on hot enough paths to notice. As usage grows, more services will need the same treatment.

Proposed fix

Make service lookup O(1) and lock-free so the caching layer becomes unnecessary:

  1. Add an indexed service slot mechanism to capy's execution_context — each service type gets a small integer key (similar to Asio's service_id). The context stores a fixed-size array (or growable vector) of service pointers indexed by key. Lookup becomes an array index — no mutex, no scan.

  2. Remove native_scheduler — concrete schedulers store their own timer_service& as a local member (grabbed once at construction via use_service), same as Asio's pattern. The base class layer disappears.

  3. Timer construction uses O(1) lookuptimer_service_direct() and its native_scheduler dependency are eliminated. timer(ctx) does a fast indexed lookup instead.

Alternative (smaller scope)

If changing capy's registry is too invasive:

  • Add a thread-local cache in find_service — first lookup per service per thread is slow, all subsequent lookups are a pointer read. This would make native_scheduler unnecessary without changing the registry's data structure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions