Reduce allocations significantly - #742
Merged
Merged
Conversation
The membership loop iterated us2.scopes and tested each element against us2.scopes — trivially true — so any two UnionScopes of equal length compared equal. Compare us1's scopes against us2's. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TaintScope and its taint Set are never mutated, so all callers can share one instance instead of allocating a fresh Set + Dict + TaintScope per call. Also makes the default scope identity-stable, which downstream fast paths can rely on (ScopedValue-free identity tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Dagger benchmarks:
|
| master | dirty | master / dirty | |
|---|---|---|---|
| array/dagger/N=1024 (block 512)/add (X + X) | 2.79 ± 1.6 ms | 2.67 ± 0.2 ms | 1.04 ± 0.61 |
| array/dagger/N=1024 (block 512)/alloc (rand) | 2.13 ± 2.4 ms | 1.48 ± 0.33 ms | 1.44 ± 1.6 |
| array/dagger/N=1024 (block 512)/broadcast (X .+ 1) | 2.15 ± 1.9 ms | 2.53 ± 3.9 ms | 0.849 ± 1.5 |
| array/dagger/N=1024 (block 512)/map (sin.(X)) | 6.63 ± 1.4 ms | 6.67 ± 0.15 ms | 0.994 ± 0.21 |
| array/dagger/N=1024 (block 512)/norm | 2.12 ± 0.65 ms | 1.12 ± 1.4 ms | 1.89 ± 2.4 |
| array/dagger/N=1024 (block 512)/reduce (sum) | 2.05 ± 2.3 ms | 4.41 ± 1.7 ms | 0.465 ± 0.55 |
| array/dagger/N=1024 (block 512)/transpose (permutedims) | 4.6 ± 1.6 ms | 4 ± 0.9 ms | 1.15 ± 0.48 |
| array/dagger/N=256 (block 256)/add (X + X) | 0.766 ± 0.051 ms | 0.798 ± 0.033 ms | 0.959 ± 0.075 |
| array/dagger/N=256 (block 256)/alloc (rand) | 0.718 ± 0.76 ms | 0.688 ± 0.14 ms | 1.04 ± 1.1 |
| array/dagger/N=256 (block 256)/broadcast (X .+ 1) | 0.632 ± 2.1 ms | 0.454 ± 0.11 ms | 1.39 ± 4.6 |
| array/dagger/N=256 (block 256)/map (sin.(X)) | 1.01 ± 0.033 ms | 1.1 ± 0.04 ms | 0.914 ± 0.045 |
| array/dagger/N=256 (block 256)/norm | 0.489 ± 0.027 ms | 0.463 ± 0.078 ms | 1.06 ± 0.19 |
| array/dagger/N=256 (block 256)/reduce (sum) | 0.871 ± 0.045 ms | 0.813 ± 0.07 ms | 1.07 ± 0.11 |
| array/dagger/N=256 (block 256)/transpose (permutedims) | 0.768 ± 0.065 ms | 0.794 ± 0.22 ms | 0.968 ± 0.27 |
| linalg/dagger/N=1024 (block 512)/cholesky | 23.2 ± 3.3 ms | 23.1 ± 4.7 ms | 1.01 ± 0.25 |
| linalg/dagger/N=1024 (block 512)/lu | 0.0448 ± 0.004 s | 0.0448 ± 0.0065 s | 0.999 ± 0.17 |
| linalg/dagger/N=1024 (block 512)/matmul (A*A) | 0.0491 ± 0.0067 s | 0.0435 ± 0.0038 s | 1.13 ± 0.18 |
| linalg/dagger/N=1024 (block 512)/matvec (A*x) | 2.84 ± 2.2 ms | 2.98 ± 0.99 ms | 0.955 ± 0.81 |
| linalg/dagger/N=1024 (block 512)/qr | 0.116 ± 0.0092 s | 0.115 ± 0.0028 s | 1.01 ± 0.084 |
| linalg/dagger/N=1024 (block 512)/solve (A\b via lu) | 0.0588 ± 0.0084 s | 0.0602 ± 0.0054 s | 0.977 ± 0.17 |
| linalg/dagger/N=1024 (block 512)/svd | 0.0403 h | 0.0399 h | 1.01 |
| linalg/dagger/N=1024 (block 512)/syrk (A'*A) | 0.034 ± 0.0081 s | 0.0338 ± 0.0057 s | 1.01 ± 0.29 |
| linalg/dagger/N=256 (block 256)/cholesky | 2.55 ± 2 ms | 4.46 ± 1.1 ms | 0.572 ± 0.48 |
| linalg/dagger/N=256 (block 256)/lu | 3.6 ± 2.3 ms | 4.03 ± 0.86 ms | 0.894 ± 0.6 |
| linalg/dagger/N=256 (block 256)/matmul (A*A) | 2.05 ± 1.1 ms | 1.91 ± 0.51 ms | 1.07 ± 0.62 |
| linalg/dagger/N=256 (block 256)/matvec (A*x) | 0.971 ± 0.067 ms | 2.41 ± 0.3 ms | 0.403 ± 0.057 |
| linalg/dagger/N=256 (block 256)/qr | 6.53 ± 1.3 ms | 4.96 ± 0.15 ms | 1.31 ± 0.27 |
| linalg/dagger/N=256 (block 256)/solve (A\b via lu) | 8.99 ± 8 ms | 10.2 ± 1.1 ms | 0.879 ± 0.79 |
| linalg/dagger/N=256 (block 256)/svd | 0.607 ± 0.0086 s | 0.649 ± 0.024 s | 0.935 ± 0.037 |
| linalg/dagger/N=256 (block 256)/syrk (A'*A) | 3.79 ± 1.3 ms | 3.43 ± 2.5 ms | 1.1 ± 0.91 |
| stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) | 7.87 ± 1 ms | 7.8 ± 1.9 ms | 1.01 ± 0.27 |
| stencil/dagger/N=1024 (block 512)/assign (const) | 1.1 ± 2 ms | 1.06 ± 0.43 ms | 1.04 ± 1.9 |
| stencil/dagger/N=1024 (block 512)/multi-expr | 4.04 ± 2.3 ms | 4.7 ± 1.8 ms | 0.859 ± 0.59 |
| stencil/dagger/N=1024 (block 512)/neighbors (Clamp) | 6.44 ± 0.48 ms | 6.3 ± 0.34 ms | 1.02 ± 0.094 |
| stencil/dagger/N=1024 (block 512)/neighbors (Pad) | 7 ± 1.4 ms | 5.82 ± 0.62 ms | 1.2 ± 0.27 |
| stencil/dagger/N=1024 (block 512)/neighbors (Reflect) | 5.28 ± 1.5 ms | 6.19 ± 0.41 ms | 0.853 ± 0.26 |
| stencil/dagger/N=1024 (block 512)/neighbors (Wrap) | 5.57 ± 0.77 ms | 5.97 ± 0.61 ms | 0.933 ± 0.16 |
| stencil/dagger/N=1024 (block 512)/update (+) | 2.15 ± 1.7 ms | 1.81 ± 1.4 ms | 1.18 ± 1.3 |
| stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) | 1.42 ± 0.42 ms | 1.34 ± 0.024 ms | 1.07 ± 0.31 |
| stencil/dagger/N=256 (block 256)/assign (const) | 0.476 ± 0.012 ms | 0.493 ± 0.026 ms | 0.966 ± 0.056 |
| stencil/dagger/N=256 (block 256)/multi-expr | 1.11 ± 0.2 ms | 0.91 ± 0.028 ms | 1.22 ± 0.23 |
| stencil/dagger/N=256 (block 256)/neighbors (Clamp) | 1.24 ± 0.039 ms | 1.29 ± 0.31 ms | 0.961 ± 0.24 |
| stencil/dagger/N=256 (block 256)/neighbors (Pad) | 1.28 ± 0.078 ms | 2.24 ± 1.3 ms | 0.571 ± 0.33 |
| stencil/dagger/N=256 (block 256)/neighbors (Reflect) | 1.27 ± 0.034 ms | 1.27 ± 0.11 ms | 1 ± 0.09 |
| stencil/dagger/N=256 (block 256)/neighbors (Wrap) | 1.19 ± 0.058 ms | 1.28 ± 0.16 ms | 0.933 ± 0.13 |
| stencil/dagger/N=256 (block 256)/update (+) | 0.577 ± 2.4 ms | 0.573 ± 0.12 ms | 1.01 ± 4.2 |
| time_to_load | 1.12 ± 0.012 s | 1.09 ± 0.0027 s | 1.02 ± 0.011 |
Allocations / memory
| master | dirty | master / dirty | |
|---|---|---|---|
| array/dagger/N=1024 (block 512)/add (X + X) | 5.38 k allocs: 8.21 MB | 5.46 k allocs: 8.22 MB | 1 |
| array/dagger/N=1024 (block 512)/alloc (rand) | 2.63 k allocs: 8.09 MB | 2.66 k allocs: 8.09 MB | 1 |
| array/dagger/N=1024 (block 512)/broadcast (X .+ 1) | 2.34 k allocs: 8.08 MB | 2.34 k allocs: 8.08 MB | 1 |
| array/dagger/N=1024 (block 512)/map (sin.(X)) | 1.99 k allocs: 8.07 MB | 2.04 k allocs: 8.07 MB | 1 |
| array/dagger/N=1024 (block 512)/norm | 2.62 k allocs: 0.0881 MB | 2.67 k allocs: 0.0898 MB | 0.982 |
| array/dagger/N=1024 (block 512)/reduce (sum) | 3.75 k allocs: 0.13 MB | 4.74 k allocs: 0.167 MB | 0.777 |
| array/dagger/N=1024 (block 512)/transpose (permutedims) | 3.2 k allocs: 8.13 MB | 3.24 k allocs: 8.13 MB | 1 |
| array/dagger/N=256 (block 256)/add (X + X) | 1.69 k allocs: 0.574 MB | 1.68 k allocs: 0.573 MB | 1 |
| array/dagger/N=256 (block 256)/alloc (rand) | 0.739 k allocs: 0.525 MB | 0.733 k allocs: 0.525 MB | 1 |
| array/dagger/N=256 (block 256)/broadcast (X .+ 1) | 0.636 k allocs: 0.522 MB | 0.636 k allocs: 0.522 MB | 1 |
| array/dagger/N=256 (block 256)/map (sin.(X)) | 0.559 k allocs: 0.519 MB | 0.559 k allocs: 0.519 MB | 1 |
| array/dagger/N=256 (block 256)/norm | 0.7 k allocs: 24.3 kB | 0.7 k allocs: 24.3 kB | 1 |
| array/dagger/N=256 (block 256)/reduce (sum) | 1.28 k allocs: 0.0464 MB | 1.29 k allocs: 0.0465 MB | 0.999 |
| array/dagger/N=256 (block 256)/transpose (permutedims) | 1.08 k allocs: 0.55 MB | 1.09 k allocs: 0.551 MB | 1 |
| linalg/dagger/N=1024 (block 512)/cholesky | 7.62 k allocs: 10.3 MB | 7.67 k allocs: 10.3 MB | 1 |
| linalg/dagger/N=1024 (block 512)/lu | 15.3 k allocs: 14.6 MB | 15.2 k allocs: 14.6 MB | 1 |
| linalg/dagger/N=1024 (block 512)/matmul (A*A) | 7.54 k allocs: 8.28 MB | 7.55 k allocs: 8.28 MB | 1 |
| linalg/dagger/N=1024 (block 512)/matvec (A*x) | 5.95 k allocs: 0.24 MB | 6.13 k allocs: 0.246 MB | 0.977 |
| linalg/dagger/N=1024 (block 512)/qr | 11.2 k allocs: 9.59 MB | 10.9 k allocs: 9.58 MB | 1 |
| linalg/dagger/N=1024 (block 512)/solve (A\b via lu) | 29 k allocs: 15.2 MB | 29.3 k allocs: 15.2 MB | 0.999 |
| linalg/dagger/N=1024 (block 512)/svd | 0.0479 M allocs: 0.197 GB | 0.0481 M allocs: 0.197 GB | 1 |
| linalg/dagger/N=1024 (block 512)/syrk (A'*A) | 8.64 k allocs: 24.4 MB | 8.7 k allocs: 24.4 MB | 1 |
| linalg/dagger/N=256 (block 256)/cholesky | 2.99 k allocs: 0.633 MB | 2.99 k allocs: 0.632 MB | 1 |
| linalg/dagger/N=256 (block 256)/lu | 5.34 k allocs: 1.24 MB | 5.34 k allocs: 1.24 MB | 1 |
| linalg/dagger/N=256 (block 256)/matmul (A*A) | 1.89 k allocs: 0.581 MB | 1.91 k allocs: 0.581 MB | 0.999 |
| linalg/dagger/N=256 (block 256)/matvec (A*x) | 2.32 k allocs: 0.0985 MB | 2.32 k allocs: 0.0984 MB | 1 |
| linalg/dagger/N=256 (block 256)/qr | 3.39 k allocs: 0.778 MB | 3.4 k allocs: 0.778 MB | 1 |
| linalg/dagger/N=256 (block 256)/solve (A\b via lu) | 11.8 k allocs: 1.53 MB | 11.8 k allocs: 1.53 MB | 1 |
| linalg/dagger/N=256 (block 256)/svd | 15.3 k allocs: 6.71 MB | 15.3 k allocs: 6.71 MB | 1 |
| linalg/dagger/N=256 (block 256)/syrk (A'*A) | 2.92 k allocs: 2.63 MB | 2.94 k allocs: 2.63 MB | 1 |
| stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) | 4.53 k allocs: 8.2 MB | 4.6 k allocs: 8.2 MB | 1 |
| stencil/dagger/N=1024 (block 512)/assign (const) | 2.03 k allocs: 0.0908 MB | 2.08 k allocs: 0.093 MB | 0.976 |
| stencil/dagger/N=1024 (block 512)/multi-expr | 4.56 k allocs: 0.205 MB | 4.63 k allocs: 0.207 MB | 0.989 |
| stencil/dagger/N=1024 (block 512)/neighbors (Clamp) | 3.7 k allocs: 0.231 MB | 3.68 k allocs: 0.23 MB | 1 |
| stencil/dagger/N=1024 (block 512)/neighbors (Pad) | 3.57 k allocs: 0.227 MB | 3.6 k allocs: 0.228 MB | 0.995 |
| stencil/dagger/N=1024 (block 512)/neighbors (Reflect) | 3.65 k allocs: 0.26 MB | 3.68 k allocs: 0.261 MB | 0.999 |
| stencil/dagger/N=1024 (block 512)/neighbors (Wrap) | 3.48 k allocs: 0.164 MB | 3.49 k allocs: 0.164 MB | 1 |
| stencil/dagger/N=1024 (block 512)/update (+) | 2.5 k allocs: 0.113 MB | 2.56 k allocs: 0.115 MB | 0.981 |
| stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) | 1.5 k allocs: 0.573 MB | 1.52 k allocs: 0.574 MB | 0.999 |
| stencil/dagger/N=256 (block 256)/assign (const) | 0.796 k allocs: 0.0405 MB | 0.796 k allocs: 0.0405 MB | 1 |
| stencil/dagger/N=256 (block 256)/multi-expr | 1.71 k allocs: 0.0866 MB | 1.7 k allocs: 0.0862 MB | 1 |
| stencil/dagger/N=256 (block 256)/neighbors (Clamp) | 1.27 k allocs: 0.0723 MB | 1.26 k allocs: 0.0722 MB | 1 |
| stencil/dagger/N=256 (block 256)/neighbors (Pad) | 1.18 k allocs: 0.0695 MB | 1.18 k allocs: 0.0695 MB | 1 |
| stencil/dagger/N=256 (block 256)/neighbors (Reflect) | 1.23 k allocs: 0.0788 MB | 1.23 k allocs: 0.0785 MB | 1 |
| stencil/dagger/N=256 (block 256)/neighbors (Wrap) | 1.19 k allocs: 0.063 MB | 1.19 k allocs: 0.063 MB | 1 |
| stencil/dagger/N=256 (block 256)/update (+) | 0.907 k allocs: 0.0457 MB | 0.913 k allocs: 0.0461 MB | 0.992 |
| time_to_load | 0.147 k allocs: 10.8 kB | 0.147 k allocs: 10.8 kB | 1 |
Plots
⚠️ Regressions (time > 25.0% and outside the reported ±spread; allocs/memory > 25.0%)
linalg/dagger/N=256 (block 256)/matvec (A*x)(time): +147.9%array/dagger/N=1024 (block 512)/reduce (sum)(memory): +28.7%array/dagger/N=1024 (block 512)/reduce (sum)(allocs): +26.4%
Within noise (6 metric(s) past threshold but inside the ±spread; not counted)
array/dagger/N=1024 (block 512)/reduce (sum)(time): 114.9%stencil/dagger/N=256 (block 256)/neighbors (Pad)(time): 75.0%linalg/dagger/N=256 (block 256)/cholesky(time): 74.7%array/dagger/N=256 (block 256)/broadcast (X .+ 1)(time): -28.1%array/dagger/N=1024 (block 512)/alloc (rand)(time): -30.5%array/dagger/N=1024 (block 512)/norm(time): -47.1%
Full results and plots (download the benchmark-results artifact).
jpsamaroo
marked this pull request as ready for review
August 25, 2026 21:54
memory_spaces(::ThreadProc/OSProc) built a fresh Set per call and processors(::CPURAMMemorySpace) rebuilt a filtered Set (with an OSProc cache lock round-trip) per call — both on datadeps planning hot paths. Cache them per owner; the processors cache is invalidated alongside OSPROC_PROCESSOR_CACHE when processor callbacks change. The returned Sets are shared and must not be mutated by callers (no caller does). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Direct construction of the (isbits) CPURAMMemorySpace instead of Set iteration + boxing tuple getindex in dynamic contexts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FireTaskSpec, do_tasks, and do_task each constructed a throwaway Context (empty procs Vector + ReentrantLock + Threads.Condition, ~13 allocations) per task/batch, purely to carry (log_sink, profile) into timespan logging. Cache one Context per (log_sink, profile) pair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every task argument move spawned a full Threads.@Spawn Task and entered a with(SCHED_MOVE=>true) ScopedValues scope — even on shared memory where the move is a no-op Chunk unwrap. Two acceleration hooks fix this without changing distributed/MPI semantics: - argument_move_may_inline: local Chunks (and plain values) moving to a ThreadProc run move_one() inline; remote Chunks stay async so network fetches still overlap, and uniform execution (MPI) keeps its rank-ordered scheduling. - with_sched_move: only MPI observes Sch.SCHED_MOVE inside moves, so the default acceleration now runs the body with no scope at all and MPIExt overrides the hook to install it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compatible_processors rebuilt ProcessScopes/ExactScopes and a Set per scheduled task, but its inputs are steady across tasks (datadeps exec scopes are LFU-cached per processor), so a bounded LFU turns it into a lookup. Also skip the second Set construction for DistributedAcceleration, whose accel_matches_proc is identically true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No field is ever assigned, so mutability is semantically inert — but as an immutable 23-field struct it was stored inline in closures and re-boxed (~200B) on every dynamic call that took it. As a mutable struct it is passed by pointer everywhere. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The procs_to_use default (which copies ctx.procs) was evaluated even when the ready list was empty, i.e. on every task completion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ctx/procs_filt/p1_resolved were outer locals assigned inside the state.lock closure, so each got a Core.Box per call. Return them from the locked block instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The steal and kick paths did randperm + a getindex broadcast (Ref, Broadcasted, two Vectors) on every attempt, which is pure overhead in the common single-processor case; shuffled_states also drops the broadcast in favor of direct indexing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
transfer_time/transfer_size are only written inside the commented-out CHUNK_CACHE block, so each task paid two Threads.Atomic allocations for counters that always read zero. Plain zeros until that code returns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-dispatch "thunk $thunk_id" interpolation allocated per task, feeding a tracked-list rename that the reuse machinery no longer performs (the tracked list is only consulted for precompile cleanliness). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
execute!(::ThreadProc, ...) spawned and fetched a second Task per thunk even though do_task already runs on a pooled task pinned to the target thread. Run inline in that case (TLS is already correct); errors are wrapped in CapturedException as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Most tokens are never waited on, and a Base.Event costs ~5 allocations (Event + condition + lock chain). Install the event on first wait via CAS; cancel! sets the cancelled flag before reading the event and waiters re-check the flag after installing, so no wakeup can be missed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
set_tls! allocated a fresh DTaskTLS per executed thunk; pooled scheduler tasks call it once per thunk and nothing retains the old TLS across thunks (get_tls() callers copy if they keep it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tochunk_pset slurped and re-splatted kwargs (rebuilding NamedTuple + Pairs per layer) and accepted-then-discarded type. Use explicit keywords ordered to match poolset's signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
with_options built a ScopedValues Scope + HAMT insert even for an empty NamedTuple (the common case in datadeps regions, where propagated options are empty). merge(prev, ()) is a no-op, so just call f(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The proc kwarg default (OSProc(), a lock + finalizer toggle) was evaluated even when move_value=false makes it unused — compute it lazily. Skip the thunk_yield closure entirely when not executing inside a DTask (its non-task branch is exactly f()). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No field of an ArgPosition is ever assigned; immutability lets it be stored inline in Argument/TypedArgument and copied by value, removing a heap allocation per argument copy. ArgPosition(::ArgPosition) and copy become the identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
signature(state, task) was recomputed (Vector{Any} + views + hashing)
at least twice per task — once when scheduling and once when recording
its runtime costs. Cache it on the Thunk; it is only computed after
inputs are resolved (signature() throws otherwise), so a premature
result can never be stored, and a benign write race stores equal
values.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hash(T, h) with T::Any is a dynamic call whose UInt return gets boxed — 16 boxes per Signature for an 8-arg task. Hash objectid(T) instead; Signature hashes never cross process boundaries (they key only the scheduler-local signature_time_cost/signature_alloc_cost tables and the task-local default cache). Also only allocate the kwarg name/type vectors when the signature actually has kwargs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
all_equal was assigned inside the worker_transfer_rate lock closure after capture (a Core.Box per call) — return it from the locked block instead. Skip the randperm-based shuffle when there is only one candidate processor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Spawn's expansion constructs a fresh Options(;...) per call, provably unaliased, so the defensive copy that spawn/typed_spawn must perform for user-supplied Options is pure overhead there (a 30-field 248B mutable struct per task). Dagger.spawn/typed_spawn still copy first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every spawn built a Symbol[] and ran append!/filter!/unique! even with no scoped options. nothing is observably identical to an empty vector (get_propagated_options returns an empty NamedTuple for both, and the populate_defaults! default is nothing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each @reuse_scope entry allocated a fresh Vector{ReuseCleanup} plus a
ScopedValues Scope + HAMT insert (and each @reuse_defer_cleanup a
RefValue + closure + ReuseCleanup + Some from the ScopedValue read).
Replace the ScopedValue with a per-task stack of pooled cleanup frames:
scope entry reuses a frame vector, registration pushes into the current
frame, and ReuseCleanup drops the RefValue for a plain mutable field.
Cleanups are always registered lexically within the scope's own body
(never from a spawned child task), so task-local storage suffices.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Renaming the errormonitor tracked-list entry on every pooled-task dispatch was an O(n) locked scan for a debugging-only list (pooled tasks are registered once at init). Also fix the non-reusable fallback error handler, which referenced an undefined variable and would have thrown UndefVarError while reporting the original error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With the flat planner's per-task cost now in the tens of microseconds, partitioned planning loses to flat at every tested single-owner scale on 8 threads (576 tasks: 26ms flat vs 84ms partitioned; 4096 tasks: 187ms vs 384ms) — the partition machinery costs more than the planning it parallelizes. Multi-owner regions still partition by data ownership. 8-thread 256x256 mul!: 189ms -> 26ms (2x faster than serial; scaling is now bounded by serial planning time). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
equiv_chunks was a WeakKeyDict{DRef,Chunk} whose value strongly rooted
its own key (chunk.handle === key), so no entry could ever be collected:
every task result — Chunk, DRef, RefState, and the result data behind
the datastore entry — leaked for the life of the eager scheduler
(measured: +64 datastore entries per 80-task datadeps region, forever).
Values are now WeakRefs, restoring equiv_chunks to a pure dedupe cache.
That immortal pin was also load-bearing: resolved input slots were
wrap_weak'd on the assumption something else kept results alive, so a
weak equiv_chunks alone would let a result expire between its producer's
teardown and a consumer firing. collect_task_inputs!/
resolve_finished_input! therefore now store the resolved result Chunk
STRONGLY in the consumer's input slot — the consumer owns its inputs
until its own teardown, giving results an exact, bounded lifetime.
After the fix the datastore is size-stable across repeated regions
(96 -> 96 entries over 10 runs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every planned task allocated a Set{ThunkSyncdep} (plus internal
arrays). On the synchronous submission path the set is provably dead
once eager submission has consumed it: reschedule_syncdeps! drains the
planner-recorded ThunkID-form entries, the submission-window loop is
the last real reader, and only nothing-tolerant debug printers look
afterwards. distribute_task! now takes sets from a global pool and
reclaims them right after enqueue, guarded by syncdeps_consumed (all
entries in WeakThunk form — deferred/batched submission leaves the
planner form in place and skips the reclaim). Also removes the stale
WeakThunk references those sets pinned, a precondition for Thunk
pooling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Argument chunks were weakened to WeakChunk on the assumption that equiv_chunks pinned the wrapper; with equiv_chunks now a weak dedupe cache (the immortal pin was the result leak), a weak slot could expire before the consumer fires (e.g. the caller drops the chunk right after @Spawn). The consuming task now owns its inputs until its own teardown, matching the resolved-result convention — and saves the WeakChunk + WeakRef pair per chunk argument. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With Thunk pooling, a stale WeakThunk could otherwise resolve to a recycled Thunk now representing a different task (silent corruption). The wrapper records the referent's id at wrap time; pool reset zeroes thunk.id, so any stale reference resolves to nothing (which unwrap_weak_checked turns into a loud assertion) instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With Thunk pooling, a second invocation (an explicit early release followed by the GC-driven DRef destructor, or a destructor double-fire) would tear down a recycled Thunk now representing someone else's task. CAS on a done flag makes teardown exactly-once. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One FutureNode per registered future and one DepNode per dataflow edge were allocated per task and dropped at the drain walks. The walks own their chain exclusively after the seal (double-seal returns nothing), so drained nodes are reset and returned to global pools; pushes that lose to a seal recycle their node instead of dropping it. Node fields became non-const to allow reset; nulling DepNode.thunk also releases the downstream Thunk reference promptly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every task allocated a fresh 16-field Thunk. task_delete! is the point where a Thunk is unreachable from every root except the caller's frame (thunk_dict and strong_thunks entries are removed here; upstream DepNodes drained at its completion; downstream input WeakThunks were replaced by strong result Chunks at resolution; syncdep WeakThunks are reclaimed post-submission), so it now returns the Thunk to a global pool — but only when the caller passes recycle=true: the failure paths (finish_task!'s failed branch, set_failed!) run schedule_dependents! AFTER deletion, and a recycled thunk's re-SEALED dependents head would silently skip failure propagation. Success-path finish_task! and unref_thunk! opt in; pool reset zeroes thunk.id so any stale WeakThunk (which validates ids) resolves loudly to nothing, and UnrefThunk's one-shot CAS prevents double teardown of a recycled thunk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TaskSpec (now mutable, with Union-typed reference fields so a pooled
spec pins nothing) plus its data Vector{Argument} and Argument cells
are borrowed from a per-ProcessorInternalState free list in fire_tasks!
(one queue-lock acquisition per batch, local fires only) and returned
at the very end of DoTaskSpec — a fully deterministic, GC-independent
death point. Guards: cancelled tasks, streaming functions, remote
fires, torn-down processor state, and in-flight istate bookkeeping all
fail open to a fresh allocation; shrinking a reused data vector nulls
the dropped cells first so pooled specs cannot pin stale values.
~6-7 allocations saved per task.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 30-field, ~250B mutable Options allocated per @Spawn now comes from a pool on the macro path (whose Options is provably unaliased) and is recycled with its owning Thunk in recycle_thunk!. Reset uses setfield! to skip setproperty!'s scope-coupling side effects, and the pooled take mirrors the @kwdef constructor's compute_scope = scope default. User-supplied Options passed to Dagger.spawn are still defensively copied and never pooled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Print an allocations/memory table alongside the median-time table, and extend the regression gate to cover allocation count and allocated bytes (BENCHMARK_ALLOC_REGRESSION_THRESHOLD, default 25%) in addition to median time. Regressions and improvements are now reported per-metric. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deep-warmup + min-of-N measurement of per-call GC allocation counts and bytes for key operations (task spawn, empty datadeps region, in-/out-of- place matmul, cholesky+solve, stencil, fft!, broadcast, sum), pinned to a single processor so results are topology-independent. Bounds are set at ~2x the current measured steady state, catching order-of-magnitude regressions while tolerating version/platform variance; the suite prints a measured-vs-bound table for recalibration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codifies the working conventions from the allocation-reduction effort: tight commits, before/after memory profiling, type-stability review, and the @nospecialize / separate-paths discipline. Agents append new lessons as they learn them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each per-processor runner captured istate.ctx once at spawn (an old FIXME: 'Context changes aren't noticed over time'). A runner created while logging was enabled kept emitting proc_run_wait/proc_run_fetch/steal events — and running the full MultiEventLog consumer battery, ~600 allocations per task — forever after disable_logging!; conversely, runners created while logging was disabled never emitted runner events after enable_logging!. ProcessorInternalState is now mutable: do_tasks refreshes istate.ctx per batch (from the same fire-time ctx_vars it already resolves), and the runner re-reads it each loop iteration. Found via the allocations test suite: running it after test/logging.jl inflated matmul! from 28k to 73k allocs/call (spawn: 234 -> 856), with Profile.Allocs pointing at TimespanLogging write_event/EventSaturation/ GCStats/CompileTimeMetrics on the runner path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Julia copies `current_task().scope` -- the `ScopedValues` chain, which is what backs `Dagger.with_options` -- into every new `Task`. That is right for a task doing work on behalf of its creator, but wrong for Dagger's long-lived and pooled tasks: the eager scheduler task, the per-processor runners, and the `ReusableTaskCache` pools that carry `fire_tasks!` payloads and execute thunks are all created lazily, by whichever call first needs them, and then serve every later caller for the rest of the session. When that first call sits inside a `with_options(...)` block, those options become the ambient scope of every task the pool later runs: `do_task` applies the task's *propagated* options via `with_options`, which merges into the ambient `NamedTuple`, so a task that asked for nothing still observes the stale ones. This is order- and topology-dependent, which is what made it show up now: with the allocations suite running first, all of its work (and hence much of the scheduler's lazy setup) happens inside `with_options(scope=ExactScope(ThreadProc(1,1)))`, and under `-t4` the extra `ThreadProc`s get their runners and executor pools created later still, from whatever scoped block first fires work at them. The visible failure was test/options.jl:37 -- a plain `@spawn` observing `scope == ExactScope(ThreadProc(1,1))` where it must see `AnyScope()` -- reproducible with `julia -t4 test/runtests.jl` and green in isolation. Fixed by clearing the inherited scope on those tasks before they start (`clear_task_scope!`; the field is only settable pre-start). Per-task option propagation is explicit, so they want no ambient scope at all. `init_eager`'s `@async` becomes `@task` + `schedule` for the same reason, keeping the sticky flag it had. test/allocations.jl now asserts no leak at the end of its scoped block, where this is deterministic and cheap to diagnose; without the fix it fails there under any topology. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`_spawn` took the `propagates` vector straight out of the ambient scoped options and then `append!`ed every scoped option name to it, `filter!`ed `:task_queue` out of it, `unique!`d it, and finally stored it as the new task's own `propagates`. That vector belongs to the caller's `with_options(propagates=[...])` block and is shared by every task spawned under it, so each spawn permanently rewrote the user's own vector -- growing it, dropping `:task_queue` from it, and racing with any concurrent spawn in the same block. The `Tuple` form already copied; only the `Vector` form aliased. The same applied to a `propagates` handed directly to a task (via `@spawn propagates=v` or a `Dagger.Options`, whose `copy` is shallow): `append!`/`unique!` mutated the caller's vector in place. Both are now copied before mutation. This costs one extra `Vector` only when `propagates` is actually set, which is off the common spawn path -- the allocation suite's bounds are unaffected. Regression test in test/options.jl asserts both vectors are unchanged after spawning. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test/diskcaching.jl` ran four `cache_setup_test.jl` children with `wait=true` and stdout inherited from the testsuite. When one of those children died inside the Julia runtime (`!!! ERROR in jl_ -- ABORTING !!!`), it dumped its object graph -- unbounded, and full of cyclic `Core.Binding`/`CodeInstance` records -- straight into the test log and then spun at 100% CPU forever. The observed instance wrote ~13 GB into the log, filled /tmp, and hung the whole `julia -t4 test/runtests.jl` run for 11 hours with no output (the parent's own stdout was still block-buffered, so the run looked merely slow). Children now write to a captured temp file and are killed if they exceed a 900s deadline; a failing or timed-out child gets a truncated copy of its output reported via `@warn` so the crash stays diagnosable, and both the timeout and the exit code are asserted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The benchmark job called any metric past its threshold a regression or an improvement, with no regard for how well the two runs were actually resolved. Benchmarks whose timing wanders by more than 10% between runs therefore reported a result on essentially every build, and the job was routinely red for no reason -- which trains everyone to ignore it. A change is now only counted when it is both large (past the threshold) and resolvable (bigger than the run-to-run spread). The noise model is the one the results table already prints -- timings are shown as `median ± (q75 - q25)` -- so a change counts only when those bands don't overlap, and the report can never contradict its own table. `BENCHMARK_NOISE_TOLERANCE` scales the band; 0 restores the previous threshold-only behaviour. Only timings carry a spread: each BenchmarkTools trial records the allocation count and byte count of a single post-warmup evaluation, so `allocs`/`memory` have no distribution and stay gated on their threshold alone. Suppressed changes aren't discarded -- they're listed in a collapsed "Within noise" section of the report and printed in the summary, so genuine drift is still visible without failing the build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test/thunk.jl` asserted `EAGER_CONTEXT[] === nothing` to check that the eager scheduler is only started on the first `@spawn`. That only holds when thunk.jl runs first in the process, so any subset run that reaches it after a suite which spawns (e.g. `--test options --test thunk`) failed there for no real reason. The canonical full-suite order works only because test/allocations.jl restores virgin state at its end. Reset to virgin state here too, and assert both halves of the property (no context before the first spawn, a context after it) so the test now checks laziness rather than the suite ordering. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`reschedule_syncdeps!` creates no dependents edge for an upstream that has
already finished -- correctly, since there is nothing left to wait on. But
resolving a consumer's input slots is done *by walking those edges*
(`schedule_dependents!` -> `resolve_finished_input!`), so a finished upstream's
slot was never resolved and kept a **weak** `Thunk` reference.
A finished dep is also deliberately never marked `sch_accessible` (that flag
would never be cleared again, leaking every completed producer named as a
syncdep -- see `eager_submit_internal!`). So as soon as the user dropped that
upstream's `DTask`, `unref_thunk!` deleted and recycled the `Thunk`, the pool
handed the object back out with a fresh id, and `unwrap_weak` started returning
`nothing`. If the consumer was still waiting on a second, unfinished upstream,
it was scheduled later -- and `collect_task_inputs!` then tripped
AssertionError: !(isweak(t)) || t_val !== nothing
inside a completion handler running on a pooled task whose error handler only
logs, so the consumer was never scheduled and the scheduler hung. Reproduced
deterministically (finished `a` + running `c` -> `a + c`, drop `a`, churn the
Thunk pool) with the exact assertion and stack seen in multithreaded CI.
Resolve the slots at submission instead, while the upstream is still alive and
holding its result -- in both the already-finished case and the case where it
finishes between the check and `deps_push!`.
Thunk pooling is what made this bite: without recycling, the weak reference
stayed resolvable until GC actually collected the Thunk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`schedule_ready!` runs on whatever thread completed an upstream: usually a pooled completion task, whose error handler only logs (`reusable_task_loop`), or -- past the inline fan-out threshold -- a detached `Threads.@spawn` whose exception nobody ever fetches. A thunk is credited to `running_count` before it gets there, so any error escaping `schedule_one!` lost the thunk silently: never fired, never finished, its credit never released, and every `fetch` on it -- plus `compute_dag`'s termination check -- blocked forever. That is how the `unwrap_weak_checked` assertion fixed in the previous commit presented in CI: a logged error, then a hung run. A scheduler bug should surface as a failed task, not as a hang. `schedule_one_guarded!` converts an escaping error into a failure of that thunk, carrying a `CapturedException` so the `DTaskFailedException` a `fetch`er sees keeps the original backtrace. A thunk that already reached `fire_tasks!` is left alone: it owns its own result and `running_count` credit, and re-releasing it would corrupt the counter. Verified by reverting the previous commit: the reproducer that used to hang now raises a `DTaskFailedException` reporting the real `AssertionError`, and the process continues. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both cost real debugging time this round: an unresolved weak input slot that only died once Thunk pooling recycled the upstream, and an assertion on a pooled completion task that turned into an 11-hour hang instead of a crash. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The streaming suite hangs, rather than fails, and only under `--code-coverage`: 9 hangs / 10 runs of `julia --code-coverage -t auto test/runtests.jl --test streaming -p 0` pinned to 4 cores, against 0 / 19 across three non-coverage topologies. `test_finishes` gives `ignore_timeout=true` tests a 10s budget, but with coverage instrumentation the *first* cold compile of the streaming submission path takes ~23s. The budget therefore expired while the test body was still inside its first `Dagger.@spawn`, so the single `cancel!(;halt_sch=true, graceful=false)` found nothing to cancel and just halted the scheduler. The `fetch(Dagger.@Spawn 1+1)` after it started a *replacement* scheduler, the streaming task the body went on to submit landed there with `stream_max_evals=1_000_000` and nobody left to stop it, and both `fetch(x)` and `fetch(t)` blocked forever -- with no `Test Summary` ever printed. It is sporadic in real CI because a full run warms the eager path first, cutting that first block to ~3.9s, under the budget. Cancel in a loop until the testing task has actually finished, and `error()` rather than `fetch()` if it never does, so a future variant of this fails loudly instead of hanging a CI job with no output. Two related margins found while measuring, same class of bug: - the teardown test's `sleep(1); @test istaskdone(x)` has a *negative* margin in a cold process (teardown measured 1.19-1.49s, 10/10 samples over 1.0s); now a `timedwait`. It failed twice over, since the following `fetch(x)` then blocked until the whole test's budget expired. - that same test is asserted to finish yet passed `ignore_timeout=true`, so it kept the 10s budget instead of the generous hang-detector budget 8aff138 introduced for finishing tests. Verified: 3/3 runs of the previously-hanging config now pass 565/565, steady at ~2m45s each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`take!` decremented the atomic `count` *before* reading the slot, and the producer treats a decremented count as free space. When the buffer is full -- exactly the case backpressure exists for -- the next slot the producer writes is the one the consumer has not read yet, so values were silently lost or duplicated. `count` is the only handshake between the two sides, so it has to be published last on both: the producer fills the slot then increments, and the consumer now reads the value out then decrements. Reproduced deterministically with a 2-slot buffer driven by two threads pushing 200k sequential integers: the consumer skipped values on the first trial every time (e.g. read 32765 where 32763 was due -- both slots overwritten). Zero mismatches after the fix across capacities 1, 2, 3 and 8, 10 trials each. `collect!` also re-read `count` per iteration while a producer could be growing it; snapshot it once instead. Adds a `ProcessRingBuffer` testset that drives the buffer from two threads at capacity 2 and checks the *value sequence* rather than counts -- the previous assertions only checked ranges and counts, which stay perfectly self-consistent under this bug. It fails on the old code and passes on the new. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`sleep` throws `EOFError` when the timer is closed out from under it during process shutdown, and `errormonitor` then prints an unhandled task error -- after the testsuite has already printed "Tests Completed!". Noise that reads like a failure on CI. Treat it as the normal exit for this loop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jpsamaroo
force-pushed
the
jps/less-allocations
branch
from
August 25, 2026 23:25
256fcdb to
b85f4a5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For too long Dagger has allocated tons of memory for even small problems, causing excessive GC load and hurting scalability. This PR is a first step towards eliminating most of these allocations, starting with cutting out about 80% of allocations on a representative GEMM benchmark.
Pairs with JuliaData/MemPool.jl#100
Written by Claude Fable and friends 🧚