Description
The diffusers backend does not cancel image generation when the requesting client disconnects. A request whose client has timed out or closed the connection still runs to completion — model load included — and the response is written to a socket nobody reads. Subsequent requests queue behind these abandoned jobs, so client-side retries (the natural reaction to a timeout) compound the problem instead of recovering from it.
Likely mechanism (from source)
In python/diffusers_server/server.py, create_image is an async def endpoint that calls the synchronous generate_images pipeline directly — no thread executor, no request.is_disconnected() check, no cancellation path. The blocking call occupies the event loop for the full generation, so the server cannot observe the disconnect even in principle, and every other request serializes behind it.
Environment
- Docker Model Runner v1.2.6 (Docker Desktop), macOS, Apple Silicon (M1 Pro, 16 GB). The handler is unchanged on current
main and in v1.2.7/v1.2.8 release notes.
- Model:
stable-diffusion (DDUF, diffusers backend, MPS float16)
Steps to reproduce
- Ensure the diffusers runner is not resident (cold start), so the first request also pays the model load.
POST /engines/diffusers/v1/images/generations with a client timeout shorter than load + generation (e.g. 120 s while the DDUF load alone takes 2+ min on a memory-pressured host).- Let the client time out and close the connection. Send a second request from another client.
Observed
- The first request's generation starts after its client is gone and runs to completion; the engine logs
200 OK to the disconnected socket. In one instance the host slept mid-queue and the engine completed the abandoned job ~6 hours after the client vanished, immediately on wake.
- The second request waits behind the abandoned job and can itself time out having never started.
- The abandoned work holds the model's residency (~7 GB for stable-diffusion) and the GPU for its full duration.
Expected
Client disconnect (or at minimum, connection close before generation starts) cancels the pending/running job — the behavior the llama.cpp chat path already exhibits via request context cancellation. Alternatively: a documented statement that diffusers jobs are uncancellable once accepted, so clients can choose long timeouts over retries.
Impact
Any client that enforces a per-request timeout (agent tool loops, gateways, proxies) turns one slow cold start into a pile-up: each timeout triggers a retry that queues behind a job whose consumer no longer exists.
Description
The diffusers backend does not cancel image generation when the requesting client disconnects. A request whose client has timed out or closed the connection still runs to completion — model load included — and the response is written to a socket nobody reads. Subsequent requests queue behind these abandoned jobs, so client-side retries (the natural reaction to a timeout) compound the problem instead of recovering from it.
Likely mechanism (from source)
In
python/diffusers_server/server.py,create_imageis anasync defendpoint that calls the synchronousgenerate_imagespipeline directly — no thread executor, norequest.is_disconnected()check, no cancellation path. The blocking call occupies the event loop for the full generation, so the server cannot observe the disconnect even in principle, and every other request serializes behind it.Environment
mainand in v1.2.7/v1.2.8 release notes.stable-diffusion(DDUF, diffusers backend, MPS float16)Steps to reproduce
POST /engines/diffusers/v1/images/generationswith a client timeout shorter than load + generation (e.g. 120 s while the DDUF load alone takes 2+ min on a memory-pressured host).Observed
200 OKto the disconnected socket. In one instance the host slept mid-queue and the engine completed the abandoned job ~6 hours after the client vanished, immediately on wake.Expected
Client disconnect (or at minimum, connection close before generation starts) cancels the pending/running job — the behavior the llama.cpp chat path already exhibits via request context cancellation. Alternatively: a documented statement that diffusers jobs are uncancellable once accepted, so clients can choose long timeouts over retries.
Impact
Any client that enforces a per-request timeout (agent tool loops, gateways, proxies) turns one slow cold start into a pile-up: each timeout triggers a retry that queues behind a job whose consumer no longer exists.