Per-service API client with identifying headers

[nicksieger]

What I did

Tag the Docker API requests that Compose makes on behalf of a specific service with HTTP headers identifying the project and service the request
originates from:

• X-Docker-Compose-Project
• X-Docker-Compose-Service

This lets the daemon (and anything in front of it) attribute an incoming API call to the Compose project/service it came from.

Why

Today every Compose request reaches the daemon indistinguishable from any other. Attributing container-lifecycle calls to their originating service makes daemon-side logging, auditing, and policy far easier, and gives operators a reliable signal for which service triggered a given create/start/pull.

How

The moby client applies custom HTTP headers per client instance, not per request — there is no per-request header hook or request-mutation callback (WithHTTPHeaders is the only lever; the sole hook, WithResponseHook, is response-side). So a single shared client can't carry service-specific headers.

This PR introduces a small per-service client layer:

• serviceClient(project, service) accessor + cache (serviceClientCache): memoizes one API client per project/service pair (guarded by a mutex, since per-service pulls run concurrently under an errgroup). Clients are owned by the composeService and closed via Close().
• newServiceClient rebuilds an equivalent client from the CLI's Docker endpoint via docker.Endpoint.ClientOpts() — mirroring how docker/cli constructs its own client (endpoint, TLS/socket/conn-helper, negotiated API version, compose/ user-agent) — then layers the identifying headers on top of any user-configured HTTPHeaders. In --dry-run mode the scoped client is re-wrapped in DryRunClient so dry-run interception is preserved.

Call sites routed through the per-service client:

• Image pull (/images/create).
• Up-path container lifecycle — ContainerCreate, the legacy-API (< 1.44) per-network NetworkConnect and its ContainerRemove cleanup, the post-create ContainerInspect, and ContainerStart.

Project-wide calls (networks, volumes, events, project-scoped container listings, Ping) intentionally stay on the shared client. Dependency health-poll inspects (isServiceHealthy/isServiceCompleted) also stay on the shared client, since they read a dependency's containers and don't have unambiguous service attribution.

Testing

• TestServiceClientHeaders: spins up an httptest daemon and asserts an /images/create request carries the project/service headers, that CLI-configured headers are preserved, that repeated calls return the cached client, and that a different service yields a distinct client.
• TestResolvePullPlatforms: table test for the pull-platform resolution helper.
• Existing TestCreateMobyContainer* tests seed the client cache with the mock so the service-scoped calls resolve to it.
• go test ./pkg/compose/ green; golangci-lint run --build-tags "e2e" reports 0 issues.

Related issue

(not mandatory) A picture of a cute animal, if possible in relation to what you did

[docker-agent]

Assessment: 🟢 APPROVE

The per-service API client implementation is well-structured and correct. Key aspects reviewed:

• Cache correctness: The mutex-guarded serviceClientCache correctly handles concurrent access from concurrent pull goroutines and container lifecycle calls. Lazy map initialization is safe (inside the lock).
• DryRun wrapping: newServiceClient correctly re-wraps the freshly-built client in DryRunClient when s.dryRun == true, preserving dry-run interception semantics.
• Resource lifecycle: Close() properly drains and closes all cached clients. The DryRunClient.Close() delegates to the inner client's Close(), so no resource leak on the dry-run path.
• Header precedence: User-configured headers are copied first, then compose-specific headers are set — ensuring compose attribution headers always win, which is the intended behavior.
• API version pinning: newServiceClient pins ClientVersion() from the shared client, ensuring consistent API version negotiation across all service clients.
• Test coverage: TestServiceClientHeaders exercises header forwarding, CLI header preservation, cache hit/miss, and close semantics. TestResolvePullPlatforms covers the extracted helper.

No bugs introduced by this PR.

[ndeloof]

On the client duplication in newServiceClient

newServiceClient re-implements docker/cli's newAPIClientFromEndpoint (cli/command/cli.go): same endpoint.ClientOpts(), same config.json HTTPHeaders merge, same client.New(). Beyond the N clients / N connection pools per project, the copy already drifts from the original: docker/cli also merges DOCKER_CUSTOM_HEADERS (withCustomHeadersFromEnv()), which this copy misses — requests made through a service-scoped client will silently drop headers that every other compose request carries. That's the cost of duplicating the construction: every future change on the docker/cli side has to be mirrored here or it becomes a behavioral difference.

The root cause is that the moby client only supports per-instance headers (WithHTTPHeaders). I've opened moby/moby#53347 adding client.WithRequestHeaders(ctx, headers) — request-scoped headers carried by the context, symmetric to the existing WithResponseHook. Once available, this whole file reduces to enriching ctx at the service-scoped call sites, keeping the single CLI-provided client (dry-run wrapping included).

Since the duplication is a temporary workaround, please mark it as such, e.g.:

// TODO(compose): temporary workaround duplicating docker/cli's client construction
// because custom headers are per-instance. Remove once request-scoped headers land
// upstream (https://github.com/moby/moby/pull/53347) and enrich ctx at call sites instead.

Alternative achieving the same today, with no moby change and no duplication

The moby client wraps its transport in otelhttp.NewTransport unconditionally (client.go). otelhttp injects whatever the global propagator produces on every request — including the W3C Baggage header, independent of any active span or tracer. So:

1. internal/tracing switches its propagator to NewCompositeTextMapPropagator(TraceContext{}, Baggage{}) (today it sets TraceContext{} only);
2. the call sites this PR already touches do ctx = baggage.ContextWith(ctx, ...) with compose.project / compose.service members;
3. the existing s.dockerCli.Client() emits Baggage: compose.project=...,compose.service=... per request.

Bonus: compose already injects the global propagator into provider-plugin child processes (pkg/compose/shellout.go), and BuildKit propagates baggage natively — attribution would follow those paths too, which dedicated X-Docker-Compose-* headers never will.

Two caveats: the daemon side must read the Baggage header (W3C, percent-encoded) instead of dedicated headers, and this relies on the unconditional otelhttp transport wrapping — an implementation choice of the moby client, not a documented guarantee, that would need to be committed to long-term if we build on it.