Skip to content

feat(ingress): opt-in PROXY protocol for the client address - #130

Merged
kvinwang merged 2 commits into
Dstack-TEE:mainfrom
kingsleydon:feat/ingress-proxy-protocol
Oct 9, 2026
Merged

kvinwang merged 2 commits into
Dstack-TEE:mainfrom
kingsleydon:feat/ingress-proxy-protocol

Conversation

@kingsleydon

Copy link
Copy Markdown
Contributor

Why

The gateway relays TCP, so HAProxy sees every connection as coming from the gateway, and the backend sees it as coming from HAProxy. Apps behind dstack-ingress therefore can't rate-limit, log, or block by client address; all clients share one address.

The dstack gateway already supports the fix. With the app's port_policy set to {"port": 443, "pp": true} (Phala Cloud: the "Proxy protocol (pp)" switch), the gateway prepends a PROXY protocol header carrying the client's address. Gateway support landed in 87ce31cd and shipped in gateway 0.5.9. dstack-ingress is the missing hop: its public bind doesn't read the header, and its backends don't receive one.

What

Two opt-in switches, both defaulting to false. With both off, the generated config is unchanged.

  • ACCEPT_PROXY_PROTOCOL=true adds accept-proxy to the public bind:

    • tls_peek in tls-alpn-01 mode;
    • tls_in in dns-01 mode.

    HAProxy then uses the header's address everywhere, including in logs. A connection without the header is refused, so the gateway's pp and this switch have to change in the same deployment.

  • SEND_PROXY_PROTOCOL=true adds send-proxy-v2 to the app backends, for both TARGET_ENDPOINT and ROUTING_MAP. For connections accepted with accept-proxy, HAProxy advertises the client's address. The ACME responder and the evidence backend never get a header.

  • In tls-alpn-01 mode the hop from tls_peek to the loopback TLS frontend already uses send-proxy-v2 / accept-proxy, so the client's address survives it unchanged.

  • Both values are validated by a new sanitize_boolean; anything other than true or false stops startup.

  • README: two env table rows and a "Client address (PROXY protocol)" section, covering the gateway side and the backend side (Caddy's proxy_protocol listener wrapper, nginx's proxy_protocol with set_real_ip_from).

Tests

  • scripts/tests/test_proxy_protocol.sh <image> (new, in the style of test_x25519mlkem768.sh). It builds the config with the real haproxy-lib.sh helpers and runs HAProxy, checking that:

    • the default config has neither keyword;
    • with both switches on, a connection carrying a PROXY v2 header from 203.0.113.9 reaches the backend as 203.0.113.9;
    • a connection without a header is refused.

    All pass against 2.6 with this branch's scripts/ copied in.

  • scripts/tests/test_sanitizers.sh now covers sanitize_boolean. All pass.

  • shellcheck -x reports no new findings against main, and the new test is clean.

Not tested here: a real CVM behind a gateway (tier 3 in TESTING.md). We intend to run that on our deployment, which will be the first to enable pp.

🤖 Generated with Claude Code

kingsleydon and others added 2 commits October 8, 2026 11:32
The gateway relays TCP, so haproxy sees connections from the gateway and the
backend from haproxy. The dstack gateway can prepend the client's address with
the PROXY protocol (the app's port_policy, `pp`), but the public bind did not
read it and the backends did not get it.

ACCEPT_PROXY_PROTOCOL=true makes the public bind (both modes) read the header,
and SEND_PROXY_PROTOCOL=true passes the client's address to the backends with
PROXY protocol v2. Both default to false; the config is otherwise unchanged.
In tls-alpn-01 mode the internal hop already carries the PROXY protocol, so the
address reaches the TLS frontend too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: the port_policy entry must name PORT, not a fixed 443; guest OS
  0.5.9+ reports the policy at registration, so public_tcbinfo is only
  needed on older images; require gateway 0.5.9+; note that anything
  reaching PORT without the gateway fails, that SEND_PROXY_PROTOCOL covers
  every ROUTING_MAP backend, and the gateway's inbound_pp_enabled when it
  sits behind an L4 load balancer.
- Warn when SEND_PROXY_PROTOCOL is set without ACCEPT_PROXY_PROTOCOL: the
  backend would get the gateway's address.
- Shorten the haproxy-lib.sh comment, which claimed the client's address
  is used in either case.
- Test backend: read the PROXY header fully instead of a single recv.

Signed-off-by: Kevin Wang <wy721@qq.com>
@kvinwang
kvinwang merged commit 7dc9e7e into Dstack-TEE:main Oct 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants