Skip to content

Pin Chromium's proxy with managed policy on filtered sessions - #442

Closed
yummybomb wants to merge 19 commits into
mainfrom
hypeship/pin-egress-proxy-policy
Closed

yummybomb wants to merge 19 commits into
mainfrom
hypeship/pin-egress-proxy-policy

Conversation

@yummybomb

@yummybomb yummybomb commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Chromium reaches the egress proxy because it is launched with --proxy-server, but Chromium applies proxy settings in the order policy, extensions, command line. An extension with the proxy permission can call chrome.proxy.settings.set({ value: { mode: "direct" } }) and every request after that leaves the VM directly, around the egress proxy and the allowlist it enforces.

This is reachable by a client that only speaks CDP. --enable-unsafe-extension-debugging is on, so Extensions.loadUnpacked goes through the DevTools proxy, and the files can be put on disk with Browser.setDownloadBehavior plus two blob downloads. Against a real allowlisted session, that chain served an unlisted host straight from its origin.

#439 closed the Target.createBrowserContext route. This closes the extension route: while the session is filtered, the VM pins Chromium's proxy with the managed ProxySettings policy, which extensions cannot override.

  • egresspolicy.Pin writes /etc/chromium/policies/managed/zz-kernel-egress.json, a file beside policy.json rather than a key inside it. Chromium merges every file in the directory, so the pin never races the writers of policy.json.
    • Where two files set the same policy, the one that sorts last wins (except for list policies named in PolicyListMultipleSourceMergeList, below), so the name keeps the pin ahead of policy.json. ProxySettings is also reserved from customer chrome_policy overrides.
    • Chromium reads every file in the directory whatever its name, so the pin is staged in the parent /etc/chromium/policies/ and renamed in. A temporary file left in the directory would be applied as policy.
  • The pin also sets WebRtcIPHandling: disable_non_proxied_udp and clears WebRtcIPHandlingUrl. WebRTC sends UDP straight to the network rather than through an HTTP proxy. With only the proxy pinned, page JS still gathered a server-reflexive candidate from a public STUN server; with this policy it gathered none. An extension cannot override it either. Chromium consults the per-URL WebRtcIPHandlingUrl rules before WebRtcIPHandling, and a chrome_policy override of [{"url":"*","handling":"default"}] brought the server-reflexive candidate back with the pin in place, so the pin sets that policy to an empty list.
    • Chromium merges a list policy from every file that sets it, rather than taking the last, when PolicyListMultipleSourceMergeList names it. A chrome_policy override naming WebRtcIPHandlingUrl there, together with a */default rule, merged that rule into the pin's empty list, and a page gathered a host UDP candidate with the pin in place. The pin also sets PolicyListMultipleSourceMergeList to an empty list; with it, the page gathered none.
  • Nothing in the pin comes from runtime flags. Once ProxySettings is set, Chromium ignores --proxy-bypass-list as well as --proxy-server, so the pin carries both. Runtime flags can be written by anyone holding the session's token, and a pinned runtime value of either would route traffic around the egress proxy.
    • The proxy server is taken from the base flags (CHROMIUM_FLAGS), where the control plane sets the egress proxy. A runtime --proxy-server has no effect while the session is filtered.
    • The bypass list is the session's private_hosts, which the control plane now sends in PUT /network/egress-policy (kernel/kernel#4714). That is the one instance endpoint customers cannot write. When the control plane sends none, the pin uses the base flags' bypass list, which is the image's default private ranges. A runtime --proxy-bypass-list has no effect while the session is filtered. The VM rejects entries containing ;, , or whitespace, which Chromium would split into extra rules.
  • The launcher ignores runtime --host-resolver-rules and --host-rules while filtered. Chromium applies them to the egress proxy's own address whatever the pin says: against Chromium 154 with the pin in place, --host-resolver-rules=MAP <proxy-ip> <other>:<port> sent every request to the other proxy. The launcher logs what it drops.
  • The pin sets DnsOverHttpsMode: off. Chromium connects to a DNS-over-HTTPS server directly, not through the proxy, whenever it resolves a name itself, as it does for a bypassed hostname. With the pin in place and a DnsOverHttpsMode: secure chrome policy naming a server, Chromium connected to that server directly; with off pinned, it made no connection. The control plane already refuses any DnsOverHttpsMode other than off together with an allowlist at create, but chrome policy can also be written at runtime.
  • The policy directory is root-owned. The wrapper used to chown /etc/chromium/policies to kernel, the user Chromium runs as, although only the root API process writes there; that left the pin in a directory the browser itself could write, where a file sorting after the pin would override it. The chown is gone, and before writing the pin the launcher makes the policy directory and the staging directory root-owned and not writable by anyone else. The handler compares the file at that path byte for byte with the pin the launcher would write for the policy and the base flags, so anything else there, including a pin for other private_hosts, gets the restart that writes the real one. Chromium running as root (RUN_AS_ROOT=true) is not covered by this.
  • The launcher writes the pin on every Chromium start, from the base flags and the recorded policy, so a change to either (for example new private_hosts) can never leave a stale pin behind. The handler derives the pin it expects the same way, through egresspolicy.BaseFlags. If the session is filtered and the pin cannot be written, or the base flags have no --proxy-server to pin, the launcher exits rather than starting Chromium unpinned.
  • PUT /network/egress-policy:
    • filtered: true with no pin restarts Chromium so the launcher writes it, and checks the pin is there before answering. Chromium would also pick up a pin written in place, since it reloads its policy directory on its own, but only after a delay, so the restart puts the pin in force before the control plane hands the session to a client.
    • filtered: true with the pin already present for the same policy does nothing, so re-applying it (e.g. on an allowlist replace) never restarts Chromium. A pin written for different private_hosts gets a restart. The check reads the pin on disk rather than the recorded policy, so a retry after a restart that failed before the launcher ran restarts again instead of reporting the old pin applied.
    • filtered: false removes the pin without a restart; Chromium reloads and hands the proxy back to the command line, so customer proxy extensions work again. Chromium normally picks the removal up within about 5s. The exception is a removal in the first moments after Chromium starts, before it has installed the watch on its policy directory (a best-effort task, so it can be late under load): Chromium notices that change when it installs the watch, but the periodic 15-minute reload it schedules next replaces the reload it scheduled for the change. A session unfiltered that soon after a restart stays pinned for up to 15 minutes, which fails closed. The e2e test waits for the watch before unfiltering.
    • The handler now holds chromiumConfigMu, since it can restart Chromium.
  • A failed restart answers 500, and the session stays recorded as filtered, so the next Chromium start pins the proxy and the CDP refusal from Refuse CDP browser contexts that set their own proxy on filtered sessions #439 stays on.

Deploy order

Ship kernel/kernel#4714 first. It sends private_hosts, which images without this change ignore. If this image ships first, sessions with an allowlist and private_hosts get the image default bypass, so their private hosts are sent through the egress proxy until the control plane catches up. Rolling the control plane back after both ship has the same effect, and also restarts Chromium on the next allowlist PATCH, because the policy no longer matches the pin.

What the pin does not cover

ProxySettings does not override the per-context proxy from Target.createBrowserContext. Checked against a real browser: with the pin in place, newContext({ proxy }) still connected directly. So the refusal in #439 remains necessary, and the two together close both routes for a CDP client.

As with #439, code running inside the VM is out of scope: it can egress without using Chromium at all. The same session JWT that writes runtime flags also reaches POST /process/exec, so the runtime-flag hardening here does not make the token holder an in-scope adversary. It removes the routes that go through Chromium's own configuration. VM-level egress enforcement is still the fix for a token holder.

Cost

On a session created with an allowlist, applying the policy restarts Chromium once: about 3s in the local Docker image, the same as a flags PATCH. It happens only the first time a session is filtered, which is at session setup, before any client is connected. That relies on the control plane filtering a session only when it is created: adding an allowlist to a running session would restart Chromium under the customer's open pages. Re-applying true and applying false do not restart.

Validation

  • go vet ./... and go test -race for the whole unit suite pass.
  • New and updated unit tests:
    • The pin follows base flags and policy: no private_hosts keeps the base bypass list (the image default, or a --proxy-bypass-list in CHROMIUM_FLAGS, as derived through BaseFlags), a list replaces it, an empty list bypasses nothing, and the last base --proxy-server wins. It refuses to pin when the base flags have no --proxy-server.
    • DropHostMappingFlags drops --host-resolver-rules and --host-rules, including single-dash and bare forms, and keeps everything else.
    • private_hosts survives an API process restart, with nil and [] kept apart, and GET /network/egress-policy reports it.
    • The pin sits beside policy.json, sorts after it, and is staged outside the managed directory. The pin's directories are closed to the group and to other users, each checked on its own. Matches rejects anything but the policy's own pin (including a pin for other private_hosts, a missing DnsOverHttpsMode and a missing PolicyListMultipleSourceMergeList), and a failed write leaves no temporary file in either directory.
    • Handler: the pin present, re-applied, and removed; changed private_hosts restart Chromium while unchanged ones do not, including on a retry after a failed restart; malformed private_hosts (;, , or any Unicode whitespace) answer 400; a failed restart answers 500 while the session stays filtered.
    • Mutation-checked: skipping the policy comparison, or ignoring private_hosts in the pin, fails the matching test. Weakening Matches to accept any pin fails both the pin and the handler retry tests.
  • TestEgressProxyPin passes against a headless image built from this revision. It first applies filtered: true without private_hosts and checks the pin keeps the image's default bypass; that request only answers 200 if the launcher and the handler agree on the pin. It then applies filtered: true with private_hosts, then writes chrome policy (WebRtcIPHandlingUrl: [{"url":"*","handling":"default"}], PolicyListMultipleSourceMergeList: ["WebRtcIPHandlingUrl"], DnsOverHttpsMode: secure) and runtime flags (--proxy-server, --proxy-bypass-list=*;exfil.example.com, --host-resolver-rules=MAP ...) through the instance API. It then checks that:
    • the pin carries the base proxy, exactly the private_hosts, an empty WebRtcIPHandlingUrl, an empty PolicyListMultipleSourceMergeList and DnsOverHttpsMode: off;
    • no process was started with --host-resolver-rules;
    • both policy directories are root 755;
    • a CDP-loaded extension reports not_controllable for the proxy and for WebRTC;
    • a loopback page gathers no UDP candidates;
    • after filtered: false, control and UDP candidates come back without a restart.
      An image built with the host-mapping drop disabled fails the test on that assertion. An image whose pin has no PolicyListMultipleSourceMergeList gathers a host UDP candidate on the loopback page and fails it too.
  • Checked by hand against Chromium 154 with temporary managed policies and two local logging proxies, before the change:
    • a pinned bypass hostname connected directly;
    • --host-resolver-rules and --host-rules MAP rules on the proxy's address sent all traffic to a second proxy;
    • a DoH policy made Chromium connect to the DoH server directly when resolving a bypassed hostname;
    • a pinned DnsOverHttpsMode: off beat secure in policy.json.
  • Checked by hand against a headless image (Chrome 152) with the pin in place: chrome_policy overrides of ProxyOverrideRules, ProxyServerMode, PolicyDictionaryMultipleSourceMergeList: ["ProxySettings"], PolicyAtomicGroupsEnabled, CloudPolicyOverridesPlatformPolicy, WebRtcLocalIpsAllowedUrls, WebRtcUdpPortRange, BuiltInDnsClientEnabled and DnsOverHttpsTemplates all left the pin's values in force in chrome://policy. A page still gathered no UDP candidates, and loading an unlisted host still failed at the proxy.
  • Against a real filtered session, before the original commit:
    • a CDP-loaded extension in direct mode reached an unlisted host;
    • writing the same ProxySettings turned that into the egress proxy's denial, live and without a restart;
    • removing the pin brought the flag's bypass back;
    • Chromium's policy loader applied a .tmp file in the managed directory, and zz-kernel-egress.json beat a conflicting policy.json.
  • Not run for this revision: the headful image, and the policy-writing e2e tests (TestChromiumConfigureExtensionLoadStrategies, TestEnterpriseExtensionInstallation, TestWebBotAuthInstallation). They passed after the chown removal in the previous revision, which this one does not touch.
  • Not changed here: filtering a session still restarts Chromium once before the configure batch, which restarts it again when it carries flags, chrome policy or a profile. Folding the two into one restart needs the control plane to send filtered with /chromium/configure, so it is left as a follow-up.

A runtime --proxy-server, which anyone holding the session's token can write, was pinned in place of the egress proxy, and a WebRtcIPHandlingUrl chrome policy was consulted before the pinned WebRtcIPHandling, turning direct UDP back on. The pin now takes the proxy server from the base flags only and sets WebRtcIPHandlingUrl to an empty list.
Chromium installs the watch on its managed policy directory in a best-effort task after it starts. A pin removed before then is found when the watch is installed, but the reload scheduled for it is replaced by the 15-minute periodic one, so under load the test could unfilter before the watch existed and time out.
…test

The service worker target can be listed before its chrome.proxy and chrome.privacy bindings exist, so the first read could throw. levelOfControl now reports a thrown exception instead of failing to decode it.
The pin took --proxy-bypass-list from the runtime flags, which anyone holding
the session's token can write, so a bypass list of "*" was pinned as policy
and routed every request around the egress proxy. The pin now keeps only the
entries the control plane accepts as private hosts and sends the rest through
the proxy.

The wrapper also chowned /etc/chromium/policies to the user Chromium runs as,
though only the root API process writes there, which left the pin in a
directory its own browser could write. The chown is gone, and the launcher
makes the policy and staging directories root-owned and closed to others
before writing the pin. Present now requires the file to be a pin, so one
that is not still gets a restart that writes the real pin.

The pin's staging directory is now explicit, and the temporary file is
removed when a write fails.
…ping and DoH routes

The pin kept any well-formed hostname from a runtime --proxy-bypass-list, so a
runtime bypass list naming an arbitrary public host still sent it around the
egress proxy. The control plane now sends the session's private_hosts with the
egress policy, which only it can write, and the pin uses exactly those, or the
base flags' bypass list when it sends none. The runtime-flag filter is gone.

While filtered, the launcher also ignores runtime --host-resolver-rules and
--host-rules, which Chromium applies to the egress proxy's own address and which
could send every request to a different proxy, and the pin sets
DnsOverHttpsMode to off, since Chromium connects to a DoH server directly.

The handler restarts Chromium when the pin was written for different private
hosts, and the pin is flushed to disk before it is renamed into place.
The handler skipped the restart when a pin was present and the recorded
policy was unchanged, but the policy is recorded before the pin is applied.
A restart that failed before the launcher ran left the old pin in place,
and a retry of the same request then reported it applied.

Compare the pin on disk with the one the launcher would write for the
policy and the base flags, which both now derive through
egresspolicy.BaseFlags.
The launcher and the egress policy handler both derive the pin from
egresspolicy.BaseFlags, but no test covered a session without private
hosts, where the bypass list comes from the base flags. Test BaseFlags
directly, and apply the policy without private hosts first in the e2e
test, which only succeeds if the launcher writes the pin the handler
expects.
Nothing checked that the image's pin sorts after policy.json in the same
directory, or that it is staged outside that directory, where Chromium
would apply the temporary file as policy.
The pin's bypass list has come from the policy's private hosts, not the
flags Chromium is launched with, since the runtime bypass list stopped
being pinned.
The endpoint documents that entries may not contain whitespace, but only
rejected spaces, tabs and line breaks.
Chromium merges a list policy from every file that sets it, rather than
taking the last, when PolicyListMultipleSourceMergeList names it. A chrome
policy naming WebRtcIPHandlingUrl there added its rules to the pin's empty
list and let a page send UDP around the egress proxy. The pin now sets that
list to empty too.
The directories were opened with mode 777, so the test passed when Sync
only checked one of the two write bits before restricting them.
When the pin cannot be written the launcher exits and supervisord restarts
it, so the log line now says Chromium is not being started.
@yummybomb yummybomb closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant