Repository navigation
Conversation
A runtime --proxy-server, which anyone holding the session's token can write, was pinned in place of the egress proxy, and a WebRtcIPHandlingUrl chrome policy was consulted before the pinned WebRtcIPHandling, turning direct UDP back on. The pin now takes the proxy server from the base flags only and sets WebRtcIPHandlingUrl to an empty list.
Chromium installs the watch on its managed policy directory in a best-effort task after it starts. A pin removed before then is found when the watch is installed, but the reload scheduled for it is replaced by the 15-minute periodic one, so under load the test could unfilter before the watch existed and time out.
…test The service worker target can be listed before its chrome.proxy and chrome.privacy bindings exist, so the first read could throw. levelOfControl now reports a thrown exception instead of failing to decode it.
The pin took --proxy-bypass-list from the runtime flags, which anyone holding the session's token can write, so a bypass list of "*" was pinned as policy and routed every request around the egress proxy. The pin now keeps only the entries the control plane accepts as private hosts and sends the rest through the proxy. The wrapper also chowned /etc/chromium/policies to the user Chromium runs as, though only the root API process writes there, which left the pin in a directory its own browser could write. The chown is gone, and the launcher makes the policy and staging directories root-owned and closed to others before writing the pin. Present now requires the file to be a pin, so one that is not still gets a restart that writes the real pin. The pin's staging directory is now explicit, and the temporary file is removed when a write fails.
…ping and DoH routes The pin kept any well-formed hostname from a runtime --proxy-bypass-list, so a runtime bypass list naming an arbitrary public host still sent it around the egress proxy. The control plane now sends the session's private_hosts with the egress policy, which only it can write, and the pin uses exactly those, or the base flags' bypass list when it sends none. The runtime-flag filter is gone. While filtered, the launcher also ignores runtime --host-resolver-rules and --host-rules, which Chromium applies to the egress proxy's own address and which could send every request to a different proxy, and the pin sets DnsOverHttpsMode to off, since Chromium connects to a DoH server directly. The handler restarts Chromium when the pin was written for different private hosts, and the pin is flushed to disk before it is renamed into place.
The handler skipped the restart when a pin was present and the recorded policy was unchanged, but the policy is recorded before the pin is applied. A restart that failed before the launcher ran left the old pin in place, and a retry of the same request then reported it applied. Compare the pin on disk with the one the launcher would write for the policy and the base flags, which both now derive through egresspolicy.BaseFlags.
The launcher and the egress policy handler both derive the pin from egresspolicy.BaseFlags, but no test covered a session without private hosts, where the bypass list comes from the base flags. Test BaseFlags directly, and apply the policy without private hosts first in the e2e test, which only succeeds if the launcher writes the pin the handler expects.
Nothing checked that the image's pin sorts after policy.json in the same directory, or that it is staged outside that directory, where Chromium would apply the temporary file as policy.
The pin's bypass list has come from the policy's private hosts, not the flags Chromium is launched with, since the runtime bypass list stopped being pinned.
The endpoint documents that entries may not contain whitespace, but only rejected spaces, tabs and line breaks.
Chromium merges a list policy from every file that sets it, rather than taking the last, when PolicyListMultipleSourceMergeList names it. A chrome policy naming WebRtcIPHandlingUrl there added its rules to the pin's empty list and let a page send UDP around the egress proxy. The pin now sets that list to empty too.
The directories were opened with mode 777, so the test passed when Sync only checked one of the two write bits before restricting them.
When the pin cannot be written the launcher exits and supervisord restarts it, so the log line now says Chromium is not being started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chromium reaches the egress proxy because it is launched with
--proxy-server, but Chromium applies proxy settings in the order policy, extensions, command line. An extension with theproxypermission can callchrome.proxy.settings.set({ value: { mode: "direct" } })and every request after that leaves the VM directly, around the egress proxy and the allowlist it enforces.This is reachable by a client that only speaks CDP.
--enable-unsafe-extension-debuggingis on, soExtensions.loadUnpackedgoes through the DevTools proxy, and the files can be put on disk withBrowser.setDownloadBehaviorplus two blob downloads. Against a real allowlisted session, that chain served an unlisted host straight from its origin.#439 closed the
Target.createBrowserContextroute. This closes the extension route: while the session is filtered, the VM pins Chromium's proxy with the managedProxySettingspolicy, which extensions cannot override.egresspolicy.Pinwrites/etc/chromium/policies/managed/zz-kernel-egress.json, a file besidepolicy.jsonrather than a key inside it. Chromium merges every file in the directory, so the pin never races the writers ofpolicy.json.PolicyListMultipleSourceMergeList, below), so the name keeps the pin ahead ofpolicy.json.ProxySettingsis also reserved from customerchrome_policyoverrides./etc/chromium/policies/and renamed in. A temporary file left in the directory would be applied as policy.WebRtcIPHandling: disable_non_proxied_udpand clearsWebRtcIPHandlingUrl. WebRTC sends UDP straight to the network rather than through an HTTP proxy. With only the proxy pinned, page JS still gathered a server-reflexive candidate from a public STUN server; with this policy it gathered none. An extension cannot override it either. Chromium consults the per-URLWebRtcIPHandlingUrlrules beforeWebRtcIPHandling, and achrome_policyoverride of[{"url":"*","handling":"default"}]brought the server-reflexive candidate back with the pin in place, so the pin sets that policy to an empty list.PolicyListMultipleSourceMergeListnames it. Achrome_policyoverride namingWebRtcIPHandlingUrlthere, together with a*/defaultrule, merged that rule into the pin's empty list, and a page gathered a host UDP candidate with the pin in place. The pin also setsPolicyListMultipleSourceMergeListto an empty list; with it, the page gathered none.ProxySettingsis set, Chromium ignores--proxy-bypass-listas well as--proxy-server, so the pin carries both. Runtime flags can be written by anyone holding the session's token, and a pinned runtime value of either would route traffic around the egress proxy.CHROMIUM_FLAGS), where the control plane sets the egress proxy. A runtime--proxy-serverhas no effect while the session is filtered.private_hosts, which the control plane now sends inPUT /network/egress-policy(kernel/kernel#4714). That is the one instance endpoint customers cannot write. When the control plane sends none, the pin uses the base flags' bypass list, which is the image's default private ranges. A runtime--proxy-bypass-listhas no effect while the session is filtered. The VM rejects entries containing;,,or whitespace, which Chromium would split into extra rules.--host-resolver-rulesand--host-ruleswhile filtered. Chromium applies them to the egress proxy's own address whatever the pin says: against Chromium 154 with the pin in place,--host-resolver-rules=MAP <proxy-ip> <other>:<port>sent every request to the other proxy. The launcher logs what it drops.DnsOverHttpsMode: off. Chromium connects to a DNS-over-HTTPS server directly, not through the proxy, whenever it resolves a name itself, as it does for a bypassed hostname. With the pin in place and aDnsOverHttpsMode: securechrome policy naming a server, Chromium connected to that server directly; withoffpinned, it made no connection. The control plane already refuses anyDnsOverHttpsModeother thanofftogether with an allowlist at create, but chrome policy can also be written at runtime./etc/chromium/policiestokernel, the user Chromium runs as, although only the root API process writes there; that left the pin in a directory the browser itself could write, where a file sorting after the pin would override it. The chown is gone, and before writing the pin the launcher makes the policy directory and the staging directory root-owned and not writable by anyone else. The handler compares the file at that path byte for byte with the pin the launcher would write for the policy and the base flags, so anything else there, including a pin for otherprivate_hosts, gets the restart that writes the real one. Chromium running as root (RUN_AS_ROOT=true) is not covered by this.private_hosts) can never leave a stale pin behind. The handler derives the pin it expects the same way, throughegresspolicy.BaseFlags. If the session is filtered and the pin cannot be written, or the base flags have no--proxy-serverto pin, the launcher exits rather than starting Chromium unpinned.PUT /network/egress-policy:filtered: truewith no pin restarts Chromium so the launcher writes it, and checks the pin is there before answering. Chromium would also pick up a pin written in place, since it reloads its policy directory on its own, but only after a delay, so the restart puts the pin in force before the control plane hands the session to a client.filtered: truewith the pin already present for the same policy does nothing, so re-applying it (e.g. on an allowlist replace) never restarts Chromium. A pin written for differentprivate_hostsgets a restart. The check reads the pin on disk rather than the recorded policy, so a retry after a restart that failed before the launcher ran restarts again instead of reporting the old pin applied.filtered: falseremoves the pin without a restart; Chromium reloads and hands the proxy back to the command line, so customer proxy extensions work again. Chromium normally picks the removal up within about 5s. The exception is a removal in the first moments after Chromium starts, before it has installed the watch on its policy directory (a best-effort task, so it can be late under load): Chromium notices that change when it installs the watch, but the periodic 15-minute reload it schedules next replaces the reload it scheduled for the change. A session unfiltered that soon after a restart stays pinned for up to 15 minutes, which fails closed. The e2e test waits for the watch before unfiltering.chromiumConfigMu, since it can restart Chromium.Deploy order
Ship kernel/kernel#4714 first. It sends
private_hosts, which images without this change ignore. If this image ships first, sessions with an allowlist andprivate_hostsget the image default bypass, so their private hosts are sent through the egress proxy until the control plane catches up. Rolling the control plane back after both ship has the same effect, and also restarts Chromium on the next allowlist PATCH, because the policy no longer matches the pin.What the pin does not cover
ProxySettingsdoes not override the per-context proxy fromTarget.createBrowserContext. Checked against a real browser: with the pin in place,newContext({ proxy })still connected directly. So the refusal in #439 remains necessary, and the two together close both routes for a CDP client.As with #439, code running inside the VM is out of scope: it can egress without using Chromium at all. The same session JWT that writes runtime flags also reaches
POST /process/exec, so the runtime-flag hardening here does not make the token holder an in-scope adversary. It removes the routes that go through Chromium's own configuration. VM-level egress enforcement is still the fix for a token holder.Cost
On a session created with an allowlist, applying the policy restarts Chromium once: about 3s in the local Docker image, the same as a flags PATCH. It happens only the first time a session is filtered, which is at session setup, before any client is connected. That relies on the control plane filtering a session only when it is created: adding an allowlist to a running session would restart Chromium under the customer's open pages. Re-applying
trueand applyingfalsedo not restart.Validation
go vet ./...andgo test -racefor the whole unit suite pass.private_hostskeeps the base bypass list (the image default, or a--proxy-bypass-listinCHROMIUM_FLAGS, as derived throughBaseFlags), a list replaces it, an empty list bypasses nothing, and the last base--proxy-serverwins. It refuses to pin when the base flags have no--proxy-server.DropHostMappingFlagsdrops--host-resolver-rulesand--host-rules, including single-dash and bare forms, and keeps everything else.private_hostssurvives an API process restart, with nil and[]kept apart, andGET /network/egress-policyreports it.policy.json, sorts after it, and is staged outside the managed directory. The pin's directories are closed to the group and to other users, each checked on its own.Matchesrejects anything but the policy's own pin (including a pin for otherprivate_hosts, a missingDnsOverHttpsModeand a missingPolicyListMultipleSourceMergeList), and a failed write leaves no temporary file in either directory.private_hostsrestart Chromium while unchanged ones do not, including on a retry after a failed restart; malformedprivate_hosts(;,,or any Unicode whitespace) answer 400; a failed restart answers 500 while the session stays filtered.private_hostsin the pin, fails the matching test. WeakeningMatchesto accept any pin fails both the pin and the handler retry tests.TestEgressProxyPinpasses against a headless image built from this revision. It first appliesfiltered: truewithoutprivate_hostsand checks the pin keeps the image's default bypass; that request only answers 200 if the launcher and the handler agree on the pin. It then appliesfiltered: truewithprivate_hosts, then writes chrome policy (WebRtcIPHandlingUrl: [{"url":"*","handling":"default"}],PolicyListMultipleSourceMergeList: ["WebRtcIPHandlingUrl"],DnsOverHttpsMode: secure) and runtime flags (--proxy-server,--proxy-bypass-list=*;exfil.example.com,--host-resolver-rules=MAP ...) through the instance API. It then checks that:private_hosts, an emptyWebRtcIPHandlingUrl, an emptyPolicyListMultipleSourceMergeListandDnsOverHttpsMode: off;--host-resolver-rules;root 755;not_controllablefor the proxy and for WebRTC;filtered: false, control and UDP candidates come back without a restart.An image built with the host-mapping drop disabled fails the test on that assertion. An image whose pin has no
PolicyListMultipleSourceMergeListgathers a host UDP candidate on the loopback page and fails it too.--host-resolver-rulesand--host-rulesMAP rules on the proxy's address sent all traffic to a second proxy;DnsOverHttpsMode: offbeatsecureinpolicy.json.chrome_policyoverrides ofProxyOverrideRules,ProxyServerMode,PolicyDictionaryMultipleSourceMergeList: ["ProxySettings"],PolicyAtomicGroupsEnabled,CloudPolicyOverridesPlatformPolicy,WebRtcLocalIpsAllowedUrls,WebRtcUdpPortRange,BuiltInDnsClientEnabledandDnsOverHttpsTemplatesall left the pin's values in force inchrome://policy. A page still gathered no UDP candidates, and loading an unlisted host still failed at the proxy.ProxySettingsturned that into the egress proxy's denial, live and without a restart;.tmpfile in the managed directory, andzz-kernel-egress.jsonbeat a conflictingpolicy.json.TestChromiumConfigureExtensionLoadStrategies,TestEnterpriseExtensionInstallation,TestWebBotAuthInstallation). They passed after the chown removal in the previous revision, which this one does not touch.filteredwith/chromium/configure, so it is left as a follow-up.