feat(windows): add credential provider mirror config for network isolated cluster#8708
feat(windows): add credential provider mirror config for network isolated cluster#8708fseldow wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the Windows CSE credential provider configuration generation to support network-isolated clusters by adding a registry mirror mapping for MCR when BootstrapProfileContainerRegistryServer is set.
Changes:
- Add a new
Config-CredentialProviderbranch that appendsmcr.microsoft.com(orMCRRepositoryBase) tomatchImages. - Add a
--registry-mirrorargument to the credential provider config whenBootstrapProfileContainerRegistryServeris set.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| elseif (![string]::IsNullOrEmpty($global:BootstrapProfileContainerRegistryServer)) { | ||
| $mcrRegistry = if ((Test-Path variable:global:MCRRepositoryBase) -and | ||
| -not [string]::IsNullOrEmpty($global:MCRRepositoryBase)) { | ||
| ([string]$global:MCRRepositoryBase).TrimEnd("/") | ||
| } | ||
| else { | ||
| "mcr.microsoft.com" | ||
| } | ||
| $credentialProviderConfig = @" | ||
| apiVersion: kubelet.config.k8s.io/v1 | ||
| kind: CredentialProviderConfig | ||
| providers: | ||
| - name: acr-credential-provider | ||
| matchImages: | ||
| - "*.azurecr.io" | ||
| - "*.azurecr.cn" | ||
| - "*.azurecr.de" | ||
| - "*.azurecr.us" | ||
| - "${mcrRegistry}" | ||
| defaultCacheDuration: "10m" | ||
| apiVersion: credentialprovider.kubelet.k8s.io/v1 | ||
| args: | ||
| - $azureConfigFile | ||
| - --registry-mirror=${mcrRegistry}:$global:BootstrapProfileContainerRegistryServer | ||
| "@ |
There was a problem hiding this comment.
we already has trim logic in aks-rp side, so prefer not to implement normalizing the variable every time.
https://msazure.visualstudio.com/CloudNativeCompute/_git/aks-rp?path=/resourceprovider/sharedlib/agentbaker/typeconversion.go&version=GBmaster&line=934&lineEnd=935&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents
do we think we should still put trim logic in cse script?
| Context 'BootstrapProfileContainerRegistryServer is set with default MCR' { | ||
| BeforeEach { | ||
| $global:BootstrapProfileContainerRegistryServer = "myregistry.azurecr.io" | ||
| # Ensure MCRRepositoryBase is not set so it falls back to mcr.microsoft.com | ||
| Remove-Variable -Name MCRRepositoryBase -Scope Global -ErrorAction SilentlyContinue | ||
| } | ||
| AfterEach { | ||
| $global:BootstrapProfileContainerRegistryServer = $null | ||
| } | ||
| It "should include mcr.microsoft.com in matchImages and registry-mirror arg" { | ||
| $expectedCredentialProviderConfig = Read-Format-Yaml ([Io.path]::Combine($credentialProviderConfigDir, "BootstrapProfileContainerRegistryServerDefault.config.yaml")) | ||
| Config-CredentialProvider -KubeDir $credentialProviderConfigDir -CredentialProviderConfPath $CredentialProviderConfPATH -CustomCloudContainerRegistryDNSSuffix "" | ||
| $acutalCredentialProviderConfig = Read-Format-Yaml $CredentialProviderConfPATH | ||
|
|
||
| $normalizedExpected = $expectedCredentialProviderConfig.Trim().Replace("`r`n", "`n") | ||
| $normalizedActual = $acutalCredentialProviderConfig.Trim().Replace("`r`n", "`n") | ||
| $normalizedActual | Should -Be $normalizedExpected | ||
| } | ||
| } |
| apiVersion: credentialprovider.kubelet.k8s.io/v1 | ||
| args: | ||
| - $azureConfigFile | ||
| - --registry-mirror=${mcrRegistry}:$global:BootstrapProfileContainerRegistryServer |
There was a problem hiding this comment.
[HIGH] Zoe — Customer-controlled registry host injected into config without validation
$global:BootstrapProfileContainerRegistryServer is expanded verbatim into a double-quoted YAML here-string and written as a privileged kubelet CredentialProviderConfig. A crafted value containing newlines or YAML metacharacters can inject additional list items or flags into args, changing how acr-credential-provider runs under SYSTEM during node bootstrap.
Suggestion: Validate $global:BootstrapProfileContainerRegistryServer against a strict hostname-only regex (e.g., ^[a-zA-Z0-9.\-]+(:[0-9]+)?$) before interpolation. Reject values with newlines, whitespace, or YAML metacharacters. Consider generating the config via structured YAML serialization rather than here-string concatenation.
Code Review — feat(windows): add credential provider mirror config for network isolated clusterReviewers: Wash (testing), Zoe (security), Simon (correctness), Kaylee (design), Book (resilience), Badger (auth/API contract) Findings Summary
High[Zoe] Customer-controlled registry host injected into config without validation —
Fix: Validate against (Inline comment posted) Medium[Wash] TrimEnd("/") normalization not tested — The custom-MCR test uses [Kaylee] Function depends on hidden global inputs —
[Kaylee] YAML config duplicated across branches (DRY violation) — Three near-identical YAML here-strings now exist, varying only in one extra Low[Wash] Typo in variable name —
No Issues Found
Process NotePR description is unfilled — the template fields ("What this PR does", issue link) are blank. Please fill them in before merge. Overall Recommendation✅ Approve with suggestions The core logic is correct and tests cover the main happy paths. The High finding (input validation) should be addressed before merge given the privileged execution context. The Medium findings are reasonable follow-ups but don't block the feature. |
| Context 'BootstrapProfileContainerRegistryServer is set with custom MCRRepositoryBase' { | ||
| BeforeEach { | ||
| $global:BootstrapProfileContainerRegistryServer = "myregistry.azurecr.io" | ||
| $global:MCRRepositoryBase = "custom.mcr.contoso.com" |
There was a problem hiding this comment.
🟢 Low / Test Coverage — This value has no trailing slash, so the .TrimEnd("/") you added in configfunc.ps1 never runs in the tests. Set it to custom.mcr.contoso.com/ and keep the fixture without the slash, so the test proves the trim works.
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #