[module-sdk] fix: prevent CRD update churn and prune unknown extensions - #128
Draft
fuldaxxx wants to merge 4 commits into
Draft
[module-sdk] fix: prevent CRD update churn and prune unknown extensions#128fuldaxxx wants to merge 4 commits into
fuldaxxx wants to merge 4 commits into
Conversation
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
fuldaxxx
force-pushed
the
fix/crd-installer-prune-unknown-fields
branch
from
August 6, 2026 14:17
e4748bf to
586d874
Compare
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Queue documents even when schema sanitization fails, while reporting the error and continuing with later documents. Normalize version defaults and overlay metadata to avoid reconcile churn and deleting ownership fields. Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
ldmonster
self-requested a review
August 7, 2026 08:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Prune CRD validation schemas to the fields the apiserver actually knows before applying
them, normalize the fields it fills in itself, and stop rewriting metadata that belongs to
other actors.
New package
pkg/crd-installer/openapi:apiextensionsv1.JSONSchemaPropsplusx-kubernetes-sensitive-data, with the union marshallers ported from upstream;Prunedecodes a schema through that type and re-encodes it, so every key that is not aschema field is dropped and whole numbers come back as
int64— exactly as the apiserverreturns them;
omitemptyincluded), union field sets, and abyte-for-byte comparison of the fork's marshalled output against upstream's.
Installer:
sanitizerewrites onlyspec.versions[*].schema.openAPIV3Schema; the rest of thedocument is passed through untouched, so a CRD field only a newer or patched apiserver
models still reaches it;
applyServerDefaultswrites what the apiserver fills in itself —spec.names.singular,spec.names.listKind, andserved/storageon every version (neither hasomitemptyupstream, so the stored object always carries both);
.spec, labels and annotations against thestate the object must end up holding, and returns early when nothing differs;
survive;
documents are still applied, all errors are joined, and a schema this build cannot decode
is sent as it came rather than keeping the CRD out of the cluster.
Why do we need it, and what problem does it solve?
Reconcile churn. Since CRDs are applied verbatim (#122), the desired
.speccarriedkeys the apiserver prunes (
x-doc-examples,x-doc-default,x-examples,x-kubernetes-immutable, …), omitted the.specfields it defaults, and encoded wholenumbers as
float64. The stored spec could therefore never equal the desired one, so everyensure_crdsrun — i.e. the startup of every module — issued a fullUpdateof nearlyevery CRD, bumping
resourceVersionand re-waking every CRD informer in the cluster. Theapiserver also logged one "unknown field" warning per pruned key on every apply.
Vendor extensions still have to survive.
x-kubernetes-sensitive-dataonly exists onthe Deckhouse apiserver, and the upstream Go type cannot hold it — dropping it would stop
the marked fields from being encrypted in etcd, RBAC-filtered and masked in audit logs.
That is the sole reason the schema type is forked instead of reusing
apiextensionsv1.JSONSchemaProps.Foreign metadata was deleted. Replacing
metadata.labelsandmetadata.annotationswholesale dropped Helm's ownership keys —
app.kubernetes.io/managed-byamong the labels,meta.helm.sh/release-nameand-namespaceamong the annotations — so the nexthelm upgradeof the chart that installed the CRD failed oninvalid ownership metadata.One bad document took the file down with it. An undecodable document aborted
processCRD, silently skipping every CRD after it in the same file, and a schema this buildcannot decode kept the CRD itself out of the cluster — which rejects every custom resource
of that kind as an unknown kind.
What is the expected behaviour?
Update,no
generationbump, no informer wakeups."unknown field" warnings are gone.
x-kubernetes-sensitive-datareaches the cluster, at every nesting position(
properties,items,additionalProperties,allOf, …).server-managed metadata (
finalizers,ownerReferences,uid) and the in-clusterspec.conversionare preserved as before.Deliberate trade-offs
.speckeys outside the schema are still sent. If this cluster's apiserver prunes one(
selectableFieldson an older cluster,x-kubernetes-sensitive-dataon a stock controlplane), that single CRD is updated on every reconcile. Accepted: the key is sent so an
apiserver that does know it gets it.
stays in the cluster; retracting one needs the field ownership the apiserver keeps for
server-side apply, which would mean switching the whole update to
Apply.understands and this build does not is dropped silently, and the drift guards only fire
when this module bumps
k8s.io/apiextensions-apiserver— keep the dependency in step withthe apiserver Deckhouse ships.
Tests
New regression subtests in
pkg/crd-installer:x-kubernetes-sensitive-datasurvives;7_churn_stored.yaml→8_churn_manifest.yaml,including a version whose
storagethe manifest omits) issues zero updates;the schema sent as it came;
the next run — on a fake client seeded by the subtest itself, so it holds in isolation.
In
pkg/crd-installer/openapi: schema round-trip tests for kept and dropped keys, the threedrift guards, and the marshalled-bytes comparison against upstream. Each fix was checked by
reverting it and confirming the matching test fails.