Nodes belonging to a BootcNodePool should not be managed if the pool is deleted

[cheesesashimi]

Description

If a BootcNodePool is created, the controller applies bootc.dev/managed= to each of the nodes within that pool, which signals that the nodes should be managed by bootc-operator as well as allows the DaemonSet controller to schedule bootc-operator daemon pods onto each of those nodes. However, if one then deletes the BootcNodePool, the controller does not remove the bootc.dev/managed= label from them, as one might expect.

Reproduction

1. Create a BootcNodePool:

kubectl apply -f - <<EOF
apiVersion: node.bootc.dev/v1alpha1
kind: BootcNodePool
metadata:
  name: worker
spec:
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker: ""
  image:
    ref: <pullspec>
EOF

2. The following expected behaviors occur:

# BootcNodes are created for the BootcNodePool
kubectl get bootcnodes
NAME                                        AGE
ip-10-0-11-42.us-west-1.compute.internal    3s
ip-10-0-20-111.us-west-1.compute.internal   3s
ip-10-0-7-245.us-west-1.compute.internal    3s

# The nodes get the bootc.dev/managed label
kubectl get nodes -l 'bootc.dev/managed='
NAME                                        STATUS   ROLES    AGE    VERSION
ip-10-0-11-42.us-west-1.compute.internal    Ready    worker   110m   v1.36.2
ip-10-0-20-111.us-west-1.compute.internal   Ready    worker   110m   v1.36.2
ip-10-0-7-245.us-west-1.compute.internal    Ready    worker   110m   v1.36.2

# Spot-check of one node
kubectl get node/ip-10-0-11-42.us-west-1.compute.internal -o json | jq '.metadata.labels["bootc.dev/managed"]'

# A DaemonSet pod is scheduled onto each node
kubectl get pods -l 'app.kubernetes.io/name=bootc-operator,app.kubernetes.io/component=daemon' -n bootc-operator
NAME                          READY   STATUS    RESTARTS   AGE
bootc-operator-daemon-b88gj   1/1     Running   0          77s
bootc-operator-daemon-hxz2g   1/1     Running   0          77s
bootc-operator-daemon-rmj8j   1/1     Running   0          77s

3. Delete the BootcNodePool:

kubectl delete bootcnodepool/worker
bootcnodepool.node.bootc.dev "worker" deleted

4. Querying for nodes with the bootc.dev/managed= label succeeds because the labels are still there, as well as the daemon pods:

# The nodes still have the label
kubectl get nodes -l 'bootc.dev/managed='
NAME                                        STATUS   ROLES    AGE    VERSION
ip-10-0-11-42.us-west-1.compute.internal    Ready    worker   110m   v1.36.2
ip-10-0-20-111.us-west-1.compute.internal   Ready    worker   110m   v1.36.2
ip-10-0-7-245.us-west-1.compute.internal    Ready    worker   110m   v1.36.2

# Spot-check of one node
kubectl get node/ip-10-0-11-42.us-west-1.compute.internal -o json | jq '.metadata.labels["bootc.dev/managed"]'
""

# The DaemonSet pods are still scheduled onto each node
kubectl get pods -l 'app.kubernetes.io/name=bootc-operator,app.kubernetes.io/component=daemon' -n bootc-operator
NAME                          READY   STATUS    RESTARTS   AGE
bootc-operator-daemon-b88gj   1/1     Running   0          77s
bootc-operator-daemon-hxz2g   1/1     Running   0          77s
bootc-operator-daemon-rmj8j   1/1     Running   0          77s

# But the BootcNodes are deleted
kubectl get bootcnodes
No resources found

Expected behavior

I would have expected the bootc.dev/managed= label to be removed from these nodes, which would look like this instead:

# No BootcNodes should be found
kubectl get bootcnodes
No resources found

# No nodes should have the label
kubectl get nodes -l 'bootc.dev/managed='
No resources found

# Spot-check of one node
kubectl get node/ip-10-0-11-42.us-west-1.compute.internal -o json | jq '.metadata.labels["bootc.dev/managed"]'
null

# No daemon pods should be running
kubectl get pods -l 'app.kubernetes.io/name=bootc-operator,app.kubernetes.io/component=daemon' -n bootc-operator
No resources found in bootc-operator namespace.