Impossible to disable `network_policy` after activation

[AndreaGiardini]

TL;DR

Disabling NetworkPolicies in a cluster is a procedure that requires a few steps, as described here :

• Disable calico in all the nodes
• Rotate all the nodes
• Disable calico in the master

This Terraform module does not seem to make a distinction between nodes and masters unfortunately. Policies are enabled/disabled everywhere using a unique var.network_policy variable:

terraform-google-kubernetes-engine/modules/beta-private-cluster/cluster.tf

Lines 349 to 351 in 6f92dde

	network_policy_config {
	disabled = !var.network_policy
	}

terraform-google-kubernetes-engine/modules/beta-private-cluster/main.tf

Lines 84 to 90 in 6f92dde

	cluster_network_policy = var.network_policy ? [{
	enabled = true
	provider = var.network_policy_provider
	}] : [{
	enabled = false
	provider = null
	}]

This means that when setting network_policies to false it's impossible to disable networkpolicies in the nodes before disabling them in the master.

Expected behavior

I expect network_policies to be disabled, or to have the possibility to disable them in the nodes before disabling them in the master.

Observed behavior

Terraform fails to apply:

╷
│ Error: googleapi: Error 400: Network policy addon cannot be removed until network policy has been disabled on the nodes.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.RequestInfo",
│     "requestId": "0xae620bac0c10b3d"
│   }
│ ]
│ , badRequest
│ 
│   with module.kubernetes-engine_private-cluster.google_container_cluster.primary,
│   on .terraform/modules/kubernetes-engine_private-cluster/modules/beta-private-cluster/cluster.tf line 22, in resource "google_container_cluster" "primary":
│   22: resource "google_container_cluster" "primary" {
│ 
╵

Terraform Configuration

module "kubernetes-engine_private-cluster" {
  source                       = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster"
  version                      = "36.3.0"
  
[...]

  network_policy          = true
  network_policy_provider = "CALICO"

[...]

}

Terraform Version

1.7.5

Terraform Provider Versions

registry.terraform.io/hashicorp/google 6.18.1

Additional information

No response