add ppc64le cluster to prow

[upodroid]

/cc @Rajalakshmi-Girish @ameukam @BenTheElder

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: upodroid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• infra/gcp/terraform/OWNERS [upodroid]
• kubernetes/OWNERS [upodroid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-infra-ci-robot]

Ran Plan for dir: infra/gcp/terraform/k8s-infra-prow workspace: default

Show Output

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.gcb_bucket.google_storage_bucket.bucket has changed
~ resource "google_storage_bucket" "bucket" {
        id                          = "k8s-infra-prow-gcb"
        name                        = "k8s-infra-prow-gcb"
        # (16 unchanged attributes hidden)

      ~ lifecycle_rule {
          - condition {
              - age                        = 7 -> null
              - days_since_custom_time     = 0 -> null
              - days_since_noncurrent_time = 0 -> null
              - matches_prefix             = [] -> null
              - matches_storage_class      = [] -> null
              - matches_suffix             = [] -> null
              - no_age                     = false -> null
              - num_newer_versions         = 0 -> null
              - send_age_if_zero           = true -> null
              - with_state                 = "ANY" -> null
            }
          + condition {
              + age                                     = 7
              + days_since_custom_time                  = 0
              + days_since_noncurrent_time              = 0
              + matches_prefix                          = []
              + matches_storage_class                   = []
              + matches_suffix                          = []
              + no_age                                  = false
              + num_newer_versions                      = 0
              + send_age_if_zero                        = true
              + send_days_since_custom_time_if_zero     = false
              + send_days_since_noncurrent_time_if_zero = false
              + send_num_newer_versions_if_zero         = false
              + with_state                              = "ANY"
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.prow.google_container_cluster.primary has changed
~ resource "google_container_cluster" "primary" {
        id                                       = "projects/k8s-infra-prow/locations/us-central1/clusters/prow"
      ~ master_version                           = "1.30.4-gke.1348000" -> "1.30.9-gke.1046000"
        name                                     = "prow"
      ~ node_version                             = "1.30.3-gke.1969001" -> "1.30.9-gke.1046000"
        # (29 unchanged attributes hidden)

      ~ cluster_autoscaling {
          + auto_provisioning_locations = []
            # (2 unchanged attributes hidden)
        }

      ~ node_pool {
            name                        = "prod-v1"
          ~ version                     = "1.30.3-gke.1969001" -> "1.30.9-gke.1046000"
            # (6 unchanged attributes hidden)

            # (5 unchanged blocks hidden)
        }

      ~ node_pool_defaults {
          ~ node_config_defaults {
              + insecure_kubelet_readonly_port_enabled = "TRUE"
                # (1 unchanged attribute hidden)

                # (1 unchanged block hidden)
            }
        }

      + secret_manager_config {
          + enabled = false
        }

        # (26 unchanged blocks hidden)
    }

  # module.prow_bucket.google_storage_bucket.bucket has changed
~ resource "google_storage_bucket" "bucket" {
        id                          = "kubernetes-ci-logs"
        name                        = "kubernetes-ci-logs"
        # (15 unchanged attributes hidden)

      + lifecycle_rule {
          + action {
              + type = "Delete"
            }
          + condition {
              + age                                     = 90
              + days_since_custom_time                  = 0
              + days_since_noncurrent_time              = 0
              + matches_prefix                          = []
              + matches_storage_class                   = []
              + matches_suffix                          = []
              + no_age                                  = false
              + num_newer_versions                      = 0
              + send_age_if_zero                        = true
              + send_days_since_custom_time_if_zero     = false
              + send_days_since_noncurrent_time_if_zero = false
              + send_num_newer_versions_if_zero         = false
              + with_state                              = "ANY"
            }
        }

        # (2 unchanged blocks hidden)
    }

  # module.testgrid_config_bucket.google_storage_bucket.bucket has changed
~ resource "google_storage_bucket" "bucket" {
        id                          = "k8s-testgrid-config"
        name                        = "k8s-testgrid-config"
        # (16 unchanged attributes hidden)

      ~ lifecycle_rule {
          - condition {
              - age                        = 90 -> null
              - days_since_custom_time     = 0 -> null
              - days_since_noncurrent_time = 0 -> null
              - matches_prefix             = [] -> null
              - matches_storage_class      = [] -> null
              - matches_suffix             = [] -> null
              - no_age                     = false -> null
              - num_newer_versions         = 0 -> null
              - send_age_if_zero           = true -> null
              - with_state                 = "ANY" -> null
            }
          + condition {
              + age                                     = 90
              + days_since_custom_time                  = 0
              + days_since_noncurrent_time              = 0
              + matches_prefix                          = []
              + matches_storage_class                   = []
              + matches_suffix                          = []
              + no_age                                  = false
              + num_newer_versions                      = 0
              + send_age_if_zero                        = true
              + send_days_since_custom_time_if_zero     = false
              + send_days_since_noncurrent_time_if_zero = false
              + send_num_newer_versions_if_zero         = false
              + with_state                              = "ANY"
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.utility_cluster.google_container_cluster.primary has changed
~ resource "google_container_cluster" "primary" {
        id                                       = "projects/k8s-infra-prow/locations/us-central1/clusters/utility"
      ~ master_version                           = "1.30.3-gke.1639000" -> "1.30.9-gke.1127000"
        name                                     = "utility"
      ~ node_version                             = "1.29.6-gke.1326000" -> "1.30.9-gke.1127000"
        # (29 unchanged attributes hidden)

      ~ cluster_autoscaling {
          + auto_provisioning_locations = []
            # (2 unchanged attributes hidden)
        }

      ~ node_pool {
            name                        = "prod-v1"
          ~ version                     = "1.29.6-gke.1326000" -> "1.30.9-gke.1127000"
            # (6 unchanged attributes hidden)

            # (5 unchanged blocks hidden)
        }

      ~ node_pool_defaults {
          ~ node_config_defaults {
              + insecure_kubelet_readonly_port_enabled = "TRUE"
                # (1 unchanged attribute hidden)

                # (1 unchanged block hidden)
            }
        }

      + secret_manager_config {
          + enabled = false
        }

        # (26 unchanged blocks hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

  # google_iam_workload_identity_pool_provider.ppc64le will be updated in-place
~ resource "google_iam_workload_identity_pool_provider" "ppc64le" {
        id                                 = "projects/k8s-infra-prow/locations/global/workloadIdentityPools/ibm-clusters/providers/ppc64le"
        name                               = "projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/providers/ppc64le"
        # (6 unchanged attributes hidden)

      ~ oidc {
          ~ allowed_audiences = [
              + "sts.googleapis.com",
            ]
            # (2 unchanged attributes hidden)
        }
    }

  # module.prow_bucket.google_storage_bucket.bucket will be updated in-place
~ resource "google_storage_bucket" "bucket" {
        id                          = "kubernetes-ci-logs"
        name                        = "kubernetes-ci-logs"
        # (15 unchanged attributes hidden)

      - lifecycle_rule {
          - action {
              - type = "Delete" -> null
            }
          - condition {
              - age                                     = 90 -> null
              - days_since_custom_time                  = 0 -> null
              - days_since_noncurrent_time              = 0 -> null
              - matches_prefix                          = [] -> null
              - matches_storage_class                   = [] -> null
              - matches_suffix                          = [] -> null
              - no_age                                  = false -> null
              - num_newer_versions                      = 0 -> null
              - send_age_if_zero                        = true -> null
              - send_days_since_custom_time_if_zero     = false -> null
              - send_days_since_noncurrent_time_if_zero = false -> null
              - send_num_newer_versions_if_zero         = false -> null
              - with_state                              = "ANY" -> null
            }
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

• ▶️ To apply this plan, comment:

  atlantis apply -d infra/gcp/terraform/k8s-infra-prow

• 🚮 To delete this plan and lock, click here
• 🔁 To plan this project again, comment:

  atlantis plan -d infra/gcp/terraform/k8s-infra-prow

Note: Objects have changed outside of Terraform
Plan: 0 to add, 2 to change, 0 to destroy.

---

• ⏩ To apply all unapplied plans from this Pull Request, comment:

  atlantis apply

• 🚮 To delete all plans and locks from this Pull Request, comment:

  atlantis unlock

[ameukam]

Maybe document where this is coming from?

[upodroid]

The google link above explains what this code is doing. As for the values, its from the kubeconfig that IBM gave us. If the cluster is rebuilt, we rerun the terraform code to fetch the latest oidc jwks keys.

[ameukam]

we should document somewhere which IBM account is used and how to access to it. From this URI, I don't know if it's community infrastructure or something else.

[ameukam]

I guess the cluster creation was handled differently since I don't see a HCL reflecting it ? So what ever happens to this cluster is not on SIG K8S Infra ?

[upodroid]

There isn't an IBM managed cluster with this architecture, so it's a cluster built from VMs. I did ask for the TF code for the infra to be merged into this repository.

[ameukam]

Do we still want to do this ? I see #7785 but I'm not sure.
cc @BenTheElder