Gateway proxy does not restart when Istio integration is enabled

[Nadine2016]

Gloo Edge Product

Open Source

Gloo Edge Version

1.18.8

Kubernetes Version

1.27

Describe the bug

When enabling the Istio integration with the Kubernetes Gateway API, the gateway proxy pod would always restart automatically to come up with the new istio-proxy container. However, somewhere between version 1.18.0-beta27 and 1.18.4, something in the Istio integration changed. The gateway proxy does not restart automatically anymore. You have to restart the gloo pod which will then trigger the restart of the gateway proxy pod.

I confirmed that the restart still works in 1.18.0-beta27, but is broken in 1.18.4.

Expected Behavior

The gateway proxy should automatically restart to add the Istio sidecar container.

Steps to reproduce the bug

1. Install Gloo Gateway.

glooctl install gateway \         
--version 1.18.0-beta27 \
--values - << EOF
discovery:
  enabled: false
gatewayProxies:
  gatewayProxy:
    disabled: true
gloo:
  disableLeaderElection: true
kubeGateway:
  enabled: true
EOF

2. Wait for pods to become available.

kubectl get pods -n gloo-system
NAME                                READY   STATUS      RESTARTS   AGE
gateway-certgen-7mwgq               0/1     Completed   0          44s
gloo-54f885cd74-fkfkl               0/1     Running     0          20s
gloo-proxy-http-5bfb4cd46d-xcjt5    1/1     Running     0          8m28s
gloo-resource-rollout-check-xz7dr   0/1     Completed   0          17s
gloo-resource-rollout-znc5q         1/1     Running     0          20s

Note that I have a gateway proxy (gloo-proxy-http) that I created with this YAML.

kubectl apply -n gloo-system -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: http
spec:
  gatewayClassName: gloo-gateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

3. Get Helm values to enable the Istio integration.

helm get values gloo -n gloo-system -o yaml > gloo-gateway.yaml
open gloo-gateway.yaml

4. Add the following values.

discovery:
  enabled: false
gatewayProxies:
  gatewayProxy:
    disabled: true
gloo:
  disableLeaderElection: true
global:
  istioIntegration:
    enableAutoMtls: true
    enabled: true
  istioSDS:
    enabled: true
kubeGateway:
  enabled: true
  gatewayParameters:
    glooGateway:
      istio:
        istioProxyContainer:
          istioDiscoveryAddress: istiod-main.istio-system.svc:15012
          istioMetaClusterId: gloo-gateway-docs-mgt
          istioMetaMeshId: gloo-gateway-docs-mgt

5. Upgrade Gloo Gateway.

helm upgrade -n gloo-system gloo gloo/gloo \
 -f gloo-gateway.yaml \
 --version=1.18.0-beta27
Release "gloo" has been upgraded. Happy Helming!
NAME: gloo
LAST DEPLOYED: Wed Feb 12 10:06:45 2025
NAMESPACE: gloo-system
STATUS: deployed
REVISION: 2
TEST SUITE: None

6. Review the pods to make sure that the gateway proxy restarts automatically.

kubectl get pods -n gloo-system                                
NAME                                  READY   STATUS      RESTARTS   AGE
gateway-certgen-cc69h                 0/1     Completed   0          65s
gloo-54f885cd74-fkfkl                 1/1     Running     0          4m45s
gloo-proxy-http-7669478f4-4slzz       2/3     Running     0          16s
gloo-proxy-http-8696455d84-q85hc      1/1     Running     0          4m24s
gloo-resource-migration-lg8h4         0/1     Completed   0          112s
gloo-resource-rollout-check-jmcxq     0/1     Completed   0          14s
gloo-resource-rollout-cleanup-5l8xn   0/1     Completed   0          81s
gloo-resource-rollout-mp5tp           0/1     Completed   0          21s

If you repeat these steps for 1.18.4, the restart does not happen automatically. You need to restart the gloo pod which will then trigger the restart of the gloo-proxy-http pod.

Additional Environment Detail

No response

Additional Context

No response

[htpvu]

@nfuden will add this to the list of GW API epic for future work