I have 4 nodes in 2 node pools on GKE. One of them has static ip(to access aws service though whitelist), so I tagged this node by hand using kubectl label. I found when enabling auto-upgrade, after upgrade, this node will be disappeared(destroyed / recreated?). No node has the static ip or tags, causing some pods unscheduled. So, I have some questions:
- Should I turn off the auto-upgrade for the particular node pool?
- What will happen if node pools runs differrent version of k8s?
- Is there a best pratice to my situation?
Best Answer
I recommend you to use Cloud NAT to prevent losing your IP address. Actually, you can assign your static IP address to the Cloud NAT, then you will not lose the IP address after scaling or auto-upgrade. Cloud NAT lets your VM instances and container pods communicate with the internet using a shared, public IP address. Cloud NAT uses NAT gateway to manage those connections.
Also, manually modifying the kubernetes labels is not a good practice. I recommend you add labels in your template. To do this, make sure to add the kubernetes node labels to a node pool during node pool creation. You can only add k8s labels at the node pool level during creation, you cannot edit the node pool to add a label.