We have kubernetes running on a cluster in google cloud. Something happened to the cluster, and all of the nodes had the status node.kubernetes.io/unreachable:NoSchedule. We attempted to delete the cluster, but it is not working.
The instance groups all have the status
This instance group is transforming.
The instances all have the status
This instance is being deleted
The kubernetes clusters all have the status
This cluster is being deleted
but when I click on a cluster it also says either
timed out waiting for Google Compute Engine operation:
operation-1554925944268-58632688f6552-473a295c-305f8d99
or
Googleapi: Error 400: There is a peering operation in progress on the
local or peer network. Try again later., badRequest.
It took a long time for the VM instances themselves to get deleted, but eventually they did.
However, the groups and clusters are not being deleted.
If I manually try to delete an instance group it says
Google Compute Engine is not ready for use yet in the project. It may
take several minutes if Google Compute Engine has just been enabled,
or if this is the first time you use Google Compute Engine in the
project.
which makes no sense whatsoever.
Any help debugging this situation would be greatly appreciated.
Best Answer
There is a public issue about instance groups stucked when trying to delete a cluster.
You could try to update your Google Kubernetes Engine cluster & nodes version.