How to remove Kubernetes namespaces stuck in a Terminating state

This document (000021065) is provided subject to the disclaimer at the end of this document.

Environment

Rancher 2.6/2.7/2.8

Situation

This situation most frequently occurs where a resource still exists within the Namespace with a finalizer that can no longer be satisfied, e.g. a custom resource controlled by an operator, with a finalizer referencing the operator, but where the operator has already been removed from the cluster.
To mitigate a namespace stuck in a terminating state, the following steps should be performed:
1) Check if any apiservice is unavailable.
2) Find all resources that still exist for this namespace and remove the finalizers.

Resolution

1) Check apiservice unavailable

Check if any apiservice is unavailable and hence does not serve its resources.

 kubectl get apiservice|grep False

2) Find resources and remove finalizers

Find all resources that still exist where "$your-ns-to-delete" is the name of the namespace stuck in a terminating state, and remove any finalizer on those resources that can no longer be met.

kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get -n $your-ns-to-delete

To remove the finalizers, you can use the following patch on the resource.

kubectl -n <namespace> patch <RessourceObject> <Object-name> -p '{"metadata":{"finalizers":null}}' --type=merge

Additional Information

Finalizers
- https://kubernetes.io/docs/concepts/overview/working-with-objects/finalizers/
- https://github.com/kubernetes/kubernetes/issues/60807

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.