Pods Stuck in Terminating status on Windows nodes

This document (000021790) is provided subject to the disclaimer at the end of this document.

Environment

Custom downstream cluster with Windows nodes

OS Name Microsoft Windows Server 2022 Standard

Version 10.0.20348 Build 20348

Situation

Windows workloads will not properly delete post rke2 upgrade. For instance, deploying a simple workload on 1.25 and up and then attempting to delete a pod spawned from that deployment post upgrade to 1.26 will result in the pod never completely terminating.

kubectl describe pod shows the following error message.

error killing pod: failed to "KillPodSandbox" for "024c126b-53af-492b-8d3c-4eca572331e7" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to remove network namespace for sandbox \"e1d38086fad91fc8609f41ce89d5aed161d04c8f63f57fb7105cb6b511feddb4\": hcnDeleteNamespace failed in Win32: The specified request is unsupported. (0x803b0015) {\"Success\":false,\"Error\":\"The specified request is unsupported. \",\"ErrorCode\":2151350293}"   4d2h         26846   opedaweb-reverse-proxy-875c9c6bc-nwsf5.182e9010b61706cc

This issue reproduces for any workload. This issue doesn't seem to be version specific, and has been reproduced for a number of different rke2 upgrade paths. This issue causes the pods to get stuck in the Terminating status

Resolution

The resolution is to restart the Windows nodes

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.