KubeErrors

Terminating: The pod has been asked to delete and is not going away

A pod stuck in Terminating has a deletion timestamp but has not been removed. It is almost always either a container ignoring SIGTERM, a volume that will not unmount, a finalizer waiting on something, or an unreachable node.

Applies to: All Kubernetes versions

What it means

Deleting a pod sets deletionTimestamp and starts a graceful shutdown: the kubelet sends SIGTERM to each container, waits up to terminationGracePeriodSeconds, then sends SIGKILL. Once the containers are gone and volumes are cleaned up, the API object is removed. A pod that stays in Terminating is blocked somewhere in that sequence. The four blockers behave differently. A slow container clears itself after the grace period. A stuck volume unmount does not — the kubelet keeps retrying. A finalizer holds the API object indefinitely regardless of what happened on the node. And an unreachable node means nobody is confirming the pod is gone at all, so the control plane cannot safely remove it.

Most common causes

How to diagnose it

  1. Check how long it has been terminating and what the grace period is: kubectl get pod POD -o jsonpath='{.metadata.deletionTimestamp}{" "}{.spec.terminationGracePeriodSeconds}'.
  2. Look for finalizers: kubectl get pod POD -o jsonpath='{.metadata.finalizers}'. If any are present, the object cannot be removed until they are.
  3. Check the node's status: kubectl get node. A NotReady node explains it immediately.
  4. Check the kubelet log on the node for repeated unmount errors: journalctl -u kubelet | grep -i unmount.
  5. Check whether the process is still alive on the node: crictl ps filtered to the pod.

How to fix it

  1. If the application ignores SIGTERM, fix the signal handling. This is the correct fix and also removes dropped connections during every rollout.
  2. If a volume is stuck, resolve the storage-side problem — an unreachable NFS server, or a network filesystem that needs its mount forcibly cleared on the node.
  3. If a finalizer is blocking removal, find out which controller owns it and why it is not running. Removing the finalizer by hand bypasses whatever cleanup it existed to perform, so treat it as a last resort and understand what you are skipping.
  4. If the node is gone for good, deleting the node object lets the control plane resolve the pods that were on it.
  5. Avoid routine use of --force --grace-period=0. It removes the API object without confirming the container stopped, which for a StatefulSet member can mean two copies running at once.

Notes

Force-deleting a pod does not stop the container — it only stops tracking it. For workloads with at-most-one semantics, such as a StatefulSet backing a database, that is precisely the situation the ordering guarantees exist to prevent.

Related

Sources

Pages on this site are written with AI assistance from the primary sources listed on each page, then checked against those sources before publishing.