KubeErrors

A plain-language reference for Kubernetes and container error states

Every entry explains what the error actually means, what usually causes it, how to confirm the cause on a real cluster, and how to fix it. Each page lists the sources it was built from.

Pod status reasons

• CrashLoopBackOff — Container keeps starting and exiting, so the kubelet is waiting longer between restarts
• ImagePullBackOff — Kubernetes could not pull the container image and is backing off before retrying
• ErrImagePull — The first failed attempt to pull a container image
• CreateContainerConfigError — The container cannot be created because something it references does not exist
• CreateContainerError — The container runtime failed to create the container
• RunContainerError — The container was created but failed to start
• ContainerCreating — The pod is scheduled and the kubelet is setting the container up, but it has not started yet
• PodInitializing — An init container is still running, so the main containers have not started
• Init:Error — An init container exited with a non-zero status
• Init:CrashLoopBackOff — An init container keeps failing, so the kubelet is backing off between retries
• InvalidImageName — The image reference in the pod spec is not a syntactically valid image name
• ErrImageNeverPull — The image is not present on the node and imagePullPolicy is Never
• Terminating — The pod has been asked to delete and is not going away

Termination reasons

• OOMKilled — The container exceeded its memory limit and was killed by the kernel
• Evicted — The kubelet removed the pod because the node ran short of a resource
• ContainerStatusUnknown — The kubelet lost track of a container's outcome and cannot say how it ended

Exit codes

• Exit code 137 — The process was killed with SIGKILL
• Exit code 143 — The process was terminated by SIGTERM
• Exit code 0 — The process exited successfully
• Exit code 1 — The application exited with a general error
• Exit code 2 — Misuse of a shell builtin, or an application-specific usage error
• Exit code 126 — The command was found but could not be executed
• Exit code 127 — The command was not found
• Exit code 128 — An invalid exit argument, or the base of the signal-encoded range
• Exit code 130 — The process was terminated by SIGINT
• Exit code 132 — The process was terminated by SIGILL — an illegal instruction
• Exit code 134 — The process was terminated by SIGABRT — it aborted itself
• Exit code 139 — The process was terminated by SIGSEGV — a segmentation fault
• Exit code 141 — The process was terminated by SIGPIPE — it wrote to a closed pipe or socket
• Exit code 255 — An out-of-range or application-specific exit status

Pod phases

• Pending — The pod has been accepted but is not running yet, usually because it cannot be scheduled
• Unknown — The control plane has lost contact with the node and cannot report the pod's state
• Completed — Every container in the pod exited successfully

Scheduling

• FailedScheduling — The scheduler could not find a node that satisfies the pod's requirements
• Insufficient cpu — No node has enough unreserved CPU for the pod's request
• Insufficient memory — No node has enough unreserved memory for the pod's request
• didn't match pod anti-affinity rules — The pod's spreading rules cannot be satisfied at the current node count
• untolerated taint — The node repels this pod and the pod has no matching toleration
• node(s) didn't match Pod's node affinity/selector — No node carries the labels the pod requires

Probes

• Liveness probe failed — The kubelet decided the container is unhealthy and restarted it
• Readiness probe failed — The pod was removed from its Service's endpoints but was not restarted
• Startup probe failed — The container did not finish starting within the startup probe's budget
• Unhealthy — The event Kubernetes emits whenever any probe fails
• Probe: connection refused — The probe reached the pod but nothing was listening on that port
• Probe: timeout — The probe connected or tried to, and got no answer in time
• Probe: HTTP 503 — The application answered the probe and reported itself unhealthy
• exec probe failed — The probe command ran inside the container and exited non-zero

Volumes and storage

• FailedMount — The kubelet could not mount a volume into the pod
• FailedAttachVolume — The volume could not be attached to the node the pod was scheduled to
• Multi-Attach error — A ReadWriteOnce volume is already attached to a different node
• FailedDetachVolume — The volume could not be detached from a node
• FailedUnmount — The kubelet could not unmount a volume from a terminating pod
• volume node affinity conflict — The volume exists somewhere the pod cannot be scheduled
• PersistentVolumeClaim Pending — The claim has not been bound to a volume
• ProvisioningFailed — The provisioner tried to create a volume and the storage backend refused
• timeout expired waiting for volumes to attach or mount — The kubelet gave up waiting for a volume and will retry
• PersistentVolume Released — The claim that owned this volume is gone, and the volume is not reusable yet
• permission denied on a mounted volume — The container's user cannot read or write the mounted volume
• CSI driver not found — The named CSI driver is not registered on the node or in the cluster
• FileSystemResizePending — The volume was expanded but the filesystem inside it has not been
• no space left on device — A write failed because the volume, the node's disk, or its inodes are full

Networking and DNS

• Service has no endpoints — The Service exists but no ready pod matches its selector
• no such host — A DNS lookup returned no record for the name
• DNS timeout — Cluster DNS did not answer the query in time
• ndots:5 slow DNS — Every external lookup tries several cluster suffixes before succeeding
• connection refused — The destination was reached and nothing was listening on that port
• i/o timeout — The connection attempt got no response at all
• connection reset by peer — An established connection was terminated abruptly by the other side
• no route to host — The network stack had nowhere to send the packet
• NetworkPolicy blocking traffic — A policy is silently dropping connections that used to work
• FailedCreatePodSandBox — The runtime could not create the pod's network sandbox
• network plugin is not ready — The kubelet cannot find a usable CNI configuration on the node
• no IP addresses available in range set — The node's pod IP pool is fully allocated
• Ingress 502 — The ingress reached a backend and got an invalid response
• Ingress 503 — The ingress had no healthy backend to send the request to
• Ingress 504 — The backend did not respond within the ingress controller's timeout
• ClusterIP unreachable — The Service has endpoints but its ClusterIP does not work from some or all pods

RBAC and admission

• Error from server (Forbidden) — The request was authenticated and RBAC denied it
• cannot list resource at the cluster scope — A cluster-wide request was made with only namespaced permissions
• Unauthorized — The API server could not establish who is making the request
• x509: certificate signed by unknown authority — The client does not trust the certificate the server presented
• admission webhook denied the request — A validating admission webhook rejected the object
• failed calling webhook — The API server could not reach an admission webhook
• PodSecurity violation — The pod does not meet the security standard enforced on its namespace
• exceeded quota — The namespace's ResourceQuota does not allow this object
• LimitRange violation — The container's resources fall outside the namespace's permitted range
• Error from server (NotFound) — The named object does not exist in the namespace that was searched
• Error from server (AlreadyExists) — An object with that name already exists in the namespace
• Error from server (Conflict) — The object was modified between reading it and writing it back
• no matches for kind — The cluster does not recognise the resource type in your manifest
• error validating data — The manifest does not match the schema for that resource
• ServiceAccount token not mounted — A pod expected in-cluster credentials and does not have them

Node conditions

• NotReady — The node is not accepting new work and its existing pods are at risk
• MemoryPressure — The node is low on available memory and has started protecting itself
• DiskPressure — The node is low on disk space or inodes
• PIDPressure — The node is running out of process IDs
• NetworkUnavailable — The node's network is not correctly configured
• SchedulingDisabled — The node has been cordoned and will not receive new pods
• PLEG is not healthy — The kubelet's container state loop is taking too long, so the node reports NotReady
• node.kubernetes.io/unreachable — The node controller cannot reach the node and has tainted it
• System OOM encountered — The kernel's OOM killer acted at the node level, not on a single container's limit
• failed to garbage collect required amount of images — The kubelet tried to reclaim disk by deleting images and could not free enough

Controllers and rollouts

• ProgressDeadlineExceeded — The Deployment made no progress within its deadline and gave up reporting success
• FailedCreate — A controller tried to create a pod and the API server rejected it
• BackoffLimitExceeded — A Job's pods failed more times than its backoffLimit allows
• DeadlineExceeded — The Job ran longer than activeDeadlineSeconds and was stopped
• too many missed start times — The CronJob controller lost track of the schedule and stopped
• StatefulSet pod stuck — One pod is blocking every pod after it in the ordinal sequence
• DaemonSet not scheduling on a node — A DaemonSet has no pod on some nodes
• unable to fetch metrics — The HorizontalPodAutoscaler cannot read the metric it scales on
• Cannot evict pod — A PodDisruptionBudget is refusing to allow the eviction
• Preempted — A higher-priority pod needed the space and this pod was removed to make room
• selector does not match template labels — The controller's selector and its pod template disagree
• orphaned pods after controller deletion — Pods outlived the controller that created them
• rollout stuck with old and new pods — A rolling update stopped partway, leaving both versions serving

Container runtime

• OCI runtime create failed — The low-level runtime refused to create the container
• failed to create shim task — containerd could not start the process that supervises the container
• exec format error — The binary is for a different architecture or is not an executable at all
• failed to reserve container name — A container with that name already exists in the runtime's state
• CRI context deadline exceeded — A call from the kubelet to the container runtime timed out
• failed to pull the sandbox image — The runtime cannot fetch the pause image every pod needs
• unknown service runtime.v1alpha2.RuntimeService — The kubelet and the container runtime disagree about which CRI version to speak
• cgroup driver mismatch — The kubelet and the container runtime are managing cgroups differently
• operation not permitted — A syscall was blocked by seccomp, capabilities, or a security module
• read-only file system — The process tried to write somewhere it cannot
• PostStartHookError — The container's postStart lifecycle hook failed
• ContainerCannotRun — The runtime accepted the container and the process could not be executed
• too many open files — A process or the node hit its file descriptor limit
• failed to stop container — The runtime could not terminate a container
• snapshot already exists — containerd's snapshotter has leftover state for a container that no longer exists

Resources and cgroups

• CPU throttling — The container hit its CPU limit and is being paused by the kernel
• cannot allocate memory — An allocation failed inside the container before any OOM kill happened

kubectl

• The connection to the server localhost:8080 was refused — kubectl has no kubeconfig and is falling back to a default that does not exist
• Unable to connect to the server — kubectl found a cluster address and could not reach it
• exec plugin failed — The credential plugin named in the kubeconfig did not produce a token
• current-context is not set — A kubeconfig exists but no context is selected
• unable to upgrade connection — kubectl exec or attach could not establish its streaming connection to the kubelet
• container is waiting to start — kubectl logs has nothing to show because the container never started
• previous terminated container not found — There is no earlier container instance whose logs can be read
• kubectl cp: tar not found — kubectl cp needs a tar binary inside the container and there is not one
• kubectl port-forward drops — The forwarded connection closes unexpectedly
• metrics not available yet — kubectl top cannot read the metrics API
• couldn't get current server API group list — kubectl could not complete API discovery
• metadata.annotations: Too long — kubectl apply's stored configuration annotation exceeded the size limit
• cannot delete DaemonSet-managed Pods — kubectl drain refuses to proceed because DaemonSet pods cannot be evicted
• OCI runtime exec failed — kubectl exec reached the container and the command could not be run inside it
• kubectl version skew — The kubectl client is too far from the cluster's version
• Apply failed with conflicts — Server-Side Apply refused because another manager owns the fields you are setting
• the server doesn't have a resource type — kubectl does not recognise the resource name you used
• logs are truncated or missing — Older log lines have been rotated away by the node