node(s) didn't match Pod's node affinity/selector: No node carries the labels the pod requires
The pod demands nodes with particular labels and none match. Usually the label is missing from the nodes, or its key or value is spelled differently in the pod than on the node.
Applies to: All Kubernetes versions
What it means
A nodeSelector or a required node affinity filters nodes by label. When no node satisfies it, every node is rejected and the pod stays Pending with this message. Almost always the cause is mechanical rather than conceptual: labels are case-sensitive and exact, so disktype=SSD and disktype=ssd are different, and a well-known label written with the wrong domain prefix matches nothing. The other frequent cause is a manually applied label lost when a node was replaced — a manifest that worked for months stops working the first time the node pool is recycled.
Most common causes
- The required label does not exist on any node.
- A typo or case difference in the label key or value.
- A well-known label with the wrong prefix, such as omitting
kubernetes.io/ortopology.kubernetes.io/. - A manually applied label lost when nodes were replaced or upgraded.
- A node pool with the required label scaled to zero.
- Multiple affinity terms combined so that no node satisfies all of them simultaneously.
How to diagnose it
- Read the pod's requirement:
kubectl get pod POD -o jsonpath='{.spec.nodeSelector}{.spec.affinity.nodeAffinity}'. - List what nodes actually carry:
kubectl get nodes --show-labels. - Test the selector directly:
kubectl get nodes -l KEY=VALUE. No results is the confirmation. - Check for case and prefix differences character by character — this is where most of these live.
- Check whether a node pool with the label exists but has scaled to zero.
How to fix it
- Label the nodes:
kubectl label node NODE KEY=VALUE. - Correct the selector in the pod spec to match the labels that exist.
- Prefer well-known labels applied automatically — architecture, operating system, zone, instance type — over hand-applied ones, since they survive node replacement.
- Apply custom labels through the node pool's configuration rather than by hand, so replacements inherit them.
- Use
preferredDuringSchedulingwhere the placement is a preference, so an unlabelled cluster still schedules the pod.
Notes
Hand-applied node labels are the fragile part of this. They vanish when a node is replaced, which happens on every upgrade and every autoscaling event, so a workload that depends on one will fail at an unpredictable future date rather than immediately.
Related
- FailedScheduling — The scheduler could not find a suitable node
- untolerated taint — The node repels this pod
Sources
- Kubernetes documentation — Assigning Pods to Nodes
- Kubernetes documentation — Labels and Selectors
- Kubernetes documentation — Kubernetes Scheduler