KubeErrors

node(s) didn't match Pod's node affinity/selector: No node carries the labels the pod requires

The pod demands nodes with particular labels and none match. Usually the label is missing from the nodes, or its key or value is spelled differently in the pod than on the node.

Applies to: All Kubernetes versions

What it means

A nodeSelector or a required node affinity filters nodes by label. When no node satisfies it, every node is rejected and the pod stays Pending with this message. Almost always the cause is mechanical rather than conceptual: labels are case-sensitive and exact, so disktype=SSD and disktype=ssd are different, and a well-known label written with the wrong domain prefix matches nothing. The other frequent cause is a manually applied label lost when a node was replaced — a manifest that worked for months stops working the first time the node pool is recycled.

Most common causes

How to diagnose it

  1. Read the pod's requirement: kubectl get pod POD -o jsonpath='{.spec.nodeSelector}{.spec.affinity.nodeAffinity}'.
  2. List what nodes actually carry: kubectl get nodes --show-labels.
  3. Test the selector directly: kubectl get nodes -l KEY=VALUE. No results is the confirmation.
  4. Check for case and prefix differences character by character — this is where most of these live.
  5. Check whether a node pool with the label exists but has scaled to zero.

How to fix it

  1. Label the nodes: kubectl label node NODE KEY=VALUE.
  2. Correct the selector in the pod spec to match the labels that exist.
  3. Prefer well-known labels applied automatically — architecture, operating system, zone, instance type — over hand-applied ones, since they survive node replacement.
  4. Apply custom labels through the node pool's configuration rather than by hand, so replacements inherit them.
  5. Use preferredDuringScheduling where the placement is a preference, so an unlabelled cluster still schedules the pod.

Notes

Hand-applied node labels are the fragile part of this. They vanish when a node is replaced, which happens on every upgrade and every autoscaling event, so a workload that depends on one will fail at an unpredictable future date rather than immediately.

Related

Sources

Pages on this site are written with AI assistance from the primary sources listed on each page, then checked against those sources before publishing.