Scheduling: affinity, taints, tolerations

Key points

• The scheduler filters the nodes that can hold the pod, scores whatever survives, then binds the winner.
• nodeSelector is the blunt version. nodeAffinity adds required and preferred rules with real operators.
• A taint on a node repels pods. Only a pod carrying a matching toleration is even allowed to land there.
• A toleration permits, it does not attract. Without affinity as well, your GPU pod happily lands on a cheap node.
• podAntiAffinity over topology.kubernetes.io/zone stops one zone outage taking every replica with it.
• topologySpreadConstraints is the cheaper modern way to say 'spread these evenly' across zones or nodes.

Example

# on the node:  kubectl taint nodes gpu-1 gpu=true:NoSchedule
spec:
  tolerations:                       # allowed onto the tainted node
    - key: gpu
      operator: Equal
      value: "true"
      effect: NoSchedule
  affinity:
    nodeAffinity:                    # and actually wants that node
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: accelerator
                operator: In
                values: ["a100"]
  topologySpreadConstraints:         # never all replicas in one zone
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app: orders

Taints keep pods off, affinity pulls pods on. Expensive hardware needs both.