Troubleshooting a pod that will not run

Kubernetes · lesson 24 of 32 · 4 min read

Read the status, then the events, then the logs. Most Kubernetes failures name themselves.

Open this lesson in the learning hub

Key points

  • Pending means never scheduled: no node has the requested CPU or memory, or a taint blocks it. The events say which.
  • ImagePullBackOff is a wrong tag, a private registry with no imagePullSecrets, or the wrong architecture.
  • CrashLoopBackOff means it starts and exits. kubectl logs --previous shows the run that actually died.
  • OOMKilled with exit code 137 is the memory limit doing its job, not a bug in your shutdown path.
  • Running but 0/1 ready is a failing readiness probe: wrong path, wrong port, or a slow boot with no startup probe.
  • kubectl debug attaches a scratch container to a running pod, which is the answer when your image has no shell.

Example

kubectl get pods -o wide
kubectl describe pod orders-7d9f8c-4xk2p    # read the Events at the bottom

# it crashed - what did the dead container say?
kubectl logs orders-7d9f8c-4xk2p --previous

# why was it killed?  137 = OOMKilled, 143 = SIGTERM
kubectl get pod orders-7d9f8c-4xk2p \
  -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'

# nothing fits on any node?
kubectl get events --sort-by=.lastTimestamp | tail -20
kubectl describe node node-1 | grep -A5 Allocated

# distroless image with no shell
kubectl debug -it orders-7d9f8c-4xk2p --image=busybox --target=app

describe for events, logs --previous for the crash, exit 137 for memory. In that order.

This is a reading copy. The full lesson — with the visual explainer, the interactive lab and a Run button for the code — lives in the Kubernetes course, and every lesson in it is listed on the Kubernetes contents page.