Consumer Lag, The Metric That Matters

Kafka · lesson 12 of 34 · 3 min read

Measure how far behind your consumers are and read the number correctly when things go wrong.

Key points

Lag = log end offset minus committed offset, per partition. It counts records behind, not seconds behind.
Steady low lag is healthy. Steadily growing lag means you consume slower than you produce. That trend is the alert.
Alert on the worst partition, not the sum. One hot key can stall a single partition while the average looks perfect.
Fix in this order: make the handler faster, then batch or parallelise, then add consumers — capped by partition count.
Lag frozen at a constant while offsets never move is a stuck consumer or a rebalance loop, not slowness.

Example

# Per-partition lag for one group
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --describe --group billing-service

# GROUP            TOPIC   PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG
# billing-service  orders  0          10482           10490           8
# billing-service  orders  1          9033            41277           32244   <-- hot partition

# Emergency skip-ahead. The group must be stopped first.
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --group billing-service --topic orders \
  --reset-offsets --to-latest --execute

Watch the slope of lag, not its value.

This is a reading copy. The full lesson — with the visual explainer, the interactive lab and a Run button for the code — lives in the Kafka course, and every lesson in it is listed on the Kafka contents page.