Retries and Dead Letter Topics

Key points

• A failing record blocks its partition. Kafka has no per-message ack, so you cannot skip one and keep the rest flowing.
• Spring Kafka DefaultErrorHandler retries in place with a back off, then hands the record to a recoverer.
• DeadLetterPublishingRecoverer republishes the failure to <topic>-dlt, with the exception and original offset in headers.
• Blocking retries stall the partition. For slow retries use @RetryableTopic, which moves the record into timed retry topics instead.
• Classify your errors. Deserialization and validation failures should go straight to the DLT; only transient faults deserve retries.
• A DLT is not a fix. Alert on it and build a replay path back into the main topic.

Example

@Bean
public DefaultErrorHandler errorHandler(KafkaTemplate<Object, Object> template) {
    // 3 retries, 1s apart, then publish to "<original-topic>-dlt"
    var recoverer = new DeadLetterPublishingRecoverer(template);
    var handler = new DefaultErrorHandler(recoverer, new FixedBackOff(1000L, 3L));

    // Never retry these -- they will fail identically every time
    handler.addNotRetryableExceptions(
        DeserializationException.class,
        IllegalArgumentException.class);

    return handler;
}

// Non-blocking alternative: retries live in their own topics,
// so the main partition keeps moving.
@RetryableTopic(attempts = "4", backoff = @Backoff(delay = 2000, multiplier = 2.0))
@KafkaListener(topics = "orders", groupId = "billing-service")
public void onOrder(OrderEvent event) {
    billing.charge(event);
}

Retry the transient, dead-letter the poison, and alert on both.