The Outbox Pattern
Fix the dual-write problem: never treat your database and Kafka as two independent writes.
Open this lesson in the learning hubKey points
- The dual write: save the order, then publish the event. If the publish fails, the database and the topic disagree permanently.
- Wrapping the send in
@Transactionaldoes not save you. A Kafka broker is not enrolled in your JDBC transaction. - The outbox: insert the event into an
outboxtable in the same transaction as the business row. - A relay then publishes it: Debezium tailing the write-ahead log, or a simple poller that marks rows as sent.
- Delivery stays at-least-once, so consumers still dedupe on the event id. What you gain is that an event can never be silently lost.
- Order is preserved if you key the record by the aggregate id, exactly as the ordering lesson requires.
Example
@Transactional
public void placeOrder(Order order) {
orderRepository.save(order);
// Same transaction, same database: both rows commit or neither does.
outboxRepository.save(new OutboxEvent(
UUID.randomUUID(), // event id -- consumers dedupe on this
"Order", // aggregate type -> target topic
order.id().toString(), // aggregate id -> record key
payload)); // the serialized OrderPlaced event
}
// A relay publishes the table. Debezium tails the WAL, or:
@Scheduled(fixedDelay = 500)
@Transactional
public void drainOutbox() {
for (OutboxEvent e : outboxRepository.findTop100BySentFalseOrderByIdAsc()) {
kafkaTemplate.send("orders", e.aggregateId(), e.payload());
e.markSent();
}
}
// NOT this -- the send lives outside the transaction and can fail alone:
// orderRepository.save(order);
// kafkaTemplate.send("orders", order.id(), event);
One transaction, one write. Let a relay do the publishing.
This is a reading copy. The full lesson — with the visual explainer, the interactive lab and a Run button for the code — lives in the Kafka course, and every lesson in it is listed on the Kafka contents page.