Run Kafka Locally in Five Minutes

Kafka · lesson 17 of 34 · 3 min read

Start a single-node KRaft broker and prove it works with the console producer and consumer.

Open this lesson in the learning hub

Key points

  • One node is plenty for learning. Format a storage directory once, start the broker, then use the shell tools that ship with it.
  • kafka-storage.sh format stamps a cluster id into log.dirs. Skip it and a KRaft broker refuses to start.
  • kafka-console-producer.sh sends one record per line of stdin; kafka-console-consumer.sh prints them back out.
  • --from-beginning matters: a brand new group has no committed offset, so without it you only see records sent from now on.
  • Docker is quicker still. The official apache/kafka image is KRaft-ready, so there is no ZooKeeper container to run.

Example

# 1. one-off: give the storage directory a cluster id
KAFKA_CLUSTER_ID=$(bin/kafka-storage.sh random-uuid)
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/server.properties

# 2. start the broker (KRaft mode -- no ZooKeeper)
bin/kafka-server-start.sh config/server.properties

# 3. second terminal: create a topic and push a few lines into it
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic demo
bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic demo

# 4. third terminal: read the whole log back
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 \
  --topic demo --from-beginning

# Or skip the tarball entirely:
#   docker run -p 9092:9092 apache/kafka:4.0.0

One broker, one topic, three terminals. That is the whole feedback loop.

Brokers, Controllers and KRaft Serializers and Poison Pills

This is a reading copy. The full lesson — with the visual explainer, the interactive lab and a Run button for the code — lives in the Kafka course, and every lesson in it is listed on the Kafka contents page.