Apache Kafka Cheatsheet

Welcome to Apache Kafka Cheatsheet! This is the most complete Apache Kafka cheatsheet. Enjoy it!

Apache Kafka Cheatsheet  Apache Kafka Cheatsheet

Start Zookeeper

bin/zookeeper-shell.sh localhost:2181 

Enter Zookeeper shell

zookeeper-shell localhost:2182 ls  

Start a Broker

bin/kafka-server-start.sh config/server1.properties 

Describe Topic

 bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-topic 

Add partitions to a topic

 bin/kafka-topics.sh --alter --zookeeper localhost:2181 --topic beacon --partitions 3 

Change topic retention

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=28800000* 

Delete Topic

bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic 

Create a Topic

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myTopic 

List topics

bin/kafka-topics.sh --list --zookeeper localhost:2181 

Start Producer

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic myTopic 

Start Consumer

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic myTopic --from-beginning 

Consume a single message

bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic  --max-messages 1 

Consume message and specify consumer group

kafka-console-consumer --topic my-topic --new-consumer --bootstrap-server localhost:9092 --consumer-property group.id=my-group 

Push messages from a file to a Topic

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic < file.log 

Purge Topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000  bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --delete-config retention.ms 

Getting the last five message of a topic

kafkacat -C -b localhost:9092 -t mytopic -p 0 -o -5 -e 

Get the earliest offset still in a topic

bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic mytopic --time -2 

Get the latest offset still in a topic

bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic mytopic --time -1 

List the consumer groups known to Kafka

bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server localhost:9092 --list 

View the details of a consumer group

bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group <group name>

Stay tuned, we will keep updating the Apache Kafka cheatsheet regularly.