How to run Apache Kafka with Docker

by

To run Apache Kafka as Docker image, the simplest way is to use the Strimzi project which provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.

The following is a docker-compose.yaml file which can be used to start a Zookeeper and a single node Apache Kafka:

version: '2'

services:

  zookeeper:
    image: strimzi/kafka:0.17.0-kafka-2.4.0
    command: [
      "sh", "-c",
      "bin/zookeeper-server-start.sh config/zookeeper.properties"
    ]
    ports:
      - "2181:2181"
    environment:
      LOG_DIR: /tmp/logs

  kafka:
    image: strimzi/kafka:0.17.0-kafka-2.4.0
    command: [
      "sh", "-c",
      "bin/kafka-server-start.sh config/server.properties --override listeners=$${KAFKA_LISTENERS} --override advertised.listeners=$${KAFKA_ADVERTISED_LISTENERS} --override zookeeper.connect=$${KAFKA_ZOOKEEPER_CONNECT}"
    ]
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      LOG_DIR: "/tmp/logs"
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

Within the configuration, we are using the following environment variables:

KAFKA_LISTENERS is a comma-separated list of listeners, and the host/ip and port to which Kafka binds to on which to listen. For more complex networking this might be an IP address associated with a given network interface on a machine. The default is 0.0.0.0, which means listening on all interfaces.

KAFKA_ADVERTISED_LISTENERS is a comma-separated list of listeners with their the host/ip and port. This is the metadata that’s passed back to clients.

KAFKA_ZOOKEEPER_CONNECT refers to the Zookeeper node which is running on port 2181

You can run the docker-compose file as follows:

docker-compose up