Installing Kafka With Zookeeper

Overview

Kafka offers two installation methods:

• With Zookeeper (Classic method): Traditional setup where Zookeeper manages configurations, leadership elections, and coordination
• KRaft Mode: Introduced in Kafka 2.8, manages metadata without Zookeeper

This guide covers the traditional Zookeeper-based installation for Windows, Mac, Linux, and Docker.

Windows Installation

Prerequisites

• Download Kafka from Apache Kafka website
• Download and install Java (Oracle JDK)
• Use PowerShell or Git Bash for commands

Installation Steps

1. Create Kafka Directory

- Create folder C:\kafka

- Extract Kafka files into this folder

1. Set Environment Variables

- KAFKA_HOME: Point to Kafka folder

- PATH: Add Kafka to system PATH

- JAVA_HOME: Point to Java installation directory

1. Start Zookeeper

- Run Zookeeper start command

- Wait for confirmation message

1. Start Kafka

- Run Kafka server start command

- Verify Kafka is running

Mac Installation

Prerequisites

• Homebrew installed
• Java (install via Homebrew if needed)

Installation Steps

1. Install Java

1brew install java

1. Set JAVA_HOME

- Add to profile configuration

- Reload profile

- Verify: java -version

1. Install Kafka

1brew install kafka

- Verify: Check Kafka version

1. Start Zookeeper

- Run Zookeeper start command

- Wait for info message confirming startup

1. Start Kafka

- Run Kafka server start command

- Verify startup confirmation message

Linux Installation

Installation Steps

1. Install Java

1sudo apt update
2sudo apt install openjdk-11-jdk
3java -version

1. Set JAVA_HOME

- Add to .bashrc file

- Reload configuration

- Confirm path is set

1. Download and Extract Kafka

- Download from Apache Kafka website

- Extract files

- Move to /opt/kafka directory

- Navigate into Kafka directory

1. Start Zookeeper

- Run Zookeeper start command

- Watch for info message

1. Start Kafka

- Run Kafka server start command

- Wait for confirmation message

Docker Installation

Setup Steps

1. Create Kafka Directory

1mkdir ~/kafka
2cd ~/kafka

1. Create docker-compose.yml

1version: '3.8'
2services:
3   zookeeper:
4      image: confluentinc/cp-zookeeper:7.3.0
5      environment:
6      ZOOKEEPER_CLIENT_PORT: 2181
7   kafka:
8      image: confluentinc/cp-kafka:7.3.0
9      ports:
10      - "9092:9092"
11      environment:
12      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
13      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092

Configuration Details

Zookeeper Service:

• Uses Confluent Zookeeper image (version 7.3.0)
• ZOOKEEPER_CLIENT_PORT: 2181: Default port for Zookeeper communication

Kafka Service:

• Uses Confluent Kafka image (version 7.3.0)
• Exposes port 9092 for client connections
• KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181: Connects to Zookeeper
• KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092: Advertises broker address to clients

Running Docker Compose

1docker-compose up -d

This starts both Zookeeper and Kafka services in detached mode.