Kafka Setup and Verification

1. Download Jdk 1.8 from https://www.oracle.com/in/java/technologies/javase/javase-jdk8-downloads.html#license-lightbox and install it.

2. set up JAVA_HOME environment variable.

3. Download Apache Kafka from https://archive.apache.org/dist/kafka/2.5.0/kafka_2.12-2.5.0.tgz and extract it to c:\.
kafka_2.12-2.5.0.tgz-------------2.12 is the scala version which has been used to develop this version of Kafka and 2.5.0 is the kafka version.
In the extracted directory, there will be 4 subdirectories.
  • bin--------------contains the binary scripts for unix platform.
  • bin\windows-----------contains the binary scripts for windows platform.
  • config--------contains the configuration files
  • libs----------contains the libraries in jar format
  • site-docs---------contains the documentation.
4. set the KAFKA_HOME environment variable(c:\kafka_2.12-2.5.0).

5. Add the %JAVA_HOME%\bin and %KAFKA_HOME%\bin\windows to the path.

6. Zookeeper is a service used by kafka to maintain its metadata. start the zookeeper.
open a new command window, and run the following command.
zookeeper-server-start %KAFKA_HOME%\config\zookeeper.properties
zookeeper by default, starts at port 2181.
Apache Kafka server start

7. start the kafka server.
open a new command window, and run the following command.
kafka-server-start %KAFKA_HOME%\config\server.properties
Zookeeper server start

kafka by default, starts at port 9092.

8. create a topic called first-topic with 4 partitions and replication factor of 1.
open a new command window, and run the following command.
kafka-topics --create --topic first-topic --partitions 4 --replication-factor 1 --zookeeper localhost:2181
if server is not properly started will get below error while creating topic.
Topic creation failure
after server starts here is the success message shows topic got created
Topic creation success

9. start a kafka console producer to send messages to the topic
kafka-console-producer --topic first-topic --bootstrap-server localhost:9092

10.start a kafka console consumer to receive messages from the topic
kafka-console-consumer --topic first-topic --bootstrap-server localhost:9092