Apache Kafka

Apache Kafka is a distributed streaming platform designed for building real-time data pipelines and streaming applications. It handles large data volumes, making it suitable for both enterprise and small-scale use.

Features of Apache Kafka:

• High throughput and scalability for real-time data handling.
• Fault-tolerant and reliable distributed architecture.
• Pub/Sub messaging model for communication between producers and consumers.
• Stream processing with Kafka Streams and Kafka Connect.
• Durability with message retention and replication.

Usage Instructions:

sudo su

Start the Kafka server:

sudo systemctl start kafka

Create a topic:

/opt/kafka/bin/kafka-topics.sh \
--create \
--topic my-topic \
--bootstrap-server localhost:9092 \
--partitions 1 \
--replication-factor 1

/opt/kafka/bin/kafka-console-producer.sh \
--topic my-topic \
--bootstrap-server localhost:9092

Type messages: hello kafka
testing message

Start Consumer message: Open another terminal and run:

/opt/kafka/bin/kafka-console-consumer.sh \
--topic my-topic \
--from-beginning \
--bootstrap-server localhost:9092

You will see the output

Disclaimer: Apache Kafka is open-source under the Apache License 2.0 and is provided "as is." Users assume all responsibility for use and compliance with licensing terms.