Apache Kafka Package by IOanyT Innovations - Ubuntu22.04

OVERVIEW

Apache Kafka is a distributed data store optimized for ingesting and processing streaming data in real-time. Kafka provides three main functions to its users:

Publish and subscribe to streams of records
Effectively store streams of records in the order in which records were generated
Process streams of records in real time

Kafka is primarily used to build real-time streaming data pipelines and applications that adapt to the data streams. It combines messaging, storage, and stream processing to allow storage and analysis of both historical and real-time data. It uses a partitioned log model, which combines messaging queue and publish subscribe approaches. It uses binary protocol over TCP.

GOALS

One-click launch
Easy configurable AMI with pre-installed Apache Kafka

INSTALLATION VIA AWS MARKETPLACE

Kindly click on the below link to install the server via AWS Marketplace:

Pending..

GET STARTED

Apache Kafka provides:

Allows data to be distributed across multiple servers, making it scalable beyond what would fit on a single server.
Kafka decouples data streams so there is very low latency, making it extremely fast
Partitions are distributed and replicated across many servers, and the data is all written to disk.
Topics are automatically replicated, but the user can manually configure topics to not be replicated

Step 1. Enable Ports on Instance

Enable the ports 2181 and 9092 port on the instance

Step 2. Start the Apache Kafka Server

Perform the below steps to enable the Apache Kafka on the server,

sudo systemctl start kafka

sudo systemctl start zookeeper

Once these services are started, go to the below path

cd /usr/local/kafka

Open new terminal and run the below command to process the producer in Apache Kafka,

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic testTopic

bin/kafka-topics.sh --list --bootstrap-server localhost:2181

Open another terminal on the same path and run the below command to process the consumer in Apache Kafka

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testTopic

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic --from-beginning