Logo-Transparent.png
partner-logo.png

Apache Kafka Package by IOanyT Innovations - AmazonLinux

Updated: Jun 20


OVERVIEW


Apache Kafka is a distributed data store optimized for ingesting and processing streaming data in real-time. Kafka provides three main functions to its users:

  • Publish and subscribe to streams of records

  • Effectively store streams of records in the order in which records were generated

  • Process streams of records in real time

Kafka is primarily used to build real-time streaming data pipelines and applications that adapt to the data streams. It combines messaging, storage, and stream processing to allow storage and analysis of both historical and real-time data. It uses a partitioned log model, which combines messaging queue and publish subscribe approaches. It uses binary protocol over TCP.



GOALS

  1. One-click launch

  2. Easy configurable AMI with pre-installed Apache Kafka


INSTALLATION VIA AWS MARKETPLACE


Kindly click on the below link to install the server via AWS Marketplace:


Pending..


GET STARTED


Apache Kafka provides:

  • Allows data to be distributed across multiple servers, making it scalable beyond what would fit on a single server.

  • Kafka decouples data streams so there is very low latency, making it extremely fast

  • Partitions are distributed and replicated across many servers, and the data is all written to disk.

  • Topics are automatically replicated, but the user can manually configure topics to not be replicated

Step 1. Enable Ports on Instance

Enable the ports 2181 and 9092 port on the instance


Step 2. Login via Kafka user

Login with kafka user using this command


su kafka

Enter Instance ID as password

password- <aws_instance_id>


Step3. Start the Apache Kafka Server

Perform the below steps to enable the Apache Kafka on the server,

sudo systemctl start zookeeper.service
sudo systemctl start kafka.service

Open new terminal and run the below command to process the producer in Apache Kafka,

~/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic testTopic
~/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testTopic

Open another terminal on the same path and run the below command to process the consumer in Apache Kafka

~/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic --from-beginning

CONTACT

Let’s Work Together

IOanyT Innovations Pvt. Ltd.

Email: aws-marketplace-support@ioanyt.com

  • LinkedIn
  • Facebook

Thanks for submitting!