Overview

Apache Kafka is an open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds.

In a nutshell, you say data to it and you can do something with that data. Here we will process data from Oracle through Kafka to big data.

So for example, you can have: Oracle → Kafka → Hadoop/Hbase

Management

To start Kafka, we can use the following script:

Start Kafka

Create a topic as follows

To create a topic we can use teh following command, knowing the zookeeper port and the name of the topic of course:

Create Topic

Send message to Kafka

Sending messages to Kafka is easy once you know the broker list and the topic to which you want them sent:

Send message to Kafka