Organizational Research By

Surprising Reserch Topic

Experts Most Trusted Topic


How To Install Apache Kafka on Ubuntu 14.04


Step 1 — Create a User for Kafka

As Kafka can handle requests over a network, you should create a dedicated user for it. This minimizes damage to your Ubuntu machine should the Kafka server be comprised.

Note: After setting up Apache Kafka, it is recommended that you create a different non-root user to perform other tasks on this server.

As root, create a user ed kafka using the useradd command:

 
useradd kafka -m
Set its password using passwd:

 
passwd kafka
Add it to the sudo group so that it has the privileges required to install Kafka's dependencies. This can be done using the adduser command:

 
adduser kafka sudo
Your Kafka user is now ready. Log into it using su:

 
su - kafka
Step 2 — Install Java

Before installing additional packages, update the list of available packages so you are installing the latest versions available in the repository:

 
sudo apt-get update
As Apache Kafka needs a Java runtime environment, use apt-get to install the default-jre package:

 
sudo apt-get install default-jre
Step 3 — Install ZooKeeper

Apache ZooKeeper is an open source service built to coordinate and synchronize configuration information of nodes that belong to a distributed system. A Kafka cluster depends on ZooKeeper to perform—among other things—operations such as detecting failed nodes and electing leaders.

Since the ZooKeeper package is available in Ubuntu's default repositories, install it using apt-get.

 
sudo apt-get install zookeeperd
After the installation completes, ZooKeeper will be started as a daemon automatically. By default, it will listen on port 2181.

To make sure that it is working, connect to it via Telnet:

 
telnet localhost 2181
At the Telnet prompt, type in ruok and press ENTER.

If everything's fine, ZooKeeper will say imok and end the Telnet session.

Step 4 — Download and Extract Kafka Binaries

Now that Java and ZooKeeper are installed, it is time to download and extract Kafka.

To start, create a directory called Downloads to store all your downloads.

 
mkdir -p ~/Downloads
Use wget to download the Kafka binaries.

 
wget "http://mirror.cc.columbia.edu/pub/software/apache/kafka/0.8.2.1/kafka_2.11-0.8.2.1.tgz" -O ~/Downloads/kafka.tgz
Create a directory called kafka and change to this directory. This will be the base directory of the Kafka installation.

 
mkdir -p ~/kafka && cd ~/kafka
Extract the archive you downloaded using the tar command.

 
tar -xvzf ~/Downloads/kafka.tgz --strip 1
Step 5 — Configure the Kafka Server

The next step is to configure the Kakfa server.

Open server.properties using vi:

 
vi ~/kafka/config/server.properties
By default, Kafka doesn't allow you to delete topics. To be able to delete topics, add the following line at the end of the file:

~/kafka/config/server.properties

delete.topic.enable = true
Save the file, and exit vi.

Step 6 — Start the Kafka Server

Run the kafka-server-start.sh script using nohup to start the Kafka server (also called Kafka broker) as a background process that is independent of your shell session.

 
nohup ~/kafka/bin/kafka-server-start.sh ~/kafka/config/server.properties > ~/kafka/kafka.log 2>&1 &
Wait for a few seconds for it to start. You can be sure that the server has started successfully when you see the following messages in ~/kafka/kafka.log:

excerpt from ~/kafka/kafka.log

...

[2015-07-29 06:02:41,736] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2015-07-29 06:02:41,776] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
You now have a Kafka server which is listening on port 9092.

asked Aug 30, 2015 in LINUX by rajesh
0 votes
215 views



Related Hot Questions



Walkin Jobs Opening



Government Jobs Opening


...