apache storm 的安装

Posted 2020-10-02 牛顿的小脑

原文： http://storm.praveendeshmane.co.in/storm/storm-0-10-0-installation-on-ubuntu-14-04.jsp

------------------------------------------

Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language. Originally created by Nathan Marz and team at BackType, the project was open sourced after being acquired by Twitter. It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed processing of streaming data. The initial release was on 17 September 2011.

A Storm application is designed as a "topology" in the shape of a directed acyclic graph (DAG) with spouts and bolts acting as the graph vertices. Edges on the graph are named streams and direct data from one node to another. Together, the topology acts as a data transformation pipeline. At a superficial level the general topology structure is similar to a MapReduce job, with the main difference being that data is processed in real time as opposed to in individual batches.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system.

2) Apcahe ZooKeeper pre installed (How to install ZooKeeper on Ubuntu 14.04)

3) Apache Storm 0.10.0 software (Download Here)

Storm Installation Steps

Step 1 - Installing java 7. Open a terminal (CTRL + ALT + T) and type the following sudo command.

 

$ sudo apt-get install openjdk-7-jdk

Check it installed properly or not.

 

$ java -version

Step 2 - Edit $HOME/.bashrc file by adding the java path.

 

$ sudo gedit $HOME/.bashrc

$HOME/.bashrc file

 

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

Step 3 - Reload your changed $HOME/.bashrc settings

 

$ source $HOME/.bashrc

Step 4 - Creating storm directory. Open a terminal (CTRL + ALT + T) and type the following $ sudo command.

 

$ sudo mkdir /usr/local/storm

Step 5 - Change the ownership and permissions of the directory /usr/local/storm. Here ‘hduser‘ is an Ubuntu username.

 

$ sudo chown -R hduser /usr/local/storm

 

$ sudo chmod -R 755 /usr/local/storm

Step 6 - Change the directory to /home/hduser/Desktop , In my case the downloaded apache-storm-0.10.0.tar.gz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.

 

$ cd /home/hduser/Desktop/

Step 7 - Untar the apache-storm-0.10.0.tar.gz file.

 

$ tar xzf apache-storm-0.10.0.tar.gz

Step 8 - Move the contents of apache-storm-0.10.0 folder to /usr/local/storm

 

$ mv apache-storm-0.10.0/* /usr/local/storm

Step 9 - Change the directory to /usr/local/storm

 

$ cd /usr/local/storm

Step 10 - Create /data, /data/nimbus and /data/supervisor directories.

 

$ sudo mkdir /usr/local/storm/data
$ sudo mkdir /usr/local/storm/data/nimbus
$ sudo mkdir /usr/local/storm/data/supervisor

Step 11 - Change the ownership and permissions of the directory /data, /data/nimbus and /data/supervisor directories.. Here ‘hduser‘ is an Ubuntu username.

 

$ sudo chown -R hduser /usr/local/storm/data
$ sudo chown -R hduser /usr/local/storm/data/nimbus
$ sudo chown -R hduser /usr/local/storm/data/supervisor

 

$ sudo chmod -R 755 /usr/local/storm/data
$ sudo chmod -R 755 /usr/local/storm/data/nimbus
$ sudo chmod -R 755 /usr/local/storm/data/supervisor

Step 12 - Change the directory to /usr/local/storm/conf

 

$ cd /usr/local/storm/conf

Step 13 - Edit storm.yaml file.

 

$ gedit storm.yaml

Step 14 - Add these below lines to storm.yaml file. Save and close.

 

storm.zookeeper.servers: 
- "localhost"
storm.local.dir: "/usr/local/storm/data"
nimbus.host: "localhost"
nimbus.thrift.port: 49627
storm.zookeeper.port: 2181
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703

Step 15 - Start ZooKeeper. Open a new terminal (CTRL + ALT + T) and start zookeeper.

 

$ /usr/local/zookeeper/bin/zkServer.sh start

Step 16 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm

 

$ cd /usr/local/storm

Step 17 - Start nimbus

 

$ ./bin/storm nimbus

Step 18 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm

 

$ cd /usr/local/storm

Step 19 - Start supervisor

 

$ ./bin/storm supervisor

Step 20 - Open a new terminal (CTRL + ALT + T). Change the directory to /usr/local/storm

 

$ cd /usr/local/storm

Step 21 - Start web UI

 

$ ./bin/storm ui

Step 22 - Check the web UI here. Open a browser and type the following URL.

 

http://localhost:8080

Please share this blog post and follow me for latest updates on