storm集群搭建

Posted 2020-07-20 鹏鹏

安装环境：

虚拟操作系统：centOS7 64位  2台

IP地址为192.168.1.10   192.168.1.11

内存：1GB内存

zookeeper版本 ：3.4.6

Storm版本：0.9.6

安装步骤：

1、安装JDK等步骤省略。

1、下载storm安装包，解压。

 

1、配置/apache-storm-0.9.6/conf/storm.yaml配置文件

 

# Licensed to the Apache Software Foundation (ASF) under one

 

# or more contributor license agreements.  See the NOTICE file

 

# distributed with this work for additional information

 

# regarding copyright ownership.  The ASF licenses this file

 

# to you under the Apache License, Version 2.0 (the

 

# "License"); you may not use this file except in compliance

 

# with the License.  You may obtain a copy of the License at

 

#

 

# http://www.apache.org/licenses/LICENSE-2.0

 

#

 

# Unless required by applicable law or agreed to in writing, software

 

# distributed under the License is distributed on an "AS IS" BASIS,

 

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 

# See the License for the specific language governing permissions and

 

# limitations under the License.

 

 

 

########### These MUST be filled in for a storm configuration

 

storm.zookeeper.servers:

 

     - "192.168.1.10"

 

     - "192.168.1.11"

 

nimbus.host : "192.168.1.10"

 

storm.local.dir : "/opt/stormtmp/data"

 

ui.port : 8080

 

supervisor.slots.ports :

 

   -6700

 

   -6701

 

   -6702

 

   -6703

 

#

 

#

 

# ##### These may optionally be filled in:

 

#    

 

## List of custom serializations

 

# topology.kryo.register:

 

#     - org.mycompany.MyType

 

#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer

 

#

 

## List of custom kryo decorators

 

# topology.kryo.decorators:

 

#     - org.mycompany.MyDecorator

 

#

 

## Locations of the drpc servers

 

# drpc.servers:

 

#     - "server1"

 

#     - "server2"

 

 

 

## Metrics Consumers

 

# topology.metrics.consumer.register:

 

#   - class: "backtype.storm.metric.LoggingMetricsConsumer"

 

#     parallelism.hint: 1

 

#   - class: "org.mycompany.MyMetricsConsumer"

 

#     parallelism.hint: 1

 

#     argument:

 

#       - endpoint: "metrics-collector.mycompany.org"

 

配置详解：

 

storm.zookeeper.servers: ZooKeeper服务器列表

 

supervisor.slots.ports :且每个端口只运行一个worker.通过这项配置可以调整每台机器上运行的worker数.(调整slot数/每机)

 

Storm.local.dir：storm使用的本地文件系统目录(必须存在并且storm进程可读写)

 

Nimbus.host：nimbus服务器地址

 

Ui.port:Storm UI的服务端口

 

nimbus.host :nimbus服务器地址

 

2、启动storm。先启动10的storm。

 

./storm nimbus &

 

./storm supervisor &

 

./storm ui &

 

IP为11结尾的机器运行

 

./storm supervisor &

 

3、查看运行状态，访问localhost:8080,进入storm ui界面，查看集群成功。