storm集群搭建

Posted 张英爱

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了storm集群搭建相关的知识,希望对你有一定的参考价值。

安装环境:

虚拟操作系统:centOS7 64位  2台

IP地址为192.168.1.10   192.168.1.11

内存:1GB内存

zookeeper版本 :3.4.6

Storm版本:0.9.6

安装步骤:

1、安装JDK等步骤省略。

1、下载storm安装包,解压。

 

1、配置/apache-storm-0.9.6/conf/storm.yaml配置文件

 

# Licensed to the Apache Software Foundation (ASF) under one

 

# or more contributor license agreements.  See the NOTICE file

 

# distributed with this work for additional information

 

# regarding copyright ownership.  The ASF licenses this file

 

# to you under the Apache License, Version 2.0 (the

 

# "License"); you may not use this file except in compliance

 

# with the License.  You may obtain a copy of the License at

 

#

 

# http://www.apache.org/licenses/LICENSE-2.0

 

#

 

# Unless required by applicable law or agreed to in writing, software

 

# distributed under the License is distributed on an "AS IS" BASIS,

 

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 

# See the License for the specific language governing permissions and

 

# limitations under the License.

 

 

 

########### These MUST be filled in for a storm configuration

 

storm.zookeeper.servers:

 

     - "192.168.1.10"

 

     - "192.168.1.11"

 

nimbus.host : "192.168.1.10"

 

storm.local.dir : "/opt/stormtmp/data"

 

ui.port : 8080

 

supervisor.slots.ports :

 

   -6700

 

   -6701

 

   -6702

 

   -6703

 

#

 

#

 

# ##### These may optionally be filled in:

 

#    

 

## List of custom serializations

 

# topology.kryo.register:

 

#     - org.mycompany.MyType

 

#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer

 

#

 

## List of custom kryo decorators

 

# topology.kryo.decorators:

 

#     - org.mycompany.MyDecorator

 

#

 

## Locations of the drpc servers

 

# drpc.servers:

 

#     - "server1"

 

#     - "server2"

 

 

 

## Metrics Consumers

 

# topology.metrics.consumer.register:

 

#   - class: "backtype.storm.metric.LoggingMetricsConsumer"

 

#     parallelism.hint: 1

 

#   - class: "org.mycompany.MyMetricsConsumer"

 

#     parallelism.hint: 1

 

#     argument:

 

#       - endpoint: "metrics-collector.mycompany.org"

 

配置详解:

 

storm.zookeeper.servers: ZooKeeper服务器列表

 

supervisor.slots.ports :且每个端口只运行一个worker.通过这项配置可以调整每台机器上运行的worker数.(调整slot数/每机)

 

Storm.local.dir:storm使用的本地文件系统目录(必须存在并且storm进程可读写)

 

Nimbus.host:nimbus服务器地址

 

Ui.port:Storm UI的服务端口

 

nimbus.host :nimbus服务器地址

 

2、启动storm。先启动10的storm。

 

./storm nimbus &

 

./storm supervisor &

 

./storm ui &

 

IP为11结尾的机器运行

 

./storm supervisor &

 

3、查看运行状态,访问localhost:8080,进入storm ui界面,查看集群成功。

 

以上是关于storm集群搭建的主要内容,如果未能解决你的问题,请参考以下文章

storm集群环境搭建

Storm集群环境搭建

storm集群搭建

storm集群搭建

Storm和Zookeeper集群搭建及在java项目中的使用

Storm实践:集群搭建