zookeeper 集群安装（单点与分布式成功安装）

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了zookeeper 集群安装（单点与分布式成功安装）相关的知识，希望对你有一定的参考价值。

参考

http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html


ZooKeeper是一个分布式开源框架，提供了协调分布式应用的基本服务，它向外部应用暴露一组通用服务——分布式同步（Distributed Synchronization）、命名服务（Naming Service）、集群维护（Group Maintenance）等，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。ZooKeeper本身可以以Standalone模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。

有关ZooKeeper的介绍，网上很多，也可以参考文章后面，我整理的一些相关链接。

下面，我们简单说明一下ZooKeeper的配置。


ZooKeeper Standalone模式


从Apache网站上（zookeeper.apache.org）下载ZooKeeper软件包，我选择了3.3.4版本的（zookeeper-3.3.4.tar.gz），在一台Linux机器上安装非常容易，只需要解压缩后，简单配置一下即可以启动ZooKeeper服务器进程。

将zookeeper-3.3.4/conf目录下面的 zoo_sample.cfg修改为zoo.cfg，配置文件内容如下所示：


[plain] view plaincopy

  1. tickTime=2000  

  2. dataDir=/home/hadoop/storage/zookeeper  

  3. clientPort=2181  

  4. initLimit=5  

  5. syncLimit=2  

上面各个配置参数的含义也非常简单，引用如下所示：



[plain] view plaincopy

  1. tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.  

  2. dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.  

  3. clientPort —— the port to listen for client connections  

下面启动ZooKeeper服务器进程：



[plain] view plaincopy

  1. cd zookeeper-3.3.4/  

  2. bin/zkServer.sh start  

通过jps命令可以查看ZooKeeper服务器进程，名称为QuorumPeerMain。


在客户端连接ZooKeeper服务器，执行如下命令：


[plain] view plaincopy

  1. bin/zkCli.sh -server dynamic:2181  

上面dynamic是我的主机名，如果在本机执行，则执行如下命令即可：

[plain] view plaincopy

  1. bin/zkCli.sh  

客户端连接信息如下所示：



接着，可以使用help查看Zookeeper客户端可以使用的基本操作命令。



ZooKeeper Distributed模式


ZooKeeper分布式模式安装（ZooKeeper集群）也比较容易，这里说明一下基本要点。

首先要明确的是，ZooKeeper集群是一个独立的分布式协调服务集群，“独立”的含义就是说，如果想使用ZooKeeper实现分布式应用的协调与管理，简化协调与管理，任何分布式应用都可以使用，这就要归功于Zookeeper的数据模型（Data Model）和层次命名空间（Hierarchical Namespace）结构，详细可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在设计你的分布式应用协调服务时，首要的就是考虑如何组织层次命名空间。

下面说明分布式模式的安装配置，过程如下所示：

第一步：主机名称到IP地址映射配置

ZooKeeper集群中具有两个关键的角色：Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务，集群中每个结点之间都互相连接，所以，在配置的ZooKeeper集群的时候，每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。

例如，我的ZooKeeper集群中每个结点的配置，以slave-01为例，/etc/hosts内容如下所示：


[plain] view plaincopy

  1. 192.168.0.179   slave-01  

  2. 192.168.0.178   slave-02  

  3. 192.168.0.177   slave-03  


ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中，只有一个Leader，其他的都是Follower，如果ZooKeeper集群在运行过程中Leader出了问题，系统会采用该算法重新选出一个Leader。因此，各个结点之间要能够保证互相连接，必须配置上述映射。

ZooKeeper集群启动的时候，会首先选出一个Leader，在Leader election过程中，某一个满足选举算的结点就能成为Leader。整个集群的架构可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。

第二步：修改ZooKeeper配置文件

在其中一台机器（slave-01）上，解压缩zookeeper-3.3.4.tar.gz，修改配置文件conf/zoo.cfg，内容如下所示：


[plain] view plaincopy

  1. tickTime=2000  

  2. dataDir=/home/hadoop/storage/zookeeper  

  3. clientPort=2181  

  4. initLimit=5  

  5. syncLimit=2  

  6. server.1=slave-01:2888:3888  

  7. server.2=slave-02:2888:3888  

  8. server.3=slave-03:2888:3888  

上述配置内容说明，可以参考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。


第三步：远程复制分发安装文件

上面已经在一台机器slave-01上配置完成ZooKeeper，现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下：


[plain] view plaincopy

  1. cd /home/hadoop/installation/  

  2. scp -r zookeeper-3.3.4/ [email protected]:/home/hadoop/installation/  

  3. scp -r zookeeper-3.3.4/ [email protected]:/home/hadoop/installation/  

第四步：设置myid


在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字，例如：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid  

  2. [email protected]:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid  

  3. [email protected]:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid  

按照上述进行配置即可。


第五步：启动ZooKeeper集群

在ZooKeeper集群的每个结点上，执行启动ZooKeeper服务的脚本，如下所示：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start  

  2. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start  

  3. [email protected]:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start  

以结点slave-01为例，日志如下所示：

[plain] view plaincopy

  1. [email protected]:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out   

  2. 2012-01-08 06:51:19,117 - INFO  [main:[email protected]] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg  

  3. 2012-01-08 06:51:19,133 - INFO  [main:[email protected]] - Defaulting to majority quorums  

  4. 2012-01-08 06:51:19,167 - INFO  [main:[email protected]] - Starting quorum peer  

  5. 2012-01-08 06:51:19,227 - INFO  [main:[email protected]] - binding to port 0.0.0.0/0.0.0.0:2181  

  6. 2012-01-08 06:51:19,277 - INFO  [main:[email protected]] - tickTime set to 2000  

  7. 2012-01-08 06:51:19,278 - INFO  [main:[email protected]] - minSessionTimeout set to -1  

  8. 2012-01-08 06:51:19,279 - INFO  [main:[email protected]] - maxSessionTimeout set to -1  

  9. 2012-01-08 06:51:19,281 - INFO  [main:[email protected]] - initLimit set to 5  

  10. 2012-01-08 06:51:19,347 - INFO  [Thread-1:[email protected]] - My election bind port: 3888  

  11. 2012-01-08 06:51:19,393 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - LOOKING  

  12. 2012-01-08 06:51:19,396 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - New election. My id =  1, Proposed zxid = 0  

  13. 2012-01-08 06:51:19,400 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)  

  14. 2012-01-08 06:51:19,416 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888  

  15. java.net.ConnectException: Connection refused  

  16.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  17.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  18.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  19.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  20.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)  

  21.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)  

  22.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)  

  23.         at java.lang.Thread.run(Thread.java:662)  

  24. 2012-01-08 06:51:19,420 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  25. java.net.ConnectException: Connection refused  

  26.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  27.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  28.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  29.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  30.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)  

  31.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)  

  32.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)  

  33.         at java.lang.Thread.run(Thread.java:662)  

  34. 2012-01-08 06:51:19,612 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888  

  35. java.net.ConnectException: Connection refused  

  36.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  37.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  38.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  39.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  40.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  41.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  42.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  43. 2012-01-08 06:51:19,615 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  44. java.net.ConnectException: Connection refused  

  45.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  46.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  47.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  48.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  49.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  50.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  51.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  52. 2012-01-08 06:51:19,616 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 400  

  53. 2012-01-08 06:51:20,019 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888  

  54. java.net.ConnectException: Connection refused  

  55.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  56.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  57.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  58.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  59.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  60.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  61.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  62. 2012-01-08 06:51:20,021 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  63. java.net.ConnectException: Connection refused  

  64.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  65.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  66.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  67.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  68.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  69.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  70.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  71. 2012-01-08 06:51:20,022 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 800  

  72. 2012-01-08 06:51:20,825 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888  

  73. java.net.ConnectException: Connection refused  

  74.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  75.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  76.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  77.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  78.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  79.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  80.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  81. 2012-01-08 06:51:20,827 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  82. java.net.ConnectException: Connection refused  

  83.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  84.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  85.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  86.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  87.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  88.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  89.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  90. 2012-01-08 06:51:20,828 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 1600  

  91. 2012-01-08 06:51:22,435 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888  

  92. java.net.ConnectException: Connection refused  

  93.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  94.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  95.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  96.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  97.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  98.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  99.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  100. 2012-01-08 06:51:22,439 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  101. java.net.ConnectException: Connection refused  

  102.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  103.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  104.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  105.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  106.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)  

  107.         at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)  

  108.         at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)  

  109. 2012-01-08 06:51:22,441 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Notification time out: 3200  

  110. 2012-01-08 06:51:22,945 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)  

  111. 2012-01-08 06:51:22,946 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Updating proposal  

  112. 2012-01-08 06:51:22,949 - WARN  [WorkerSender Thread:[email protected]] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888  

  113. java.net.ConnectException: Connection refused  

  114.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  

  115.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)  

  116.         at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)  

  117.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)  

  118.         at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)  

  119.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)  

  120.         at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)  

  121.         at java.lang.Thread.run(Thread.java:662)  

  122. 2012-01-08 06:51:22,951 - INFO  [WorkerReceiver Thread:[email protected]] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)  

  123. 2012-01-08 06:51:23,156 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - FOLLOWING  

  124. 2012-01-08 06:51:23,170 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - TCP NoDelay set to: true  

  125. 2012-01-08 06:51:23,206 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT  

  126. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:host.name=slave-01  

  127. 2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.version=1.6.0_30  

  128. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.vendor=Sun Microsystems Inc.  

  129. 2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre  

  130. 2012-01-08 06:51:23,209 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:[email protected]] - Server environment:

    以上是关于zookeeper 集群安装（单点与分布式成功安装）的主要内容，如果未能解决你的问题，请参考以下文章

    zookeeper安装部署-单点和集群

    小象精华导航帖zookeeper技术专栏

    ELK之七------Zookeeper和Kafka安装与配置

    分布式服务管理框架-Zookeeper安装与配置（单机集群）

    分布式服务管理框架-Zookeeper安装与配置（单机集群）

    Zookeeper安装与配置