Hadoop HA的搭建

Posted 蜡笔小兴

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Hadoop HA的搭建相关的知识,希望对你有一定的参考价值。

1.首先添加hosts文件

vim /etc/hosts

10.0.110.56  MSJTVL-DSJC-H01
10.0.110.57  MSJTVL-DSJC-H03
10.0.110.58  MSJTVL-DSJC-H05
10.0.110.60  MSJTVL-DSJC-H02
10.0.110.61  MSJTVL-DSJC-H04

2.几台机器做互信

Setup passphraseless ssh

Now check that you can ssh to the localhost without a passphrase:

  $ ssh localhost
If you cannot ssh to localhost without a passphrase, execute the following commands:

  $ ssh-keygen -t dsa -P ‘‘ -f ~/.ssh/id_dsa
  $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

把其他几台机器的秘钥文件复制到MSJTVL-DSJC-H01的authorized_keys文件中

[[email protected] .ssh]$ scp [email protected]:/hadoop/.ssh/id_dsa.pub ./id_dsa.pub2
[[email protected] .ssh]$ scp [email protected]:/hadoop/.ssh/id_dsa.pub ./id_dsa.pub3
[[email protected] .ssh]$ scp [email protected]:/hadoop/.ssh/id_dsa.pub ./id_dsa.pub4
[[email protected] .ssh]$ scp [email protected]:/hadoop/.ssh/id_dsa.pub ./id_dsa.pub5

[[email protected] .ssh]$ cat ~/.ssh/id_dsa.pub2 >> ~/.ssh/authorized_keys
[[email protected] .ssh]$ cat ~/.ssh/id_dsa.pub3 >> ~/.ssh/authorized_keys
[[email protected] .ssh]$ cat ~/.ssh/id_dsa.pub4 >> ~/.ssh/authorized_keys
[[email protected] .ssh]$ cat ~/.ssh/id_dsa.pub5 >> ~/.ssh/authorized_keys

以上操作实现了MSJTVL-DSJC-H02,3,4,5对MSJTVL-DSJC-H01的无密码登录

要是实现MSJTVL-DSJC-H01-5的全部互信则把MSJTVL-DSJC-H01上的authorized_keys文件COPY到其他机器上去

[[email protected] ~]$ scp [email protected]:/hadoop/.ssh/authorized_keys /hadoop/.ssh/authorized_keys

 

下载相应的tar包

wget http://apache.fayea.com/hadoop/common/hadoop-2.6.4/hadoop-2.6.4.tar.gz

 

以上是关于Hadoop HA的搭建的主要内容,如果未能解决你的问题,请参考以下文章

搭建HA高可用hadoop-2.3(部署配置HBase)

hadoop2 环境的搭建(自动HA)

Hadoop HA集群的搭建

hadoop2.8 ha 集群搭建

Hadoop HA + HBase环境搭建————zookeeper和hadoop环境搭建

Hadoop HA高可用搭建流程