Resourcemanager的高可用

Posted 2020-10-15 kevin&learn

Resourcemanager的高可用：

1. 架构
   
   
   
   
   
2. 配置yarn-site.xml
   

   <configuration>
   <property>
   <!-- 1.开始yarn的高可用-->
   <name>yarn.resourcemanager.ha.enabled</name>
   <value>true</value>
   </property>
   <property>
   <!-- 2.指定RM的cluster id-->
   <name>yarn.resourcemanager.cluster-id</name>
   <value>yarn-ha</value>
   </property>
   <property>
   <!-- 3.指定开启高可用的RM别名-->
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>krm1,krm2</value>
   </property>
   <property>
   <!-- 4.分别指定RM的地址-->
   <name>yarn.resourcemanager.hostname.krm1</name>
   <value>kmaster1</value>
   </property>
   <property>
   <name>yarn.resourcemanager.hostname.krm2</name>
   <value>kmaster2</value>
   </property>
   <property>
   <name>yarn.resourcemanager.recovery.enabled</name>
   <value>true</value>
   </property>
   <property>
   <name>yarn.resourcemanager.store.class</name>
   <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
   </property>
   <property>
   <!-- 5.指定zookeeper的地址-->
   <name>yarn.resourcemanager.zk-address</name>
   <value>kslave5:2181,kslave6:2181,kslave7:2181</value>
   </property>
   <property>
   <!-- 6.为 mapreduce 分配 yarn 服务-->
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
   </property>
   <property>
   <!-- 7.nodemanager 本机计算任务的临时文件-->
   <name>yarn.nodemanager.local-dirs</name>
   <value>file:///home/hadoop/disk1/nodemgr/local,file:///home/hadoop/disk2/nodemgr/local</value>
   </property>
   <property>
   <!-- 8.nodemanager 日志输出-->
   <name>yarn.nodemanager.log-dirs</name>
   <value>file:///home/hadoop/disk1/log/hadoop-yarn/containers</value>
   </property>
   <property>
   <!-- 9.远程任务的输出-->
   <name>yarn.nodemanager.remote-app-log-dir</name>
   <value>file:///home/hadoop/disk1/log/hadoop-yarn/apps</value>
   </property>
   <property>
   <!-- 10.日志汇集-->
   <name>yarn.log-aggregation-enable</name>
   <value>true</value>
   </property>
   </configuration>

    

3. 然后两台机器分别启动resourcemanager即可

4. 检查

   yarn rmadmin -getServiceState krm1
    active
   yarn rmadmin -getServiceState krm2
    standby

    高可用RM配置成功

    

5. 学习博客