EMR目录

Posted 2021-03-21 hongfeng2019

EMR目录：

配置文件目录:
/etc/ecm/hadoop-conf
#看gateway机器配置是否同步,直接看配置文件,如:
/etc/ecm/hadoop-conf/core-site.xml
然后输入命令hadoop classpath 打印一下classpath, 看引用的conf目录是否为/etc/ecm/hadoop-conf,以及是否包含其他目录

系统已经设置好的环境:
echo $JAVA_HOME /usr/lib/jvm/java-1.8.0
echo $HADOOP_HOME /usr/lib/hadoop-current
echo $HADOOP_CONF_DIR /etc/ecm/hadoop-conf
echo $HADOOP_OG_DIR
echo $YARN_LOG_DIR /var/log/hadoop-yarn
echo $HIVE_HOME /usr/lib/hive-current
echo $HIVE_CONF_DIR /etc/ecm/hive-conf
echo $PIG_HOME /usr/lib/pig-current
echo $PIG_CONF_DIR /etc/ecm/pig-conf

#做的软链

配置文件目录在/etc/ecm

例如 :core-site.xml在$HADOOP_CONF_DIR

/etc/ecm/hadoop-conf/core-site.xml

而是实际地址是: /etc/ecm/hadoop-conf-2.8.5-1.4.0/core-site.xml

又如:

vi /etc/ecm/hadoop-conf/fair-scheduler.xml

#这里面的配置文件记录集群的一些配置,调用jar等信息;

/usr/local/emr/emr-bin/conf

ln -s /opt/apps/hive-current /usr/lib/hive-current
/usr/lib/hadoop-current/share/hadoop/common/hadoop-common-2.8.5.jar
/usr/lib/hadoop-current/share/hadoop/common/lib/hadoop-auth-2.8.5.jar