cdh5.14.2-安装文档

Posted 2021-01-06 blablablabla

cdh5.14.2-安装文档

一、版本信息：

CM/CDH: 5.14.2-1

Linux: Centos7.4

二、准备原料，vmware虚拟机环境：

cm150	10.0.1.150
cdh151	10.0.1.151
cdh152	10.0.1.152
cdh153	10.0.1.153

三、软件列表

编号	类别	名称
01	jdk	jdk-6u31-linux-amd64.rpm
02	jdk	oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
03	driver	mysql-connector-java.jar
04	database	MySQL-5.7.21-SETUP.zip
05	cm	cloudera-manager-centos7-cm5.14.2_x86_64.tar.gz
06	cdh	CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel
07	cdh	CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel.sha
08	cdh	manifest.json
09	rpm	MySQL-python-1.2.5-1.el7.x86_64.rpm
10	rpm	mariadb-libs-5.5.56-2.el7.x86_64.rpm

如图：

四、更新系统，安装组件，重启系统(CM/CDH执行)：

yum install -y epel-release && yum install -y python-httplib2.noarch python-devel libxslt httpd mod_ssl unzip p7zip lsof telnet jemalloc jemalloc-devel bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs openssl-devel python-psycopg2 MySQL-python redhat-lsb-core.x86_64 && yum update -y && reboot

五、防火墙设置(CM/CDH执行)：

systemctl disable firewalld.service && systemctl stop firewalld.service

六、修改系统安全项(CM/CDH执行)：

1，替换文件内容

sed -i ‘s/ SELINUX=enforcing/ SELINUX=disabled/‘ /etc/selinux/config

2，执行命令1

setenforce 0

3,执行命令2

echo "10" > /proc/sys/vm/swappiness

echo "never" > /sys/kernel/mm/transparent_hugepage/defrag

echo "never" > /sys/kernel/mm/transparent_hugepage/enabled

七、配置主机名(CM/CDH执行)：

echo -e "10.0.1.150 cm150 10.0.1.151 cdh151 10.0.1.152 cdh152 10.0.1.153 cdh153 " >> /etc/hosts

八、配置SSH无密码访问(CM/CDH执行)：

ssh-keygen -t rsa

ssh-copy-id -i /root/.ssh/id_rsa.pub cm150

ssh-copy-id -i /root/.ssh/id_rsa.pub cdh151

ssh-copy-id -i /root/.ssh/id_rsa.pub cdh152

ssh-copy-id -i /root/.ssh/id_rsa.pub cdh153

九、创建用户(CM/CDH执行)：

useradd cloudera-scm -s /sbin/nologin

十、上传软件(CM执行)

过程略

十一、解压软件(CM执行)

tar xf cloudera-manager-centos7-cm5.14.2_x86_64.tar.gz

十二、移动文件(CM执行)

mv cloudera /opt/

mv cm-5.14.2/ /opt/

mv CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel* /opt/cloudera/parcel-repo/

mv manifest.json /opt/cloudera/parcel-repo/

十三、修改配置文件(CM执行)：

sed -i ‘s/server_host=localhost/server_host=cm150/‘ /opt/cm-5.14.2/etc/cloudera-scm-agent/config.ini

十四、复制驱动文件(CM执行):

cp mysql-connector-java.jar /opt/cm-5.14.2/share/cmf/lib/

十五、创建文件夹(CM/CDH执行)：

mkdir -p /usr/share/java

十六、复制驱动文件(CM执行):

cp /root/mysql-connector-java.jar /usr/share/java/

十七、复制文件(CDH执行)

scp -r [email protected]:/opt/* /opt/

scp -r [email protected]:/root/*.rpm /root

scp [email protected]:/usr/share/java/mysql-connector-java.jar /usr/share/java

十八、安装JDK(CM/CDH执行)：

rpm -ivh *.rpm

十九、配置环境变量(CM/CDH执行)：

echo -e "export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera export JAVA_BIN=/usr/java/jdk1.7.0_67-cloudera/bin export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export JAVA_HOME JAVA_BIN PATH CLASSPATH" >> /etc/profile && source /etc/profile

二十、数据库安装(可安装在CM节点，也可单独安装)

安装过程略

二十一、创建数据库(数据库执行):

CREATE DATABASE IF NOT EXISTS hue DEFAULT CHARSET gbk COLLATE gbk_bin;

CREATE DATABASE IF NOT EXISTS oozie DEFAULT CHARSET gbk COLLATE gbk_bin;

CREATE DATABASE IF NOT EXISTS hive DEFAULT CHARSET gbk COLLATE gbk_bin;

CREATE DATABASE IF NOT EXISTS scm DEFAULT CHARSET gbk COLLATE gbk_bin;

二十二、用户授权(数据库执行):

grant all on *.* to ‘cdhuser‘@‘%‘ identified by ‘cdhpass‘;

flush privileges;

二十三、若数据库开启了GTID事务，那么需将其关闭(数据库执行):

如图：

重启数据库服务，service mysqld restart

二十四、初始化数据库(CM执行)：

/opt/cm-5.14.2/share/cmf/schema/scm_prepare_database.sh mysql scm -h cm6 cdhuser cdhpass --scm-host cm6 scm scm scm

二十五、启动CM服务(CM执行)：

/opt/cm-5.14.2/etc/init.d/cloudera-scm-server start

查看日志

tail -f /opt/cm-5.14.2/log/cloudera-scm-server/cloudera-scm-server.log

二十六、启动CM-Agent端（CDH执行）：

/opt/cm-5.14.2/etc/init.d/cloudera-scm-agent start

查看日志

tail -f /opt/cm-5.14.2/log/cloudera-scm-agent/cloudera-scm-agent.log