centos7中安装cdh5.15

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了centos7中安装cdh5.15相关的知识,希望对你有一定的参考价值。

cdh搭建
1.软件准备

软件 版本
os Centos 7.3(64位)
cm cloudera-manager-centos7-cm5.15.0_x86_64.tar.gz
cdh CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel & CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha1 & manifest.json
mysql mysql-community-common-5.7.27-1.el7.x86_64.rpm & mysql-community-libs-5.7.27-1.el7.x86_64.rpm & mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm & mysql-community-devel-5.7.27-1.el7.x86_64.rpm & mysql-community-client-5.7.27-1.el7.x86_64.rpm & mysql-community-server-5.7.27-1.el7.x86_64.rpm
jdk jdk-8u221-linux-x64.rpm
jdbc mysql-connector-java-5.1.48.tar

2.主机配置规划

ip 配置 角色 需执行以下步骤
192.168.16.236 slave1 3,4,5,6,9
192.168.16.237 master 3,4,5,6,7,8,9,10,11
192.168.16.238 slave2 3,4,5,6,9

3.配置host

shell> vi /etc/hosts

在host文件中加入以下内容:
127.0.0.1 localhost
192.168.16.237 master
192.168.16.236 slave1
192.168.16.238 slave2
4.关闭防火墙和selinux安全工具

shell> systemctl stop firewalld.service(关闭防火墙)
shell> systemctl disable firewalld.service(禁止开机启动)
shell> firewall-cmd --state(状态查询)

关闭linux SELINUX安全内核

shell> setenforce 0(临时有效)

修改配置文件使其永久有效

shell> vi /etc/selinux/config

在config文件中找到SELINUX并设置为:
SELINUX=disabled

shell> reboot
shell> sestatus(状态查询)

5.配置ssh免密登陆和ntp时钟同步(参考https://blog.51cto.com/abezoo/2394680 中软件应用部分)
6.安装jdk及配置java环境变量
查看系统是否自带有jdk并将其卸载

shell> rpm -qa |grep java
shell> rpm -e -nodeps 包名

卸载后安装下载的jdk

shell> rpm -ivh jdk-8u221-linux-x64.rpm

配置环境java环境变量

shell> vi /etc/profile

在profile文件末尾加入以下内容:
export JAVA_HOME=/usr/java/jdk1.8.0_221-amd64
export PATH=.:$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
然后激活上面的配置项

shell> source /etc/profile
shell> echo $JAVA_HOME(查看是否配置成功)

7.安装mysql(参考https://blog.51cto.com/abezoo/2402906)
mysql这里需要注意两点:
(1)mysql的密码策略,mysql默认的中级,我们在这里除了要把其改为低级外,还要将密码的长度设置为3,因为我们后面cdh中通用的scm库的密码通常正好是3位

mysql> set global validate_password_policy=LOW;
mysql> set validate_password_number_count=0;
mysql> set validate_password_length=3;
mysql> SHOW VARIABLES LIKE ‘validate_password%‘; (查看密码策略中的各项参数)

(2)给root用户和scm用户赋权

mysql> grant all privileges on *.* to ‘root‘@‘localhost‘ identified by ‘root‘ with grant option;
mysql> grant all privileges on *.* to ‘root‘@‘127.0.0.1‘ identified by ‘root‘ with grant option;
mysql> grant all privileges on *.* to ‘root‘@‘%‘ identified by ‘root‘ with grant option;

mysql> create user scm;
mysql> grant all privileges on *.* to ‘scm‘@‘localhost‘ identified by ‘scm‘ with grant option;
mysql> grant all privileges on *.* to ‘scm‘@‘127.0.0.1‘ identified by ‘scm‘ with grant option;
mysql> grant all privileges on *.* to ‘scm‘@‘%‘ identified by ‘scm‘ with grant option;
mysql> flush privileges;

8.cdh所需数据库创建

mysql> create database hive default charset utf8;
mysql> create database oozie default charset utf8;
mysql> create database hue default charset utf8;
mysql> create database scm default charset utf8;

9.依赖软件安装

shell> yum install -y psmisc libxslt zlib sqlite fuse fuse-libs redhat-lsb cyrus-sasl-plain cyrus-sasl-gassapi chkconfig bind-utils perl

10.cm安装
在每台服务器上创建CM使用的用户

shell> useradd --system --home=/opt/cm-5.15.0/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
shell> mv CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha1 CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha
shell> tar -zxvf cloudera-manager-centos7-cm5.15.0_x86_64.tar.gz -C /opt/
shell> mv CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha manifest.json /opt/cloudera/parcel-repo

解压jdbc,并将其移到相应位置

shell> tar -xvf mysql-connector-java-5.1.48.tar
shell> cd mysql-connector-java-5.1.48
shell> cp mysql-connector-java-5.1.48-bin.jar /opt/cm-5.15.0/share/cmf/lib
shell> cp mysql-connector-java-5.1.48-bin.jar /usr/share/java

初始化cm服务器

shell> /opt/cm-5.15.0/share/cmf/schema/scm_prepare_database.sh mysql -hlocalhost -uroot -proot --scm-host localhost scm scm scm

直到出现your scm database is configured correctly!则表示初始化成功,才可接着配置agent

shell> vi /opt/cm-5.15.0/etc/cloudera-scm-agent/config.ini

找到文件中的server_host并设置为:
server_host=master

将cm复制到slave上

shell> scp -r /opt/cm-5.15.0 root@slave1:/opt
shell> scp -r /opt/cm-5.15.0 root@slave2:/opt

启动服务

shell> /opt/cm-5.15.0/etc/init.d/cloudera-scm-server start
shell> /opt/cm-5.15.0/etc/init.d/cloudera-scm-server statue(查看是否成功启动)
shell> /opt/cm-5.15.0/etc/init.d/cloudera-scm-agent start

注意:master、slave1、slave2三台机器都需要启动agent
输入:http://192.168.16.237:7180/ 就可以访问到CM的管理界面用户名密码都是:admin

11.组件安装

以上是关于centos7中安装cdh5.15的主要内容,如果未能解决你的问题,请参考以下文章

centos7中安装python3

centos7中安装zabbix

01 CentOS7中安装和启动solr

开发人员学Linux:VirtualBox中安装CentOS7系统设置

在 Docker 的 CentOS7 镜像 中安装 mysql

在 Docker 的 CentOS7 镜像 中安装 mysql