Cloudera 5.8.2 Installation guide

Posted 2020-08-16

Envireronment:

VMware® Workstation 12 Pro 12.0.1 build-3160714(Host: Windows 7 Ultimate)

Centos 6.5 x64

Cloudera 5.8.2

Resource:

1. Cloudera Manager 5 Requirements and Supported Versions
2. Installation Path B - Manual Installation Using Cloudera Manager Packages
3. http://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.16-linux-glibc2.5-x86_64.tar
4. http://archive.cloudera.com/cdh5/
5. CHD5 5.8.2 EL6
1. CDH-5.8.2-1.cdh5.8.2.p0.3-el6.parcel
2. CDH-5.8.2-1.cdh5.8.2.p0.3-el6.parcel.sha1
3. manifest.json
6. CM RPMS
1. cloudera-manager-agent-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
2. cloudera-manager-daemons-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
3. cloudera-manager-server-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
4. cloudera-manager-server-db-2-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
5. enterprise-debuginfo-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
6. jdk-6u31-linux-amd64.rpm
7. oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
7. Repro
1. cm5.8.2-centos6.tar.gz
2. cm5.8.2-centos6.tar.gz.md5
3. cm5.8.2-centos6.tar.gz.sha1

规划

角色	数量	内存	CPU
namenode	1	20GB	1*1 core
datanode	3	2GB	1*1 core

 

Envireronment setup：

• 在VMware中安装4台虚拟机，操作系统为Centos 6.5。
  • nn
  • n1
  • n2
  • n3
•  网络设置（每台）

• vi /etc/sysconfig/network
  vi /etc/sysconfig/network-scripts/ifcfg-eth0
  
  

  IPADDR=192.168.254.151~154
  GATEWAY=192.168.254.2
  DNS1=192.168.254.2

  service iptables stop
  chkconfig iptables off

  vi /etc/selinux/config
  
  shutdown -r now

• 检查

  • hostname
    ifconfig

    getenforce
    service iptables status

    ping cn.pool.ntp.org

    vi /etc/hosts

    192.168.254.151 nn
    192.168.254.152 n1
    192.168.254.153 n2
    192.168.254.154 n3

  • ping n1
    ping n2
    ping n3

•  系统配置（每台）
  • 配置NTP

  • ntpstat
    service ntpd start
    chkconfig ntpd on
    watch ntpq -p 

  • SSH免密码

  • ssh-keygen -t rsa
    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    chmod 600 ~/.ssh/authorized_keys
    
    scp ~/.ssh/authorized_keys [email protected]:~/.ssh/
    scp ~/.ssh/authorized_keys [email protected]:~/.ssh/
    scp ~/.ssh/authorized_keys [email protected]:~/.ssh/

Install CM：

• NameNode

• cloudera-manager-agent-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  cloudera-manager-daemons-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  cloudera-manager-server-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  cloudera-manager-server-db-2-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  enterprise-debuginfo-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  jdk-6u31-linux-amd64.rpm
  oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
  cloudera-manager-installer.bin

  yum localinstall --nogpgcheck *.rpm

• DataNode

• cloudera-manager-agent-5.8.2-1.cm582.p0.17.el6.x86_64.rpm
  cloudera-manager-daemons-5.8.2-1.cm582.p0.17.el6.x86_64.rpm

  jdk-6u31-linux-amd64.rpm
  oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

  yum localinstall --nogpgcheck *.rpm

  vi /etc/cloudera-scm-agent/config.ini

  server_host=nn

• Download CDH parcels and manifest.json from https://archive.cloudera.com/cdh5/parcels/

• mkdir /tmp/parcel
  cp /home/admin/Downloads/CDH-5.5.2-1.cdh5.5.2.p0.4-el7.parcel /tmp/parcel/
  cp /home/admin/Downloads/ manifest.json /tmp/parcel/

• python -m SimpleHTTPServer 8900

• http://n1:7180/
• 设置Parcels远程目录为http://192.168.254.151:8900
• 检查（所有主机）

• echo never > /sys/kernel/mm/transparent_hugepage/defrag
  echo 0 > /proc/sys/vm/swappiness

• 安装结束