hadoop

Posted 2020-12-11 寂寞沙洲冷

1 hadoop 在虚拟机上的安装与配置

下载地址 http://archive.cloudera.com/cdh5/cdh/5/

演示的下载包 http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.15.1.tar.gz

下载解压

 环境centos7.6

 安装jdk1.8

配置ssh无密码登陆

ssh -keygen -t rsa 一路回车

cd ~/.ssh  有id_rsa ，id_rsa.pub，known_hosts

 

 

 cat  id_rsa.pub >> authorized_keys

chomd 600 authorized_keys

vim /etc/hosts

设置 ip  hadoop000

即 

 

 

 

vim /etc/profile

 

 

 

export JAVA_HOME=/usr/local/java/jdk/jdk1.8.0_11/
export JRE_HOME=/usr/local/java/jdk/jdk1.8.0_11//jre
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export CLASSPATH=$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

#hadoop
export HADOOP_HOME=/work/hadoop/hadoop-2.6.0-cdh5.15.1/
export PATH=$PATH:HADOOP_HOME/bin:$HADOOP_HOME/bin

 

vim /work/hadoop/hadoop-2.6.0-cdh5.15.1/etc/hadoop/hdfs-site.xml

mkdir tmp

 

 

 

vim /work/hadoop/hadoop-2.6.0-cdh5.15.1/etc/hadoop/core-site.xml

 

 

 

<configuration>

<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop000:8020</value>
</property>
</configuration>

 

vim /work/hadoop/hadoop-2.6.0-cdh5.15.1/etc/hadoop/hadoop-env.sh

 

 

vim /work/hadoop/hadoop-2.6.0-cdh5.15.1/etc/hadoop/slaves

 

 

/sbin/stop-all.sh #先停止hadoop相关进程

/bin/hdfs namenode -format #重新格式化文件系统，因为之前的文件系统被误删除

firewall-cmd --state   查看防火墙的运行状态

开启/关闭防火墙(重启服务器后又会重新关闭)

[root@localhost HMK]# systemctl start/stop firewalld.service

设置防火墙开机自动启动

[root@localhost HMK]# systemctl enable firewalld.service   

重启防火墙

[root@localhost HMK]# systemctl restart firewalld.service

访问
http://192.168.19.128:50070