Centos 7.4 下hadoop环境配置

Posted 2020-10-17 我是畅游海

以后的工作中需要用到hadoob分布式系统，所以博主预先学习了一下hadoob的安装与环境配置，特此记录。

• 首先，是准备一台Linux开发机，这里使用的是centos 7.4

[[email protected] ~]# cat /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core) 
[[email protected] ~]# 

• 配置ssh免密码登录

[[email protected] ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
以下部分省略

• 将秘钥copy到本机

[[email protected] ~]# ssh-copy-id xxx.xx.xxx.xx
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]‘s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh ‘xxx.xx.xxx.xx‘"
and check to make sure that only the key(s) you wanted were added.

• 配置java环境

%建立三个文件夹存放环境
cd /home
mkdir tools
mkdir softwares
mkdir data

　　这里需要安装一个文件上传下载插件

yum install -y lrzsz

　　上传已经下载的jdk文件，这里你也可以选择其他的安装jdk方法。

　　解压jdk文件

[[email protected] tools]# tar -zxvf jdk-8u151-linux-x64.tar.gz -C ../softwares/

　　配置环境变量：在配置文件中加入环境变量

[[email protected] jdk1.8.0_151]# vim /etc/profile

export JAVA_HOME=/home/softwares/jdk1.8.0_151
export PATH=$PATH:$JAVA_HOME/bin

　　添加用户

adduser user1
passwd user1

• 下载安装hadoop

wget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.9.0/hadoop-2.9.0-src.tar.gz

　　解压

• 安装依赖

　　

maven 然后配置环境变量
findbugs 然后配置环境变量

cd ../protobuf-2.5.0
./configure
安装其他依赖
yum -y install autoconf automake libtool cmake ncurses-devel openssl-devel lzo-devel zlib-devel gcc gcc-c++
make install

• 进行编译

cd ../hadoop-2.9.0
mvn package -Pdist,native,docs -DskipTests -Dtar
%docs 可去除

• 进行基本配置