如何获取Hadoop Native Lib

Posted 2023-05-05

最近需要安装CDH4.5, 在重启后提示缺失native lib, 经过一番折腾后,终于安装成功,下面记录下native lib的获取过程.

@Author duangr

@Website http //my oschina net/duangr/blog/180905

1.相关环境
OS
Centos6.4
Arch
x86_64

Hadoop
hadoop-2.0.0-cdh4.5.0

[主]duangr-1

192.168.56.10

[主]duangr-2

192.168.56.11

[主]duangr-3

192.168.56.12

2.获取代码

hadoop-2.0.0-cdh4.5.0.tar.gz

hadoop-2.0.0+1518-1.cdh4.5.0.p0.24.el6.x86_64.rpm

3.安装hadoop

安装hadoop-2.0.0-cdh4.5.0.tar.gz, 此处略

4.安装Native Lib

将hadoop-2.0.0+1518-1.cdh4.5.0.p0.24.el6.x86_64.rpm在PC上使用压缩工具打开后,取出\usr\lib\hadoop\lib\native
下面的代码,上传到主机 $HADOOP_HOME/lib/native 下面

cd $HADOOP_HOME/lib/native rm libhadoop.so rm libsnappy.so rm libsnappy.so.1 ln -s libhadoop.so.1.0.0 libhadoop.so ln -s libsnappy.so.1.1.3 libsnappy.so.1 ln -s libsnappy.so.1.1.3 libsnappy.so
cd .. tar -zcf native.tar.gz native
scp native.tar.gz duangr-2:$HADOOP_HOME/lib ssh duangr-2 "cd $HADOOP_HOME/lib; tar -zxf native.tar.gz;"
scp native.tar.gz duangr-3:$HADOOP_HOME/lib ssh duangr-3 "cd $HADOOP_HOME/lib; tar -zxf native.tar.gz;"

5.安装完成

重启hadoop即可 参考技术A 最近需要安装CDH在重启后提示缺失nativelib经过一番折腾后终于安装成功下面记录下nativelib的获取过程@Authorduangr@Website相关环境获取代码hadoopcdhtargzhadoop+cdhpelx_rpm安装hadoop安装hadoopcdhtargz此处略安装NativeLib

解决Unable to load native-hadoop library for your platform

使用hadoop fs相应命令时候总是出现

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

系统环境下

添加

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"