关于java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.的问题

Posted a-du

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了关于java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.的问题相关的知识,希望对你有一定的参考价值。

报错如下:

300 [main] DEBUG org.apache.hadoop.util.Shell  - Failed to detect a valid hadoop home directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set

 

解决办法一:

根据 http://blog.csdn.net/baidu_19473529/article/details/54693523 配置hadoop_home变量

下载winutils地址https://github.com/srccodes/hadoop-common-2.2.0-bin下载解压

技术图片

技术图片

 

 

依然报相同的错误

 

解决办法二:

在java程序中加入

System.setProperty("hadoop.home.dir", "/usr/local/hadoop-2.6.0");

依然报错

 

解决办法三:

根据该提示:

 

技术图片

依然报错

--------------------------------5.19.早晨------------------------------------------

今天早上再次执行该程序,将上传到HDFS部分的代码给注释掉后发现日志报错信息如下:

132 [main] DEBUG org.apache.hadoop.util.NativeCodeLoader  - Trying to load the custom-built native-hadoop library...
134 [main] DEBUG org.apache.hadoop.util.NativeCodeLoader  - Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path

1)check Hadoop library:

[email protected]:/usr/local/hadoop-2.6.0/lib/native# file libhadoop.so.1.0.0

技术图片

It‘s 64-bite library

2)Try adding the HADOOP_OPTS environment variable:

[email protected]:~# vi /etc/profile                            //environment variable
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native/"

It doesn‘t work, and reports the same error.

3)Try adding the HADOOP_OPTS and HADOOP_COMMON_LIB_NATIVE_DIR environment variable:

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/"

It still doesn‘t work, and reports the same error.

4)Adding the Hadoop library into LD_LIBRARY_PATH

[email protected]:~# vi .bashrc 
export LD_LIBRARY_PATH=/usr/local/hadoop/lib/native/:$LD_LIBRARY_PATH

It doesn‘t work, and reports the same error.

5)append word native to HADOOP_OPTS like this

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

It doesn‘t work, and reports the same error.

-----------------------------------5.20早晨-------------------------

win下依旧无法使用java代码上传文件到HDFS

 ----------------------------------5.24------------

copy了一份hadoop-2.6.0文件到本机,更改bin目录和path ,显示无效的path,遂更改回去

---------------------2017.5.26-------------------

今天早上开机 试了一下 发现它已经好了  其实上述的设置已经可以解决大部分遇到这类问题的人群

 技术图片

 

以上是关于关于java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.的问题的主要内容,如果未能解决你的问题,请参考以下文章

关于java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.的问题

java.io.IOException Too many open files 解决

java.io.IOException: There appears to be a gap in the edit log. We expected txid ***, but got txid(示

Android java.io.IOException: write failed: EBADF (Bad file number)

解决Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory

React Native:错误:[消息/未知] java.io.IOException:java.util.concurrent.ExecutionException:java.io.IOExcept