Hadoop遇到的坑运行hadoop自带的例子报错 Error: Could not find or load main class org.apache.hadoop.mapred.YarnCh

Posted 2022-08-01 jiangxiaoju

原创不易，未经允许，请勿转载。

博客主页：https://xiaojujiang.blog.csdn.net/

2021-05-09 22:31:33,652 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
2021-05-09 22:31:34,985 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/atguigu/.staging/job_16205706
2021-05-09 22:31:35,147 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2021-05-09 22:31:36,157 INFO input.FileInputFormat: Total input files to process : 1
2021-05-09 22:31:36,236 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2021-05-09 22:31:36,380 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2021-05-09 22:31:36,430 INFO mapreduce.JobSubmitter: number of splits:1
2021-05-09 22:31:36,801 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2021-05-09 22:31:36,915 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1620570661027_0001
2021-05-09 22:31:36,915 INFO mapreduce.JobSubmitter: Executing with tokens: []
2021-05-09 22:31:37,558 INFO conf.Configuration: resource-types.xml not found
2021-05-09 22:31:37,559 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2021-05-09 22:31:37,928 INFO impl.YarnClientImpl: Submitted application application_1620570661027_0001
2021-05-09 22:31:38,478 INFO mapreduce.Job: The url to track the job: http://hadoop103:8088/proxy/application_1620570661027_0001/
2021-05-09 22:31:38,479 INFO mapreduce.Job: Running job: job_1620570661027_0001
2021-05-09 22:31:46,632 INFO mapreduce.Job: Job job_1620570661027_0001 running in uber mode : false
2021-05-09 22:31:46,633 INFO mapreduce.Job:  map 0% reduce 0%
2021-05-09 22:31:50,715 INFO mapreduce.Job: Task Id : attempt_1620570661027_0001_m_000000_0, Status : FAILED
[2021-05-09 22:31:49.268]Exception from container-launch.
Container id: container_1620570661027_0001_01_000002
Exit code: 1

[2021-05-09 22:31:49.297]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild


[2021-05-09 22:31:49.298]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild



2021-05-09 22:31:53,796 INFO mapreduce.Job: Task Id : attempt_1620570661027_0001_m_000000_1, Status : FAILED
[2021-05-09 22:31:52.393]Exception from container-launch.
Container id: container_1620570661027_0001_01_000003
Exit code: 1

[2021-05-09 22:31:52.398]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild


[2021-05-09 22:31:52.399]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild

首先先执行下面这个命令，获取所有的类路径。

hadoop classpath

然后在etc/hadoop/mapred-site.xml中加入下面这个标签。其中<value>里面的值就是上面hadoop classpath运行后的输出内容

<property>
<name>mapreduce.application.classpath</name>
<value>/opt/module/hadoop-3.1.3/etc/hadoop:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/common/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn:/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn/*</value>
</property>

拒绝白嫖从一键三连开始！

原创不易，未经允许，请勿转载。

博客主页：https://xiaojujiang.blog.csdn.net/