原创大叔问题定位分享（33）oozie提交任务报错ArithmeticException: / by zero

Posted 2022-03-18 barneywill

oozie提交workflow后执行task报错：

2019-07-04 17:19:00,559 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: ERROR IN CONTACTING RM. 
java.lang.ArithmeticException: / by zero
    at org.apache.hadoop.mapreduce.v2.app.rm.ResourceCalculatorUtils.computeAvailableContainers(ResourceCalculatorUtils.java:38)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$ScheduledRequests.assign(RMContainerAllocator.java:981)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$ScheduledRequests.access$200(RMContainerAllocator.java:873)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:252)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:282)
    at java.lang.Thread.run(Thread.java:748)

查看代码

org.apache.hadoop.mapreduce.v2.app.rm.ResourceCalculatorUtils

    public static int computeAvailableContainers(Resource available, Resource required, EnumSet<SchedulerResourceTypes> resourceTypes) 
        return resourceTypes.contains(SchedulerResourceTypes.CPU)?Math.min(available.getMemory() / required.getMemory(), available.getVirtualCores() / required.getVirtualCores()):available.getMemory() / required.getMemory();
    

应该是以下两者之一为0

required.getMemory()
required.getVirtualCores()

注意之前的日志还有一行

2019-07-04 17:18:58,557 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:0, vCores:1>

问题在于required.getMemory()=0

 

从job history server查看失败task对应的application的conf发现问题

              <tr>
                <td>
                  mapreduce.map.memory.mb
                </td>
                <td>
                  0
                </td>
                <td>
                  job.xml &#11013; programatically
                </td>
              </tr>

查看oozie代码发现

org.apache.oozie.action.hadoop.JavaActionExecutor

    private static void injectLauncherProperties(Configuration srcConf, Configuration launcherConf) 
        for (Map.Entry<String, String> entry : srcConf) 
            if (entry.getKey().startsWith("oozie.launcher.")) 
                String name = entry.getKey().substring("oozie.launcher.".length());
                String value = entry.getValue();
                // setting original KEY
                launcherConf.set(entry.getKey(), value);
                // setting un-prefixed key (to allow Hadoop job config
                // for the launcher job
                launcherConf.set(name, value);
            
        
    

所有的hadoop配置，都需要增加 oozie.launcher. 前缀（大量的组件都是这么搞得），即需要传递参数

oozie.launcher.mapreduce.map.memory.mb=1024

问题解决