Ambari hadoop 安装失败

Posted 2023-04-18

【中文标题】Ambari hadoop 安装失败

【英文标题】：Ambari hadoop installation failed

【发布时间】：2017-12-21 08:40:06

【问题描述】：

017-12-21 13:46:55,297 - 堆栈功能版本信息：集群堆栈=2.6，集群当前版本=无，命令堆栈=无，命令版本=无 -> 2.6 2017-12-21 13:46:55,317 - 使用 hadoop conf 目录：/usr/hdp/current/hadoop-client/conf hostLevelParams 中缺少用户组映射 (user_group) 2017-12-21 13:46:55,319 - 组 ['hadoop'] 2017-12-21 13:46:55,320 - 组 ['users'] 2017-12-21 13:46:55,321 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,323 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh zookeeper'] 2017-12-21 13:46:55,334 - 呼叫返回 (0, '1002') 2017-12-21 13:46:55,335 - User['zookeeper'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1002 2017-12-21 13:46:55,337 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,338 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh infra-solr'] 2017-12-21 13:46:55,349 - 呼叫返回 (0, '1013') 2017-12-21 13:46:55,350 - 用户 ['infra-solr'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1013 2017-12-21 13:46:55,351 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,351 - 呼叫['/var/lib/ambari-agent/tmp/changeUid.sh oozie'] 2017-12-21 13:46:55,360 - 呼叫返回 (0, '1003') 2017-12-21 13:46:55,360 - User['oozie'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': 1003 2017-12-21 13:46:55,363 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,364 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh ams'] 2017-12-21 13:46:55,375 - 呼叫返回 (0, '1004') 2017-12-21 13:46:55,376 - User['ams'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1004 2017-12-21 13:46:55,378 - User['ambari-qa'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None 2017-12-21 13:46:55,380 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,381 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh kafka'] 2017-12-21 13:46:55,390 - 呼叫返回 (0, '1020') 2017-12-21 13:46:55,390 - User['kafka'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1020 2017-12-21 13:46:55,391 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,392 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh tez'] 2017-12-21 13:46:55,401 - 呼叫返回 (0, '1006') 2017-12-21 13:46:55,402 - User['tez'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': 1006 2017-12-21 13:46:55,403 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,404 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh hdfs'] 2017-12-21 13:46:55,415 - 呼叫返回 (0, '1007') 2017-12-21 13:46:55,416 - User['hdfs'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1007 2017-12-21 13:46:55,418 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,418 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh sqoop'] 2017-12-21 13:46:55,426 - 呼叫返回 (0, '1008') 2017-12-21 13:46:55,427 - User['sqoop'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1008 2017-12-21 13:46:55,429 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,429 - 调用['/var/lib/ambari-agent/tmp/changeUid.sh yarn'] 2017-12-21 13:46:55,439 - 呼叫返回 (0, '1009') 2017-12-21 13:46:55,440 - User['yarn'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1009 2017-12-21 13:46:55,441 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,442 - 调用 ['/var/lib/ambari-agent/tmp/changeUid.sh mapred'] 2017-12-21 13:46:55,452 - 呼叫返回 (0, '1010') 2017-12-21 13:46:55,452 - User['mapred'] 'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1010 2017-12-21 13:46:55,453 - 文件 ['/var/lib/ambari-agent/tmp/changeUid.sh'] 'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555 2017-12-21 13:46:55,456 - 执行 ['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] 'not_if': '(test $(id -u ambari-qa) -gt 1000) || （错误的）' 2017-12-21 13:46:55,462 - 跳过执行 ['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home /ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] 由于 not_if 2017-12-21 13:46:55,462 - 组 ['hdfs'] 2017-12-21 13:46:55,463 - 用户 ['hdfs'] 'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs'] 2017-12-21 13:46:55,463 - FS 类型： 2017-12-21 13:46:55,464 - 目录 ['/etc/hadoop'] 'mode': 0755 2017-12-21 13:46:55,465 - 目录 ['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] 'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777 2017-12-21 13:46:55,483 - 初始化 2 个存储库 2017-12-21 13:46:55,484 - 存储库 ['HDP-2.6'] 'base_url': 'http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.3.0', 'action': ['create'], 'components': [u'HDP', 'main']，'repo_template'：'package_type base_url components'，'repo_file_name'：'HDP'，'mirror_list'：无 2017-12-21 13:46:55,491 - 文件 ['/tmp/tmp8SS8_V'] 'content': 'deb http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.3.0 HDP main' 2017-12-21 13:46:55,492 - 写入文件 ['/tmp/tmp8SS8_V'] 因为内容不匹配 2017-12-21 13:46:55,492 - 文件['/tmp/tmpceZ8Dh'] 'content': StaticFile('/etc/apt/sources.list.d/HDP.list') 2017-12-21 13:46:55,493 - 写入文件 ['/tmp/tmpceZ8Dh'] 因为内容不匹配 2017-12-21 13:46:55,570 - 存储库 ['HDP-UTILS-1.1.0.21'] 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu16', 'action': ['create'], 'components': [u 'HDP-UTILS'，'main']，'repo_template'：'package_type base_url components'，'repo_file_name'：'HDP-UTILS'，'mirror_list'：无 2017-12-21 13:46:55,573 - 文件 ['/tmp/tmpabpRd8'] 'content': 'deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu16 HDP-UTILS main' 2017-12-21 13:46:55,573 - 写入文件 ['/tmp/tmpabpRd8'] 因为内容不匹配 2017-12-21 13:46:55,574 - 文件['/tmp/tmpAKCJ9S'] 'content': StaticFile('/etc/apt/sources.list.d/HDP-UTILS.list') 2017-12-21 13:46:55,574 - 写入文件 ['/tmp/tmpAKCJ9S'] 因为内容不匹配 2017-12-21 13:46:55,616 - 包 ['unzip'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:55,671 - 跳过安装现有软件包解压缩 2017-12-21 13:46:55,672 - 包 ['curl'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:55,720 - 跳过现有包 curl 的安装 2017-12-21 13:46:55,721 - 包 ['hdp-select'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:55,767 - 跳过现有软件包 hdp-select 的安装 2017-12-21 13:46:56,025 - 使用 hadoop conf 目录：/usr/hdp/current/hadoop-client/conf 2017-12-21 13:46:56,028 - 堆栈功能版本信息：集群堆栈 = 2.6，集群当前版本 = 无，命令堆栈 = 无，命令版本 = 无 - > 2.6 2017-12-21 13:46:56,067 - 使用 hadoop conf 目录：/usr/hdp/current/hadoop-client/conf 2017-12-21 13:46:56,091-checked_call['dpkg -s hdp-select | grep 版本 | awk 'print $2''] 'stderr': -1 2017-12-21 13:46:56,136 - check_call 返回 (0, '2.6.1.0-129', '') 2017-12-21 13:46:56,146 - 包 ['hadoop-2-6-1-0-129-client'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,195 - 跳过现有软件包 hadoop-2-6-1-0-129-client 的安装 2017-12-21 13:46:56,196 - 包 ['hadoop-2-6-1-0-129-hdfs-datanode'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,245 - 跳过安装现有包 hadoop-2-6-1-0-129-hdfs-datanode 2017-12-21 13:46:56,246 - 包 ['hadoop-2-6-1-0-129-hdfs-journalnode'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,293 - 跳过现有软件包 hadoop-2-6-1-0-129-hdfs-journalnode 的安装 2017-12-21 13:46:56,294 - 包 ['hadoop-2-6-1-0-129-hdfs-namenode'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,348 - 跳过现有软件包 hadoop-2-6-1-0-129-hdfs-namenode 的安装 2017-12-21 13:46:56,349 - 包 ['hadoop-2-6-1-0-129-hdfs-secondarynamenode'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,396 - 跳过安装现有包 hadoop-2-6-1-0-129-hdfs-secondarynamenode 2017-12-21 13:46:56,397 - 包 ['hadoop-2-6-1-0-129-hdfs-zkfc'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,443 - 跳过现有软件包 hadoop-2-6-1-0-129-hdfs-zkfc 的安装 2017-12-21 13:46:56,444 - 包 ['libsnappy1'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:46:56,489 - 安装包 libsnappy1 ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes安装 libsnappy1') 2017-12-21 13:47:05,876 - 包 ['libsnappy-dev'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:47:05,922 - 安装软件包 libsnappy-dev ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume -yes 安装 libsnappy-dev') 2017-12-21 13:47:14,593 - 包 ['libhdfs0-2-6-1-0-129'] 'retry_on_repo_unavailability': False, 'retry_count': 5 2017-12-21 13:47:14,640 - 跳过现有软件包 libhdfs0-2-6-1-0-129 的安装 2017-12-21 13:47:14,642 - 目录 ['/etc/security/limits.d'] 'owner': 'root', 'create_parents': True, 'group': 'root' 2017-12-21 13:47:14,658 - 文件 ['/etc/security/limits.d/hdfs.conf'] 'content': Template('hdfs.conf.j2'), 'owner': 'root '，'组'：'根'，'模式'：0644 2017-12-21 13:47:14,658 - XmlConfig ['hadoop-policy.xml'] 'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/usr/hdp/current/ hadoop-client/conf', 'configuration_attributes': , 'configurations': ... 2017-12-21 13:47:14,670 - 生成配置：/usr/hdp/current/hadoop-client/conf/hadoop-policy.xml 2017-12-21 13:47:14,671 - 文件 ['/usr/hdp/current/hadoop-client/conf/hadoop-policy.xml'] 'owner': 'hdfs', 'content': InlineTemplate(. ..), 'group': 'hadoop', 'mode': 无, 'encoding': 'UTF-8'

1 次尝试后命令失败

【参考方案1】：

看起来是存储库的问题。尝试配置和使用本地仓库。

检查这个：Setting Up a Local Repository with No Internet Access。