Hdp安装问题杂解

Posted 菜鸟的进击

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Hdp安装问题杂解相关的知识,希望对你有一定的参考价值。

5.在安装的时候遇到的问题

5.1使用ambari-server start的时候出现ERROR: Exiting with exit code -1.

5.1.1REASON: Ambari Server java process died with exitcode 255. Check /var/log/ambari-server/ambari-server.out for more information

 

解决:

由于是重新安装,所以在使用/etc/init.d/postgresql  initdb初始化数据库的时候会出现这个错误,所以需要

先用yum –y remove postgresql*命令把postgresql卸载

然后把/var/lib/pgsql/data目录下的文件全部删除

然后再配置postgresql数据库(执行1.6章节内容)

然后再次安装(3章节内容)

 

5.1.2在日志中有如下错误:ERROR [main] AmbariServer:820 - Failed to run the Ambari Server

 

com.google.inject.ProvisionException: Guice provision errors:

 

1) Error injecting method, java.lang.NullPointerException

  at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)

  at org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)

  while locating org.apache.ambari.server.api.services.AmbariMetaInfo

    for field at org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)

  at org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)

  while locating org.apache.ambari.server.controller.AmbariServer

 

1 error

        at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)

        at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)

        at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)

Caused by: java.lang.NullPointerException

        at org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)

        at org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)

        at org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)

        at org.apache.ambari.server.stack.StackManager.(StackManager.java:119)

        at org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()

        at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)

        at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)

        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)

        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

        at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

        at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

        at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

        at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)

        at com.sun.proxy.$Proxy26.create(Unknown Source)

        at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)

5.2安装HDFS和HBASE的时候出现/usr/hdp/current/hadoop-client/conf  doesn‘t exist

5.2.1/etc/Hadoop/conf文件链接存在

是由于/etc/hadoop/conf和/usr/hdp/current/hadoop-client/conf目录互相链接,造成死循环,所以要改变一个的链接

cd /etc/hadoop

rm -rf conf

ln -s /etc/hadoop/conf.backup /etc/hadoop/conf

 

HBASE也会遇到同样的问题,解决方式同上

cd /etc/hbase

rm -rf conf

ln -s /etc/hbase/conf.backup /etc/hbase/conf

 

ZooKeeper也会遇到同样的问题,解决方式同上

cd /etc/zookeeper

rm -rf conf

ln -s /etc/zookeeper/conf.backup /etc/zookeeper/conf

 

5.2.2/etc/Hadoop/conf文件链接不存在

查看正确的配置,发现缺少两个目录文件config.backup和2.4.0.0-169,把文件夹拷贝到/etc/hadoop目录下

 

 

重新创建/etc/hadoop目录下的conf链接:

cd /etc/hadoop

rm -rf conf

ln -s /usr/hdp/current/hadoop-client/conf conf

 

问题解决

 

5.3在认证机器(Confirm Hosts)的时候出现错误Ambari agent machine hostname (localhost) does not match expected ambari server hostname

Ambari配置时在Confirm Hosts的步骤时,中间遇到一个很奇怪的问题:总是报错误:

Ambari agent machine hostname (localhost.localdomain) does not match expected ambari server hostname (xxx).

后来修改的/etc/hosts文件中

 

修改前:

127.0.0.1   localhost dsj-kj1
::1         localhost dsj-kj1

10.13.39.32     dsj-kj1

10.13.39.33     dsj-kj2

10.13.39.34     dsj-kj3

10.13.39.35     dsj-kj4

10.13.39.36     dsj-kj5

修改后:

127.0.0.1  localhost localhost.localdomain localhost4 localhost4.localdomain4
::1          localhost localhost.localdomain localhost6 localhost6.localdomain6


10.13.39.32     dsj-kj1

10.13.39.33     dsj-kj2

10.13.39.34     dsj-kj3

10.13.39.35     dsj-kj4

10.13.39.36     dsj-kj5

感觉应该是走的ipv6协议,很奇怪,不过修改后就可以了。

5.4ambary-server重装

删除使用脚本删除

注意删除后要安装两个系统组件

yum -y install ruby*

yum -y install redhat-lsb*

yum -y install snappy*

 

安装参考3

 

5.5Ambari连接mysql设置

在主节点把mysql数据库连接包拷贝在/var/lib/ambary-server/resources目录下并改名为mysql-jdbc-driver.jar

cp /usr/share/java/mysql-connector-java-5.1.17.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar

 

再在图形界面下启动hive

5.6在注册机器(Confirm Hosts)的时候出现错误Failed to start ping port listener of: [Errno 98] Address already in use

 

某个端口或者进程一直陪占用

解决方法:
发现df命令一直执行没有完成,

[[email protected] ~]# netstat -lanp|grep 8670
tcp        0      0 0.0.0.0:8670                0.0.0.0:*                   LISTEN      2587/df

[[email protected] ~]# kill -9 2587
kill后,再重启ambari-agent问题解决

[[email protected] ~]# service ambari-agent restart
Verifying Python version compatibility...
Using python  /usr/bin/python2.6
ambari-agent is not running. No PID found at /var/run/ambari-agent/ambari-agent.pid
Verifying Python version compatibility...
Using python  /usr/bin/python2.6
Checking for previously running Ambari Agent...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /var/run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log

5.7在注册机器(Confirm Hosts)的时候出现错误The following hosts have Transparent HugePages (THP) enabled。THP should be disabled to avoid potential Hadoop performance issues


解决方法:
在Linux下执行:

echo never >/sys/kernel/mm/redhat_transparent_hugepage/defrag

echo never >/sys/kernel/mm/redhat_transparent_hugepage/enabled

echo never >/sys/kernel/mm/transparent_hugepage/enabled

echo never >/sys/kernel/mm/transparent_hugepage/defrag

 

5.8启动hive的时候出现错误unicodedecodeerror ambari in position 117

 

查看/etc/sysconfig/i18n文件,发现内容如下:

LANG=”zh_CN.UTF8”

原来系统字符集设置成了中文,改成如下内容,问题解决:

LANG="en_US.UTF-8"

 

 

5.9安装Metrics的时候报如下错误,安装包找不到

1.failure: Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.

 

在ftp源服务器上执行命令:

cd /var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6

mkdir Updates-ambari-2.2.1.0

cp -r /var/www/html/ambari/Updates-ambari-2.2.1.0/ambari /var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0

 

然后重新生成repodata

cd /var/www/html/ambari

rm -rf repodata

createrepo ./

 

2.failure: HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.

 

在/etc/yum.repos.d目录下删除mnt.repo,并使用yum clean all命令来清空yum的缓存

cd /ec/yum.repos.d

rm -rf mnt.repo

yum clean all


5.11jps 报process information unavailable解决办法

4791 -- process information unavailable

 

解决办法:

 

进入tmp目录,

 

cd /tmp

 

删除该目录下

 

名称为hsperfdata_{ username}的文件夹

 

然后jps,清净了。

 

脚本:

cd /tmp

ls -l | grep hsperf | xargs rm -rf

ls -l | grep hsperf

 

5.12namenode启动报错在日志文件中ERROR namenode.NameNode (NameNode.java:main(1712)) - Failed to start namenode

日志中还有java.net.BindException: Port in use: gmaster:50070

Caused by: java.net.BindException: Address already in use

判断原因是50070上一次没有释放,端口占用

 

netstat下time_wait状态的tcp连接: 
1.这是一种处于连接完全关闭状态前的状态; 
2.通常要等上4分钟(windows server)的时间才能完全关闭; 
3.这种状态下的tcp连接占用句柄与端口等资源,服务器也要为维护这些连接状态消耗资源; 
4.解决这种time_wait的tcp连接只有让服务器能够快速回收和重用那些TIME_WAIT的资源:修改注册表[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters]添加dword值TcpTimedWaitDelay=30(30也为微软建议值;默认为2分钟)和MaxUserPort:65534(可选值5000 - 65534); 
5.具体tcpip连接参数配置还可参照这里:http://technet.microsoft.com/zh-tw/library/cc776295%28v=ws.10%29.aspx 
6.linux下: 
vi /etc/sysctl.conf 
新增如下内容: 
net.ipv4.tcp_tw_reuse = 1 
net.ipv4.tcp_tw_recycle = 1 
net.ipv4.tcp_syncookies=1 

net.ipv4.tcp_fin_timeout=30

net.ipv4.tcp_keepalive_time=1800

net.ipv4.tcp_max_syn_backlog=8192


使内核参数生效: 
[[email protected] ~]# sysctl -p 
readme: 
net.ipv4.tcp_syncookies=1 打开TIME-WAIT套接字重用功能,对于存在大量连接的Web服务器非常有效。 
net.ipv4.tcp_tw_recyle=1 
net.ipv4.tcp_tw_reuse=1 减少处于FIN-WAIT-2连接状态的时间,使系统可以处理更多的连接。 
net.ipv4.tcp_fin_timeout=30 减少TCP KeepAlive连接侦测的时间,使系统可以处理更多的连接。 
net.ipv4.tcp_keepalive_time=1800 增加TCP SYN队列长度,使系统可以处理更多的并发连接。 
net.ipv4.tcp_max_syn_backlog=8192


5.13在启动的时候报错误resource_management.core.exceptions.Fail: Execution of ‘ambari-sudo.sh  -H -E /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh

在日志中有如下内容:

2016-03-31 13:55:28,090 INFO  security.ShellBasedIdMapping (ShellBasedIdMapping.java:updateStaticMapping(322)) - Not doing static UID/GID mapping because ‘/etc/nfs.map‘ does not exist.

2016-03-31 13:55:28,096 INFO  nfs3.WriteManager (WriteManager.java:(92)) - Stream timeout is 600000ms.

2016-03-31 13:55:28,096 INFO  nfs3.WriteManager (WriteManager.java:(100)) - Maximum open streams is 256

2016-03-31 13:55:28,096 INFO  nfs3.OpenFileCtxCache (OpenFileCtxCache.java:(54)) - Maximum open streams is 256

2016-03-31 13:55:28,259 INFO  nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:(205)) - Configured HDFS superuser is

2016-03-31 13:55:28,261 INFO  nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:clearDirectory(231)) - Delete current dump directory /tmp/.hdfs-nfs

2016-03-31 13:55:28,269 WARN  fs.FileUtil (FileUtil.java:deleteImpl(187)) - Failed to delete file or dir [/tmp/.hdfs-nfs]: it still exists.

说明hdfs这个用户对/tmp没有权限

赋予权限给hdfs用户:

chown  hdfs:hadoop /tmp

 

再启动问题解决

5.14在安装ranger组件的时候出现错误连接不上mysql数据库rangeradmin用户和不能赋权的问题

在数据库中先删除所有rangeradmin用户,注意使用drop user命令:

drop user ‘rangeradmin‘@‘%‘;

drop user ‘rangeradmin‘@‘localhost‘;

drop user ‘rangeradmin‘@‘gmaster‘;

drop user ‘rangeradmin‘@‘gslave1‘;

drop user ‘rangeradmin‘@‘gslave2‘;

FLUSH PRIVILEGES;

 

再创建用户(注意gmaster是ranger安装的服务器机器名)

CREATE USER ‘rangeradmin‘@‘%‘ IDENTIFIED BY ‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO ‘rangeradmin‘@‘%‘  with grant option;

CREATE USER ‘rangeradmin‘@‘localhost‘ IDENTIFIED BY ‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO ‘rangeradmin‘@‘localhost‘  with grant option;

CREATE USER ‘rangeradmin‘@‘gmaster‘ IDENTIFIED BY ‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO ‘rangeradmin‘@‘gmaster‘  with grant option;

FLUSH PRIVILEGES;

 

再查看权限:

SELECT DISTINCT CONCAT(‘User: ‘‘‘,user,‘‘‘@‘‘‘,host,‘‘‘;‘) AS query FROM mysql.user

 

select * from mysql.user where user=‘rangeradmin‘ \G;

 

问题解决

5.15在ambari启动的时候出现错误:AmbariServer:820 - Failed to run the Ambari Server

这个问题困扰了我很久,最后通过查看源码找到了问题所在:

在/var/log/ambari-server/ambary-server.log文件中报有错误:

13 Apr 2016 14:16:01,723  INFO [main] StackDirectory:458 - Stack ‘/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS‘ doesn‘t contain an upgrade directory

13 Apr 2016 14:16:01,723  INFO [main] StackDirectory:468 - Stack ‘/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS‘ doesn‘t contain config upgrade pack file

13 Apr 2016 14:16:01,744  INFO [main] StackDirectory:484 - Role command order info was loaded from file: /var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS/role_command_order.json

13 Apr 2016 14:16:01,840  INFO [main] StackDirectory:484 - Role command order info was loaded from file: /var/lib/ambari-server/resources/stacks/HDP/2.4/role_command_order.json

13 Apr 2016 14:16:01,927 ERROR [main] AmbariServer:820 - Failed to run the Ambari Server

com.google.inject.ProvisionException: Guice provision errors:

 

1) Error injecting method, java.lang.NullPointerException

  at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)

  at org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)

  while locating org.apache.ambari.server.api.services.AmbariMetaInfo

    for field at org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)

  at org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)

  while locating org.apache.ambari.server.controller.AmbariServer

 

1 error

         at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)

         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)

         at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)

Caused by: java.lang.NullPointerException

         at org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)

         at org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)

         at org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)

         at org.apache.ambari.server.stack.StackManager.(StackManager.java:119)

         at org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()

         at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)

         at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)

         at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)

         at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

         at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

         at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

         at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)

         at com.sun.proxy.$Proxy26.create(Unknown Source)

         at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)

         at org.apache.ambari.server.api.services.AmbariMetaInfo$$FastClassByGuice$$202844bc.invoke()

         at com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)

         at com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:56)

         at com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:90)

         at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)

         at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)

         at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

         at com.google.inject.Scopes$1$1.get(Scopes.java:65)

         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)

         at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)

         at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)

         at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)

         at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

         at com.google.inject.Scopes$1$1.get(Scopes.java:65)

         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)

         at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)

         at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

         ... 2 more

 

解决方法:

后来发现在文件/var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml中的内容os这行原来的如下:

 

改成:

 

问题解决

 

5.16在启动hive的时候出现错误Error: Duplicate key name ‘PCS_STATS_IDX‘ (state=42000,code=1061)

在HIVE安装的机器cd /var/lib/ambari-agent/data目录下有日志,日志中报如下错误:

Traceback (most recent call last):

  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 245, in

    HiveMetastore().execute()

  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute

    method(env)

  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 58, in start

    self.configure(env)

  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py", line 72, in configure

    hive(name = ‘metastore‘)

  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk

    return fn(*args, **kwargs)

  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py", line 292, in hive

    user = params.hive_user

  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__

    self.env.run()

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run

    self.run_action(resource, action)

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action

    provider_action()

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run

    tries=self.resource.tries, try_sleep=self.resource.try_sleep)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner

    result = function(command, **kwargs)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call

    tries=tries, try_sleep=try_sleep)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper

    result = _call(command, **kwargs_copy)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call

    raise Fail(err_msg)

resource_management.core.exceptions.Fail: Execution of ‘export HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/conf.server ; /usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql -userName hive -passWord [PROTECTED]‘ returned 1. WARNING: Use "yarn jar" to launch YARN applications.

Metastore connection URL:        jdbc:mysql://a2slave1/hive?createDatabaseIfNotExist=true

Metastore Connection Driver :    com.mysql.jdbc.Driver

Metastore connection User:       hive

Starting metastore schema initialization to 1.2.1000

Initialization script hive-schema-1.2.1000.mysql.sql

Error: Duplicate key name ‘PCS_STATS_IDX‘ (state=42000,code=1061)

org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

*** schemaTool failed ***

 

解决方法:

HIVE安装的机器拷贝脚本hive-schema-1.2.1000.mysql.sql到:

[[email protected] /]# scp /usr/hdp/2.4.0.0-169/hive/scripts/metastore/upgrade/mysql/hive-schema-1.2.1000.mysql.sql [email protected]:/usr/local/mysql

hive-schema-1.2.1000.mysql.sql                                                                   100%   34KB  34.4KB/s   00:00

 

在HIVE安装机器使用hive用户登陆,进入hive数据库,执行这个脚本

[[email protected] conf.server]# mysql -uhive -p

Enter password:

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 505

Server version: 5.6.26-log MySQL Community Server (GPL)

 

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

 

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

 

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

 

mysql> use hive;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

 

Database changed

mysql> source /usr/local/mysql/hive-schema-1.2.1000.mysql.sql;

问题解决

 

5.17在启动hive的时候出现错误Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

在/var/log/hive中的日志文件hiveserver.log中记录有:

2016-04-15 10:45:20,446 INFO  [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(405)) - Starting HiveServer2

2016-04-15 10:45:20,573 INFO  [main]: metastore.ObjectStore (ObjectStore.java:initialize(294)) - ObjectStore, initialize called

2016-04-15 10:45:20,585 INFO  [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:(140)) - Using direct SQL, underlying DB is MYSQL

2016-04-15 10:45:20,585 INFO  [main]: metastore.ObjectStore (ObjectStore.java:setConf(277)) - Initialized ObjectStore

2016-04-15 10:45:20,590 WARN  [main]: metastore.ObjectStore (ObjectStore.java:getDatabase(577)) - Failed to get database default, returning NoSuchObjectException

2016-04-15 10:45:20,591 ERROR [main]: bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000

2016-04-15 10:45:20,600 WARN  [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultDB(623)) - Retrying creating default database after error: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

javax.jdo.JDOUserException: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

         at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)

         at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)

         at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

         at org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

         at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

         at com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

         at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

         at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

         at sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

         at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

         at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

         at org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

         at org.apache.hive.service.CompositeService.init(CompositeService.java:59)

         at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

         at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

         at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

         at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

         at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

NestedThrowablesStackTrace:

Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

org.datanucleus.exceptions.NucleusUserException: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

         at org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)

         at org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)

         at org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)

         at org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)

         at org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)

         at org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)

         at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)

         at org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)

         at org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)

         at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)

         at org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)

         at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)

         at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)

         at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)

         at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)

         at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)

         at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

         at org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

         at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

         at com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

         at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

         at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

         at sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

         at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

         at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

         at org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

         at org.apache.hive.service.CompositeService.init(CompositeService.java:59)

         at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

         at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

         at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

         at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

         at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

2016-04-15 10:45:20,607 WARN  [main]: metastore.ObjectStore (ObjectStore.java:getDatabase(577)) - Failed to get database default, returning NoSuchObjectException

2016-04-15 10:45:20,609 ERROR [main]: bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000

2016-04-15 10:45:20,617 INFO  [main]: server.HiveServer2 (HiveServer2.java:stop(371)) - Shutting down HiveServer2

2016-04-15 10:45:20,618 WARN  [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(442)) - Error starting HiveServer2 on attempt 29, will retry in 60 seconds

java.lang.RuntimeException: Error applying authorization policy on hive configuration: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

         at org.apache.hive.service.cli.CLIService.init(CLIService.java:114)

         at org.apache.hive.service.CompositeService.init(CompositeService.java:59)

         at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

         at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

         at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

         at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

         at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:494)

         at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

         at org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

         ... 12 more

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

         at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

         ... 14 more

Caused by: java.lang.reflect.InvocationTargetException

         at sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

         ... 20 more

Caused by: javax.jdo.JDOUserException: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

NestedThrowables:

org.datanucleus.exceptions.NucleusUserException: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

         at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)

         at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)

         at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

         at org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

         at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

         at com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:625)

         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

         at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

         at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

         ... 24 more

Caused by: org.datanucleus.exceptions.NucleusUserException: Could not create "increment"/"table" value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

         at org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)

         at org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)

         at org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)

         at org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)

         at org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)

         at org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)

         at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)

         at org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)

         at org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)

         at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)

         at org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)

         at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)

         at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)

         at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)

         at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)

         at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)

         ... 39 more

最后发现是mysql数据库的binlog_format参数设置不正确,

原来设置的是STATEMENT,修改为MIXED

修改方法,在/etc/my.cnf文件中加上binlog_format=MIXED

然后重启mysql数据库

问题解决

 

5.18在使用hive命令进入hive的时候报错误Permission denied: user=root, access=WRITE, inode="/user/root":hdfs:hdfs:drwxr-xr-x  hive

解决方法:

1.       使用HDFS的命令行接口修改相应目录的权限,hadoop fs -chmod 777 /user,后面的/user是要上传文件的路径,不同的情况可能不一样,比如要上传的文件路径为hdfs://namenode/user/xxx.doc,则这样的修改可以,如果要上传的文件路径为hdfs://namenode/java/xxx.doc,则要修改的为hadoop fs -chmod 777 /java或者hadoop fs -chmod 777 /,java的那个需要先在HDFS里面建立Java目录,后面的这个是为根目录调整权限。

 

脚本

su - hdfs

hadoop fs -chmod 777 /user

 

 

2.       在/etc/profile文件中加上系统的环境变量或java JVM变量里面添加export HADOOP_USER_NAME=hdfs(ambari使用的hadoop用户是hdfs),这个值具体等于多少看自己的情况,以后会运行HADOOP上的Linux的用户名。

 

export HADOOP_USER_NAME=hdfs

5.19Spark Thrift Server组件启动后自动关闭

Spark Thrift Server组件在启动后自动关闭,查看/var/log/spark下的日志文件spark-hive-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-a2master.out,中有如下内容:

16/04/18 10:26:10 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.

16/04/18 10:26:10 INFO Client: Requesting a new application from cluster with 3 NodeManagers

16/04/18 10:26:10 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (512 MB per container)

16/04/18 10:26:10 ERROR SparkContext: Error initializing SparkContext.

java.lang.IllegalArgumentException: Required executor memory (1024+384 MB) is above the max threshold (512 MB) of this cluster! Please check the values of ‘yarn.scheduler.maximum-allocation-mb‘ and/or ‘yarn.nodemanager.resource.memory-mb‘.

        at org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:283)

        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:139)

        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)

        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)

        at org.apache.spark.SparkContext.(SparkContext.scala:530)

        at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:56)

        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:76)

        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)

        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)

        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

16/04/18 10:26:11 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/kill,null}

分析:原因是spark的内存设置的太小

解决方法:在前台界面修改内存为1536M,并把配置在其它机器上更新,重启spark服务

 

 

5.20hbase在启动的时候出现错误Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461130860883":hdfs:hdfs:drwxr-xr-x

在日志中有如下内容:

2016-04-20 15:42:11,640 INFO  [regionserver/gslave2/192.168.1.253:16020] hfile.CacheConfig: Allocating LruBlockCache size=401.60 MB, blockSize=64 KB

2016-04-20 15:42:11,648 INFO  [regionserver/gslave2/192.168.1.253:16020] hfile.CacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=433080, freeSize=420675048, maxSize=421108128, heapSize=433080, minSize=400052704, minFactor=0.95, multiSize=200026352, multiFactor=0.5, singleSize=100013176, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false

2016-04-20 15:42:11,704 INFO  [regionserver/gslave2/192.168.1.253:16020] wal.WALFactory: Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider

2016-04-20 15:42:11,729 INFO  [regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: STOPPED: Failed initialization

2016-04-20 15:42:11,729 ERROR [regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: Failed init

org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

         at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

         at java.security.AccessController.doPrivileged(Native Method)

         at javax.security.auth.Subject.doAs(Subject.java:415)

         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

 

         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

         at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)

         at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)

         at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)

         at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)

         at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)

         at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)

         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

         at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)

         at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)

         at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)

         at org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)

         at org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)

         at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)

         at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)

         at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

         at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

         at java.security.AccessController.doPrivileged(Native Method)

         at javax.security.auth.Subject.doAs(Subject.java:415)

         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

 

         at org.apache.hadoop.ipc.Client.call(Client.java:1411)

         at org.apache.hadoop.ipc.Client.call(Client.java:1364)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)

         at com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

         at com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)

         at com.sun.proxy.$Proxy16.mkdirs(Unknown Source)

         at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)

         ... 15 more

2016-04-20 15:42:11,732 FATAL [regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: ABORTING region server gslave2,16020,1461138130424: Unhandled: Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

         at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

         at java.security.AccessController.doPrivileged(Native Method)

         at javax.security.auth.Subject.doAs(Subject.java:415)

         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

 

org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

         at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

         at java.security.AccessController.doPrivileged(Native Method)

         at javax.security.auth.Subject.doAs(Subject.java:415)

         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

 

         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

         at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)

         at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)

         at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)

         at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)

         at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)

         at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)

         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

         at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)

         at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)

         at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)

         at org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)

         at org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)

         at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)

         at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)

         at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)

         at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hbase, access=WRITE, inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

         at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

         at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

         at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

         at java.security.AccessController.doPrivileged(Native Method)

         at javax.security.auth.Subject.doAs(Subject.java:415)

         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

 

         at org.apache.hadoop.ipc.Client.call(Client.java:1411)

         at org.apache.hadoop.ipc.Client.call(Client.java:1364)

         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)

         at com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

         at com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:606)

         at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)

         at com.sun.proxy.$Proxy16.mkdirs(Unknown Source)

         at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)

         ... 15 more

2016-04-20 15:42:11,732 FATAL [regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []

2016-04-20 15:42:11,744 INFO  [regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: Dump of metrics as JSON on abort: {

 

解决方法:

su - hdfs

hdfs dfs -chown -R hbase:hbase /apps/hbase

 

以上是关于Hdp安装问题杂解的主要内容,如果未能解决你的问题,请参考以下文章

HDP Ambari 安装失败

HortonWorks HDP 2.6:NameNode 通过 Ambari 安装问题

从 HDF 3.4 通过 HDP 3.1 安装时如何获得适当的 Kafka 版本

ambari 2.6.2 安装 hdp 2.6.5.0 遇到的问题

HDP安装教程笔记

Powershell windows server 2012 .Path 安装 Hortonworks 数据平台 (HDP) 时出错