使用JDBC 连接 hive 报错 Invalid status 72

Posted 2023-05-12

参考技术A 使用JDBC 连接 hive 报错 Invalid status 72

出现该问题是由于 JDBC url 配置问题，核对 hive 下列属性配置：

更改为

无法使用 Zookeeper 连接字符串通过 JDBC 连接到 Hive

【中文标题】无法使用 Zookeeper 连接字符串通过 JDBC 连接到 Hive

【英文标题】：Can't connect to Hive via JDBC using Zookeeper connection string

【发布时间】：2018-05-23 16:21:18

【问题描述】：

val driverClassName = "org.apache.hive.jdbc.HiveDriver";
Class.forName(driverClassName);
// The below connection string works
// val jdbcUrl = String.format("jdbc:hive2://sandbox-hdp.hortonworks.com:10000/%s", database);
// The next one doesn't...
val jdbcUrl = String.format("jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2", database);
val jdbcUsername = "amy_ds";
val jdbcPassword = "amy_ds";
return DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword);

我正在使用上述代码建立与 Hortonworks Data Platform Sandbox 2.6.4 打包的 Hive 实例的 JDBC 连接。如果我使用第一个（已注释掉的）连接字符串进行连接，则该连接已建立并可稍后在该方法中使用。但是，如果我尝试通过 Zookeeper 进行连接，则会引发异常：

2018-05-23 10:14:08 INFO  Utils:310 - Supplied authorities: sandbox-hdp.hortonworks.com:2181
2018-05-23 10:14:08 INFO  CuratorFrameworkImpl:230 - Starting
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:host.name=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.version=1.8.0_172
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.vendor=Oracle Corporation
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.home=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.class.path=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.io.tmpdir=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:java.compiler=<NA>
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:os.name=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:os.arch=amd64
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:os.version=10.0
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:user.name=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:user.home=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:100 - Client environment:user.dir=[OMITTED FOR SECURITY]
2018-05-23 10:14:13 INFO  ZooKeeper:438 - Initiating client connection, connectString=sandbox-hdp.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@17935b13
2018-05-23 10:14:13 INFO  ClientCnxn:975 - Opening socket connection to server sandbox-hdp.hortonworks.com/[OMITTED FOR SECURITY]:2181. Will not attempt to authenticate using SASL (unknown error)
2018-05-23 10:14:13 INFO  ClientCnxn:852 - Socket connection established to sandbox-hdp.hortonworks.com/[OMITTED FOR SECURITY]:2181, initiating session
2018-05-23 10:14:13 INFO  ClientCnxn:1235 - Session establishment complete on server sandbox-hdp.hortonworks.com/[OMITTED FOR SECURITY]:2181, sessionid = 0x163891b08d300ed, negotiated timeout = 60000
2018-05-23 10:14:13 INFO  ConnectionStateManager:228 - State change: CONNECTED
2018-05-23 10:14:14 INFO  ZooKeeperHiveClientHelper:83 - Selected HiveServer2 instance with uri: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox-hdp.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
2018-05-23 10:14:14 INFO  ZooKeeper:684 - Session: 0x163891b08d300ed closed
2018-05-23 10:14:14 INFO  ClientCnxn:512 - EventThread shut down
2018-05-23 10:14:14 INFO  Utils:397 - Resolved authority: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox-hdp.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
2018-05-23 10:14:14 INFO  HiveConnection:203 - Will try to open client transport with JDBC Uri: jdbc:hive2://hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox-hdp.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
May 23, 2018 10:14:14 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause
java.lang.NullPointerException
    at org.apache.thrift.transport.TSocket.open(TSocket.java:170)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)
    at mil.navy.navair.ecbm.widgets.example.TestController.getHiveJdbcConnection(TestController.java:59)
    at mil.navy.navair.ecbm.widgets.example.TestController.allRecords(TestController.java:30)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

有什么想法吗？我看到的大多数文献都说要么更改 JDBC 驱动程序版本，要么尝试不同的连接字符串格式。我尝试了 1.2.1、2.3.0 和 3.0.0，但都没有工作（除非我使用已知的工作连接字符串直接连接到 Hive）。

我怀疑 JDBC 驱动程序在解释 Zookeeper 返回的 URL 时遇到问题，因为它说如果我尝试使用 Zookeeper 从日志中返回的 URL，它就找不到主机。

【问题讨论】：

同样的问题，如果你有答案，可以在这里发布答案吗？

【参考方案1】：

我遇到了同样的问题，正在努力解决它。发现 jdbc url 无法处理分号，尝试在端点和端口之后添加引号。

jdbc:hive2://sandbox-hdp.hortonworks.com:2181/';serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2'

这解决了我的问题。