无法从主机连接到 ZooKeeper/Hive 到 Sandbox Hortonworks HDP VM
Posted
技术标签:
【中文标题】无法从主机连接到 ZooKeeper/Hive 到 Sandbox Hortonworks HDP VM【英文标题】：Can not connect to ZooKeeper/Hive from host to Sandbox Hortonworks HDP VM 【发布时间】：2018-01-16 16:33:38 【问题描述】：我前段时间下载了 HDP-Sandbox（在 Oracle VirtualBox VM 中），从未经常使用它，现在我正在尝试使用 Hive HDBC 从外部世界访问数据。
我使用来自 apache 的 hive-jdbc 1.2.2，我从 mvnrepository 获得，所有依赖项都在类路径中，或者 hortonworks JDBC 从他们的存储库获得
http://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/1.2.2
或
http://repo.hortonworks.com/content/repositories/releases/org/apache/hive/hive-jdbc/1.2.1000.2.6.4.0-91
我写了这个非常基本的程序
import java.sql.Connection;
import java.sql.DriverManager;
public class Main
public static void main(final String[] args)
try
final Class<?> driver = Class.forName("org.apache.hive.jdbc.HiveDriver");
DriverManager.setLoginTimeout(10);
try (final Connection connection = DriverManager.getConnection("jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"))
//
catch (final Exception e)
e.printStackTrace();
连接因空指针异常或超时而失败（请参阅下面的输出日志，我使用的每个 JDBC 驱动程序都有一个），我猜这是由配置错误引起的——即使我使用的是开箱即用的沙箱。
我的沙盒版本
Sandbox information:
Created on: 25_10_2016_08_11_26 for
Hadoop stack version: Hadoop 2.7.3.2.5.0.0-1245
Ambari Version: 2.4.0.0-1225
Ambari Hash: 59175b7aa1ddb74b85551c632e3ce42fed8f0c85
Ambari build: Release : 1225
Java version: 1.8.0_111
OS Version: CentOS release 6.8 (Final)
我的虚拟机屏幕
HDP 2.5
http://hortonworks.com
To initiate your Hortonworks Sandbox session,
please open a browser and enter this address
in the browser's address field:
http://127.0.0.1:8888/
我的主机文件：
…/…
127.0.0.1 sandbox.hortonworks.com
…/…
来自 Ambari 屏幕的 Hive 摘要
Hive Metastore Started No alerts
HiveServer2 Started No alerts
WebHCat Server Started No alerts
Hive Client 1 Hive Client Installed
HiveServer2 JDBC URL jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
动物园管理员总结
ZooKeeper Server Started No alerts
ZooKeeper Client 1 ZooKeeper Client Installed
我被卡住了，任何帮助将不胜感激！
谢谢！
程序输出：
使用 apache hive jdbc 1.2.2：
16:52:06,378 INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:52:06,500 INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:52:06,503 DEBUG curator.CuratorZookeeperClient: Starting
16:52:06,503 DEBUG curator.ConnectionState: Starting
16:52:06,503 DEBUG curator.ConnectionState: reset
16:52:06,515 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
16:52:06,515 INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:52:06,515 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:52:06,515 INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:52:06,516 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@24269709
16:52:06,521 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:52:06,631 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:52:06,633 INFO zookeeper.ClientCnxn: Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
16:52:06,635 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:52:06,645 INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0008, negotiated timeout = 40000
16:52:06,650 INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:52:06,663 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 1,12 replyHeader:: 1,1347,0 request:: '/hiveserver2,F response:: v'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,s467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340
16:52:06,669 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,1347,0 request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340
16:52:06,670 INFO hive.jdbc.ZooKeeperHiveClientHelper: Selected HiveServer2 instance with uri: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,670 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:52:06,671 DEBUG curator.CuratorZookeeperClient: Closing
16:52:06,671 DEBUG curator.ConnectionState: Closing
16:52:06,672 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0008
16:52:06,672 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0008
16:52:06,674 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 3,-11 replyHeader:: 3,1348,0 request:: null response:: null
16:52:06,674 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0008
16:52:06,674 INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0008 closed
16:52:06,674 INFO hive.jdbc.Utils: Resolved authority: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,675 INFO zookeeper.ClientCnxn: EventThread shut down
16:52:06,731 INFO hive.jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
16:52:06,733 DEBUG thrift.transport.TSaslTransport: opening transport org.apache.thrift.transport.TSaslClientTransport@49049a04
java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:170)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:206)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)
使用 Hortonworks hive jdbc 1.2.1000.2.6.4.0-91
16:49:57,414 INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:49:57,544 INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:49:57,546 DEBUG curator.CuratorZookeeperClient: Starting
16:49:57,546 DEBUG curator.ConnectionState: Starting
16:49:57,546 DEBUG curator.ConnectionState: reset
16:49:57,555 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-91--1, built on 01/04/2018 10:14 GMT
16:49:57,555 INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:49:57,555 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:49:57,555 INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:49:57,556 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@1190200a
16:49:57,560 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:49:57,668 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:49:57,673 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:17993, server: 127.0.0.1/127.0.0.1:2181
16:49:57,674 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:49:57,787 INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0006, negotiated timeout = 40000
16:49:57,807 INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:49:58,378 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 1,12 replyHeader:: 1,1343,0 request:: '/hiveserver2,F response:: v'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,s467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340
16:49:58,400 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,1343,0 request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340
16:49:58,402 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:49:58,403 DEBUG curator.CuratorZookeeperClient: Closing
16:49:58,403 DEBUG curator.ConnectionState: Closing
16:49:58,405 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0006
16:49:58,405 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0006
16:49:58,466 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 3,-11 replyHeader:: 3,1344,0 request:: null response:: null
16:49:58,466 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0006
16:49:58,466 INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0006 closed
16:49:58,467 INFO hive.jdbc.Utils: Resolved authority: sandbox.hortonworks.com:10000
16:49:58,467 INFO zookeeper.ClientCnxn: EventThread shut down
16:49:58,545 DEBUG thrift.transport.TSaslTransport: opening transport org.apache.thrift.transport.TSaslClientTransport@2794eab6
16:49:58,549 DEBUG thrift.transport.TSaslClientTransport: Sending mechanism name PLAIN and initial response of length 20
16:49:58,550 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status START and payload length 5
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status COMPLETE and payload length 20
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Start message handled
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Main negotiation loop complete
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: SASL Client receiving last message
16:50:08,552 WARN hive.jdbc.HiveConnection: Failed to connect to sandbox.hortonworks.com:10000
16:50:08,553 INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:50:08,554 DEBUG curator.CuratorZookeeperClient: Starting
16:50:08,554 DEBUG curator.ConnectionState: Starting
16:50:08,554 DEBUG curator.ConnectionState: reset
16:50:08,554 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@46cdf8bd
16:50:08,557 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:50:08,558 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:18018, server: 127.0.0.1/127.0.0.1:2181
16:50:08,558 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:50:08,592 INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0007, negotiated timeout = 40000
16:50:08,592 INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:50:08,594 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 1,12 replyHeader:: 1,1345,0 request:: '/hiveserver2,F response:: v'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,s467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340
16:50:08,594 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:50:08,594 DEBUG curator.CuratorZookeeperClient: Closing
16:50:08,594 DEBUG curator.ConnectionState: Closing
16:50:08,594 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0007
16:50:08,594 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0007
16:50:08,605 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 2,-11 replyHeader:: 2,1346,0 request:: null response:: null
16:50:08,605 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0007
16:50:08,605 INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0007 closed
16:50:08,605 INFO zookeeper.ClientCnxn: EventThread shut down
16:50:08,605 ERROR hive.jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
java.sql.SQLException: Could not open client transport for any of the Server URI's in ZooKeeper: java.net.SocketTimeoutException: Read timed out
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:228)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:166)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
at org.apache.thrift.transport.TiostreamTransport.read(TIOStreamTransport.java:129)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/java.net.SocketInputStream.socketRead0(Native Method)
at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
... 10 more
【问题讨论】：
来自 Docker，beeline -u "jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" 工作。
【参考方案1】：
好吧，我无法解决这个问题。
但后来，我下载了 2.6.4 沙盒，神奇的是，它就像一个魅力！
【讨论】：
以上是关于无法从主机连接到 ZooKeeper/Hive 到 Sandbox Hortonworks HDP VM的主要内容，如果未能解决你的问题，请参考以下文章
无法从我的 Docker 容器内部连接到主机的 localhost
出于开发目的，无法从本地主机连接到 AWS RDS Postgresql