Apache Sqoop 通信链路故障
Posted
技术标签:
【中文标题】Apache Sqoop 通信链路故障【英文标题】：Apache Sqoop communication link failure 【发布时间】：2012-10-12 05:18:01 【问题描述】：当我在一个集群 (aaaaaa1) 上连接到 mysql 并在另一台服务器 (aaaaaa2) 上连接到 sqooop 时，我收到以下错误，尽管在 mySql 中创建了一个表并授予了另一个集群的完全权限。
请帮助我确定导致此问题的原因。谢谢！
**$sqoop import --connect jdbc:mysql://(address of mysql):11000/sqoopdemo --table widgets**
12/10/12 10:42:23 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/10/12 10:42:23 INFO tool.CodeGenTool: Beginning code generation
12/10/12 10:42:23 ERROR manager.SqlManager: Error executing statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2332)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2369)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:207)
at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:684)
at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:53)
at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:473)
at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:496)
at com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:195)
at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:179)
at com.cloudera.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:116)
at com.cloudera.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1226)
at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1051)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:370)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:456)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:218)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:294)
... 32 more
12/10/12 10:42:23 ERROR manager.CatalogQueryManager: Failed to list columns
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2332)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2369)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:207)
at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:684)
at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:53)
at com.cloudera.sqoop.manager.CatalogQueryManager.getColumnNames(CatalogQueryManager.java:147)
at com.cloudera.sqoop.orm.ClassWriter.getColumnNames(ClassWriter.java:1198)
at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1053)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:370)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:456)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:218)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:294)
... 28 more
12/10/12 10:42:23 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.cloudera.sqoop.manager.CatalogQueryManager.getColumnNames(CatalogQueryManager.java:163)
at com.cloudera.sqoop.orm.ClassWriter.getColumnNames(ClassWriter.java:1198)
at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1053)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:370)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:456)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2332)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2369)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:207)
at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:684)
at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:53)
at com.cloudera.sqoop.manager.CatalogQueryManager.getColumnNames(CatalogQueryManager.java:147)
... 11 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:218)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:294)
【问题讨论】：
【参考方案1】：您似乎没有使用参数 --username 和 --password 来指定可疑的 MySQL 用户名和密码。我建议明确设置凭据。此外 MySQL 默认运行在端口 3306 而不是 11000 上，因此请确保端口正确。
之后，请尝试使用 mysql 命令行实用程序从运行 Sqoop 的同一台机器连接到您的 MySQL 机器：
mysql -h (mysql的地址) -p 11000 -u (用户名) -p(密码) sqoopdemo
如果您可以共享此命令的输出以防您无法连接到 MySQL 机器，那就太好了。
【讨论】：
【参考方案2】：在服务器 (aaaaaa1) 上的 my.cnf 中查找绑定地址设置，并确保它正在侦听您的目标 IP 地址（或主机名）（aaaaaa1 的 IP）。您可能还需要确保跳过网络未打开！ 或者（并且不太安全！）以下将设置它以侦听所有地址 - 本地和远程： 绑定地址 = 0.0.0.0
这意味着将 my.cnf 文件中的 bind-address = 127.0.0.1 替换为 bind-address = 0.0.0.0。 my.cnf 文件位于 /etc/mysql 文件夹中。请重启你的mysql服务器 在您尝试使用 IP 地址或机器名称之前。 只是为了测试尝试在aaaaaa1的命令行上运行它：mysql -h -u -p
如果以上运行正常，则在 aaaaaa2 上运行 sqoop 命令。 确保您已创建用户 () 并已授予该用户所有权限。
这对我有用。希望，它也适合你！ :)
参考：http://pavanchalla.blogspot.in/2013/10/mapreduce.html
【讨论】：
【参考方案3】：请运行以下 sqoop 命令
sqoop import --connect jdbc:mysql://(mysql的地址):3306/sqoopdemo --table widgets
【讨论】：
【参考方案4】：我的问题通过授予 IP 地址适当的权限得到解决： 将 . 上的所有权限授予 root@192.168.42.1 IDENTIFIED BY 'root' WITH GRANT OPTION;
有不同的IPV4 ip地址，请确保您选择正确的ip地址
【讨论】：
【参考方案5】：我有解决方案是下载最新的 Mysql 连接器。
试试这个，你会成功的 - https://***.com/a/55365072/8642144
下载后只需将 jar 文件复制到您的 sqoop/lib/ 目录中并尝试此命令
sqoop import --connect jdbc:mysql://localhost:3306/sqooptest --username newuser --password password --table sqoopcheck
localhost - 是因为我在本地运行。
3306 - mysql 默认端口号
sqooptest - 这是我的数据库名称（确保创建数据库 通过任何用户或 root 手动操作）。
newuser - 这是我的 mysql 用户名（你也可以创建新用户 并确保赋予他权利）。
密码 - 这是我的新用户密码。
sqoopcheck - 这是我数据库中的表 (sqooptest)。
【讨论】：
以上是关于Apache Sqoop 通信链路故障的主要内容，如果未能解决你的问题，请参考以下文章
DatabaseException CommunicationsException：通信链路故障
使用 JDBC 和 MySQL 解决“通信链路故障”[重复]