phoenix SQLNestedException: Cannot create PoolableConnectionFactory

Posted 2021-02-01 Nucky_yang

java通过phoenix的jdbc链接hbase数据库遇到如下情况：

查看日志发现phoenix维护的表system.function 的文件缺失了（在hdfs上），就是有节点掉了。

用命令  

$HBASE_PATH/bin/hbase hbck

查看确实有blk丢失，就是应为node掉了。

hdfs.DFSClient: Could not obtain BP-886382026-192.168.1.20-1515141616941:blk_1074071596_330784 from any node: 
java.io.IOException: No live nodes contain current block No live nodes contain current block Block locations: Dead nodes: . Will get new block locations from namenode and retry...

 

在用 hdfs dfsadmin -report 

查看是哪个节点掉了，重新启动掉线的节点以及守护进程，之后在运行代码，完美通过！

 

以下是报错的堆栈信息：

 

19/01/08 11:13:49 INFO zookeeper.ClientCnxn: EventThread shut down
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (SYSTEM.FUNCTION)
    at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
    at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
    at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
    at com.rs.java.db.CreatConnect.getPhoeConnect(CreatConnect.java:45)
    at com.rs.java.mapreduce.oneTuoN.largebandwidth.TheTest.main(TheTest.java:25)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: org.apache.phoenix.exception.PhoenixIOException: SYSTEM.FUNCTION
    at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:116)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1153)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1434)
    at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2624)
    at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:1040)
    at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:212)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:393)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:376)
    at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:374)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:363)
    at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1707)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.createOtherSystemTables(ConnectionQueryServicesImpl.java:2555)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.access$900(ConnectionQueryServicesImpl.java:264)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2466)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2389)
    at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2389)
    at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
    at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
    at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
    at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
    at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
    at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
    at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
    ... 10 more
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableExistsException): SYSTEM.FUNCTION
    at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.prepareCreate(CreateTableProcedure.java:286)
    at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:107)
    at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:59)
    at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139)
    at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499)
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1148)
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:943)
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:896)
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:78)
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:498)
Exception in thread "main" java.lang.NullPointerException
    at com.rs.java.mapreduce.oneTuoN.largebandwidth.TheTest.main(TheTest.java:27)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:153)