ZK—zookeeper无法启动，报“Unable to load database on dis”

Posted 2020-11-03

问题描述：
部署zookeeper的服务器磁盘满了，引起服务异常，将zookeeper进程杀掉之后，重新启动zookeeper启动的时候显示成功了：

[[email protected] bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

但是zookeeper的进程和端口都没有起来。

查看zookeeper.out，发现有报错信息如下：

2018-05-07 10:33:58,556 [myid:3] - INFO  [main:[email protected]] - Reading snapshot /opt/data/zookeeper/version-2/snapshot.1900356f19
2018-05-07 10:33:58,966 [myid:3] - ERROR [main:[email protected]] - Last transaction was partial.
2018-05-07 10:33:58,968 [myid:3] - ERROR [main:[email protected]] - Unable to load database on disk
java.io.IOException: The accepted epoch, 19 is less than the current epoch, 3a
    at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:645)
    at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:591)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:164)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
2018-05-07 10:33:58,970 [myid:3] - ERROR [main:[email protected]] - Unexpected exception, exiting abnormally
java.lang.RuntimeException: Unable to run quorum server 
    at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:649)
    at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:591)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:164)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: java.io.IOException: The accepted epoch, 19 is less than the current epoch, 3a
    at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:645)
    ... 4 more
[[email protected] bin]# 

经过网上查询，得到解决方法：
将zookeeper的version-2目录备份一份，再重新创建一个新的version-2目录，然后重新启动zookeeper。

操作如下：

[[email protected] zookeeper]# ll
total 28
-rw-r--r-- 1 root root     2 Aug 30  2017 myid
drwxr-xr-x 2 root root 24576 May  7 10:36 version-2
-rw-r--r-- 1 root root     5 May  7 10:42 zookeeper_server.pid
[[email protected] zookeeper]# mv version-2 version-2.bak
[[email protected] zookeeper]# ll
total 28
-rw-r--r-- 1 root root     2 Aug 30  2017 myid
drwxr-xr-x 2 root root 24576 May  7 10:36 version-2.bak
-rw-r--r-- 1 root root     5 May  7 10:42 zookeeper_server.pid
[[email protected] zookeeper]# mkdir version-2
[[email protected] zookeeper]# ll
total 28
-rw-r--r-- 1 root root     2 Aug 30  2017 myid
drwxr-xr-x 2 root root     6 May  7 10:44 version-2
drwxr-xr-x 2 root root 24576 May  7 10:36 version-2.bak
-rw-r--r-- 1 root root     5 May  7 10:42 zookeeper_server.pid

然后重新启动zookeeper：

[[email protected] bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[[email protected] bin]# netstat -tlunp|grep 2181
tcp6       0      0 :::2181                 :::*                    LISTEN      32695/java