Zookeeper日志清理

Posted 2023-04-01

参考技术A 周末的时候在家收到了工作的告警通知，远程登录后发现是自运维的CK集群硬盘空间不足，通过命令查看各个目录的大小，确定是Zookeeper的日志及snapshot占用了绝大部分空间。
Zookeeper主要存放了两类文件，snapshot和log，前者是内存数据的快照，后者是记录修改数据相关的操作记录，类似于mysql的binlog。
正常的运行过程中，zk会不断的将快照数据和日志记录输出到这两个目录，并且zk不会自动清理这些文件，所以需要我们手动清理。清理的脚本如下：

从3.4.0开始，zookeeper提供了自动清理snapshotlog的功能，在zoo.cfg文件中配置如下参数：

zookeeper清理日志

一、描述

    大早上刚到公司就发现报警，磁盘容量告警，zookeeper集群服务器报警。

二、处理过程

1. 查看磁盘空间

df -h

2.确认那个目录占用空间比较多，发现是zookeeper

修改zookeeper参数文件（版本3.4.6）

vi zoo.cfg

# The number of snapshots to retain in dataDir

autopurge.snapRetainCount=20    #保留log和snapshot各20个

# Purge task interval in hours

# Set to "0" to disable auto purge feature

autopurge.purgeInterval=1    #每小时清理一次

3.查看状态

$ /usr/local/RoilandGroup/zookeeper/bin/zkServer.sh status

JMX enabled by default

Using config: /usr/local/RoilandGroup/zookeeper/bin/../conf/zoo.cfg

Mode: leader

4.关闭服务

$ /usr/local/RoilandGroup/zookeeper/bin/zkServer.sh stop

JMX enabled by default

Using config: /usr/local/RoilandGroup/zookeeper/bin/../conf/zoo.cfg

Stopping zookeeper ... STOPPED

5.重启服务

$ /usr/local/RoilandGroup/zookeeper/bin/zkServer.sh start

JMX enabled by default

Using config: /usr/local/RoilandGroup/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

[[email protected] conf]$ jps

8118449 QuorumPeerMain

6.zookeeper集群中每一个节点都需要修改参数文件，然后重启服务.

至于为什么这么修改，是否还有其他的方式，大家可以看官方文档，或者比较懒就百度。

7.理解每一个参数文件，看注释就行

less zoo.cfg

# The number of milliseconds of each tick

tickTime=2000

# The number of ticks that the initial

# synchronization phase can take

initLimit=10

# The number of ticks that can pass between

# sending a request and getting an acknowledgement

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just

# example sakes.

dataDir=/data/zookeeper

# the port at which the clients will connect

clientPort=2181

# the maximum number of client connections.

# increase this if you need to handle more clients

#maxClientCnxns=60

#

# Be sure to read the maintenance section of the

# administrator guide before turning on autopurge.

#

# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain in dataDir

autopurge.snapRetainCount=20

# Purge task interval in hours

# Set to "0" to disable auto purge feature

autopurge.purgeInterval=1

#####################################################

#####################################################

#####################################################

server.1=10.10.10.11:2888:3888

server.2=10.10.10.12:2888:3888

server.3=10.10.10.13:2888:3888

server.4=10.10.10.14:2888:3888

server.5=10.10.10.15:2888:3888

本文出自 “roidba” 博客，请务必保留此出处http://roidba.blog.51cto.com/12318731/1923375