kafka集群启动和停止
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了kafka集群启动和停止相关的知识，希望对你有一定的参考价值。
参考技术A 1、启动先启动zk集群
然后在所有节点执行脚本
nohup kafka-server-start.sh /kkb/install/kafka/config/server.properties >/dev/null 2>&1 &
一键启动kafka
start_kafka.sh
for host in node01 node02 node03
do
ssh host kafka is running"
done
2、停止
所有节点执行关闭kafka脚本
kafka-server-stop.sh
一键停止kafka
stop_kafka.sh
for host in node01 node02 node03
do
ssh host kafka is stopping"
done
3、一键启动和停止脚本
kafkaCluster.sh
case host "source /etc/profile; nohup /kkb/install/kafka/bin/kafka-server-start.sh /kkb/install/kafka/config/
server.properties > /dev/null 2>&1 &"
echo "$host kafka is running..."
done
;;
"stop")
for host in node01 node02 node03
do
ssh host kafka is stopping..."
done
;;
esac
启动
sh kafkaCluster.sh start
停止
sh kafkaCluster.sh stop
脚本一键启动zk，kafka集群
一 zk的一键启动脚本
注意： 停止 Kafka 集群时，一定要等 Kafka 所有节点进程全部停止后再停止 Zookeeper 集群。因为 Zookeeper 集群当中记录着 Kafka 集群相关信息， Zookeeper 集群一旦先停止， Kafka 集群就没有办法再获取停止进程的信息，只能手动杀死 Kafka 进程了。1.1 脚本内容
#!/bin/bash
case $1 in
"start")
for i in 192.168.152.136 192.168.152.138 192.168.152.140
do
echo ------------- zookeeper $i 启动 ------------
ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
art"
done
;;
"stop")
for i in 192.168.152.136 192.168.152.138 192.168.152.140
do
echo ------------- zookeeper $i 停止 ------------
ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
op"
done
;;
"status")
for i in 192.168.152.136 192.168.152.138 192.168.152.140
do
echo ------------- zookeeper $i 状态 ------------
ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
atus"
done
;;
esac
1.2 赋予执行权限
[root@localhost servers]# ls -l
total 8
drwxr-xr-x. 7 root root 118 Feb 13 22:25 kafka_2.12-2.1.0
-rw-r--r--. 1 root root 659 Mar 1 22:18 zk.sh
drwxr-xr-x. 12 root root 4096 Feb 6 17:43 zookeeper-3.4.10
[root@localhost servers]# chmod +x zk.sh
[root@localhost servers]# ls -l
total 8
drwxr-xr-x. 7 root root 118 Feb 13 22:25 kafka_2.12-2.1.0
-rwxr-xr-x. 1 root root 659 Mar 1 22:18 zk.sh
drwxr-xr-x. 12 root root 4096 Feb 6 17:43 zookeeper-3.4.10
1.3 启动脚本
[root@localhost servers]# zk.sh start
1.4 启动效果
[root@localhost servers]# sh zk.sh start
------------- zookeeper 192.168.152.136 启动 ------------
root@192.168.152.136's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
------------- zookeeper 192.168.152.138 启动 ------------
root@192.168.152.138's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
------------- zookeeper 192.168.152.140 启动 ------------
root@192.168.152.140's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost servers]# jps
3347 QuorumPeerMain
3381 Jps
在其他节点138查看：
[root@localhost bin]# jps
3347 QuorumPeerMain
3671 Jps
在其他节点140查看：
[root@localhost bin]# jps
3347 QuorumPeerMain
3671 Jps
注意：因为没有配置节点之间ssh的免登陆，所以在启动一个个节点时候，还需要手动输入密码。
1.5 关闭zk
1.在140上执行关闭脚本
[root@localhost servers]# ./zk.sh stop
------------- zookeeper 192.168.152.136 停止 ------------
root@192.168.152.136's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
------------- zookeeper 192.168.152.138 停止 ------------
root@192.168.152.138's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
------------- zookeeper 192.168.152.140 停止 ------------
root@192.168.152.140's password:
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@localhost servers]# jps
5847 Jps
[root@localhost servers]#
2.在138上查看zk进程
[root@localhost servers]# jps
5847 Jps
[root@localhost servers]#
3.在140上查看zk进程
[root@localhost servers]# jps
5847 Jps
[root@localhost servers]#
二 kafka的一键启动脚本
2.1 脚本内容
创建kf.sh脚本： touch kf.sh
#!/bin/bash
case $1 in
"start")
for i in 192.168.152.136 192.168.152.138 192.168.152.140
do
echo " --------启动 $i Kafka-------"
ssh $i "nohup /root/export/servers/kafka_2.12-2.1.0/bin/kafka-server-start.sh
/root/export/servers/kafka_2.12-2.1.0/config/server.properties 2>&1 &"
done
;;
"stop")
for i in 192.168.152.136 192.168.152.138 192.168.152.140
do
echo " --------停止 $i Kafka-------"
ssh $i "/root/export/servers/kafka_2.12-2.1.0/bin/kafka-server-stop.sh "
done
;;
esac
2.2 赋予执行权限
[root@localhost servers]# ls -l
total 12
drwxr-xr-x. 7 root root 118 Feb 13 22:25 kafka_2.12-2.1.0
-rw-r--r--. 1 root root 461 Mar 1 22:38 kf.sh
-rwxr-xr-x. 1 root root 659 Mar 1 22:29 zk.sh
drwxr-xr-x. 12 root root 4096 Feb 6 17:43 zookeeper-3.4.10
[root@localhost servers]# chmod +x kf.sh
[root@localhost servers]# ls -l
total 12
drwxr-xr-x. 7 root root 118 Feb 13 22:25 kafka_2.12-2.1.0
-rwxr-xr-x. 1 root root 461 Mar 1 22:38 kf.sh
-rwxr-xr-x. 1 root root 659 Mar 1 22:29 zk.sh
drwxr-xr-x. 12 root root 4096 Feb 6 17:43 zookeeper-3.4.10
2.3 启动脚本
[root@localhost servers]# sh kf.sh start
--------启动 192.168.152.136 Kafka-------
root@192.168.152.136's password:
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
[2022-03-01 22:39:00,424] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2022-03-01 22:39:00,540] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.nio.file.NoSuchFileException: config/server.properties
。。。。。。。
注意：因为没有配置节点之间ssh的免登陆，所以在启动一个个节点时候，还需要手动输入密码。
2.4 查看启动效果
1.在136结果：
[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps
2.在138上查看
[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps
3.在140上查看
[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps
2.5 查看主题
1.136上查看
[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]#
2.在138上查看
[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]#
3.在140上查看
[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]#
2.6 关闭kafka
1.在140上执行脚本，关闭kafka
[root@localhost servers]# ./kf.sh stop
--------停止 192.168.152.136 Kafka-------
root@192.168.152.136's password:
--------停止 192.168.152.138 Kafka-------
root@192.168.152.138's password:
--------停止 192.168.152.140 Kafka-------
root@192.168.152.140's password:
查看进程号：
[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]#
注意执行脚本后，需要稍等一下，再查看
2.在136上关闭kafka
[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]#
3.在140上关闭kafka
[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]#
以上是关于kafka集群启动和停止的主要内容，如果未能解决你的问题，请参考以下文章