kafka在broker中新增副本因子

Posted 2022-04-12 健康平安的活着

一 增加副本因子

1.1 背景描述

在生产环境当中，由于某个主题的重要等级需要提升，我们考虑增加副本。副本数的 增加需要先制定计划，然后根据计划执行。 例子： 1.创建一个分区，3个副本的topic主题： kafka_four

bin/kafka-topics.sh --create --zookeeper 192.168.152.138,192.168.152.140,192.168.153.136 --replication-factor 1 --partitions 3 --topic kafka_four

2.查看主题副本信息 [root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh --describe --zookeeper 192.168.152.140:2181 --topic kafka_four
Topic:kafka_four    PartitionCount:3    ReplicationFactor:1    Configs:
    Topic: kafka_four    Partition: 0    Leader: 0    Replicas: 0    Isr: 0
    Topic: kafka_four    Partition: 1    Leader: 1    Replicas: 1    Isr: 1
    Topic: kafka_four    Partition: 2    Leader: 2    Replicas: 2    Isr: 2
[root@localhost kafka_2.12-2.1.0]# 

1.2 手动增加副本存储增加副本因子

现在需要再增加两个副本，变为3个副本，操作如下： 1创建副本存储计划（所有副本都指定存储在 broker0 、 broker1 、 broker2 中）。操作命令如下： [root@localhost kafka_2.12-2.1.0]# vim define-create-partition.json 文件内容为：

"version":1,"partitions":["topic":"kafka_four","partition":0,"replicas":[0,1,2],"topic":"kafka_four","partition":1,"replicas":[0,1,2],"topic":"kafka_four","partition":2,"replicas":[0,1,2]]

2.执行副本存储计划。

[root@localhost kafka_2.12-2.1.0]# bin/kafka-reassign-partitions.sh --zookeeper  192.168.152.138,192.168.152.140,192.168.153.136 --reassignment-json-file define-create-partition.json --execute
Current partition replica assignment

"version":1,"partitions":["topic":"kafka_four","partition":2,"replicas":[2],"log_dirs":["any"],"topic":"kafka_four","partition":1,"replicas":[1],"log_dirs":["any"],"topic":"kafka_four","partition":0,"replicas":[0],"log_dirs":["any"]]

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.

3.验证执行计划

[root@localhost kafka_2.12-2.1.0]# bin/kafka-reassign-partitions.sh --zookeeper 192.168.152.138,192.168.152.140,192.168.153.136 --reassignment-json-file define-create-partition.json --verify
Status of partition reassignment: 
Reassignment of partition kafka_four-0 completed successfully
Reassignment of partition kafka_four-1 completed successfully
Reassignment of partition kafka_four-2 completed successfully
[root@localhost kafka_2.12-2.1.0]# 

4.查看现在的副本分区

[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh --describe --zookeeper 192.168.152.140:2181 --topic kafka_four
Topic:kafka_four    PartitionCount:3    ReplicationFactor:3    Configs:
    Topic: kafka_four    Partition: 0    Leader: 0    Replicas: 0,1,2    Isr: 0,2,1
    Topic: kafka_four    Partition: 1    Leader: 1    Replicas: 0,1,2    Isr: 1,2,0
    Topic: kafka_four    Partition: 2    Leader: 2    Replicas: 0,1,2    Isr: 2,0,1
[root@localhost kafka_2.12-2.1.0]#