Kafka动态增加Topic的副本

Posted 2021-10-12 雅冰石

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Kafka动态增加Topic的副本相关的知识，希望对你有一定的参考价值。

一需求描述

一个分区可以有多个副本，这些副本保存在不同的broker上，可实现kafka高可用。每个分区的副本中都会有一个作为Leader。当一个broker失败时，Leader在这台broker上的分区都会变得不可用，kafka会自动移除Leader，再其他副本中选一个作为新的Leader。

假如我们目前kafka集群环境中的topic没有副本，可通过如下方法增加topic的副本

二操作步骤

2.1 查看某个topic当前的分区及副本信息

[root@Centos7-Mode-V8 kafka]# bin/kafka-topics.sh --describe --zookeeper 192.168.144.247:3292,192.168.144.251:3292,192.168.144.253:3292 --topic mubai

Topic:mubai PartitionCount:3 ReplicationFactor:1 Configs:

Topic: mubai Partition: 0 Leader: 0 Replicas: 0 Isr: 0

Topic: mubai Partition: 1 Leader: 1 Replicas: 1 Isr: 1

Topic: mubai Partition: 2 Leader: 0 Replicas: 0 Isr: 0

2.2 制定分区及副本分配策略

注意修改下面文件中的topic为自己的topic，注意修改里面的partition，replicas，值需要与上面2.1步骤中查询出来的自己topic的现有值保持一致

vi /opt/test.json

添加：

{

    "version": 1,

    "partitions": [

        {

            "topic": "mubai",

            "partition": 0,

            "replicas": [

                0,

                1,

                2

            ]

        },

        {

            "topic": "mubai",

            "partition": 1,

            "replicas": [

                1,

                2,

                0

            ]

        },

        {

            "topic": "mubai",

            "partition": 2,

            "replicas": [

                2,

                0,

                1

            ]

        }

    ]

}

如果为只有一个分区的topic创建三个副本，则这样配置：

{
 
    "version": 1,
 
    "partitions": [
 
        {
 
            "topic": "baidd",
 
            "partition": 0,
 
            "replicas": [
 
                0,
 
                1,
 
                2
 
            ]
 
        }
 
 
    ]
 
}

2.3 扩副本

[root@Centos7-Mode-V8 kafka]# bin/kafka-reassign-partitions.sh --zookeeper 192.168.144.247:3292,192.168.144.251:3292,192.168.144.253:3292 --reassignment-json-file /opt/test.json --execute

Current partition replica assignment

{"version":1,"partitions":[{"topic":"mubai","partition":0,"replicas":[0],"log_dirs":["any"]},{"topic":"mubai","partition":2,"replicas":[0],"log_dirs":["any"]},{"topic":"mubai","partition":1,"replicas":[1],"log_dirs":["any"]}]}

Save this to use as the --reassignment-json-file option during rollback

Successfully started reassignment of partitions.