kafka之partition分区及副本replica升级

Posted 2020-09-27 zhzhang

修改kafka的partition分区

bin/kafka-topics.sh --zookeeper datacollect-2:2181 --alter --partitions 3 --topic client-agent-1 

修改kafka副本数

官网解释如下：

Increasing replication factor

Increasing the replication factor of an existing partition is easy. Just specify the extra replicas in the custom reassignment json file and use it with the --execute option to increase the replication factor of the specified partitions.
For instance, the following example increases the replication factor of partition 0 of topic foo from 1 to 3. Before increasing the replication factor, the partition‘s only replica existed on broker 5. As part of increasing the replication factor, we will add more replicas on brokers 6 and 7.

The first step is to hand craft the custom reassignment plan in a json file:


> cat increase-replication-factor.json
{"version":1,
"partitions":[{"topic":"foo","partition":0,"replicas":[5,6,7]}]}

 

准备json文件：

此文件结构没错，partition不够，需要根据实际情况完善此json文件。
{
    "partitions":
        [
        {
            "topic": "speech-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "speech-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "speech-1",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-1",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-1",
            "partition": 2,
            "replicas": [0,2,3]
        }
        ],
    "version":1
}

 

开始执行副本升级：

bin/kafka-reassign-partitions.sh --zookeeper datacollect-2:2181 --reassignment-json-file json.json --execute

 

验证是否完成：

　　

bin/kafka-reassign-partitions.sh --zookeeper datacollect-2:2181 --reassignment-json-file json.json --verify

 

参考：

　　kafka修改分区和副本数 http://blog.csdn.net/haifeng000haifeng/article/details/50914197

 　  kafka运维--增加topic备份因子 http://blog.csdn.net/yanshu2012/article/details/53761284

　　kafka迁移与扩容 http://liyonghui160com.iteye.com/blog/2193430

　　Kafka 消息不能接收的问题 http://blog.csdn.net/lg772ef/article/details/65633359