kafka 扩展partition和replication-factor

Posted 2020-11-12 wangyh702

 问题：

1. kafka的topic 是程序自己建立，默认只建立8个partitions，1个replication-factor

目的： 扩展partitions 到9个， replicatoion-factor  3个

原因：因为kafka是三台，如果replicatoion-factor设置1的话，kafka坏掉一个会有影响

 

 (1). 扩展partitions 到9个

/kafka-topics.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --alter --topic Message --partitions 9

 

查看结果：

／kafka-topics.sh --describe  --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message 

Topic:Message  PartitionCount:9 ReplicationFactor:1 Configs:
Topic: Message  Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic:Message  Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 2 Leader: 3 Replicas: 3 Isr: 3
Topic: Message  Partition: 3 Leader: 1 Replicas: 1 Isr: 1
Topic: Message Partition: 4 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 5 Leader: 3 Replicas: 3 Isr: 3
Topic: Message Partition: 6 Leader: 1 Replicas: 1 Isr: 1
Topic: Message  Partition: 7 Leader: 2 Replicas: 2 Isr: 2

Topic: Message  Partition: 8 Leader: 3 Replicas: 3 Isr: 3

 

（2）扩展ReplicationFactor 为3

vim increace-factor.json

{"version":1,
"partitions":[
{"topic":"Message","partition":0,"replicas":[1,2,3]},
{"topic":"Message","partition":1,"replicas":[1,2,3]},
{"topic":"Message","partition":2,"replicas":[1,2,3]},
{"topic":"Message","partition":3,"replicas":[1,2,3]},
{"topic":"Message","partition":4,"replicas":[1,2,3]},
{"topic":"Message","partition":5,"replicas":[1,2,3]},
{"topic":"Message","partition":6,"replicas":[1,2,3]},
{"topic":"Message","partition":7,"replicas":[1,2,3]}

{"topic":"Message","partition":8,"replicas":[1,2,3]}
]}

执行如下命令：

/kafka-reassign-partitions.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --reassignment-json-file increace-factor.json --execute

 

[root@java]#/kafka-topics.sh --describe --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message
Topic:Message PartitionCount:9 ReplicationFactor:3 Configs:
Topic: Message Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 3 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 4 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message  Partition: 5 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: cMessage Partition: 7 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3

Topic: Message  Partition: 8 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3