kafka集群搭建

Posted 2020-11-15 xingxin666

前提：1、准备3台linux虚机  2、已搭建zk环境

以下是实操命令步骤

一、下载kafka
[app@kafkanode01 ~]$ mkdir kafka
[app@kafkanode01 ~]$ cd kafka
[app@kafkanode01 kafka]$ mkdir kafka-logs
[app@kafkanode01 kafka]$ wget http://mirrors.hust.edu.cn/apache/kafka/2.0.0/kafka_2.11-2.0.0.tgz

二、解压并配置kafka
[app@kafkanode01 kafka]$ tar -xzvf kafka_2.11-2.0.0.tgz
[app@kafkanode01 kafka]$ cd kafka_2.11-2.0.0
[app@kafkanode01 kafka_2.11-2.0.0]$ cd config

[app@kafkanode01 config]$ vi server.properties

node1修改为

broker.id=1
listeners=PLAINTEXT://10.10.30.238:9092
log.dirs=/home/app/kafka/kafka-logs
zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

node2修改为

broker.id=2
listeners=PLAINTEXT://10.10.30.239:9092
log.dirs=/home/app/kafka/kafka-logs
zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

node3修改为

broker.id=3
listeners=PLAINTEXT://10.10.30.240:9092
log.dirs=/home/app/kafka/kafka-logs
zookeeper.connect=10.10.30.238:2181,10.10.30.239:2181,10.10.30.240:2181

三、启动kafka
cd /home/app/kafka/kafka_2.11-2.0.0/bin
./kafka-server-start.sh -daemon ../config/server.properties
tail -100f /home/app/kafka/kafka_2.11-2.0.0/logs/server.log

四、创建topic
cd /home/app/kafka/kafka_2.11-2.0.0/bin

[app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --list
[app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 2 --partitions 3 --topic my-topic
Created topic "my-topic".
[app@kafkanode01 bin]$ ./kafka-topics.sh --zookeeper localhost:2181 --list
my-topic

五、测试生产和消费消息

[app@kafkanode03 bin]$ ./kafka-console-producer.sh --broker-list 10.10.30.240:9092 --topic my-topic
>rr
>
[app@kafkanode01 bin]$ ./kafka-console-consumer.sh --bootstrap-server 10.10.30.239:9092 --topic my-topic --from-beginning

kafka集群搭建完毕。