kafka安装

Posted 2021-03-05 yinchh

kafka依赖zookeeper，需要先安装zookeeper，参考:https://www.cnblogs.com/yinchh/p/12494681.html

单机

wget https://archive.apache.org/dist/kafka/1.1.0/kafka_2.11-1.1.0.tgz
tar -zxzf kafka_2.11-1.1.0.tgz
cd kafka_2.11-1.1.0
bin/kafka-server-start.sh -daemon config/server.properties
# 查看zookeeper节点
bin/zkCli.sh
ls /
ls /brokers/ids

集群

三台服务器分别为
192.168.0.201
192.168.0.202
192.168.0.203

1.分别修改3台机器的配置文件

# 192.168.0.201
broker.id=1
listeners=PLAINTEXT://192.168.0.201:9092
zookeeper.connect=192.168.0.201:2181,192.168.0.202:2181,192.168.0.203:2181
# 192.168.0.202
broker.id=2
listeners=PLAINTEXT://192.168.0.202:9092
zookeeper.connect=192.168.0.201:2181,192.168.0.202:2181,192.168.0.203:2181
# 192.168.0.203
broker.id=3
listeners=PLAINTEXT://192.168.0.203:9092
zookeeper.connect=192.168.0.201:2181,192.168.0.202:2181,192.168.0.203:2181

2.启动服务
bin/kafka-server-start.sh -daemon config/server.properties
3.创建topic（只创建一个分区，且一主两从，主题名称为test-topic）
bin/kafka-topics.sh --create --zookeeper 192.168.0.201:2181 --replication-factor 3 --partitions 1 --topic test-topic
4.查看topic信息
bin/kafka-topics.sh --describe --zookeeper 192.168.0.201:2181 --topic test-topic
5.常用命令

# 发送消息
bin/kafka-console-producer.sh --broker-list 192.168.0.201:9092 --topic test-topic
# 消费消息（组名为testGroup）
bin/kafka-console-consumer.sh --bootstrap-server 192.168.0.201:9092  --consumer-property group.id=testGroup --consumer-property client.id=consumer-1  --topic test-topic
# 查看组名
bin/kafka-consumer-groups.sh --bootstrap-server 192.168.0.201:9092 --list --new-consumer
# 查看组的偏移量
bin/kafka-consumer-groups.sh --bootstrap-server 192.168.0.201:9092 --describe --group testGroup