kafka伪分布式安装

Posted 2022-02-05 Mr.zhou_Zxy

kafka伪分布式安装

启动Kafka前需先启动zookeeper，如果已装zookeeper，请忽略

1.预备工作

1.一台服务器
2.伪分布式kafka 1,2,3
3.创建Kafka目录结构
	kafkalog_1/2/3 用于存放不同节点的日志
4.将kafka目录解压到此处并修改为kafka_2.11
5.配置环境变量：vim /etc/profile
	添加 export KAFKA_HOME=/zxy/apps/kafka/kafka_2.11
		export PATH=$PATH:$ZK_HOME/bin:$KAFKA_HOME/bin
	刷新资源 source /etc/profile

2.修改配置文件

[root@hadoop_zxy config]#
cp server.properties ./server_1.properties 
cp server.properties ./server_2.properties
cp server.properties ./server_3.properties

## 主要修改或添加一下内容
broker.id=1
port=9092
listeners=PLAINTEXT://hadoop_zxy:9092
log.dirs=/zxy/apps/kafka/kafkalog_1
zookeeper.connect=hadoop_zxy:2181,hadoop_zxy:2182,hadoop_zxy:2183

3.启动Kafka

[root@hadoop_zxy kafka_2.11]#
bin/kafka-server-start.sh -daemon config/server_1.properties
bin/kafka-server-start.sh -daemon config/server_2.properties
bin/kafka-server-start.sh -daemon config/server_3.properties

4.查看进程

[root@hadoop_zxy kafka_2.11]# jps
5104 Kafka
5936 ZooKeeperMain
29858 QuorumPeerMain
29715 QuorumPeerMain
12069 Jps
4678 Kafka
29977 QuorumPeerMain
5564 Kafka
[root@hadoop_zxy kafka_2.11]#

5.客户端查看

[root@hadoop_zxy bin]# sh zookeeper-shell.sh hadoop_zxy:2181
Connecting to hadoop_zxy:2181
Welcome to ZooKeeper!
JLine support is disabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
ls /
[cluster, controller, controller_epoch, brokers, zookeeper, admin, isr_change_notification, consumers, log_dir_event_notification, latest_producer_id_block, config]
ls /brokers
[ids, topics, seqid]
ls /brokers/ids
[1, 2, 3]

6.创建主题

[root@hadoop_zxy kafka_2.11]# bin/kafka-topics.sh --create --zookeeper hadoop_zxy:2181 --replication-factor 2 --partitions 3 --topic test
Created topic "test".
[root@hadoop_zxy kafka_2.11]#

7.生产者和消费者交互(到此已成功)

生产者

[root@hadoop_zxy kafka_2.11]# bin/kafka-console-producer.sh --broker-list hadoop_zxy:9092 --topic test
>zxy
>

消费者

[root@hadoop_zxy kafka_2.11]# sh $KAFKA_HOME/bin/kafka-console-consumer.sh --topic test --bootstrap-server hadoop_zxy:9092
zxy

8.脚本管理(拓展)

开启Kafka

[root@hadoop_zxy scripts]# sh start-kafka.sh k1
[root@hadoop_zxy scripts]# sh start-kafka.sh k2
[root@hadoop_zxy scripts]# sh start-kafka.sh k3
[root@hadoop_zxy scripts]# jps
20545 Kafka
29858 QuorumPeerMain
29715 QuorumPeerMain
20197 Kafka
29977 QuorumPeerMain
20574 Jps
19806 Kafka
[root@hadoop_zxy scripts]#

开启producer

[root@hadoop_zxy scripts]# sh start-producter.sh test
>zxy

开启consumer

[root@hadoop_zxy scripts]# sh start-consumer.sh test
zxy

仅server配置文件和脚本管理