zookeeper+kafka

Posted 2021-10-27 sunshinea121

准备三台主机
192.168.233.167 master
192.168.233.168 slave1
192.168.233.169 slave2
下载jdk，zookeeper，kafka,并解压到opt下重命名

tar xf jdk-8u181-linux-x64.tar.gz -C /opt/
tar xf zookeeper-3.4.9.tar.gz -C /opt/
tar xf kafka_2.11-0.10.2.1.tar.gz -C /opt/
mv jdk1.8.0_181 java
mv kafka_2.11-0.10.2.1 kafka
mv zookeeper-3.4.9 zookeeper

设置java环境变量，并验证

vim /etc/profile
JAVA_HOME=/opt/java
PATH=$PATH:$JAVA_HOME/bin/
source /etc/profile
java -version

把opt下所有文件同步到slave两台机器上，并同步profile配置文件

scp -r * 192.168.233.168:/opt/
scp -r * 192.168.233.169:/opt/
scp /etc/profile 192.168.233.168:/etc
scp /etc/profile 192.168.233.169:/etc

进入/opt下zookeeper目录配置zookeeper，更改zoo.cfg配置文件

cd zookeeper/
mkdir data
cd conf/
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zookeeper/data
clientPort=2181
server.1=192.168.233.167:2888:3888
server.2=192.168.233.168:2888:3888
server.3=192.168.233.169:2888:3888
echo 1 > /opt/zookeeper/data/myid
scp -r /opt/zookeeper/ 192.168.233.168:/opt/
scp -r /opt/zookeeper/ 192.168.233.169:/opt/

更改myid得在另外两台机器得值，启动zookeeper并查看状态

/opt/zookeeper/bin/zkServer.sh start
/opt/zookeeper/bin/zkServer.sh status

配置kafka配置文件

cd /opt/kafka/config/
vim server.properties 
broker.id=1         #id每台机器不能一样
delete.topic.enable=true
listeners=PLAINTEXT://192.168.233.167:9092  #填写本机IP地址，填写主机名要可以解析
advertised.listeners=PLAINTEXT://192.168.233.167:9092
host.name=192.168.233.167
advertised.host.name=192.168.233.167
num.network.threads=3
num.io.threads=9
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/opt/kafka/logs    #日志目录
num.partitions=9
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.233.167:2181,192.168.233.168:2181,192.168.233.169:2181    #zookeeper集群地址
zookeeper.connection.timeout.ms=12000
auto.create.topics.enable=false
unclean.leader.election.enable=false

同步配置文件到其他两台机器并更改相应id，启动kafka

/opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties