centos单机安装zookeeper+kafaka
Posted yy
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了centos单机安装zookeeper+kafaka相关的知识,希望对你有一定的参考价值。
环境如下:
CentOS-7-x86_64
zookeeper-3.4.11
kafka_2.12-1.1.0
一.zookeeper下载与安装
1)下载zookeeper
[root@localhost opt]# cd /opt/ [root@localhost opt]# wget https://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz
2)解压
[root@localhost opt]# tar zxvf zookeeper-3.4.11.tar.gz [root@localhost opt]# ls zookeeper-3.4.11 zookeeper-3.4.11.tar.gz
3)配置
[root@localhost opt]# cd zookeeper-3.4.11 [root@localhost zookeeper-3.4.11]# ll total 1596 drwxr-xr-x. 2 502 games 149 Nov 1 14:52 bin -rw-r--r--. 1 502 games 87943 Nov 1 14:47 build.xml drwxr-xr-x. 2 502 games 77 Nov 1 14:52 conf drwxr-xr-x. 10 502 games 130 Nov 1 14:47 contrib drwxr-xr-x. 2 502 games 4096 Nov 1 14:54 dist-maven drwxr-xr-x. 6 502 games 4096 Nov 1 14:52 docs -rw-r--r--. 1 502 games 1709 Nov 1 14:47 ivysettings.xml -rw-r--r--. 1 502 games 8197 Nov 1 14:47 ivy.xml drwxr-xr-x. 4 502 games 4096 Nov 1 14:52 lib -rw-r--r--. 1 502 games 11938 Nov 1 14:47 LICENSE.txt -rw-r--r--. 1 502 games 3132 Nov 1 14:47 NOTICE.txt -rw-r--r--. 1 502 games 1585 Nov 1 14:47 README.md -rw-r--r--. 1 502 games 1770 Nov 1 14:47 README_packaging.txt drwxr-xr-x. 5 502 games 47 Nov 1 14:47 recipes drwxr-xr-x. 8 502 games 211 Nov 1 14:52 src -rw-r--r--. 1 502 games 1478279 Nov 1 14:49 zookeeper-3.4.11.jar -rw-r--r--. 1 502 games 195 Nov 1 14:52 zookeeper-3.4.11.jar.asc -rw-r--r--. 1 502 games 33 Nov 1 14:49 zookeeper-3.4.11.jar.md5 -rw-r--r--. 1 502 games 41 Nov 1 14:49 zookeeper-3.4.11.jar.sha1 [root@localhost zookeeper-3.4.11]# cp -rf conf/zoo_sample.cfg conf/zoo.cfg [root@localhost zookeeper-3.4.11]# vi conf/zoo.cfg
修改或添加zoo.cfg文件中如下两个配置项:
dataDir=/opt/zookeeper-3.4.11/zkdata #这个目录是预先创建的 dataLogDir=/opt/zookeeper-3.4.11/zkdatalog #这个目录是预先创建的
创建zk数据存储和zk日志存储目录:
[root@localhost zookeeper-3.4.11]# mkdir /opt/zookeeper-3.4.11/zkdata [root@localhost zookeeper-3.4.11]# mkdir /opt/zookeeper-3.4.11/zkdatalog [root@localhost zookeeper-3.4.11]# ll total 1596 drwxr-xr-x. 2 502 games 149 Nov 1 14:52 bin -rw-r--r--. 1 502 games 87943 Nov 1 14:47 build.xml drwxr-xr-x. 2 502 games 92 Mar 31 11:12 conf drwxr-xr-x. 10 502 games 130 Nov 1 14:47 contrib drwxr-xr-x. 2 502 games 4096 Nov 1 14:54 dist-maven drwxr-xr-x. 6 502 games 4096 Nov 1 14:52 docs -rw-r--r--. 1 502 games 1709 Nov 1 14:47 ivysettings.xml -rw-r--r--. 1 502 games 8197 Nov 1 14:47 ivy.xml drwxr-xr-x. 4 502 games 4096 Nov 1 14:52 lib -rw-r--r--. 1 502 games 11938 Nov 1 14:47 LICENSE.txt -rw-r--r--. 1 502 games 3132 Nov 1 14:47 NOTICE.txt -rw-r--r--. 1 502 games 1585 Nov 1 14:47 README.md -rw-r--r--. 1 502 games 1770 Nov 1 14:47 README_packaging.txt drwxr-xr-x. 5 502 games 47 Nov 1 14:47 recipes drwxr-xr-x. 8 502 games 211 Nov 1 14:52 src drwxr-xr-x. 2 root root 6 Mar 31 11:13 zkdata drwxr-xr-x. 2 root root 6 Mar 31 11:13 zkdatalog -rw-r--r--. 1 502 games 1478279 Nov 1 14:49 zookeeper-3.4.11.jar -rw-r--r--. 1 502 games 195 Nov 1 14:52 zookeeper-3.4.11.jar.asc -rw-r--r--. 1 502 games 33 Nov 1 14:49 zookeeper-3.4.11.jar.md5 -rw-r--r--. 1 502 games 41 Nov 1 14:49 zookeeper-3.4.11.jar.sha1
4)配置环境变量
[root@localhost zookeeper-3.4.11]# vi /etc/profile
配置项如下:
# config java class path export JAVA_HOME=/usr/local/java/jdk1.8.0_161 export JRE_HOME=${JAVA_HOME}/jre export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$ZOOKEEPER_HOME/lib: export PATH=${JAVA_HOME}/bin:$ZOOKEEPER_HOME/bin:$PATH # config zookeeper install path export ZOOKEEPER_HOME=/opt/zookeeper-3.4.11
5)启动zookeeper
[root@localhost bin]# cd /opt/zookeeper-3.4.11/bin [root@localhost bin]# ll total 36 -rwxr-xr-x. 1 502 games 232 Nov 1 14:47 README.txt -rwxr-xr-x. 1 502 games 1937 Nov 1 14:47 zkCleanup.sh -rwxr-xr-x. 1 502 games 1056 Nov 1 14:47 zkCli.cmd -rwxr-xr-x. 1 502 games 1534 Nov 1 14:47 zkCli.sh -rwxr-xr-x. 1 502 games 1628 Nov 1 14:47 zkEnv.cmd -rwxr-xr-x. 1 502 games 2696 Nov 1 14:47 zkEnv.sh -rwxr-xr-x. 1 502 games 1089 Nov 1 14:47 zkServer.cmd -rwxr-xr-x. 1 502 games 6773 Nov 1 14:47 zkServer.sh [root@localhost bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.11/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
二.kafka下载与安装
1).下载kafka:
[root@localhost bin]# cd /opt/ [root@localhost opt]# wget http://apache.fayea.com/kafka/1.1.0/kafka_2.12-1.1.0.tgz --2018-03-31 11:21:52-- http://apache.fayea.com/kafka/1.1.0/kafka_2.12-1.1.0.tgz Resolving apache.fayea.com (apache.fayea.com)... 202.115.175.188, 202.115.175.187 Connecting to apache.fayea.com (apache.fayea.com)|202.115.175.188|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 50326212 (48M) [application/x-gzip] Saving to: ‘kafka_2.12-1.1.0.tgz’ 100%[=============================================================================================================================>] 50,326,212 442KB/s in 1m 44s 2018-03-31 11:23:36 (473 KB/s) - ‘kafka_2.12-1.1.0.tgz’ saved [50326212/50326212] [root@localhost opt]# ll total 84964 -rw-r--r--. 1 root root 50326212 Mar 28 08:05 kafka_2.12-1.1.0.tgz drwxr-xr-x. 15 502 games 4096 Mar 31 11:20 zookeeper-3.4.11 -rw-r--r--. 1 root root 36668066 Nov 8 13:24 zookeeper-3.4.11.tar.gz
2) 解压:
tar -zxvf kafka_2.12-1.1.0.tgz
3) 配置:
进入kafka安装工程根目录编辑config/server.properties
[root@localhost opt]# cd /opt/kafka_2.12-1.1.0/config/ [root@localhost config]# ll total 64 -rw-r--r--. 1 root root 906 Mar 23 18:51 connect-console-sink.properties -rw-r--r--. 1 root root 909 Mar 23 18:51 connect-console-source.properties -rw-r--r--. 1 root root 5807 Mar 23 18:51 connect-distributed.properties -rw-r--r--. 1 root root 883 Mar 23 18:51 connect-file-sink.properties -rw-r--r--. 1 root root 881 Mar 23 18:51 connect-file-source.properties -rw-r--r--. 1 root root 1111 Mar 23 18:51 connect-log4j.properties -rw-r--r--. 1 root root 2730 Mar 23 18:51 connect-standalone.properties -rw-r--r--. 1 root root 1221 Mar 23 18:51 consumer.properties -rw-r--r--. 1 root root 4727 Mar 23 18:51 log4j.properties -rw-r--r--. 1 root root 1919 Mar 23 18:51 producer.properties -rw-r--r--. 1 root root 6851 Mar 23 18:51 server.properties -rw-r--r--. 1 root root 1032 Mar 23 18:51 tools-log4j.properties -rw-r--r--. 1 root root 1023 Mar 23 18:51 zookeeper.properties [root@localhost config]# mkdir /opt/kafka_2.12-1.1.0/kafka_log
添加或者修改以下两个配置项:
log.dirs=/opt/kafka_2.12-1.1.0/kafka_log #(提前创建) listeners=PLAINTEXT://192.168.0.111:9092
config/server.properties修改后:
1 [root@localhost config]# more server.properties 2 # Licensed to the Apache Software Foundation (ASF) under one or more 3 # contributor license agreements. See the NOTICE file distributed with 4 # this work for additional information regarding copyright ownership. 5 # The ASF licenses this file to You under the Apache License, Version 2.0 6 # (the "License"); you may not use this file except in compliance with 7 # the License. You may obtain a copy of the License at 8 # 9 # http://www.apache.org/licenses/LICENSE-2.0 10 # 11 # Unless required by applicable law or agreed to in writing, software 12 # distributed under the License is distributed on an "AS IS" BASIS, 13 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 # See the License for the specific language governing permissions and 15 # limitations under the License. 16 17 # see kafka.server.KafkaConfig for additional details and defaults 18 19 ############################# Server Basics ############################# 20 21 # The id of the broker. This must be set to a unique integer for each broker. 22 broker.id=0 23 24 ############################# Socket Server Settings ############################# 25 26 # The address the socket server listens on. It will get the value returned from 27 # java.net.InetAddress.getCanonicalHostName() if not configured. 28 # FORMAT: 29 # listeners = listener_name://host_name:port 30 # EXAMPLE: 31 # listeners = PLAINTEXT://your.host.name:9092 32 #listeners=PLAINTEXT://:9092 33 listeners=PLAINTEXT://192.178.0.111:9092 34 35 # Hostname and port the broker will advertise to producers and consumers. If not set, 36 # it uses the value for "listeners" if configured. Otherwise, it will use the value 37 # returned from java.net.InetAddress.getCanonicalHostName(). 38 #advertised.listeners=PLAINTEXT://your.host.name:9092 39 40 # Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details 41 #listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL 42 43 # The number of threads that the server uses for receiving requests from the network and sending responses to the network 44 num.network.threads=3 45 46 # The number of threads that the server uses for processing requests, which may include disk I/O 47 num.io.threads=8 48 49 # The send buffer (SO_SNDBUF) used by the socket server 50 socket.send.buffer.bytes=102400 51 52 # The receive buffer (SO_RCVBUF) used by the socket server 53 socket.receive.buffer.bytes=102400 54 55 # The maximum size of a request that the socket server will accept (protection against OOM) 56 socket.request.max.bytes=104857600 57 58 59 ############################# Log Basics ############################# 60 61 # A comma separated list of directories under which to store log files 62 #log.dirs=/tmp/kafka-logs 63 log.dirs=/opt/kafka_2.12-1.1.0/kafka_log 64 65 # The default number of log partitions per topic. More partitions allow greater 66 # parallelism for consumption, but this will also result in more files across 67 # the brokers. 68 num.partitions=1 69 70 # The number of threads per data directory to be used for log recovery at startup and flushing at shutdown. 71 # This value is recommended to be increased for installations with data dirs located in RAID array. 72 num.recovery.threads.per.data.dir=1 73 74 ############################# Internal Topic Settings ############################# 75 # The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state" 76 # For anything other than development testing, a value greater than 1 is recommended for to ensure availability such as 3. 77 offsets.topic.replication.factor=1 78 transaction.state.log.replication.factor=1 79 transaction.state.log.min.isr=1 80 81 ############################# Log Flush Policy ############################# 82 83 # Messages are immediately written to the filesystem but by default we only fsync() to sync 84 # the OS cache lazily. The following configurations control the flush of data to disk. 85 # There are a few important trade-offs here: 86 # 1. Durability: Unflushed data may be lost if you are not using replication. 87 # 2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush. 88 # 3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to excessive seeks. 89 # The settings below allow one to configure the flush policy to flush data after a period of time or 90 # every N messages (or both). This can be done globally and overridden on a per-topic basis. 91 92 # The number of messages to accept before forcing a flush of data to disk 93 #log.flush.interval.messages=10000 94 95 # The maximum amount of time a message can sit in a log before we force a flush 96 #log.flush.interval.ms=1000 97 98 ############################# Log Retention Policy ############################# 99 100 # The following configurations control the disposal of log segments. The policy can 101 # be set to delete segments after a period of time, or after a given size has accumulated. 102 # A segment will be deleted whenever *either* of these criteria are met. Deletion always happens 103 # from the end of the log. 104 105 # The minimum age of a log file to be eligible for deletion due to age 106 log.retention.hours=168 107 108 # A size-based retention policy for logs. Segments are pruned from the log unless the remaining 109 # segments drop below log.retention.bytes. Functions independently of log.retention.hours. 110 #log.retention.bytes=1073741824 111 112 # The maximum size of a log segment file. When this size is reached a new log segment will be created. 113 log.segment.bytes=1073741824 114 115 # The interval at which log segments are checked to see if they can be deleted according 116 # to the retention policies 117 log.retention.check.interval.ms=300000 118 119 ############################# Zookeeper ############################# 120 121 # Zookeeper connection string (see zookeeper docs for details). 122 # This is a comma separated host:port pairs, each corresponding to a zk 123 # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002". 124 # You can also append an optional chroot string to the urls to specify the 125 # root directory for all kafka znodes. 126 zookeeper.connect=localhost:2181 127 128 # Timeout in ms for connecting to zookeeper 129 zookeeper.connection.timeout.ms=6000 130 131 132 ############################# Group Coordinator Settings ############################# 133 134 # The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance. 135 # The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms. 136 # The default value for this is 3 seconds. 137 # We override this to 0 here as it makes for a better out-of-the-box experience for development and testing. 138 # However, in production environments the default value of 3 seconds is more suitable as this will help to avoid unnecessary, and potentially expensive, rebalances dur 139 ing application startup. 140 group.initial.rebalance.delay.ms=0
4)启动kafka
[root@localhost kafka_2.12-1.1.0]# cd /opt/kafka_2.12-1.1.0/ [root@localhost kafka_2.12-1.1.0]# ll total 48 drwxr-xr-x. 3 root root 4096 Mar 23 18:55 bin drwxr-xr-x. 2 root root 4096 Mar 31 11:30 config drwxr-xr-x. 2 root root 6 Mar 31 11:31 kafka_log drwxr-xr-x. 2 root root 4096 Mar 31 11:26 libs -rw-r--r--. 1 root root 28824 Mar 23 18:51 LICENSE drwxr-xr-x. 2 root root 182 Mar 31 11:33 logs -rw-r--r--. 1 root root 336 Mar 23 18:51 NOTICE drwxr-xr-x. 2 root root 44 Mar 23 18:55 site-docs [root@localhost kafka_2.12-1.1.0]# sh ./bin/kafka-server-start.sh ./config/server.properties & [2018-03-31 11:35:47,198] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.KafkaException: Socket server failed to bind to 192.178.0.111:9092: Cannot assign requested address. at kafka.network.Acceptor.openServerSocket(SocketServer.scala:404) at kafka.network.Acceptor.<init>(SocketServer.scala:308) at kafka.network.SocketServer.$anonfun$createAcceptorAndProcessors$1(SocketServer.scala:126) at kafka.network.SocketServer.$anonfun$createAcceptorAndProcessors$1$adapted(SocketServer.scala:122) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at kafka.network.SocketServer.createAcceptorAndProcessors(SocketServer.scala:122) at kafka.network.SocketServer.startup(SocketServer.scala:84) at kafka.server.KafkaServer.startup(KafkaServer.scala:247) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38) at kafka.Kafka$.main(Kafka.scala:92) at kafka.Kafka.main(Kafka.scala) Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) at kafka.network.Acceptor.openServerSocket(SocketServer.scala:400) ... 12 more
备注:上边是启动失败信息,启动失败原因是我的服务器9092端口未开启
此时,检测2181与9092端口
[root@localhost kafka_2.12-1.1.0]# netstat -tunlp|egrep "(2181|9092)" tcp6 0 0 :::2181 :::* LISTEN 8896/java
CentOS 7.0默认使用的是firewall作为防火墙,使用iptables必须重新设置一下
1、直接关闭防火墙
systemctl stop firewalld.service #停止firewall systemctl disable firewalld.service #禁止firewall开机启动
2、设置 iptables service
[root@localhost kafka_2.12-1.1.0]# yum -y install iptables-services Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.163.com * extras: mirrors.163.com * updates: mirrors.cn99.com Resolving Dependencies --> Running transaction check ---> Package iptables-services.x86_64 0:1.4.21-18.3.el7_4 will be installed --> Processing Dependency: iptables = 1.4.21-18.3.el7_4 for package: iptables-services-1.4.21-18.3.el7_4.x86_64 --> Running transaction check ---> Package iptables.x86_64 0:1.4.21-18.0.1.el7.centos will be updated ---> Package iptables.x86_64 0:1.4.21-18.3.el7_4 will be an update --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================= Package Arch Version Repository Size ======================================================================================================================================================================= Installing: iptables-services x86_64 1.4.21-18.3.el7_4 updates 51 k Updating for dependencies: iptables x86_64 1.4.21-18.3.el7_4 updates 428 k Transaction Summary ======================================================================================================================================================================= Install 1 Package Upgrade ( 1 Dependent package) Total download size: 479 k Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. (1/2): iptables-services-1.4.21-18.3.el7_4.x86_64.rpm | 51 kB 00:00:00 (2/2): iptables-1.4.21-18.3.el7_4.x86_64.rpm | 428 kB 00:00:01 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 447 kB/s | 479 kB 00:00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : iptables-1.4.21-18.3.el7_4.x86_64 1/3 Installing : iptables-services-1.4.21-18.3.el7_4.x86_64 2/3 Cleanup : iptables-1.4.21-18.0.1.el7.centos.x86_64 3/3 Verifying : iptables-1.4.21-18.3.el7_4.x86_64 1/3 Verifying : iptables-services-1.4.21-18.3.el7_4.x86_64 2/3 Verifying : iptables-1.4.21-18.0.1.el7.centos.x86_64 3/3 Installed: iptables-services.x86_64 0:1.4.21-18.3.el7_4 Dependency Updated: iptables.x86_64 0:1.4.21-18.3.el7_4 Complete! [root@localhost kafka_2.12-1.1.0]#
备注:默认iptables是没有安装的,需要先安装iptables
如果要修改防火墙配置,如增加防火墙端口 9092
vi /etc/sysconfig/iptables
增加规则 -A INPUT -m state --state NEW -m tcp -p tcp --dport 9092 -j ACCEPT
[root@localhost kafka_2.12-1.1.0]# vi /etc/sysconfig/iptables [root@localhost kafka_2.12-1.1.0]# more /etc/sysconfig/iptables # sample configuration for iptables service # you can edit this manually or use system-config-firewall # please do not ask us to add additional ports/services to this default configuration *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 9092 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT
保存退出后
systemctl restart iptables.service #重启防火墙使配置生效 systemctl enable iptables.service #设置防火墙开机启动 systemctl start firewalld.service systemctl enable firewalld.service
最后重启系统使设置生效即可。
重新启动zk,重新启动kafaka,启动后检测端口是否通:
[root@localhost opt]# netstat -tunlp|egrep "(2181|9092)" tcp6 0 0 192.178.0.111:9092 :::* LISTEN 10299/java tcp6 0 0 :::2181 :::* LISTEN 8896/java
5)新建一个TOPIC
--创建topic
/opt/kafka_2.12-1.1.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1 --topic kafkatopic
此时kafaka服务器开启窗口(执行[root@localhost kafka_2.12-1.1.0]# sh ./bin/kafka-server-start.sh ./config/server.properties &的窗口)会有变化:
--查看所有topic
/opt/kafka_2.12-1.1.0/bin/kafka-topics.sh --list --zookeeper localhost:2181
--查看指定topic
/opt/kafka_2.12-1.1.0/bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic logTopic100
6) 把KAFKA的生产者启动起来:
/opt/kafka_2.12-1.1.0/bin/kafka-console-producer.sh --broker-list 192.178.0.111:9092 --sync --topic kafkatopic
7)另开一个终端,把消费者启动起来:
sh /opt/kafka_2.12-1.1.0/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic kafkatopic --from-beginning
也可以采用:
sh /opt/kafka_2.12-1.1.0/bin/kafka-console-consumer.sh --bootstrap-server 192.178.0.111:9092 --topic kafkatopic --from-beginning
(--from beginning 是从头开始消费,不加则是消费当前正在发送到该topic的消息)
8)使用
在发送消息的终端输入aaa,则可以在消费消息的终端显示
生产者生产:
[root@localhost ~]# /opt/kafka_2.12-1.1.0/bin/kafka-console-producer.sh --broker-list 192.178.0.111:9092 --topic kafkatopic
>a
>b
>c
>d
>
消费者接收:
[root@localhost ~]# /opt/kafka_2.12-1.1.0/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic kafkatopic --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
a
b
c
d
参考资料如下:
1)《 CentOS 7 开放防火墙端口命令》
2)《 kafka+zookeeper环境配置(linux环境单机版)》
以上是关于centos单机安装zookeeper+kafaka的主要内容,如果未能解决你的问题,请参考以下文章
debeziumkafka connector 解析 mysql binlog 到 kafak
zookeeper集群+kafak集群 + KafkaOffsetMonitor 监控