Kafka是由LinkedIn开发的一个分布式的消息系统，使用Scala编写，它以可水平扩展和高吞吐率而被广泛使用。目前越来越多的开源分布式处理系统如Cloudera、Apache Storm、Spark都支持与Kafka集成。
Kafka拓扑结构
安装和启动
Download
Download the 0.10.2.1 release and un-tar it.
tar -xzf kafka_2.11-0.10.2.1.tgz
cd kafka_2.11-0.10.2.1
Start the server
Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don\'t already have one. You can use the convenience script packaged with kafka to get a quick-and-dirty single-node ZooKeeper instance.
bin/zookeeper-server-start.sh config/zookeeper.properties
Now start the Kafka server:
bin/kafka-server-start.sh config/server.properties
Create a topic
bin/kafka-topics.sh --create --zookeeper localhost:8181 --replication-factor 1 --partitions 1 --topic test
查看topic
bin/kafka-topics.sh --list --zookeeper localhost:8181
test
Send some messages
bin/kafka-console-producer.sh --broker-list localhost:8092 --topic test
Start a consumer
bin/kafka-console-consumer.sh --bootstrap-server localhost:8092 --topic test --from-beginning
common commands
- list topic command
bin/kafka-topics.sh --list --zookeeper localhost:8181
- 查询topic
bin/kafka-topics.sh --describe --zookeeper localhost:8181 --topic my-replicated-topic
Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: my-replicated-topic Partition: 0 Leader: 0 Replicas: 0,2,1 Isr: 0