http://blog.csdn.net/jingshuigg/article/details/24439637
1. su root : chown -R hadoop:hadoop version-2/
drwxr-xr-x. 2 hadoop hadoop 4096 Dec 13 21:19 version-2
drwxr-xr-x. 2 hadoop hadoop 4096 Dec 13 16:50 kafka-logs
2.启动服务器
修改： vi zookeeper-shell.sh
注意：本文在试验的时候出现异常，最后发现原因是无法识别UseCompressedOops指令，导致无法创建JVM。经过查找发现该指令位于kafka-run-class.sh文件的153行，
可以将 -XX:+UseCompressedOops 整体删除。该指令的作用主要作用是用于指针的压缩，主要用于JVM优化，优化只是对于64位的，
本文使用的是32位，所以去掉后应该没事，对于UseCompressedOops 的了解可以参考
文章1：http://agapple.iteye.com/blog/1038336
文章2：http://kenwublog.com/compressedoops
也可以参考《深入理解JAVA虚拟机JVM高级特性与最佳实践》中的1.4.5节-----64位虚拟机
经过上面的处理后，再启动服务器
先启动zookeeper服务器
bin/zookeeper-server-start.sh config/zookeeper.properties &[[email protected] kafka]$ cd bin
[[email protected] bin]$ ll
total 72
-rwxrwxr-x. 1 hadoop hadoop 901 Apr 22 2014 kafka-console-consumer.sh
-rwxrwxr-x. 1 hadoop hadoop 901 Apr 22 2014 kafka-console-producer.sh
-rwxrwxr-x. 1 hadoop hadoop 901 Apr 22 2014 kafka-consumer-perf-test.sh
-rwxrwxr-x. 1 hadoop hadoop 884 Apr 22 2014 kafka-preferred-replica-election.sh
-rwxrwxr-x. 1 hadoop hadoop 901 Apr 22 2014 kafka-producer-perf-test.sh
-rwxrwxr-x. 1 hadoop hadoop 872 Apr 22 2014 kafka-reassign-partitions.sh
-rwxrwxr-x. 1 hadoop hadoop 866 Apr 22 2014 kafka-replay-log-producer.sh
-rwxrwxr-x. 1 hadoop hadoop 4194 Apr 22 2014 kafka-run-class.sh
-rwxrwxr-x. 1 hadoop hadoop 1238 Apr 22 2014 kafka-server-start.sh
-rwxrwxr-x. 1 hadoop hadoop 890 Apr 22 2014 kafka-server-stop.sh
-rwxrwxr-x. 1 hadoop hadoop 907 Apr 22 2014 kafka-simple-consumer-perf-test.sh
-rwxrwxr-x. 1 hadoop hadoop 868 Apr 22 2014 kafka-simple-consumer-shell.sh
-rwxrwxr-x. 1 hadoop hadoop 861 Apr 22 2014 kafka-topics.sh
drwxrwxr-x. 2 hadoop hadoop 4096 Apr 22 2014 windows
-rwxrwxr-x. 1 hadoop hadoop 1275 Apr 22 2014 zookeeper-server-start.sh
-rwxrwxr-x. 1 hadoop hadoop 875 Apr 22 2014 zookeeper-server-stop.sh
-rwxrwxr-x. 1 hadoop hadoop 968 Apr 22 2014 zookeeper-shell.sh
[[email protected] bin]$ vi zookeeper-shell.sh
[[email protected] bin]$ vi kafka-run-class.sh
[[email protected] bin]$ cd ..
[[email protected] kafka]$ ll
total 28
drwxr-xr-x. 3 hadoop hadoop 4096 Dec 16 23:02 bin
drwxr-xr-x. 2 hadoop hadoop 4096 Dec 16 22:59 config
drwxr-xr-x. 2 hadoop hadoop 4096 Apr 22 2014 libs
-rw-rw-r--. 1 hadoop hadoop 11358 Apr 22 2014 LICENSE
-rw-rw-r--. 1 hadoop hadoop 162 Apr 22 2014 NOTICE
[[email protected] kafka]$ bin/zookeeper-server-start.sh config/zookeeper.properties &
[1] 2638
[[email protected] kafka]$ [2017-12-16 23:06:05,784] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.Quoru
[2017-12-16 23:06:05,787] WARN Either no config or no quorum defined in config, running in standalone mode (org.apache.zookeeper.server.quorum.QuorumP
[2017-12-16 23:06:05,935] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2017-12-16 23:06:05,942] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
[2017-12-16 23:06:05,989] INFO Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.server.ZooKeeper
[2017-12-16 23:06:05,989] INFO Server environment:host.name=alamps (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:05,989] INFO Server environment:java.version=1.7.0_65 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,024] INFO Server environment:java.vendor=Oracle Corporation (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,024] INFO Server environment:java.home=/home/hadoop/jdk1.7.0_65/jre (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,026] INFO Server environment:java.class.path=:/home/hadoop/app/kafka/bin/../core/build/dependant-libs-2.8.0/*.jar:/home/hadoop/appf_2.8.0*.jar:/home/hadoop/app/kafka/bin/../clients/build/libs//kafka-clients*.jar:/home/hadoop/app/kafka/bin/../examples/build/libs//kafka-examples*.jahadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/home/hadoop/app/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/hople-3.2.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1-javadoc.jar:/home/hadoop/apcaladoc.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1-sources.jar:/home/hadoop/app/kafka/bin/../libs/log4j-1.2.15.jar:/home/hadoop/app/kafkme/hadoop/app/kafka/bin/../libs/scala-library-2.10.1.jar:/home/hadoop/app/kafka/bin/../libs/slf4j-api-1.7.2.jar:/home/hadoop/app/kafka/bin/../libs/snap/bin/../libs/zkclient-0.3.jar:/home/hadoop/app/kafka/bin/../libs/zookeeper-3.3.4.jar:/home/hadoop/app/kafka/bin/../core/build/libs/kafka_2.8.0*.jar (orr)
[2017-12-16 23:06:06,026] INFO Server environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperSer
[2017-12-16 23:06:06,027] INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,027] INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,028] INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,028] INFO Server environment:os.arch=i386 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,029] INFO Server environment:os.version=2.6.32-358.el6.i686 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,029] INFO Server environment:user.name=hadoop (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,029] INFO Server environment:user.home=/home/hadoop (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,029] INFO Server environment:user.dir=/home/hadoop/app/kafka (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,064] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,064] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,065] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-12-16 23:06:06,168] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NioserverCnxn)
[2017-12-16 23:06:06,459] INFO Snapshotting: 0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2017-12-16 23:37:54,795] INFO Accepted socket connection from /0:0:0:0:0:0:0:1:46109 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:37:54,827] INFO Client attempting to establish new session at /0:0:0:0:0:0:0:1:46109 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:37:54,836] INFO Creating new log file: log.1 (org.apache.zookeeper.server.persistence.FileTxnLog)
[2017-12-16 23:37:54,901] INFO Established session 0x160634b49c50000 with negotiated timeout 6000 for client /0:0:0:0:0:0:0:1:46109 (org.apache.zookeep
[2017-12-16 23:37:55,039] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x4 zxid:0xfffffffffffffffe brokers Error:KeeperErrorCode = NoNode for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:37:55,090] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0xa zxid:0xfffffffffffffffe config Error:KeeperErrorCode = NoNode for /config (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:37:55,103] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x10 zxid:0xfffffffffffffffe/admin Error:KeeperErrorCode = NoNode for /admin (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:37:56,124] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:setData cxid:0x1a zxid:0xfffffffffffffff:/controller_epoch Error:KeeperErrorCode = NoNode for /controller_epoch (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:37:56,369] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:delete cxid:0x27 zxid:0xfffffffffffffffe/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election (org.apache.zookeeper.server.PrepRequestProcesso
[2017-12-16 23:39:31,578] INFO Accepted socket connection from /127.0.0.1:48119 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:39:31,586] INFO Client attempting to establish new session at /127.0.0.1:48119 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:39:31,591] INFO Established session 0x160634b49c50001 with negotiated timeout 30000 for client /127.0.0.1:48119 (org.apache.zookeeper.se
[2017-12-16 23:39:32,603] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50001 type:setData cxid:0x3 zxid:0xfffffffffffffffe/config/topics/test Error:KeeperErrorCode = NoNode for /config/topics/test (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:39:32,673] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50001 type:create cxid:0x4 zxid:0xfffffffffffffffe config/topics Error:KeeperErrorCode = NodeExists for /config/topics (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:39:32,736] INFO Processed session termination for sessionid: 0x160634b49c50001 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:39:32,743] INFO Closed socket connection for client /127.0.0.1:48119 which had sessionid 0x160634b49c50001 (org.apache.zookeeper.server.
[2017-12-16 23:39:32,886] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x3b zxid:0xfffffffffffffffe/brokers/topics/test/partitions/0 Error:KeeperErrorCode = NoNode for /brokers/topics/test/partitions/0 (org.apache.zookeeper.server.PrepRequestProcesso
[2017-12-16 23:39:32,892] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x3c zxid:0xfffffffffffffffe/brokers/topics/test/partitions Error:KeeperErrorCode = NoNode for /brokers/topics/test/partitions (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:40:17,412] INFO Accepted socket connection from /0:0:0:0:0:0:0:1:46112 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:40:17,422] INFO Client attempting to establish new session at /0:0:0:0:0:0:0:1:46112 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:40:17,428] INFO Established session 0x160634b49c50002 with negotiated timeout 30000 for client /0:0:0:0:0:0:0:1:46112 (org.apache.zookee
[2017-12-16 23:40:17,646] INFO Processed session termination for sessionid: 0x160634b49c50002 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:40:17,649] INFO Closed socket connection for client /0:0:0:0:0:0:0:1:46112 which had sessionid 0x160634b49c50002 (org.apache.zookeeper.s
[2017-12-16 23:42:24,414] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:setData cxid:0x43 zxid:0xfffffffffffffff:/config/topics/testpro Error:KeeperErrorCode = NoNode for /config/topics/testpro (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:42:24,420] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x44 zxid:0xfffffffffffffffe/config/topics Error:KeeperErrorCode = NodeExists for /config/topics (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:42:24,638] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x4f zxid:0xfffffffffffffffe/brokers/topics/testpro/partitions/0 Error:KeeperErrorCode = NoNode for /brokers/topics/testpro/partitions/0 (org.apache.zookeeper.server.PrepRequestPr
[2017-12-16 23:42:24,644] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x50 zxid:0xfffffffffffffffe/brokers/topics/testpro/partitions Error:KeeperErrorCode = NoNode for /brokers/topics/testpro/partitions (org.apache.zookeeper.server.PrepRequestProces
[2017-12-16 23:42:24,666] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50000 type:create cxid:0x54 zxid:0xfffffffffffffffe/brokers/topics/testpro/partitions/1 Error:KeeperErrorCode = NoNode for /brokers/topics/testpro/partitions/1 (org.apache.zookeeper.server.PrepRequestPr
[2017-12-16 23:44:31,088] INFO Accepted socket connection from /127.0.0.1:48124 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:44:31,101] INFO Client attempting to establish new session at /127.0.0.1:48124 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:44:31,104] INFO Established session 0x160634b49c50003 with negotiated timeout 6000 for client /127.0.0.1:48124 (org.apache.zookeeper.ser
[2017-12-16 23:44:31,346] INFO Accepted socket connection from /127.0.0.1:48125 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:44:31,347] INFO Client attempting to establish new session at /127.0.0.1:48125 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-12-16 23:44:31,350] INFO Established session 0x160634b49c50004 with negotiated timeout 30000 for client /127.0.0.1:48125 (org.apache.zookeeper.se
[2017-12-16 23:44:31,439] INFO Processed session termination for sessionid: 0x160634b49c50004 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:44:31,444] INFO Closed socket connection for client /127.0.0.1:48125 which had sessionid 0x160634b49c50004 (org.apache.zookeeper.server.
[2017-12-16 23:44:31,898] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x2 zxid:0xfffffffffffffffe consumers/console-consumer-96946/ids Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/ids (org.apache.zookeeper.server.PrepRequestP
[2017-12-16 23:44:31,901] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x3 zxid:0xfffffffffffffffe consumers/console-consumer-96946 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946 (org.apache.zookeeper.server.PrepRequestProcessor
[2017-12-16 23:44:32,726] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x17 zxid:0xfffffffffffffffe/consumers/console-consumer-96946/owners/testpro Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/owners/testpro (org.apache.zookee
[2017-12-16 23:44:32,774] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x18 zxid:0xfffffffffffffffe/consumers/console-consumer-96946/owners Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/owners (org.apache.zookeeper.server.PrepR
[[email protected] kafka]$ [2017-12-16 23:45:31,300] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:setData cxid:0x23 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-96946/offsets/testpro/0 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/offsets/testpro/0 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:45:31,311] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x24 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-96946/offsets Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/offsets (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:45:31,336] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:setData cxid:0x28 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-96946/offsets/testpro/1 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-96946/offsets/testpro/1 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-12-16 23:45:31,350] INFO Got user-level KeeperException when processing sessionid:0x160634b49c50003 type:create cxid:0x29 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-96946/offsets/testpro Error:KeeperErrorCode = NodeExists for /consumers/console-consumer-96946/offsets/testpro (org.apache.zookeeper.server.PrepRequestProcessor)
再启动kafka服务器
bin/kafka-server-start.sh config/server.properties &[[email protected] app]$ cd kafka/
[[email protected] kafka]$ bin/kafka-server-start.sh config/server.properties &
[1] 2828
[[email protected] kafka]$ [2017-12-16 23:37:54,213] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,408] INFO Property broker.id is overridden to 0 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,408] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,409] INFO Property log.dirs is overridden to /tmp/kafka-logs (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,409] INFO Property log.retention.check.interval.ms is overridden to 60000 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,413] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,414] INFO Property log.segment.bytes is overridden to 536870912 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,414] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,419] INFO Property num.network.threads is overridden to 2 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,420] INFO Property num.partitions is overridden to 2 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,421] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,427] INFO Property socket.receive.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,432] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,433] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,434] INFO Property zookeeper.connect is overridden to localhost:2181 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,434] INFO Property zookeeper.connection.timeout.ms is overridden to 1000000 (kafka.utils.VerifiableProperties)
[2017-12-16 23:37:54,549] INFO [Kafka Server 0], starting (kafka.server.KafkaServer)
[2017-12-16 23:37:54,559] INFO [Kafka Server 0], Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2017-12-16 23:37:54,654] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-12-16 23:37:54,690] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,690] INFO Client environment:host.name=alamps (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,690] INFO Client environment:java.version=1.7.0_65 (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,690] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,690] INFO Client environment:java.home=/home/hadoop/jdk1.7.0_65/jre (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,691] INFO Client environment:java.class.path=:/home/hadoop/app/kafka/bin/../core/build/dependant-libs-2.8.0/*.jar:/home/hadoop/app/kafka/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/home/hadoop/app/kafka/bin/../clients/build/libs//kafka-clients*.jar:/home/hadoop/app/kafka/bin/../examples/build/libs//kafka-examples*.jar:/home/hadoop/app/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/home/hadoop/app/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/home/hadoop/app/kafka/bin/../libs/jopt-simple-3.2.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1-javadoc.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1-scaladoc.jar:/home/hadoop/app/kafka/bin/../libs/kafka_2.10-0.8.1.1-sources.jar:/home/hadoop/app/kafka/bin/../libs/log4j-1.2.15.jar:/home/hadoop/app/kafka/bin/../libs/metrics-core-2.2.0.jar:/home/hadoop/app/kafka/bin/../libs/scala-library-2.10.1.jar:/home/hadoop/app/kafka/bin/../libs/slf4j-api-1.7.2.jar:/home/hadoop/app/kafka/bin/../libs/snappy-java-1.0.5.jar:/home/hadoop/app/kafka/bin/../libs/zkclient-0.3.jar:/home/hadoop/app/kafka/bin/../libs/zookeeper-3.3.4.jar:/home/hadoop/app/kafka/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,691] INFO Client environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,691] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:os.arch=i386 (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:os.version=2.6.32-358.el6.i686 (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:user.name=hadoop (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:user.home=/home/hadoop (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,692] INFO Client environment:user.dir=/home/hadoop/app/kafka (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,697] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 [email protected] (org.apache.zookeeper.ZooKeeper)
[2017-12-16 23:37:54,763] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 (org.apache.zookeeper.ClientCnxn)
[2017-12-16 23:37:54,789] INFO Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-12-16 23:37:54,905] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x160634b49c50000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2017-12-16 23:37:54,934] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2017-12-16 23:37:55,322] INFO Log directory ‘/tmp/kafka-logs‘ not found, creating it. (kafka.log.LogManager)
[2017-12-16 23:37:55,373] INFO Starting log cleanup with a period of 60000 ms. (kafka.log.LogManager)
[2017-12-16 23:37:55,386] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2017-12-16 23:37:55,610] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2017-12-16 23:37:55,619] INFO [Socket Server on Broker 0], Started (kafka.network.SocketServer)
[2017-12-16 23:37:55,931] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2017-12-16 23:37:56,091] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2017-12-16 23:37:56,683] INFO Registered broker 0 at path /brokers/ids/0 with address alamps:9092. (kafka.utils.ZkUtils$)
[2017-12-16 23:37:56,780] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2017-12-16 23:37:56,993] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2017-12-16 23:39:33,216] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions [test,0] (kafka.server.ReplicaFetcherManager)
[2017-12-16 23:39:33,381] INFO Completed load of log test-0 with log end offset 0 (kafka.log.Log)
[2017-12-16 23:39:33,388] INFO Created log for partition [test,0] in /tmp/kafka-logs with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, cleanup.policy -> delete, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 604800000}. (kafka.log.LogManager)
[2017-12-16 23:39:33,391] WARN Partition [test,0] on broker 0: No checkpointed highwatermark is found for partition [test,0] (kafka.cluster.Partition)
[2017-12-16 23:42:24,430] INFO Topic creation {"version":1,"partitions":{"1":[0],"0":[0]}} (kafka.admin.AdminUtils$)
[2017-12-16 23:42:24,445] INFO [KafkaApi-0] Auto creation of topic testpro with 2 partitions and replication factor 1 is successful! (kafka.server.KafkaApis)
[2017-12-16 23:42:24,650] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor)
[2017-12-16 23:42:24,722] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions [testpro,0],[testpro,1] (kafka.server.ReplicaFetcherManager)
[2017-12-16 23:42:24,776] INFO Completed load of log testpro-0 with log end offset 0 (kafka.log.Log)
[2017-12-16 23:42:24,777] INFO Created log for partition [testpro,0] in /tmp/kafka-logs with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, cleanup.policy -> delete, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 604800000}. (kafka.log.LogManager)
[2017-12-16 23:42:24,784] WARN Partition [testpro,0] on broker 0: No checkpointed highwatermark is found for partition [testpro,0] (kafka.cluster.Partition)
[2017-12-16 23:42:24,818] INFO Completed load of log testpro-1 with log end offset 0 (kafka.log.Log)
[2017-12-16 23:42:24,825] INFO Created log for partition [testpro,1] in /tmp/kafka-logs with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, cleanup.policy -> delete, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 604800000}. (kafka.log.LogManager)
注意 & 符号的使用
3.创建topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
可以使用如下命令查看topic
bin/kafka-topics.sh --list --zookeeper localhost:2181
test
4.发送message
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
[[email protected] kafka]$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testpro
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
aaa
bbb
ccc
ddd
5.启动consumer
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
[[email protected] kafka]$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic testpro --from-beginning
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
aaa
bbb
ccc
ddd
6，jps
3018 ConsoleProducer
3038 ConsoleConsumer
3071 Jps
2828 Kafka
2638 QuorumPeerMain
[[email protected] kafka]$