使用kafka作为生产者生产数据_到_hbase

Posted 瓶子xf

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了使用kafka作为生产者生产数据_到_hbase相关的知识,希望对你有一定的参考价值。

配置文件:

agent.sources = r1
agent.sinks = k1
agent.channels = c1

## sources config
agent.sources.r1.type = org.apache.flume.source.kafka.KafkaSource
agent.sources.r1.kafka.bootstrap.servers = 192.168.80.128:9092,192.168.80.129:9092,192.168.80.130:9092
agent.sources.r1.kafka.topics =1713
agent.sources.r1.migrateZookeeperOffsets =false
agent.sources.r1.kafka.consumer.timeout.ms = 1000
#agent.sources.r1.kafka.consumer.group.id = consumer-group

## channels config
agent.channels.c1.type = memory
agent.channels.c1.capacity = 1000
agent.channels.c1.transactionCapacity = 100
agent.channels.c1.byteCapacityBufferPercentage = 60
agent.channels.c1.byteCapacity = 1280
agent.channels.c1.keep-alive = 60


# Describe the sink
agent.sinks.k1.type =asynchbase //sink类型到hbase
agent.sinks.k1.table = tb_words3 //表
agent.sinks.k1.columnFamily = words //列族
agent.sinks.k1.serializer.payloadColumn=wd //列名
agent.sinks.k1.serializer =org.apache.flume.sink.hbase.SimpleAsyncHbaseEventSerializer

# Use a channel which buffers events in memory
agent.channels.c1.type = memory
agent.channels.c1.capacity = 1000
agent.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
agent.sources.r1.channels = c1
agent.sinks.k1.channel = c1

以上是关于使用kafka作为生产者生产数据_到_hbase的主要内容,如果未能解决你的问题,请参考以下文章

使用kafka作为生产者生产数据到hdfs(单节点)

使用kafka作为生产者生产数据到hdfs(单节点)

Apache NiFi之Kafka流数据到HBase

Java程序创建Kafka Topic,以及数据生产消费,常用的命令

Kafka生产者

kafka 常用命令行操作(查看主题,生产数据,消费数据)