Hadoop实战-Flume之Hdfs Sink

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Hadoop实战-Flume之Hdfs Sink相关的知识,希望对你有一定的参考价值。

a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444

# Describe the sink
#a1.sinks.k1.type = logger
a1.sinks.k1.type = hdfs
a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S
#官方缺少一个配置 a1.sinks.k1.hdfs.useLocalTimeStamp
= true a1.sinks.k1.hdfs.filePrefix = events- a1.sinks.k1.hdfs.round = true a1.sinks.k1.hdfs.roundValue = 10 a1.sinks.k1.hdfs.roundUnit = minute # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1

 


以上是关于Hadoop实战-Flume之Hdfs Sink的主要内容,如果未能解决你的问题,请参考以下文章

flume简介与监听文件目录并sink至hdfs实战

Hadoop实战-Flume之自定义Sink(十九)

Flume案例之采集特定目录的数据到HDFS

Hadoop之Flume

Hadoop之Flume

0#1hadoop生态圈之日志采集框架Flume入门