错误 node.PollingPropertiesFileConfigurationProvider：无法加载配置数据。例外如下。 org.apache.flume.FlumeException：

Posted 2023-02-16

【中文标题】错误 node.PollingPropertiesFileConfigurationProvider：无法加载配置数据。例外如下。 org.apache.flume.FlumeException：

【英文标题】：ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows. org.apache.flume.FlumeException:

【发布时间】：2016-12-08 10:05:48

【问题描述】：

here moved snapshot jar in lib folder在运行flume命令后出现这个错误， 即

user@ubuntu:~/hadoop_store/apache-flume-1.4.0-bin/bin$ ./flume-ng agent -n TwitterAgent -c conf -f ../conf/twitter1.conf

也试过了

user@ubuntu:~/hadoop_store/apache-flume-1.4.0-bin/bin$ ./flume-ng agent -n TwitterAgent -c conf -f /home/user/hadoop_store/apache-flume-1.4。 0-bin/conf/twitter1.conf

因为在一个类似的问题中提到过， 这正是它所说的

16/12/08 04:41:26 INFO source.DefaultSourceFactory: Creating instance of source Twitter, type org.apache.flume.source.twitter.TwitterSource
16/12/08 04:41:26 ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Unable to load source type: org.apache.flume.source.twitter.TwitterSource, class: org.apache.flume.source.twitter.TwitterSource
    at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:67)
    at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:40)
    at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:327)
    at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
    at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.flume.source.twitter.TwitterSource
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:195)
    at org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:65)
    ... 11 more

这是我的 twitter1.conf 文件，是否有任何错误

# Naming the components on the current agent. 
TwitterAgent.sources = Twitter 
TwitterAgent.channels = MemChannel 
TwitterAgent.sinks = HDFS

# Describing/Configuring the source 
TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.consumerKey = ###
TwitterAgent.sources.Twitter.consumerSecret = ###
TwitterAgent.sources.Twitter.accessToken = ###
TwitterAgent.sources.Twitter.accessTokenSecret = ###
TwitterAgent.sources.Twitter.keywords = lalalala

# Describing/Configuring the sink 

TwitterAgent.sinks.HDFS.type = hdfs 
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:8020/user/Hadoop/twitter_data/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream 
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text 
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 

# Describing/Configuring the channel 
TwitterAgent.channels.MemChannel.type = memory 
TwitterAgent.channels.MemChannel.capacity = 10000 
TwitterAgent.channels.MemChannel.transactionCapacity = 100

# Binding the source and sink to the channel 
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sinks.HDFS.channel = MemChannel 

无法弄清楚这里有什么问题

【问题讨论】：

@A. T : 我在你之前的问题中已经回答了，请进入那个

这不是同一个，做了更正，

您收到 java.lang.ClassNotFoundException: org.apache.flume.source.twitter.TwitterSource 错误...您必须将 flume-source jar 放在 flume lib 目录中

yes 更正了仅在上一个问题中，flume.en.sh 中提到的问题仍然存在相同的错误

谢谢 farooque，虽然我不知道如何检查 jar 文件中的 org.apache.flume.source.twitter.TwitterSource 类，但我所做的是安装新的水槽版本并下载新的快照 jar file.maybe 它有 org.apache.flume.source.twitter.TwitterSource 类存在于它的工作中

【参考方案1】：

在 flume/lib 文件夹中添加 flume-sources-1.0-SNAPSHOT.jar 并将 Twitter4j3.0.3.jar 替换为 Twitter4j2.2.6.jar 文件（包括核心、媒体和流文件）它可能会工作