Hadoop map减少了java
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Hadoop map减少了java相关的知识,希望对你有一定的参考价值。
public static class TokenizerMapper extends Mapper<Object, Text, Text, Text> {
public void map(Object key, Text value, Context context) throws IOException, InterruptedException {
StringTokenizer itr = new StringTokenizer(value.toString(), " *$&#/
f"'\,.:;?![](){}<>~-_");
while (itr.hasMoreTokens()) {
String term = itr.nextToken().toLowerCase();
List<Pair<String, Pair<Integer, Integer>>> map = new ArrayList<Pair<String, Pair<Integer, Integer>>>();
/*here i am performing some operations*/
for (Pair<String, Pair<Integer, Integer>> i : map){
String w1 = i.getKey();
Text word = new Text(w1);
Pair<Integer, Integer> newValue = i.getValue();
String merge = String.valueOf(newValue.getKey()) + " " + String.valueOf(newValue.getValue());
Text val = new Text(merge);
/*sending both the arguments as text into my context.*/
context.write(word, val);
}
}
}
}
public static class Reducer1 extends Reducer<Text, Text, Text, Text> {
public void reduce(Text key, Text values, Context context) throws IOException, InterruptedException {
/* here i want to extract the values, i tried using for loop but its saying cannot iterate, its expecting something iterable.*/
for (Text t : values)
{
/*this is not working. I know we can use Iterable<IntWritable> for integers but in my case it is text.
}
//context.write(key, values);
}
}
请参阅注释行以更好地了解我的问题。有没有办法在reducer中提取Text值。 For
循环期待iterable
。
答案
在你的reducer中,你会得到一个Iterable<Text>
,你必须循环并在一个空格上拆分,为mapper写的每个merge
字符串重新创建String值。
注意:如果地图中的newValue.getKey()
本身包含字符串,则拆分空间将不会非常可靠
以上是关于Hadoop map减少了java的主要内容,如果未能解决你的问题,请参考以下文章