• hadoop wrong key class: class org.apache.hadoop.io.Tex


    java.io.IOException: wrong key class: class org.apache.hadoop.io.Text is not class com.sogou.web.selector.updana.wapPc.test.TextPair
    	at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:172)
    	at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1119)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner$OutputConverter.write(Task.java:1440)
    	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:28)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:1)
    	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:195)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1461)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1334)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1198)
    	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:608)
    	at org.apache.had
    java.io.IOException: wrong key class: class org.apache.hadoop.io.Text is not class com.sogou.web.selector.updana.wapPc.test.TextPair
    	at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:172)
    	at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1119)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner$OutputConverter.write(Task.java:1440)
    	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:28)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:1)
    	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:195)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1461)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1334)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1198)
    	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:608)
    	at org.apache.had
    java.io.IOException: wrong key class: class org.apache.hadoop.io.Text is not class com.sogou.web.selector.updana.wapPc.test.TextPair
    	at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:172)
    	at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1119)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner$OutputConverter.write(Task.java:1440)
    	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:28)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:1)
    	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:195)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1461)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1334)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1198)
    	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:608)
    	at org.apache.had
    java.io.IOException: wrong key class: class org.apache.hadoop.io.Text is not class com.sogou.web.selector.updana.wapPc.test.TextPair
    	at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:172)
    	at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1119)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner$OutputConverter.write(Task.java:1440)
    	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:28)
    	at com.sogou.web.selector.updana.wapPc.WapPcReducer.reduce(WapPcReducer.java:1)
    	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:195)
    	at org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1461)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1334)
    	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1198)
    	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:608)
    	at org.apache.had
    修改方法:去掉combiner

    //设置combiner
    //job.setCombinerClass(Reducer.class);

    去掉上述代码就行了,原因应该是设置的Reducer cannot be same due to inconsistency in the output key and value types.

  • 相关阅读:
    【转】[完结] 取结构偏移 和 取地址符号的 思考
    资料备忘 【攀枝花】 我的百度文库
    static_cast reinterprt_cast 区别
    c++ 四种转换 cast 列表小结 (等待补充)2012 3月
    对偏移表达式的 思考过程—how offset macro is think out
    【转】常见面试题思想方法整理 原来果然有双指针遍历
    [转]好的习惯 提高你开发效率的十五个Visual Studio 2010使用技巧
    多种方案 测试 有无符号数包括 不适用大小于符号判断符号数
    [转] 仅有USB线拷贝无SD卡小米照相文件方法 简言adb shell
    iframe 高度自动调节
  • 原文地址:https://www.cnblogs.com/csxf/p/3785728.html
Copyright © 2020-2023  润新知