• hadoop常见错误集锦


    1  Hadoop namenode -format后Incompatible namespaceIDS 错误解决办法

    重新格式话namenode后,会重新分配一个namespaceId, 这个namespaceID就会和存储在datanode上的namespaceID不一致,

    解决办法:

    a  修改datanode中的namespaceID, 在“dfs.data.dir”/current/VERSION中进行修改,新的namespaceID可以在log中寻找;

    b  删除datanode,  dfs.data.dir中的文件,或者更改dfs.data.dir目录为新目录  

     至于如何恢复系统的数据可以参考该文章: http://www.cnblogs.com/Richardzhu/p/3435989.html

    2   SafeModeException

    error: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/feixiang/hadoop-0.20.2/mapred/system. Name node is in safe mode.
    The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
    org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/feixiang/hadoop-0.20.2/mapred/system. Name node is in safe mode.
    The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.

    原因:namenode每次启动的时候,会进入安全模式,接受datanode上报块数,如果上报的块数损失率达到某个数值(默认是1-0.9990),会一直处于只读不能写的安全模式。namenode每次启动的时候为什么要删除mapred/system文件???

    有两种方式离开安全模式:  

    a  在hdfs-site.xml中, 修改dfs.safemode.threshold.pct的值,但是具体为多少,要看log里面的ratio of reported blocks

    b  强制离开。命令: hadoop dfsadmin -safemode leave

    参考:

    http://www.linuxidc.com/Linux/2012-12/76782.htm

    http://blog.csdn.net/rzhzhz/article/details/7056793

     

    2014-03-28 21:15:26,118 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9003, call addBlock(/home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_-690631423_1, null) from 127.0.0.1:36367: error: java.io.IOException: File /home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
    java.io.IOException: File /home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

    mapred文件夹下没有建立system目录,namenode -format再启动就会出现这个问题,建立这个目录后,重新格式话,再启动就ok,不知道什么原因。

  • 相关阅读:
    Day 03
    Day 02
    Day 01
    re正则表达四
    python学习05之数据可视化
    python学习04之柱形图和热图
    python学习03之线图表
    python的学习02之数据处理
    python的学习01之csv文件处理
    中国大学排名实例
  • 原文地址:https://www.cnblogs.com/kangls/p/3580691.html
Copyright © 2020-2023  润新知