• java.io.IOException: Incompatible clusterIDs


    启动Hadoop集群的时候,所有的datanode启动不了,报错如下

    java.io.IOException: Incompatible clusterIDs in /home/xiaoqiu/hadoop_tmp/dfs/data:
    namenode clusterID = CID-7ecadf3f-9aa7-429a-8013-4e3ad1f28870; 
    datanode clusterID = CID-77fab491-d173-4dd3-8bc4-f36c0cb28b29
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:777)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:300)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:416)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:395)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:573)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1393)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1358)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:216)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:637)
            at java.lang.Thread.run(Thread.java:745)

    解决办法:

    分别进入各个节点的临时目录,找到data和name目录下面的VERSION,将data的ClusterID改成和name 一致

    master节点s150:

    name:

    [xiaoqiu@s150 /home/xiaoqiu/hadoop_tmp/dfs/name/current]$ cat VERSION
    #Sun Dec 31 00:29:38 EST 2017
    namespaceID=685530356
    clusterID=CID-cd569893-3a8e-4837-8c10-bdb93fd50d65
    cTime=0
    storageType=NAME_NODE
    blockpoolID=BP-907694094-192.168.109.150-1514698178308
    layoutVersion=-63

    data:

    xiaoqiu@s150 /home/xiaoqiu/hadoop_tmp/dfs/data/current]$ cat VERSION
    #Sun Dec 31 00:27:44 EST 2017
    storageID=DS-a5caee40-5e97-4751-bcec-dc4f7a7e3fda
    clusterID=CID-576629e1-43c9-4669-a6ee-74c5344be3df//不一致
    cTime=0
    datanodeUuid=b8cfe998-2d55-4fcc-9fc5-e849017cbceb
    storageType=DATA_NODE
    layoutVersion=-56

    修改data的clusterID

    [xiaoqiu@s150 /home/xiaoqiu/hadoop_tmp/dfs/data/current]$ cat VERSION
    #Sun Dec 31 00:27:44 EST 2017
    storageID=DS-a5caee40-5e97-4751-bcec-dc4f7a7e3fda
    clusterID=CID-cd569893-3a8e-4837-8c10-bdb93fd50d65//修改成和name一致
    cTime=0
    datanodeUuid=b8cfe998-2d55-4fcc-9fc5-e849017cbceb
    storageType=DATA_NODE
    layoutVersion=-56

    启动datanode

    [xiaoqiu@s150 /home/xiaoqiu/hadoop_tmp/dfs/data/current]$ start-dfs.sh

    节点s151:

    name:

    [xiaoqiu@s151 /home/xiaoqiu/hadoop_tmp/dfs/name/current]$ cat VERSION
    #Mon Dec 25 15:20:38 EST 2017
    namespaceID=875672388
    clusterID=CID-45abf7d9-2dec-4f77-b800-f20ddab41a1b
    cTime=0
    storageType=NAME_NODE
    blockpoolID=BP-1336727972-192.168.109.151-1514233238518
    layoutVersion=-63

    data:

    [xiaoqiu@s151 /home/xiaoqiu/hadoop_tmp/dfs/data/current]$ cat VERSION
    #Sun Dec 24 10:58:58 EST 2017
    storageID=DS-421723b4-ab06-486c-aece-a5a0b3f2d25e
    #clusterID=CID-77fab491-d173-4dd3-8bc4-f36c0cb28b29
    clusterID=CID-afd6244d-a77a-4ffe-a5ef-ce1a810145a7//修改为CID-45abf7d9-2dec-4f77-b800-f20ddab41a1b
    cTime=0
    datanodeUuid=e7800fda-3197-4ab9-ad34-24a1293f8097
    storageType=DATA_NODE
    layoutVersion=-56

    启动datanode

    [xiaoqiu@s151 /home/xiaoqiu/hadoop_tmp/dfs/data/current]$ hadoop-daemon.sh start datanode

    同理,对其他节点亦如此,将每个节点的data的VERSION的clusterID 改成每个节点自己对应的name的VERSION的clusterID

    欢迎关注我的公众号:小秋的博客 CSDN博客:https://blog.csdn.net/xiaoqiu_cr github:https://github.com/crr121 联系邮箱:rongchen633@gmail.com 有什么问题可以给我留言噢~
  • 相关阅读:
    使用LAMP创建基于wordpress的个从博客网站
    【solr基础教程之一】Solr相关知识点串讲
    solr源码导入eclipse
    【Nutch2.2.1基础教程之2.2】集成Nutch/Hbase/Solr构建搜索引擎之二:内容分析
    设置secureCRT中vim的字体颜色
    何时使用hadoop fs、hadoop dfs与hdfs dfs命令
    Hadoop常见异常及其解决方案
    Java HashMap遍历的两种方式
    android json 解析简单实例
    Android客户端与数据库交互数据的简单学习
  • 原文地址:https://www.cnblogs.com/flyingcr/p/10326967.html
Copyright © 2020-2023  润新知