• 多次format namenode后,datanode无法正常启动


    DataNode报错日志如下:

    2021-07-05 16:30:53,595 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-21/192.168.2.21:8020. Exiting. 
    java.io.IOException: All specified directories are failed to load.
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:477)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1361)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1326)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:316)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801)
            at java.lang.Thread.run(Thread.java:748)
    2021-07-05 16:30:53,597 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-21/192.168.2.21:8020
    2021-07-05 16:30:53,595 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-22/192.168.2.22:8020. Exiting. 
    java.io.IOException: Cluster IDs not matched: dn cid=CID-ae173266-e1e1-4871-9b15-f29a7c16e302 but ns cid=CID-0a73391e-ebcc-42ca-99a6-1454d873bb5e; bpid=BP-1650383988-192.168.2.22-1625473706063
            at org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:717)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1319)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:316)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801)
            at java.lang.Thread.run(Thread.java:748)
    2021-07-05 16:30:53,598 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-22/192.168.2.22:8020
    2021-07-05 16:30:53,699 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
    2021-07-05 16:30:55,699 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
    2021-07-05 16:30:55,701 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
    2021-07-05 16:30:55,703 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: 

    DataNode无法正常启动的话,可以先删除对应的 /xxxx/dfs/data/current 目录  因为version版本不一致导致启动失败。删除成功后,然后重启DataNode。发现重启成功。

    然后访问hadoop namenode节点50070端口,发现active和standby节点展示DataNode节点不一样(或者有其中一个直接不展示),则需要同步namenode节点信息保持一致。

    如果是在hadoop01节点执行的格式化操作:hdfs namenode -format ,则需要在另一台namenode节点执行同步:hdfs namenode -bootstrapStandby

    参考:

    https://blog.csdn.net/QMW19910301/article/details/78055601

    https://blog.51cto.com/hsbxxl/2066487

  • 相关阅读:
    免费分享各大快递接口代码
    js 拖动div
    前端开发入门学习笔记(一)
    2017年2月14日
    html表格中的tr td th用法
    转:上传图片并给图片加水印(公共类积累)
    通用防SQL注入代码ASP版
    Firebug 在IE Opera Safari 下这样使用
    [正则表达式]文本框输入内容控制
    图片上传添加水印类
  • 原文地址:https://www.cnblogs.com/-courage/p/14973725.html
Copyright © 2020-2023  润新知