DataNode报错日志如下:
2021-07-05 16:30:53,595 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-21/192.168.2.21:8020. Exiting. java.io.IOException: All specified directories are failed to load. at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:477) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1361) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1326) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:316) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801) at java.lang.Thread.run(Thread.java:748) 2021-07-05 16:30:53,597 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-21/192.168.2.21:8020 2021-07-05 16:30:53,595 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-22/192.168.2.22:8020. Exiting. java.io.IOException: Cluster IDs not matched: dn cid=CID-ae173266-e1e1-4871-9b15-f29a7c16e302 but ns cid=CID-0a73391e-ebcc-42ca-99a6-1454d873bb5e; bpid=BP-1650383988-192.168.2.22-1625473706063 at org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:717) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1319) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:316) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801) at java.lang.Thread.run(Thread.java:748) 2021-07-05 16:30:53,598 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to test-hadoop-2-22/192.168.2.22:8020 2021-07-05 16:30:53,699 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned) 2021-07-05 16:30:55,699 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode 2021-07-05 16:30:55,701 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0 2021-07-05 16:30:55,703 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
DataNode无法正常启动的话,可以先删除对应的 /xxxx/dfs/data/current 目录 因为version版本不一致导致启动失败。删除成功后,然后重启DataNode。发现重启成功。
然后访问hadoop namenode节点50070端口,发现active和standby节点展示DataNode节点不一样(或者有其中一个直接不展示),则需要同步namenode节点信息保持一致。
如果是在hadoop01节点执行的格式化操作:hdfs namenode -format ,则需要在另一台namenode节点执行同步:hdfs namenode -bootstrapStandby
参考: