• hadoop 使用ip配置导致hdfs启动失败


    dataNode 有守护进行,但hdfs web页面上显示没有live node。

    错误日志:

    2017-06-21 17:44:59,513 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-1394689615-10.85.123.43-1498038283287 (Datanode Uuid null) service to /10.85.123.43:9000 Datanode denied communication with namenode because hostname cannot be resolved (ip=10.85.123.44, hostname=10.85.123.44): DatanodeRegistration(0.0.0.0:50010, datanodeUuid=e086ba2d-fe65-4ba7-a6c2-a4829ac9e708, infoPort=50075, infoSecurePort=0, ipcPort=50020, storageInfo=lv=-56;cid=CID-9bfeb191-6823-4f9e-9daa-e31c797e70df;nsid=523094968;c=0)
            at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:873)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:4529)
            at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:1286)
            at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:96)
            at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:28752)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

    由于配置hadoop没有使用host+hostName的配置方式,导致了hadoop无法解析DataNode,对DataNode的注册失败。

    解决方案:hdfs-site.xml中添加

      <property>
        <name>dfs.namenode.datanode.registration.ip-hostname-check</name>
        <value>false</value>
      </property>
  • 相关阅读:
    $().click()和$(document).on("click","selector",function(){})的区别
    orm 常用字段及参数
    前端vue 跨组件传参,cokies,axion
    drf 异常 响应 解析 三大模块
    drf 视图家族
    表断关系,和modlesserializers序列化,反序列化
    drf ___jwt插件
    drf 排序过滤分页
    django drf cors 跨域问题
    redis 数据库
  • 原文地址:https://www.cnblogs.com/suanec/p/7061485.html
Copyright © 2020-2023  润新知