• hadoop balancer 二


    1.每次迭代一个datanode会移动不超过如下两者中较小的一个的数据量

       1)10G

       2)能力阈值 dfs.datanode.balance.bandwidthPerSec

      每次迭代时间不会超过20分钟

    <p>The tool moves blocks from highly utilized datanodes to poorly
     * utilized datanodes iteratively. In each iteration a datanode moves or
     * receives no more than the lesser of 10G bytes or the threshold fraction
     * of its capacity. Each iteration runs no more than 20 minutes.
     * At the end of each iteration, the balancer obtains updated datanodes
     * information from the namenode.

    2.balancer自动停止的情景

     * <p>The balancer automatically exits when any of the following five
     * conditions is satisfied:
     * <ol>
     * <li>The cluster is balanced;
     * <li>No block can be moved;
     * <li>No block has been moved for five consecutive iterations;
     * <li>An IOException occurs while communicating with the namenode;
     * <li>Another balancer is running.
     * </ol>

     <p>Upon exit, a balancer returns an exit code and prints one of the
     * following messages to the output file in corresponding to the above exit
     * reasons:
     * <ol>
     * <li>The cluster is balanced. Exiting
     * <li>No block can be moved. Exiting...
     * <li>No block has been moved for 5 iterations. Exiting...
     * <li>Received an IO exception: failure reason. Exiting...
     * <li>Another balancer is running. Exiting...
     * </ol>

    3.一个datanode同时可用于balance的移动最大数

     /** The maximum number of concurrent blocks moves for
       * balancing purpose at a datanode
       */
      public static final int MAX_NUM_CONCURRENT_MOVES = 5;
      private static final int MAX_NO_PENDING_BLOCK_ITERATIONS = 5;
     

    4.

  • 相关阅读:
    freemarker 获取当前日期
    获取Map的key和value的两种方法
    maven install中依赖关系打包failed
    cxf动态调用外部web service 报告异常java.lang.NoSuchFieldErr
    Java的Annotation标签
    webservice cxf
    serversocket和socket
    Problem 1 珠江夜游 (cruise .cpp)———2019.10.6
    Problem 2 旅行计划 (travelling .cpp)———2019.10.6
    Problem 3 基站建设 (station.cpp)———2019.10.6
  • 原文地址:https://www.cnblogs.com/roger888/p/6703742.html
Copyright © 2020-2023  润新知