• hadoop InputSplit


    /**
     * <code>InputSplit</code> represents the data to be processed by an 
     * individual {@link Mapper}. 
     * InputSplit 代表可以被Mapper处理的数据
     * <p>Typically, it presents a byte-oriented view on the input and is the 
     * responsibility of {@link RecordReader} of the job to process this and present
     * a record-oriented view.
     *  
     * @see InputFormat
     * @see RecordReader
     */
    @InterfaceAudience.Public
    @InterfaceStability.Stable
    public abstract class InputSplit {
      /**
       * Get the size of the split, so that the input splits can be sorted by size.
       * @return the number of bytes in the split
       * @throws IOException
       * @throws InterruptedException
       * split的长度用byte表示
       */
      public abstract long getLength() throws IOException, InterruptedException;
    
      /**
       * Get the list of nodes by name where the data for the split would be local.
       * The locations do not need to be serialized.
       * 获取split所在的节点
       * @return a new array of the node nodes.
       * @throws IOException
       * @throws InterruptedException
       */
      public abstract 
        String[] getLocations() throws IOException, InterruptedException;
      
      /**
       * Gets info about which nodes the input split is stored on and how it is
       * stored at each location.
       * 返回split所在的节点信息以及在该节点上如何存储 memory
       * @return list of <code>SplitLocationInfo</code>s describing how the split
       *    data is stored at each location. A null value indicates that all the
       *    locations have the data stored on disk.
       * @throws IOException
       */
      @Evolving
      public SplitLocationInfo[] getLocationInfo() throws IOException {
        return null;
      }
    }
  • 相关阅读:
    【URAL 1004】 floyd最小环
    【UVA 10881】 经典模拟题
    【HDU 1541】 树状数组(入门题)
    【HDU 4000】 树状数组
    【HDU 3391 && HDU 4431】 dfs+模拟
    【HDU 1058 & HDU 3199 类似丑数】 简单DP思想
    Acdream原创群赛3(部分题解)
    vfor实现双层循环嵌套
    vue获取当前时间并实时刷新时间
    vue+element ui实现左侧导航栏动态路由跳转
  • 原文地址:https://www.cnblogs.com/fantiantian/p/9340187.html
Copyright © 2020-2023  润新知