• hadoop InputSplit


    /**
     * <code>InputSplit</code> represents the data to be processed by an 
     * individual {@link Mapper}. 
     * InputSplit 代表可以被Mapper处理的数据
     * <p>Typically, it presents a byte-oriented view on the input and is the 
     * responsibility of {@link RecordReader} of the job to process this and present
     * a record-oriented view.
     *  
     * @see InputFormat
     * @see RecordReader
     */
    @InterfaceAudience.Public
    @InterfaceStability.Stable
    public abstract class InputSplit {
      /**
       * Get the size of the split, so that the input splits can be sorted by size.
       * @return the number of bytes in the split
       * @throws IOException
       * @throws InterruptedException
       * split的长度用byte表示
       */
      public abstract long getLength() throws IOException, InterruptedException;
    
      /**
       * Get the list of nodes by name where the data for the split would be local.
       * The locations do not need to be serialized.
       * 获取split所在的节点
       * @return a new array of the node nodes.
       * @throws IOException
       * @throws InterruptedException
       */
      public abstract 
        String[] getLocations() throws IOException, InterruptedException;
      
      /**
       * Gets info about which nodes the input split is stored on and how it is
       * stored at each location.
       * 返回split所在的节点信息以及在该节点上如何存储 memory
       * @return list of <code>SplitLocationInfo</code>s describing how the split
       *    data is stored at each location. A null value indicates that all the
       *    locations have the data stored on disk.
       * @throws IOException
       */
      @Evolving
      public SplitLocationInfo[] getLocationInfo() throws IOException {
        return null;
      }
    }
  • 相关阅读:
    对话框事件
    C# 实现retry
    Linq中的group by多表多字段,Sum求和
    词干提取(stemming)和词形还原(lemmatization)
    GIT简单应用实例
    像素、分辨率、dpi的区别和联系
    二值图像、灰度图像、彩色图像
    更好的做科研
    数据挖掘竞赛利器-Stacking和Blending方式
    如何成为一名成功的博士生
  • 原文地址:https://www.cnblogs.com/fantiantian/p/9340187.html
Copyright © 2020-2023  润新知