• hadoopMR自定义输入类型


    hadoop中的输入输出数据类型:

    BooleanWritable:标准布尔型数值

    ByteWritable:单字节数值

    DoubleWritable:双字节数值

    FloatWritable:浮点数

    常用的:

    IntWritable:整型数

    LongWritable:长整型数

    Text:使用UTF8格式存储的文本

    NullWritable:当<key, value>中的key或value为空时使用

    hadoop中的数据类型都实现了writable接口,以便这些类型的数据可以在网络传输和文件存储

    自定义数据类型:

    ①实现WritableComparable接口,实现write和readFields方法,完成可序列化和反序列化

    ②如果该数据类型需要有比较顺序时,实现CompareTo()方法

    ③必须声明一个无参构造方法,为了方便反射,创建对象

    package com.neworigin.temperature;

    import java.io.DataInput;
    import java.io.DataOutput;
    import java.io.IOException;

    import org.apache.hadoop.io.WritableComparable;

    public class Weather implements WritableComparable <Weather> {
    private int year;
    private int month;
    private int day;
    private int wd;

    public int getYear() {
    return year;
    }
    public void setYear(int year) {
    this.year = year;
    }
    public int getMonth() {
    return month;
    }
    public void setMonth(int month) {
    this.month = month;
    }
    public int getDay() {
    return day;
    }
    public void setDay(int day) {
    this.day = day;
    }
    public int getWd() {
    return wd;
    }
    public void setWd(int wd) {
    this.wd = wd;
    }
    public void write(DataOutput out) throws IOException {
    out.writeInt(year);
    out.writeInt(month);
    out.writeInt(day);
    out.writeInt(wd);

    }
    public void readFields(DataInput in) throws IOException {
    year=in.readInt();
    month=in.readInt();
    day=in.readInt();
    wd=in.readInt();
    }
    //将年月日按升序排序
    public int compareTo(Weather w) {
    int c1 = Integer.compare(this.year,w.getYear());
    if (c1==0)
    {
    int c2 = Integer.compare(this.month,w.getMonth());
    if (c2==0)
    {
    return Integer.compare(this.wd,w.getWd());
    }
    return c2;
    }
    return c1;
    }

    }

  • 相关阅读:
    数据库 proc编程三
    数据库 Proc编程二
    数据库 Proc编程一
    数据库 Oracle数据库对象二
    Your local changes to the following files would be overwritten by merge: ... Please, commit your changes or stash them before you can merge
    生活感悟关键字
    科3
    NGINX 健康检查和负载均衡机制分析
    django模板里关闭特殊字符转换,在前端以html语法渲染
    django 获取前端获取render模板渲染后的html
  • 原文地址:https://www.cnblogs.com/chengdonghui/p/7831127.html
Copyright © 2020-2023  润新知