• Hadoop源码分析3: 序列化


    1. Java 默认序列化

    public class Block1 implements Serializable {
        private static finallong serialVersionUID = 1276464248616673062L;
        privatelong blockId;
        privatelong numBytes;
        privatelong generationStamp; 

        publicBlock1() { 
        }

        publicBlock1(final long blkid, final long len, final longgenerationStamp) {
          this.blockId = blkid;
          this.numBytes = len;
          this.generationStamp =genStamp;
       } 
        @Override
        publicString toString() {
          return "Block1 [blockId=" +blockId + ", numBytes=" + numBytes
              +", generationStamp=" + generationStamp + "]";
       } 
          // setter getter...............

    }

    2.Hadoop 序列化

    public class Block2 implements Writable 

           static { // register a ctor
       WritableFactories.setFactory(Block2.class,new WritableFactory() {
          public Writable newInstance(){
              return new Block2();
          }
          });
        }

        privatelong blockId;
        privatelong numBytes;
        privatelong generationStamp;

        publicBlock2() { 
        }

        publicBlock2(final long blkid, final long len, final longgenerationStamp) {
          this.blockId = blkid;
          this.numBytes = len;
          this.generationStamp =genStamp;
        }
     
        publicvoid set(long blkid, long len, long genStamp) {
          this.blockId = blkid;
          this.numBytes = len;
          this.generationStamp =genStamp;
       } 
     
       @Override
       publicvoid write(DataOutput out) throws IOException {
          out.writeLong(blockId);
          out.writeLong(numBytes);
          out.writeLong(generationStamp);
       }

       @Override
       publicvoid readFields(DataInput in) throws IOException{
          this.blockId = in.readLong();
          this.numBytes =in.readLong();
          this.generationStamp =in.readLong();
          if (numBytes < 0) {
              throw new IOException("Unexpected blocksize: " + numBytes);
          }
       
           // settergetter...............
      @Override
        publicString toString() {
          return "Block2 [blockId=" +blockId + ", numBytes=" + numBytes
             + ", generationStamp=" +generationStamp + "]";
       } 
    }

    2. 比较 

    public class BlockMain {

        publicstatic void main(String[] args) throws IOException {

          Block1 block1 = newBlock1(7806259420524417791L, 39447755L, 56736651L);
           ByteArrayOutputStream out1 = newByteArrayOutputStream();
           ObjectOutputStream objOut1 = newObjectOutputStream(out1);
           objOut1.writeObject(block1);
           objOut1.close();
           System.out.println("writeObject (block1) with "+ out1.size() + " bytes:");
           SerializationExample.print16(out1.toByteArray(),out1.size());

           System.out.println();

           Block2 block2 = new Block2(7806259420524417791L,39447755L, 56736651L);
           ByteArrayOutputStream out2 = newByteArrayOutputStream();
           ObjectOutputStream objOut2 = newObjectOutputStream(out2);
           block2.write(objOut2);
           objOut2.close();
           System.out.println("writeObject (block2) with "+ out2.size() + " bytes:");
           SerializationExample.print16(out2.toByteArray(),out2.size());

        }
    }

    打印:

    writeObject (block1) with 113 bytes:
    AC ED 00 05 73 72 00 1D  6F 72 67 2E 68 61 646F    ....sr.. org.hado
    6F 70 69 6E 74 65 72 6E  61 6C 2E 73 65 72 2E42    opintern al.ser.B
    6C 6F 63 6B 31 11 B6 E9  78 9B 48 5F 26 02 0003    lock1... x.H_&...
    4A 00 07 62 6C 6F 63 6B  49 64 4A 00 0F 67 656E    J..block IdJ..gen
    65 72 61 74 69 6F 6E 53  74 61 6D 70 4A 00 086E    erationS tampJ..n
    75 6D 42 79 74 65 73 78  70 6C 55 67 95 68 E792    umBytesx plUg.h..
    FF 00 00 00 00 03 61 BB  8B 00 00 00 00 02 59EC    ......a. ......Y.
    CB                                          .

    writeObject (block2) with 30 bytes:
    AC ED 00 05 77 18 6C 55  67 95 68 E7 92 FF 0000    ....w.lU g.h.....
    00 00 02 59 EC CB 00 00  00 00 03 61 BB 8B        ...Y.... ...a..

    可见使用Writable的数据量比Serializable 小得多: 30/113 约为 26.6%。

    3.文件读写对象

    public class BlockMain {

    public static void main(String[] args) throwsIOException, ClassNotFoundException {

        Filefile =new File("d:\1.txt");
       FileOutputStream fos = newFileOutputStream(file);
       ObjectOutputStream objOut = newObjectOutputStream(fos);
       FileInputStream fis = newFileInputStream(file);
       ObjectInputStream objIn = newObjectInputStream(fis);
        Block1block1 = new Block1(7806259420524417791L, 39447755L,56736651L);
       objOut.writeObject(block1);
       objOut.close();

       Block1 block1Retrive =(Block1) objIn.readObject();
       System.out.println("writeObject(block1) : " + block1Retrive + " ");
       objIn.close();
       System.out.println();
       
        file=new File("d:\2.txt");
        fos =new FileOutputStream(file);
        objOut= new ObjectOutputStream(fos);
        fis =new FileInputStream(file);
        objIn =new ObjectInputStream(fis);
        Block2block2 = new Block2(7806259420524417791L, 39447755L,56736651L);
       block2.write(objOut);
       objOut.close();

       Block2 block2Retrive = newBlock2();
       block2Retrive.readFields(objIn);
       objIn.close();
       System.out.println("writeObject(block2) : " + block2Retrive + " ");
     
    }

    }
    打印:
    writeObject (block1) : Block1 [blockId=7806259420524417791,numBytes=39447755,generationStamp=56736651] 

    writeObject (block2) : Block2 [blockId=7806259420524417791,numBytes=39447755,generationStamp=56736651] 

    文件内容:
    d:1.txt 113 字节


    d:2.txt 30 字节


  • 相关阅读:
    结构体初体验
    hdu 2068 RPG的错排
    hdu 2160 母猪的故事
    python爬虫从入门到放弃(一)——试用bs4, request爬百度股票
    python web开发c6——阿里云上ubuntu+flask+gunicorn+nginx服务器部署(一)简单测试
    python web开发——c5 图片上传 flask-uploads 云储存sdk
    python web开发——c3 数据库交互和flask-SQLALchemy
    python web开发——c2 flask框架和flask_script
    javar入门——基本数据类型
    java入门——面向对象
  • 原文地址:https://www.cnblogs.com/leeeee/p/7276540.html
Copyright © 2020-2023  润新知