Java读取大文件的操作【转】

转自http://aronlulu.iteye.com/blog/1018370

读取文件大小：1.45G 
第一种，OldIO: 

public static void oldIOReadFile() throws IOException{
    BufferedReader br = new BufferedReader(new FileReader("G://lily_947.txt"));
    PrintWriter pw = new PrintWriter("G://oldIO.tmp");
    char[] c = new char[100*1024*1024];
    for(;;){
        if(br.read(c)!=-1){
            pw.print(c);
        }else{
            break;
        }
    }
    pw.close();
    br.close();
}

耗时70.79s 


第二种，newIO: 

public static void newIOReadFile() throws IOException{
        FileChannel read = new RandomAccessFile("G://lily_947.txt","r").getChannel();
        FileChannel writer = new RandomAccessFile("G://newIO.tmp","rw").getChannel();
        ByteBuffer bb = ByteBuffer.allocate(200*1024*1024);
        while(read.read(bb)!=-1){
            bb.flip();
            writer.write(bb);
            bb.clear();
        }
        read.close();
        writer.close();

    }

耗时47.24s 


第三种，RandomAccessFile: 

public static void randomReadFile() throws IOException{
        RandomAccessFile read = new RandomAccessFile("G://lily_947.txt","r");
        RandomAccessFile writer = new RandomAccessFile("G://random.tmp","rw");
        byte[] b = new byte[200*1024*1024];
        while(read.read(b)!=-1){
            writer.write(b);
        }
        writer.close();
        read.close();
    }

耗时46.65 

第四种，MappedByteBuffer： 

public static void mappedBuffer() throws IOException{
        FileChannel read = new FileInputStream("G://lily_947.txt").getChannel();
        FileChannel writer = new RandomAccessFile("G://buffer.tmp","rw").getChannel();
        long i = 0;
        long size = read.size()/30;
        ByteBuffer bb,cc = null;
        while(i<read.size()&&(read.size()-i)>size){
            bb = read.map(FileChannel.MapMode.READ_ONLY, i, size);
            cc = writer.map(FileChannel.MapMode.READ_WRITE, i, size);
            cc.put(bb);
            i+=size;
            bb.clear();
            cc.clear();
        }
        bb = read.map(FileChannel.MapMode.READ_ONLY, i, read.size()-i);
        cc.put(bb);
        bb.clear();
        cc.clear();
        read.close();
        writer.close();

    }

耗时：36 

前三种读法对应的资源占用图如下： 
相对于最后一种内存直接映射方式前面的测试其实无意义，基本秒杀。。。。。 
对于很大的文件直接分块映射时内存会不够，这是因为MappedByteBuffer未被释放造成的，sun未提供直接回收MappedByteBuffer区域的方法，这个时候有两种方法解决，第一种比较愚笨的： 

          System.gc();
         System.runFinalization();
         try {
    Thread.sleep(3000);
} catch (InterruptedException e) {

    e.printStackTrace();
}

第二种网上找来的，利用反射调用clean方法： 

public static void unmap(final MappedByteBuffer buffer) {
        if (buffer == null) {
            return;
        }
        AccessController.doPrivileged(new PrivilegedAction<Object>() {
            public Object run() {
                try {
                    Method getCleanerMethod = buffer.getClass().getMethod("cleaner", new Class[0]);
                    if (getCleanerMethod != null) {
                        getCleanerMethod.setAccessible(true);
                        Object cleaner = getCleanerMethod.invoke(buffer, new Object[0]);
                        Method cleanMethod = cleaner.getClass().getMethod("clean", new Class[0]);
                        if (cleanMethod != null) {
                            cleanMethod.invoke(cleaner, new Object[0]);
                        }
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
                return null;
            }

        });
    }

以上两种方法感觉都别扭，还有就是可以自己分割成物理文件再循环调用，这个也不太美观。 
速度也会减慢好多。