• 1 weekend110的hdfs源码跟踪之打开输入流 + hdfs源码跟踪之打开输入流总结


      

    3种形式的元数据,fsimage是在磁盘上,meta.data是在内存上,

    我们继续,前面呢,断点是打在这一行代码处,

    FileSystem fs = FileSystem.get(conf);

    weekend110的hdfs下载数据源码跟踪铺垫  +  hdfs下载数据源码分析-getFileSystem

    现在,开始weekend110的hdfs源码跟踪之打开输入流

    1、  清掉之前,所有的断点,

    dfs是fs的成员,

    此刻,断点过不去了,为什么?因为,这需要在服务器上运行。

    由此可见,newInfo里面就是block信息。

    LocatedBlocks{

      fileLength=143588167

      underConstruction=false

      blocks=[LocatedBlock{BP-168439838-192.168.80.110-1468829197712:blk_1073741856_1032; getBlockSize()=134217728; corrupt=false; offset=0; locs=[192.168.80.110:50010]}, LocatedBlock{BP-168439838-192.168.80.110-1468829197712:blk_1073741857_1033; getBlockSize()=9370439; corrupt=false; offset=134217728; locs=[192.168.80.110:50010]}]

      lastLocatedBlock=LocatedBlock{BP-168439838-192.168.80.110-1468829197712:blk_1073741857_1033; getBlockSize()=9370439; corrupt=false; offset=134217728; locs=[192.168.80.110:50010]}

      isLastBlockComplete=true}

    BP-168439838-192.168.80.110-1468829197712

    这是机器集群的信息,

    [hadoop@weekend110 app]$ cd hadoop-2.4.1/

    [hadoop@weekend110 hadoop-2.4.1]$ ll

    total 60

    drwxr-xr-x. 2 hadoop hadoop  4096 Jul 18 15:48 bin

    drwxrwxr-x. 4 hadoop hadoop  4096 Jul 18 20:37 data

    drwxr-xr-x. 3 hadoop hadoop  4096 Jun 21  2014 etc

    drwxr-xr-x. 2 hadoop hadoop  4096 Jun 21  2014 include

    drwxr-xr-x. 3 hadoop hadoop  4096 Jun 21  2014 lib

    drwxr-xr-x. 2 hadoop hadoop  4096 Jun 21  2014 libexec

    -rw-r--r--. 1 hadoop hadoop 15458 Jun 21  2014 LICENSE.txt

    drwxrwxr-x. 3 hadoop hadoop  4096 Jul 22 11:20 logs

    -rw-r--r--. 1 hadoop hadoop   101 Jun 21  2014 NOTICE.txt

    -rw-r--r--. 1 hadoop hadoop  1366 Jun 21  2014 README.txt

    drwxr-xr-x. 2 hadoop hadoop  4096 Jun 21  2014 sbin

    drwxr-xr-x. 4 hadoop hadoop  4096 Jun 21  2014 share

    [hadoop@weekend110 hadoop-2.4.1]$ clear

    [hadoop@weekend110 hadoop-2.4.1]$ cd data/

    [hadoop@weekend110 data]$ ll

    total 8

    drwxrwxr-x.  5 hadoop hadoop 4096 Jul 18 16:15 dfs

    drwxr-xr-x. 12 hadoop hadoop 4096 Jul 22 11:20 nm-local-dir

    [hadoop@weekend110 data]$ cd dfs/

    [hadoop@weekend110 dfs]$ ll

    total 12

    drwx------. 3 hadoop hadoop 4096 Jul 18 23:07 data

    drwxrwxr-x. 3 hadoop hadoop 4096 Jul 18 23:06 name

    drwxrwxr-x. 3 hadoop hadoop 4096 Jul 18 23:07 namesecondary

    [hadoop@weekend110 dfs]$ cd data/

    [hadoop@weekend110 data]$ ll

    total 8

    drwxrwxr-x. 3 hadoop hadoop 4096 Jul 18 16:14 current

    -rw-rw-r--. 1 hadoop hadoop   16 Jul 22 11:19 in_use.lock

    [hadoop@weekend110 data]$ cd current/

    [hadoop@weekend110 current]$ ll

    total 8

    drwx------. 4 hadoop hadoop 4096 Jul 22 11:20 BP-168439838-192.168.80.110-1468829197712

    -rw-rw-r--. 1 hadoop hadoop  229 Jul 22 11:19 VERSION

    [hadoop@weekend110 current]$ cd BP-168439838-192.168.80.110-1468829197712/

    [hadoop@weekend110 BP-168439838-192.168.80.110-1468829197712]$ ll

    total 12

    drwxrwxr-x. 4 hadoop hadoop 4096 Jul 18 23:02 current

    -rw-rw-r--. 1 hadoop hadoop    0 Jul 22 11:20 dncp_block_verification.log.curr

    -rw-rw-r--. 1 hadoop hadoop  348 Jul 22 11:20 dncp_block_verification.log.prev

    drwxrwxr-x. 2 hadoop hadoop 4096 Jul 22 11:19 tmp

    [hadoop@weekend110 BP-168439838-192.168.80.110-1468829197712]$ cd current/

    [hadoop@weekend110 current]$ ll

    total 16

    -rw-rw-r--. 1 hadoop hadoop   23 Jul 18 23:02 dfsUsed

    drwxrwxr-x. 2 hadoop hadoop 4096 Jul 20 15:20 finalized

    drwxrwxr-x. 2 hadoop hadoop 4096 Jul 20 15:20 rbw

    -rw-rw-r--. 1 hadoop hadoop  131 Jul 22 11:19 VERSION

    [hadoop@weekend110 current]$ cd finalized/

    [hadoop@weekend110 finalized]$ ll

    total 141340

    -rw-rw-r--. 1 hadoop hadoop 134217728 Jul 19 09:59 blk_1073741856

    -rw-rw-r--. 1 hadoop hadoop   1048583 Jul 19 09:59 blk_1073741856_1032.meta

    -rw-rw-r--. 1 hadoop hadoop   9370439 Jul 19 09:59 blk_1073741857

    -rw-rw-r--. 1 hadoop hadoop     73215 Jul 19 09:59 blk_1073741857_1033.meta

    -rw-rw-r--. 1 hadoop hadoop       168 Jul 19 13:05 blk_1073741858

    -rw-rw-r--. 1 hadoop hadoop        11 Jul 19 13:05 blk_1073741858_1034.meta

    -rw-rw-r--. 1 hadoop hadoop        71 Jul 20 15:20 blk_1073741866

    -rw-rw-r--. 1 hadoop hadoop        11 Jul 20 15:20 blk_1073741866_1042.meta

    [hadoop@weekend110 finalized]$

    以上是weekend110的hdfs源码跟踪之打开输入流

    下面,继续,weekend110的hdfs源码跟踪之打开输入流总结

  • 相关阅读:
    Oracle数据库的经典问题 snapshot too old是什么原因引起的
    在服务器上排除问题的头五分钟
    MySQL的redo log结构和SQL Server的log结构对比
    MySQL优化---DBA对MySQL优化的一些总结
    事务分类
    扩展HT for Web之HTML5表格组件的Renderer和Editor
    iOS平台快速发布HT for Web拓扑图应用
    HT for Web的HTML5树组件延迟加载技术实现
    Zip 压缩、解压技术在 HTML5 浏览器中的应用
    百度地图、ECharts整合HT for Web网络拓扑图应用
  • 原文地址:https://www.cnblogs.com/zlslch/p/5894914.html
Copyright © 2020-2023  润新知