HDFS文件操作

文件操作

Hadoop-fs 帮助文档

[root@hadoop01 current]# hadoop fs –help

-ls [-d] [-h] [-R] [<path> ...] :

  List the contents that match the specified file pattern. If path is not

  specified, the contents of /user/<currentUser> will be listed. Directory entries

  are of the form:

        permissions - userId groupId sizeOfDirectory(in bytes)

  modificationDate(yyyy-MM-dd HH:mm) directoryName

  

  and file entries are of the form:

        permissions numberOfReplicas userId groupId sizeOfFile(in bytes)

  modificationDate(yyyy-MM-dd HH:mm) fileName

                                                                                 

  -d  Directories are listed as plain files.                                     

  -h  Formats the sizes of files in a human-readable fashion rather than a number

      of bytes.                                                                  

  -R  Recursively list the contents of directories.

查看所有文件

hadoop fs -ls /

下载文件

hadoop fs -get hdfs://hadoop01:9000/jdk* /home/software/

从hdfs的根目录下载以jdk开头的文件，下载到/home/software目录

copyFromLocal

从本地copy一个文件到hdfs系统中

[root@hadoop01 software]# hadoop fs -copyFromLocal /home/software/jdk-7u55-linux-i586.tar.gz  /

说明：源路径只能是本地的一个文件

put

Hadoop fs –put /home/software/a.txt /input

把/home/software下面的a.txt文件上传到hdfs中的input文件夹中

文件个数

hadoop fs -count /

统计文件大小

这种情况下计算出来的大小就一个数据

[root@hadoop01 software]# hadoop fs -du -s -h hdfs://hadoop01:9000/

Java HotSpot(TM) Client VM warning: You have loaded library /home/app/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

15/03/24 01:07:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

133.0 M  hdfs://hadoop01:9000/

这种情况下计算出来的大小是每一个文件的

[root@hadoop01 software]# hadoop fs -du -s -h hdfs://hadoop01:9000/*

Java HotSpot(TM) Client VM warning: You have loaded library /home/app/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

15/03/24 01:07:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

133.0 M  hdfs://hadoop01:9000/jdk-7u55-linux-i586.tar.gz

9  hdfs://hadoop01:9000/slaves

删除文件

[root@hadoop08 ~]# hadoop fs -rm -r /output