hadoop集群由于磁盘损坏、节点挂机、机架掉电难免造成一些文件块损坏,通过hdfs fsck命令,检查HDFS上⽂件和⽬录的健康状态、获取⽂件的block信息和位置信息
如,出现报错There are 41 missing blocks. The following files may be corrupted
有坏块启动不起来直接
hdfs fsck / -delete
直接删除命令,只会删除坏块,正常的块不会删除。
hdfs fsck <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-includeSnapshots] [-storagepolicies] [-blockId <blk_Id>]
<path> 检测的目录 -move 把损坏的文件移动到/lost+found -delete 直接删除损坏的文件 -files 打印被检测的文件 -openforwrite print out files opened for write -includeSnapshots include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it -list-corruptfileblocks print out list of missing blocks and files they belong to -blocks 打印 block 的信息 -locations 打印 block 的位置,即在哪个节点 -racks print out network topology for data-node locations -storagepolicies 打印 block 存储的策略信息 -blockId 打印指定blockId所属块的状况 -replicaDetails print out each replica details