模拟namenode崩溃，将name目录的内容全部删除，然后通过secondary namenode恢复namenode。
环境：OS：Centos 6.5 x64 & Soft：Hadoop 1.2.1
1、进入name目录下，删除name目录内容。
[huser@master name]$ pwd /home/huser/hadoop/tmp/dfs/name [huser@master name]$ ll drwxrwxr-x 2 huser huser 4096 4月 16 20:16 current drwxrwxr-x 2 huser huser 4096 4月 16 17:24 image -rw-rw-r-- 1 huser huser 0 4月 16 20:10 in_use.lock drwxrwxr-x 2 huser huser 4096 4月 16 18:55 previous.checkpoint [huser@master name]$ rm -R * [huser@master name]$ ls
2、停止集群，然后重启集群，发现nameNode失败。
[huser@master hadoop-1.2.1]$ bin/stop-all.sh [huser@master hadoop-1.2.1]$ bin/start-all.sh [huser@master hadoop-1.2.1]$ jps 7160 SecondaryNameNode 7229 JobTracker 7369 Jps
3、停止集群格式化namenode。
[huser@master hadoop-1.2.1]$ bin/stop-all.sh [huser@master hadoop-1.2.1]$ bin/hadoop namenode -format 14/04/16 21:17:39 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = master/192.168.1.115 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 1.2.1 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013 STARTUP_MSG: java = 1.7.0_51 ************************************************************/ Re-format filesystem in /home/huser/hadoop/tmp/dfs/name ? (Y or N) Y 14/04/16 21:17:42 INFO util.GSet: Computing capacity for map BlocksMap 14/04/16 21:17:42 INFO util.GSet: VM type = 64-bit 14/04/16 21:17:42 INFO util.GSet: 2.0% max memory = 1013645312 14/04/16 21:17:42 INFO util.GSet: capacity = 2^21 = 2097152 entries 14/04/16 21:17:42 INFO util.GSet: recommended=2097152, actual=2097152 14/04/16 21:17:43 INFO namenode.FSNamesystem: fsOwner=huser 14/04/16 21:17:43 INFO namenode.FSNamesystem: supergroup=supergroup 14/04/16 21:17:43 INFO namenode.FSNamesystem: isPermissionEnabled=true 14/04/16 21:17:43 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100 14/04/16 21:17:43 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s) 14/04/16 21:17:43 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0 14/04/16 21:17:43 INFO namenode.NameNode: Caching file names occuring more than 10 times 14/04/16 21:17:43 INFO common.Storage: Image file /home/huser/hadoop/tmp/dfs/name/current/fsimage of size 111 bytes saved in 0 seconds. 14/04/16 21:17:43 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/home/huser/hadoop/tmp/dfs/name/current/edits 14/04/16 21:17:43 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/home/huser/hadoop/tmp/dfs/name/current/edits 14/04/16 21:17:44 INFO common.Storage: Storage directory /home/huser/hadoop/tmp/dfs/name has been successfully formatted. 14/04/16 21:17:44 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at master/192.168.1.115 ************************************************************/
4、从datanode节点获取namespace的ID。
[huser@master hadoop-1.2.1]$ ssh slave1 [huser@slave1 current]$ pwd /home/huser/hadoop/tmp/dfs/data/current [huser@slave1 current]$ ll -rw-rw-r-- 1 huser huser 49184 4月 16 18:43 blk_-1800088935645150399 -rw-rw-r-- 1 huser huser 395 4月 16 18:43 blk_-1800088935645150399_1013.meta -rw-rw-r-- 1 huser huser 25 4月 16 18:43 blk_269963827714855400 -rw-rw-r-- 1 huser huser 11 4月 16 18:43 blk_269963827714855400_1014.meta -rw-rw-r-- 1 huser huser 16353 4月 16 18:43 blk_4611281727215307463 -rw-rw-r-- 1 huser huser 135 4月 16 18:43 blk_4611281727215307463_1015.meta -rw-rw-r-- 1 huser huser 769 4月 16 19:32 dncp_block_verification.log.curr -rw-rw-r-- 1 huser huser 158 4月 16 19:51 VERSION [huser@slave1 current]$ cat VERSION #Wed Apr 16 19:51:23 CST 2014 namespaceID=589801292 storageID=DS-1065963269-192.168.1.111-50010-1397640950581 cTime=0 storageType=DATA_NODE layoutVersion=-41
5、修改namenode的VERSION文件中namespaceID。
[huser@slave1 current]$ exit logout [huser@master current]$ pwd /home/huser/hadoop/tmp/dfs/name/current [huser@master current]$ vi VERSION #Wed Apr 16 21:17:43 CST 2014 namespaceID=589801292 cTime=0 storageType=NAME_NODE layoutVersion=-41
6、删除namenode节点下的fsinage文件。
[huser@master current]$ rm fsimage [huser@master current]$ ll -rw-rw-r-- 1 huser huser 4 4月 16 21:17 edits -rw-rw-r-- 1 huser huser 8 4月 16 21:17 fstime -rw-rw-r-- 1 huser huser 100 4月 16 21:30 VERSION
7、复制secondarynamenode节点的fsimage文件到namenode节点下。
[huser@master current]$ pwd /home/huser/hadoop/tmp/dfs/namesecondary/current [huser@master current]$ ll -rw-rw-r-- 1 huser huser 4 4月 16 20:16 edits -rw-rw-r-- 1 huser huser 2259 4月 16 20:16 fsimage -rw-rw-r-- 1 huser huser 8 4月 16 20:16 fstime -rw-rw-r-- 1 huser huser 100 4月 16 20:16 VERSION [huser@master current]$ cp fsimage /home/huser/hadoop/tmp/dfs/name/current/ [huser@master current]$ cd /home/huser/hadoop/tmp/dfs/name/current/ [huser@master current]$ ll -rw-rw-r-- 1 huser huser 4 4月 16 21:17 edits -rw-rw-r-- 1 huser huser 2259 4月 16 21:37 fsimage -rw-rw-r-- 1 huser huser 8 4月 16 21:17 fstime -rw-rw-r-- 1 huser huser 100 4月 16 21:30 VERSION
8、重启集群并检查运行情况。
[huser@master hadoop-1.2.1]$ jps 7927 SecondaryNameNode 7773 NameNode 8017 JobTracker 8123 Jps