Hpsa OOM错误:
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190266] active_anon:850084 inactive_anon:1372017 isolated_anon:0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190267] active_file:3177085 inactive_file:1515641 isolated_file:0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190269] unevictable:0 dirty:67378 writeback:0 unstable:0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190270] free:448723 slab_reclaimable:293611 slab_unreclaimable:346392
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190272] mapped:6605 shmem:1750450 pagetables:16953 bounce:0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190279] Node 0 DMA free:15728kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0k
B present:15324kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190301] lowmem_reserve[]: 0 1814 15954 15954
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190326] Node 0 DMA32 free:259816kB min:5084kB low:6352kB high:7624kB active_anon:115648kB inactive_anon:229520kB active_file:438304kB inactive_file:193924kB unevictable:0kB isolate
d(anon):0kB isolated(file):0kB present:1857552kB mlocked:0kB dirty:11844kB writeback:0kB mapped:592kB shmem:277680kB slab_reclaimable:115160kB slab_unreclaimable:104852kB kernel_stack:1136kB pagetables:1192kB unstable:0kB bounce:0kB writ
eback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190348] lowmem_reserve[]: 0 0 14140 14140
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190373] Node 0 Normal free:1204844kB min:39656kB low:49568kB high:59484kB active_anon:1553884kB inactive_anon:2515388kB active_file:5868168kB inactive_file:2222352kB unevictable:0k
B isolated(anon):0kB isolated(file):0kB present:14479360kB mlocked:0kB dirty:125688kB writeback:0kB mapped:10824kB shmem:3224972kB slab_reclaimable:394928kB slab_unreclaimable:552088kB kernel_stack:4312kB pagetables:45636kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190396] lowmem_reserve[]: 0 0 0 0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190420] Node 1 Normal free:314064kB min:45320kB low:56648kB high:67980kB active_anon:1731220kB inactive_anon:2743160kB active_file:6401336kB inactive_file:3646288kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:16547836kB mlocked:0kB dirty:131980kB writeback:0kB mapped:15004kB shmem:3499148kB slab_reclaimable:664356kB slab_unreclaimable:728628kB kernel_stack:3144kB pagetables:20984kB unstable:0kB b
ounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190442] lowmem_reserve[]: 0 0 0 0
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190466] Node 0 DMA: 2*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15728kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190525] Node 0 DMA32: 57072*4kB 2712*8kB 520*16kB 49*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 259936kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190591] Node 0 Normal: 246319*4kB 19953*8kB 3232*16kB 313*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1206628kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190649] Node 1 Normal: 24071*4kB 17889*8kB 4013*16kB 402*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 316596kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190706] 6444684 total pagecache pages
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190711] 1209 pages in swap cache
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190718] Swap cache stats: add 20603325, delete 20602116, find 248979552/249268961
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190724] Free swap = 1280992kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.190729] Total swap = 6143992kB
Aug 24 03:47:36 LIVE-117 kernel: [3983420.385835] 8355838 pages RAM
Aug 24 03:47:36 LIVE-117 kernel: [3983420.385841] 178918 pages reserved
Aug 24 03:47:36 LIVE-117 kernel: [3983420.385844] 4937078 pages shared
Aug 24 03:47:36 LIVE-117 kernel: [3983420.385846] 3055834 pages non-shared
Aug 24 03:47:36 LIVE-117 kernel: [3983420.386050] hpsa 0000:0a:00.0: out of memory
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914301] .hpacucli: page allocation failure. order:4, mode:0xd0
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914310] Pid: 12573, comm: .hpacucli Not tainted 2.6.32-279.9.1.el6.x86_64 #1
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914315] Call Trace:
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914335] [<ffffffff8112789f>] ? __alloc_pages_nodemask+0x77f/0x940
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914347] [<ffffffff811620a2>] ? kmem_getpages+0x62/0x170
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914355] [<ffffffff81162cba>] ? fallback_alloc+0x1ba/0x270
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914363] [<ffffffff8116270f>] ? cache_grow+0x2cf/0x320
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914370] [<ffffffff81162a39>] ? ____cache_alloc_node+0x99/0x160
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914378] [<ffffffff8116327a>] ? kmem_cache_alloc_notrace+0xfa/0x130
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914402] [<ffffffffa005746d>] ? hpsa_update_scsi_devices+0x8d/0x1c60 [hpsa]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914410] [<ffffffff8115c51a>] ? alloc_pages_current+0xaa/0x110
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914420] [<ffffffffa00f9b6b>] ? sg_build_indirect+0x18b/0x320 [sg]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914432] [<ffffffffa0059147>] ? hpsa_scan_start+0x107/0x150 [hpsa]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914441] [<ffffffffa00f9e1a>] ? sg_open+0x9a/0x630 [sg]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914450] [<ffffffff81193bca>] ? dput+0x9a/0x150
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914461] [<ffffffffa005a9cc>] ? hpsa_ioctl+0x12c/0x8d0 [hpsa]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914470] [<ffffffff8117f3b5>] ? chrdev_open+0x125/0x230
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914478] [<ffffffff8117f290>] ? chrdev_open+0x0/0x230
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914488] [<ffffffff81365bd3>] ? scsi_ioctl+0xc3/0x380
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914497] [<ffffffffa00faebe>] ? sg_ioctl+0x20e/0xd20 [sg]
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914506] [<ffffffff8118c500>] ? do_filp_open+0x780/0xd60
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914513] [<ffffffff8118e25c>] ? vfs_ioctl+0x7c/0xa0
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914520] [<ffffffff8118e3a4>] ? do_vfs_ioctl+0x84/0x580
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914527] [<ffffffff8118e921>] ? sys_ioctl+0x81/0xa0
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914538] [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
Aug 24 03:52:43 LIVE-117 kernel: [3983726.914542] Mem-Info:
解决办法:
系统无法分配连续的内存地址,增加系统预留内存可降低该问题的出现概率。
# echo "vm.min_free_kbytes = 1048576" >> /etc/sysctl.conf /预留1G。
# sysctl -p