【转载+部分翻译】Win32 调试时堆内部结构

原文地址：http://www.nobugs.org/developer/win32/debug_crt_heap.html

调试程序时有时会遇到HEAP CORRUPTION DETECTED，但是其内部是怎么检测堆溢出的呢。该文章就描述了其内部实现细节

When you compile programs with DevStudio in debug mode, all of your calls to malloc() and free() use a special “debug” implementation.

Rather than being blazingly fast, the debug heap concerns itself with spotting heap errors. It achieves this by surrounding your memory blocks with guard bytes (aka “no mans land”, 0xFD) so that it can detect buffer overruns and underruns. It also initialises newly allocated memory to a fixed value (0xCD) to aid reproducability. Finally, it sets free()d blocks to a known value (0xDD) so that it can detect that people are writing through dangling pointers.

Mnemonics for remembering what each fill-pattern means:

• The newly allocated memory (0xCD) is Clean memory.（新申请的内存使用0xCD初始化
• The free()d memory (0xDD) is Dead memory.> 释放的空间使用0xDD覆盖
• The guard bytes (0xFD) are like Fences around your memory.> 申请的空间两边各有4个字节作为边界，用来检测堆溢出，使用0xFD初始化

The debug CRT heap defers most of the heavy work to the Win32 heap functions HeapAlloc() and HeapFree(). Therefore, you won’t see any first-fit or “buddy system” code in the debug CRT. The 4Gb virtual memory space which you process has is sliced up and managed by the Win32 heap inside kernel32.dll.

If you call malloc(8), the debug CRT will request a 48 byte block from HeapAlloc(). It needs the extra 40 bytes to store information about the memory blocks – such as the file/line where malloc() was called, and space for links to the next/prev heap block. In the table below, all of the debug CRT information is colored in shades of red.

HeapAlloc() itself needs bookkeeping information. The HeapAlloc(40) call will, in turn, reserve a total of 80 bytes from your process’s address space. Eight bytes of bookkeeping appear below the requested 40 bytes, and the other 32 bytes appear above it. In the table below, the Win32 heap bookkeeping is colored grey. The memory which you get back from HeapAlloc() is always initialised to the 4 byte pattern 0xBAADF00D.

malloc申请8个字节的空间（浅红色），实际使用48个字节，其中40个字节为控制信息（红色），而malloc中调用的HeapAlloc也会保留40个字节用来存储控制信息（灰色，8字节在下，另外32字节在上面），见下图（拷贝过来表颜色已经错了，可以看原链接的表格）

橙色的为边界

(As an aside, when you request pages from the VM manager via VirtualAlloc, they are initialized to zero, so HeapAlloc is actually doing additional work to re-initialize them to this pattern).

Once the debug CRT has got it’s 40 byte block, it will fill in it’s book-keeping information. The first 2 words are links to the previous and next blocks on the CRT heap. The choice of names is confusing, because the “next” pointer actually takes you the block which was allocated before this one chronologically, while the “previous” pointer takes you to the one allocatedsubsequently. The reason for the naming is that the linked list starts at the last-allocated block, and progresses back in time as you follow “next” links. The debug CRT heap also internally maintains pointers to the first and last blocks (_pFirstBlock and _pLastBlock) to allow heap-checking code to traverse all the blocks.

If the filename and line number of the malloc() call are known, they are stored in the next 2 words. Following that, the next word tells you how many bytes were requested. The next word gives a type field. Usually this is “1” which means a normal block, allocated by malloc/new. It will be “2” for blocks allocate by the CRT for its own internal purposes, and “0” for blocks which have been freed but not released back to the win32 heap (see below for more info). The final word is a simple counter which increases everytime an allocation is made.

Surrounding the 8-byte malloc()‘d memory there are areas of “no mans land”. These are filled with a known value (0xFD), and when the block is free()d, the CRT will check that they still have the right value. If they’ve changed, then your program contains an error. Unfortunately, the corruption may have happened a long time ago. You can use Purify or Boundschecker to stop at the corruption point, or if you don’t fancy spending any money, you can wait a few days until I write an article telling you how to do it using only a bit of ingenuity!

The eight bytes which were originally requested are initialised with 0xCD. If you see this pattern appearing in your variables, you have forgotten to initialise something.

When you call free() on the 8-byte block, the CRT sets the whole 48-byte block (including its bookkeeping) to 0xDDDDDDDD. This means that it can tell if the block gets subsequently altered (ie. via a dangling pointer) by checking that the pattern is still there.

At this point, two things can happen. Normally, HeapFree() will be called to return the block to the win32 heap. This causes the block to be overwritten with the win32 heap’s “freed memory” pattern, which is 0xFEEEFEEE. Note that the debug CRT does not maintain any “free lists” – all of that is done within the black box of HeapFree().

However, you can also tell the debug heap to keep hold of free()d blocks. You do this by passing the _CRTDBG_DELAY_FREE_MEM_DF flag to _CrtSetDbgFlag(). In this case, the debug CRT will keep hold of the block. This is useful if you are trying to track down a dangling pointer error, since memory blocks will not be reused and you should expect them to remain filled with 0xDDDDDDDD unless someone is writing to the free()d block. You can call _CrtCheckMemory() and it will tell you if any of these values have been tampered with.

Here’s an allocation I prepared earlier …

I called malloc(8) followed by free() and stepped through the CRT calls to see how the memory was changed. Read the columns from left to right, and you will see what values appear in memory at various stages during malloc() and free(). The call to malloc(8) returned a block at address 0×00321000, and I’ve included offsets from that address so that you can find out the information for one of your allocations. 

Address	Offset	After HeapAlloc()	After malloc()申请空间后	During free()释放空间后	After HeapFree()	Comments
0×00320FD8	-40	0×01090009	0×01090009	0×01090009	0×0109005A	Win32 heap info
0×00320FDC	-36	0×01090009	0×00180700	0×01090009	0×00180400	Win32 heap info
0×00320FE0	-32	0xBAADF00D	0×00320798	0xDDDDDDDD	0×00320448	Ptr to next CRT heap block (allocated earlier in time)  前一个块
0×00320FE4	-28	0xBAADF00D	0×00000000	0xDDDDDDDD	0×00320448	Ptr to prev CRT heap block (allocated later in time)   前一个块
0×00320FE8	-24	0xBAADF00D	0×00000000	0xDDDDDDDD	0xFEEEFEEE	Filename of malloc() call
0×00320FEC	-20	0xBAADF00D	0×00000000	0xDDDDDDDD	0xFEEEFEEE	Line number of malloc() call
0×00320FF0	-16	0xBAADF00D	0×00000008	0xDDDDDDDD	0xFEEEFEEE	Number of bytes to malloc()  长度
0×00320FF4	-12	0xBAADF00D	0×00000001	0xDDDDDDDD	0xFEEEFEEE	Type (0=Freed, 1=Normal, 2=CRT use, etc)
0×00320FF8	-8	0xBAADF00D	0×00000031	0xDDDDDDDD	0xFEEEFEEE	Request #, increases from 0
0×00320FFC	-4	0xBAADF00D	0xFDFDFDFD	0xDDDDDDDD	0xFEEEFEEE	No mans land  边界
0×00321000	+0	0xBAADF00D	0xCDCDCDCD	0xDDDDDDDD	0xFEEEFEEE	The 8 bytes you wanted（实际空间
0×00321004	+4	0xBAADF00D	0xCDCDCDCD	0xDDDDDDDD	0xFEEEFEEE	The 8 bytes you wanted 实际空间
0×00321008	+8	0xBAADF00D	0xFDFDFDFD	0xDDDDDDDD	0xFEEEFEEE	No mans land  边界
0×0032100C	+12	0xBAADF00D	0xBAADF00D	0xDDDDDDDD	0xFEEEFEEE	Win32 heap allocations are rounded up to 16 bytes
0×00321010	+16	0xABABABAB	0xABABABAB	0xABABABAB	0xFEEEFEEE	Win32 heap bookkeeping
0×00321014	+20	0xABABABAB	0xABABABAB	0xABABABAB	0xFEEEFEEE	Win32 heap bookkeeping
0×00321018	+24	0×00000010	0×00000010	0×00000010	0xFEEEFEEE	Win32 heap bookkeeping
0×0032101C	+28	0×00000000	0×00000000	0×00000000	0xFEEEFEEE	Win32 heap bookkeeping
0×00321020	+32	0×00090051	0×00090051	0×00090051	0xFEEEFEEE	Win32 heap bookkeeping
0×00321024	+36	0xFEEE0400	0xFEEE0400	0xFEEE0400	0xFEEEFEEE	Win32 heap bookkeeping
0×00321028	+40	0×00320400	0×00320400	0×00320400	0xFEEEFEEE	Win32 heap bookkeeping
0×0032102C	+44	0×00320400	0×00320400	0×00320400	0xFEEEFEEE	Win32 heap bookkeeping

(I’ve tried to helpfully color-code things. Blue and grey is for Win32 heap stuff, and reds are for the debug crt heap stuff)