• windbg


    http://www.codemachine.com/article_x64deepdive.html

    https://sites.google.com/site/jozsefbekes/Home/windows-programming/windbg

    http://www.codeproject.com/Articles/Toby-Opferman#Article

    http://www.nynaeve.net/?cat=2

    https://blogs.msdn.microsoft.com/alejacma/2009/07/24/managed-debugging-with-windbg-call-stacks-part-1/

    https://blogs.msdn.microsoft.com/jmstall/page/2/

    https://blogs.msdn.microsoft.com/alejacma

    Debugging Tools for Windows (WinDbg, KD, CDB, NTSD)

    https://msdn.microsoft.com/zh-cn/library/windows/hardware/ff551063(v=vs.85).aspx

    Yeah, that’s not very helpful. Here’s how to get from that terse description to something useful.

    As an example, let’s use the following trivial program:

    using System;
    
    public class Simple
    {
       public static int Main()
       {
           Console.WriteLine( "Hello World!" );
           Console.ReadLine();
           
           return 0;
       }
    }
    

    Dump this snippet into Notepad, save as simple.cs file and compile it. We want to see what it’ll be like when the user runs it, so we’ll do a release build. From the VS2005 Command Prompt, run csc /debug:pdbonly /o+ simple.cs. (I find it’s quicker to do command-line compiles for simple test programs than firing up Visual Studio. It’s not as if we’re designing a Form.) If you’re doing this on a 64–bit computer, like me, add /platform:x86 so we get consistent output across platforms. (Marking as x86 means that the executable headers are set to indicate a 32–bit program. The IL is the same whatever you set here, but if you leave it as anycpu, the default, the .NET Framework will create a 64–bit process which obviously gives different output.)

    Why call Console.ReadLine? Well, the JIT compiler is helpful (or at least it thinks it is). If you start a program under the debugger, it will generate less optimized code because it thinks you want to debug it, but that changes the code from what the user will see. So I want to add a stop point in the program so I can attach the debugger after the process has started.

    The next thing you’ll need is the Debugging Tools for Windows kit. Grab the 32–bit kit – the 64–bit kit can debug 32–bit processes, but SOS (the debugger extension DLL which implements the commands we’re going to use) doesn’t appear to work. We’re going to use WinDBG, which is slightly friendlier than the other debuggers although not a lot!

    Open WinDBG. Run simple.exe and, when it’s waiting for input, go to File/Attach to a Process in WinDBG. You’ll see a list of other processes (and if running Windows Vista with UAC enabled, or on XP as a standard user, a load of access denied errors). Select simple.exe from the list and hit OK. WinDBG automatically stops once you’ve attached to a process so you can start manipulating the program straight away. This is the output I got:

    Microsoft (R) Windows Debugger Version 6.8.0004.0 X86
    Copyright (c) Microsoft Corporation. All rights reserved.
    
    *** wait with pending attach
    Symbol search path is: C:Windows;C:WindowsSystem32;C:WindowsSysWOW64;SRV*C:WebSymbols*http://msdl.microsoft.com/download/symbols
    Executable search path is: 
    ModLoad: 008c0000 008c8000   C:UsersMikeDocumentsProgrammingsimple.exe
    ModLoad: 776a0000 77800000   C:WindowsSysWOW64
    tdll.dll
    ModLoad: 79000000 79046000   C:Windowssystem32mscoree.dll
    ModLoad: 75b70000 75c80000   C:Windowssyswow64KERNEL32.dll
    ModLoad: 76ce0000 76da6000   C:Windowssyswow64ADVAPI32.dll
    ModLoad: 76e10000 76f00000   C:Windowssyswow64RPCRT4.dll
    ModLoad: 75850000 758b0000   C:Windowssyswow64Secur32.dll
    ModLoad: 75a80000 75ad8000   C:Windowssyswow64SHLWAPI.dll
    ModLoad: 77110000 771a0000   C:Windowssyswow64GDI32.dll
    ModLoad: 77040000 77110000   C:Windowssyswow64USER32.dll
    ModLoad: 76c30000 76cda000   C:Windowssyswow64msvcrt.dll
    ModLoad: 75d00000 75d60000   C:Windowssystem32IMM32.DLL
    ModLoad: 76940000 76a08000   C:Windowssyswow64MSCTF.dll
    ModLoad: 75a70000 75a79000   C:Windowssyswow64LPK.DLL
    ModLoad: 759c0000 75a3d000   C:Windowssyswow64USP10.dll
    ModLoad: 746c0000 7485e000   C:WindowsWinSxSx86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6001.18000_none_5cdbaa5a083979cccomctl32.dll
    ModLoad: 79e70000 7a3ff000   C:WindowsMicrosoft.NETFrameworkv2.0.50727mscorwks.dll
    ModLoad: 75390000 7542b000   C:WindowsWinSxSx86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.1434_none_d08b6002442c891fMSVCR80.dll
    ModLoad: 75e30000 7693f000   C:Windowssyswow64shell32.dll
    ModLoad: 771b0000 772f4000   C:Windowssyswow64ole32.dll
    ModLoad: 790c0000 79bf6000   C:WindowsassemblyNativeImages_v2.0.50727_32mscorlib5b3e3b0551bcaa722c27dbb089c431e4mscorlib.ni.dll
    ModLoad: 79060000 790b6000   C:WindowsMicrosoft.NETFrameworkv2.0.50727mscorjit.dll
    (1dc.12b4): Break instruction exception - code 80000003 (first chance)
    eax=7efaf000 ebx=00000000 ecx=00000000 edx=7770d2d4 esi=00000000 edi=00000000
    eip=776b0004 esp=04c6fa00 ebp=04c6fa2c iopl=0         nv up ei pl zr na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
    ntdll!DbgBreakPoint:
    776b0004 cc              int     3
    

    You enter your debugging commands in the edit box at the bottom, next to the prompt 0:003>. This means we’re debugging the 0th process we’re attached to, and our commands by default apply to thread number 3. Four threads? Well, thread 0 is our main thread, thread 1 was created by the CLR’s debugging support in case we attached a debugger, thread 2 is the finalizer thread, and thread 3 was just created by WinDBG so it could stop the process safely. (You can see this by running ~* k, although you’ll need to be set up to get debugging symbols from the symbol server to get good stack traces.) The Debugging Tools debuggers automatically stop the process when you attach, so that you can start manipulating the process straight away.

    The first thing we need to do is ask WinDBG to load the SOS extension. This is installed with the CLR, so we ask it to load from the same folder that mscorwks.dll (the DLL which implements the virtual machine, effectively the guts of the CLR) lives in:

    0:003> .loadby sos.dll mscorwks

    Now we can see what state of the managed threads are in by running !threads. Any command beginning ! comes from a debugger extension DLL, and they can be disambiguated if necessary, but they’re searched in reverse order loaded (last loaded = first searched) so anything we want will be found in SOS anyway.

    Right. To see what we can do, run !help.

    0:003> !help
    -------------------------------------------------------------------------------
    SOS is a debugger extension DLL designed to aid in the debugging of managed
    programs. Functions are listed by category, then roughly in order of
    importance. Shortcut names for popular functions are listed in parenthesis.
    Type "!help " for detailed info on that function. 
    
    Object Inspection                  Examining code and stacks
    -----------------------------      -----------------------------
    DumpObj (do)                       Threads
    DumpArray (da)                     CLRStack
    DumpStackObjects (dso)             IP2MD
    DumpHeap                           U
    DumpVC                             DumpStack
    GCRoot                             EEStack
    ObjSize                            GCInfo
    FinalizeQueue                      EHInfo
    PrintException (pe)                COMState
    TraverseHeap                       BPMD 
    
    Examining CLR data structures      Diagnostic Utilities
    -----------------------------      -----------------------------
    DumpDomain                         VerifyHeap
    EEHeap                             DumpLog
    Name2EE                            FindAppDomain
    SyncBlk                            SaveModule
    DumpMT                             GCHandles
    DumpClass                          GCHandleLeaks
    DumpMD                             VMMap
    Token2EE                           VMStat
    EEVersion                          ProcInfo 
    DumpModule                         StopOnException (soe)
    ThreadPool                         MinidumpMode 
    DumpAssembly                       
    DumpMethodSig                      Other
    DumpRuntimeTypes                   -----------------------------
    DumpSig                            FAQ
    RCWCleanupList
    DumpIL

    To find out a little more about !U, let’s run !help U:

    0:003> !help U
    -------------------------------------------------------------------------------
    !U [-gcinfo] [-ehinfo] <MethodDesc address> | <Code address> 
    
    Presents an annotated disassembly of a managed method when given a MethodDesc
    pointer for the method, or a code address within the method body. Unlike the
    debugger "U" function, the entire method from start to finish is printed,
    with annotations that convert metadata tokens to names.
    
    
    ...
    03ef015d b901000000       mov     ecx,0x1
    03ef0162 ff156477a25b     call   dword ptr [mscorlib_dll+0x3c7764 (5ba27764)] (System.Console.InitializeStdOutError(Boolean), mdToken: 06000713)
    03ef0168 a17c20a701       mov     eax,[01a7207c] (Object: SyncTextWriter)
    03ef016d 89442414         mov     [esp+0x14],eax
    
    If you pass the -gcinfo flag, you'll get inline display of the GCInfo for
    the method. You can also obtain this information with the !GCInfo command.
    
    If you pass the -ehinfo flag, you'll get inline display of exception info
    for the method. (Beginning and end of try/finally/catch handlers, etc.).
    You can also obtain this information with the !EHInfo command.

    Right, so we need the address of a MethodDesc structure, or the address of the code itself. How can we get one of these? There’s a command called DumpMD…

    0:003> !help dumpmd
    -------------------------------------------------------------------------------
    !DumpMD 
    
    This command lists information about a MethodDesc. You can use !IP2MD to turn 
    a code address in a managed function into a MethodDesc:
    
    0:000> !dumpmd 902f40
    Method Name: Mainy.Main()
    Class: 03ee1424
    MethodTable: 009032d8
    mdToken: 0600000d
    Module: 001caa78
    IsJitted: yes
    m_CodeOrIL: 03ef00b8
    
    If IsJitted is "yes," you can run !U on the m_CodeOrIL pointer to see a 
    disassembly of the JITTED code. You can also call !DumpClass, !DumpMT, 
    !DumpModule on the Class, MethodTable and Module fields above.
    

    We’re getting a bit closer. Let’s try DumpModule…

    0:003> !help DumpModule
    -------------------------------------------------------------------------------
    !DumpModule [-mt] 
    
    You can get a Module address from !DumpDomain, !DumpAssembly and other 
    functions. Here is sample output:
    
    0:000> !dumpmodule 1caa50
    Name: C:pubunittest.exe
    Attributes: PEFile
    Assembly: 001ca248
    LoaderHeap: 001cab3c
    TypeDefToMethodTableMap: 03ec0010
    TypeRefToMethodTableMap: 03ec0024
    MethodDefToDescMap: 03ec0064
    FieldDefToDescMap: 03ec00a4
    MemberRefToDescMap: 03ec00e8
    FileReferencesMap: 03ec0128
    AssemblyReferencesMap: 03ec012c
    MetaData start address: 00402230 (1888 bytes)
    
    The Maps listed map metadata tokens to CLR data structures. Without going into 
    too much detail, you can examine memory at those addresses to find the 
    appropriate structures. For example, the TypeDefToMethodTableMap above can be 
    examined:
    
    0:000> dd 3ec0010
    03ec0010  00000000 00000000 0090320c 0090375c
    03ec0020  009038ec ...
    
    This means TypeDef token 2 maps to a MethodTable with the value 0090320c. You 
    can run !DumpMT to verify that. The MethodDefToDescMap takes a MethodDef token 
    and maps it to a MethodDesc, which can be passed to !DumpMD.
    
    There is a new option "-mt", which will display the types defined in a module,
    and the types referenced by the module. For example:
    
    0:000> !dumpmodule -mt 1aa580
    Name: C:pubunittest.exe
    ......
    MetaData start address: 0040220c (1696 bytes)
    
    Types defined in this module
    
          MT    TypeDef Name
    ------------------------------------------------------------------------------
    030d115c 0x02000002 Funny
    030d1228 0x02000003 Mainy
    
    Types referenced in this module
    
          MT    TypeRef Name
    ------------------------------------------------------------------------------
    030b6420 0x01000001 System.ValueType
    030b5cb0 0x01000002 System.Object
    030fceb4 0x01000003 System.Exception
    0334e374 0x0100000c System.Console
    03167a50 0x0100000e System.Runtime.InteropServices.GCHandle
    0336a048 0x0100000f System.GC

    We still need a Module to pass to this command, but we’re getting there. Maybe DumpDomain can help us? I’ll skip the help text this time.

    0:003> !DumpDomain
    --------------------------------------
    System Domain: 7a3bc8b8
    LowFrequencyHeap: 7a3bc8dc
    HighFrequencyHeap: 7a3bc934
    StubHeap: 7a3bc98c
    Stage: OPEN
    Name: None
    --------------------------------------
    Shared Domain: 7a3bc560
    LowFrequencyHeap: 7a3bc584
    HighFrequencyHeap: 7a3bc5dc
    StubHeap: 7a3bc634
    Stage: OPEN
    Name: None
    Assembly: 0052f848
    --------------------------------------
    Domain 1: 00516800
    LowFrequencyHeap: 00516824
    HighFrequencyHeap: 0051687c
    StubHeap: 005168d4
    Stage: OPEN
    SecurityDescriptor: 00517d30
    Name: simple.exe
    Assembly: 0052f848 [C:WindowsassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll]
    ClassLoader: 0050e470
    SecurityDescriptor: 0051f5d0
      Module Name
    790c2000 C:WindowsassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll
    
    Assembly: 00535ac0 [C:UsersMikeDocumentsProgrammingsimple.exe]
    ClassLoader: 0050e630
    SecurityDescriptor: 00535a38
      Module Name
    000f2c3c C:UsersMikeDocumentsProgrammingsimple.exe

    At last, we have the address of something we can use. Let’s get the module information for simple.exe. For that we need the address next to it in the module list for the simple.exe assembly.

    0:003> !dumpmodule -mt f2c3c
    Name: C:UsersMikeDocumentsProgrammingsimple.exe
    Attributes: PEFile 
    Assembly: 00535ac0
    LoaderHeap: 00000000
    TypeDefToMethodTableMap: 000f0038
    TypeRefToMethodTableMap: 000f0040
    MethodDefToDescMap: 000f005c
    FieldDefToDescMap: 000f0068
    MemberRefToDescMap: 000f006c
    FileReferencesMap: 000f0088
    AssemblyReferencesMap: 000f008c
    MetaData start address: 008c206c (740 bytes)
    
    Types defined in this module
    
          MT    TypeDef Name
    ------------------------------------------------------------------------------
    000f3030 0x02000002 Simple
    
    Types referenced in this module
    
          MT    TypeRef Name
    ------------------------------------------------------------------------------
    790fd0f0 0x01000001 System.Object
    79101118 0x01000006 System.Console
    

    Now we can get the MethodTable for the Simple class:

    0:003> !dumpmt -md f3030
    EEClass: 000f1174
    Module: 000f2c3c
    Name: Simple
    mdToken: 02000002  (C:UsersMikeDocumentsProgrammingsimple.exe)
    BaseSize: 0xc
    ComponentSize: 0x0
    Number of IFaces in IFaceMap: 0
    Slots in VTable: 6
    --------------------------------------
    MethodDesc Table
       Entry MethodDesc      JIT Name
    79371278   7914b928   PreJIT System.Object.ToString()
    7936b3b0   7914b930   PreJIT System.Object.Equals(System.Object)
    7936b3d0   7914b948   PreJIT System.Object.GetHashCode()
    793624d0   7914b950   PreJIT System.Object.Finalize()
    00310070   000f3020      JIT Simple.Main()
    000fc01c   000f3028     NONE Simple..ctor()

    Finally we have something we can pass to !U. Note the JIT column. Here, PreJIT means that the code has come from a native image generated by ngen, JIT means that the JIT compiler in this process has generated the code, and NONE means that the method hasn’t been compiled yet. We didn’t override any of System.Object’s virtual methods, so they occupy the first four places in our MethodTable.

    I placed the call to Console.ReadLine at the end of the program so everything has already been compiled – remember, the JIT compiles code on-demand, one method at a time, when that method is called (barring very simple methods which might be inlined into their callers). Let’s just get on and show the code for Main.

    0:003> !U f3020
    Normal JIT generated code
    Simple.Main()
    Begin 00310070, size 36
    00310070 833d8c10920300  cmp     dword ptr ds:[392108Ch],0
    00310077 750a            jne     00310083
    00310079 b901000000      mov     ecx,1
    0031007e e8e1580579      call    mscorlib_ni+0x2a5964 (79365964) (System.Console.InitializeStdOutError(Boolean), mdToken: 06000770)
    00310083 8b0d8c109203    mov     ecx,dword ptr ds:[392108Ch] (Object: System.IO.TextWriter+SyncTextWriter)
    00310089 8b153c309203    mov     edx,dword ptr ds:[392303Ch] ("Hello World!")
    0031008f 8b01            mov     eax,dword ptr [ecx]
    00310091 ff90d8000000    call    dword ptr [eax+0D8h]
    00310097 e84c750d79      call    mscorlib_ni+0x3275e8 (793e75e8) (System.Console.get_In(), mdToken: 0600076e)
    0031009c 8bc8            mov     ecx,eax
    0031009e 8b01            mov     eax,dword ptr [ecx]
    003100a0 ff5064          call    dword ptr [eax+64h]
    003100a3 33c0            xor     eax,eax
    003100a5 c3              ret

    Interesting. The JIT clearly sees Console.WriteLine(string) and Console.ReadLine as trivial and has inlined the calls. In turn it’s inlined Console.get_Out. The reference to SyncTextWriter isn’t the JIT being clever – it’s SOS telling us that the address 0x0392108C is the base of SyncTextWriter’s vtable, as this call to TextWriter.WriteLine(string) is virtual.

    I realise, and hope, that this isn’t everyday usage. More often than not you’ll have crashed somewhere and want to find out where (for which see !IP2MD). But it can be useful to see just how your C# code turns into machine instructions.

    ========================

    Introduction 

    WinDbg is the most powerful debugger for windows, and it is licensed with the OS, so once you paid money for the OS, no extra money has to be paid for windbg.

    Article about WinDbg: http://www.debuginfo.com/articles/easywindbg.html

    Technique for analysing exceptions: http://blogs.msdn.com/jmstall/archive/2005/01/18/355697.aspx

    Collection of debugging techniques: http://www.dumpanalysis.org/

    This is another good tutorial: http://www.codeproject.com/KB/debug/windbg_part1.aspx

    Memory leak detection using windbg http://www.codeproject.com/Articles/31382/Memory-Leak-Detection-Using-Windbg

    Installing Windbg

    #  Download windbg and install (http://www.microsoft.com/whdc/DevTools/Debugging/default.mspx)

    #  Add the installation folder to the path (for me it is "c:Program FilesDebugging Tools for Windows")
    #  Create a new folder for symbols (I chose to use c:symbols, this is going to be used in this document)
    # Create an environment variable called _NT_SYMBOL_PATH and set _NT_SYMBOL_PATH to the following: CACHE*c:Symbols;c:CustomSymbols;srv*http://msdl.microsoft.com/download/symbols  (substitute your folder for c:symbols and c:customsymbols. CustomSymbols can be used to manually copy pdb files into - if you run !sym noisy you will be able to see how it is used.)

    _NT_SYMBOL_PATH is used for getting the pdb files for windows dlls, you will see the function names in the call-stack for windows dlls if you configure this setting properly.

    You can also download symbol files for  windows dlls manually using the command (issue in the command prompt):

    symchk /r c:windowssystem32 /s SRV*c:symbols*http://msdl.microsoft.com/download/symbols

    where symchk.exe is installed as part of the WinDbg software into c:Program FilesDebugging Tools for Windows. You can also issue commands to download symbol files from within windbg, see the documentation of WinDbg for details, and this address: http://support.microsoft.com/kb/311503.
     
    You can set windbg to be the post-mortem debugger by running
    windbg -I 

    Using built in help

    .hh <search str>

    General stuff

    !wow64exts.sw                 - Switch between 32 and 64 bit modes (obviously only if you have a 64 bit windbg)
    !verifier                              - verifier has a built in command in windbg, useful when you investigate a bsod caused by verifier
    !verifier FF <drivername> - turn on full info about a driver

    Debugging unmanaged (native, ordinary c++) code

    When a software crashes, or asserts on a machine that does not have Visual Studio installed it is possible to locate the crash or assert by using WinDbg to debug the application, see threads running and examine the call stack. To achieve this you will need to make sure that the application was built in debug mode, you have the generated symbol (pdb) files and the source files.

     
    #    Start "c:Program FilesDebugging Tools for Windowswindbg.exe" (or from the location where it has been installed)
    #    Push Ctrl+P, add the relevant source folders to the list ( select the folder of the solution)
    #    Push F6, select the process
    #    Push Alt+9 (threads window pops up) and Alt+6 (callstack window pops up)

    If you see a message like this: *** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:WINDOWSsystem32RPCRT4.dll, the symbol files cannot be found. If the error mentions one of your dlls,  make sure that the pdb file is in the same folder as the dll that was mentioned in the error message. If a windows dll is mentioned, the _NT_SYMBOL_PATH is not configured correctly. 

    Using the watch

    The shortcut to the watch window is alt+2.

    You can use the watch just like in VS, but you should try to prefix your variables, otherwise the debugging process might become too long. For a local variable you should try the prefix $!. So for getting the value if a variable i, you should write $!i into the watch window. A member variable of a class can be referenced as a local variable if the debugger is positioned on a line of a non-static member function.

    For a global variable you could try <modulename>!variablename. 

    Investigating the call stack in managed (vb.net, C#) code

    #  Map a drive to the shared src drive

    # Start "c:Program FilesDebugging Tools for Windowswindbg.exe" (or from the location where it has been installed)
    # Push Ctrl+P, add the relevant source folders to the list (from the mapped drive)
    # Push F6, select LCShell.exe
    # Push Alt+1 for the command window
    # Type: .load c:WINDOWSMicrosoft.NETFrameworkv1.1.4322SOS.dll (dll location might be different, depends on .net version)
    # Instead of the previous step this might also work: .loadby sos mscorwks (tells WinDbg to look up which directory mscorwks.dll was loaded from and load sos.dll from there)
    # Type: !threads (this will list the threads with managed code)
    # Type: ~*e !clrstack (this will list the callstack in the managed code)

    Useful commands

    (some from http://www.codeproject.com/KB/debug/windbg_part1.aspx)

    Common windbg commands: http://windbg.info/doc/1-common-cmds.html

    NATIVE + MANAGED

    Start DML (more user friendly display):
    .dml_start

    Start a new temporary command window (with command history):

    .browse

    Add a new folder to the list of symbol paths:

    .sympath +c:SomeFolderSomeSubfolder
    .reload
    !sym noisy # turn on detailed logging about symbols
      
    loads all the necessary symbols
    .reload /f
     
    .reload /f mydr.sys # reload all symbols for mydr
    .reload /unl /f mydr.sys # reload all symbols for mydr even though mydr has been unloaded at this stage. Useful is e.g. worker threads are not shut down during driver unload and the crash is due to unloading the module with active calls in place.
     
    Which is the current address of execution
    ln
     
    Which is the current function of execution
    ln eip

    Automatic helper

    !analyze -v

    If the PC is hanging and you're attached to it remotely with windbg, this is useful:

    !analyze -v -hang

    Get a description of the handles held by the process

    !handle
     

    NATIVE

    Processes
     
     
    !dml_proc            - list all processes & PID and process address & threads and switch context to this process etc.
    !peb <address>    - display process environment  block
    .attach PID          - attach to a process
    .detach                - ends the debugging session
    .process <address>  - you can attach to process and dump callstacks etc. works only in kdb mode (not in user mode)
    !process <address> - displays info (threads etc.) about the process specified by address
    !process 0 7         - dumps all callstacks in all processes (takes ages), better to attach to a process: !process <id> where id can be retrieved from !dml_proc. Then you can run !process -1 7, it will only dump the process you're attached to.
    !findstack xxx       - finds all callstacks that contain xxx in the symbols. !Uext.help  for more info. Works only in user mode (not in kdb mode)
    ~*kb                    - display all callstacks in all threads in the attached process
     
    This is interesting about attaching to processes: http://analyze-v.com/?p=336
     
    Threads
    !teb                     - dump thread environment  block
    !peb                     - dump process environment  block
    ~                         - list threads (only in user mode)
     

    !stacks 2 bla        - list all callstacks that contain bla (kernel mode only?)

    .thread <address>         - switch to the context of this thread
    !runaway -           - for performance, in usermode (or usermode dump) sorts threads according to how busy they are. Not tested yet.
     
    Stack trace (Displays stack trace of current thread (x frames))
    K x
    KB x # (also shows the first 3 words of params)
    KV x # (also shows the calling convention) kP n # (the most useful, prints variable types and values as well) ~*kL # prints stacktrace from all treads kf # Automatically do the ChildEBP - Previous ChildEBP calculation to see which function call is eating up all the stack .ecxr to switch the current context to an exceptions callstack
    When the callstack seems wrong this can be useful: http://www.dumpanalysis.org/blog/index.php/2007/04/03/crash-dump-analysis-patterns-part-11/ Good explanation about what is the stack & how it works: http://www.codeproject.com/KB/debug/cdbntsd2.aspx !stack - to display stacks, see example below in hang analysis
    !stacks 2 <drivername> - will list stacks that contain code from that driver
    If you have a code address and have proper symbols, source file location all that, you can open the source file that has the code at that code address by .open -a <address>
    To see the assembly of a function implementation, you can do .browse uf <func> e.g. .browse uf ExFreePoolWithTag
     
    When analyzing a hang, use these
     
    !stacks
    You can do !stacks 2 <drivername> and it will list stacks that contain code from that driver
    !locks
              To see the callstack for who is holding a lock: 
    !locks <drivername>!<global lock variable name>
     
    Will say
     
    Resource @ <drivername>!<global lock variable name> (0xfffff801948d2d80)    Exclusively owned
         Threads: ffffe001b38b2840-01<*>
    1 total locks, 1 locks currently held
     
    So you can do
     
    !thread ffffe001b38b2840
     
    Which will print the stack.
     
    !analyze -hang
    To see threads and all that
     
    Variables
     
    With some commands here if you do not specify an address, they will move forward. So if you do db <address>, you will see the byte at the specified memory at address. Further db commands w/o an address will display bytes following the previous byte and the "window will move forward".
    dv
    dv /t - prints all variables on the currently selected stack-frame
    dv - without arguments prints all locals
    dv /v - locals with their addresses
     
    dt <variablename> -  print, -r will print structures recursively
    in case the variablename does not work (e.g. for a structure member of a structure) then you can do dt <TypeName> <Address>
    dd <module>!<variablename> # dump  data - this one will print the value of the variable
    db <address> - dump the contents of memory at address (dumps a byte), repeatedly issue db and it will print the next bytes
    du <address> - display unicode string at address. Usually dt returns the address of a string buffer if the variable is a wchar_t* or char*.
    !for_each_frame dv /t - prints all variables on all stack-frames in current thread
    ed < module>!<variablename>  0x02000FFF # ed means edit data so this one will overwrite the value of  the variable in memory
     
    !stl <variablename> - displays std::wstring (possibly also std::string) contents
     
    Memory
    !memusage # Overview of OS memory info
    !vm # virtual memory usage details !poolused # displays pool usage summaries, report includes the tag  used for each pool allocation. See help for sorting capabilities. !pool <address> # Whether a particular block of memory is allocated correctly or is just some random bit of memory. !poolfind # finds all instances of a specific pool tag in either nonpaged or paged memory pools. Takes ages! !heap -s # will print heap overview info about allocations. Then the address of an entry can be fed to the !heap command again to get further details. !vadump # dump list of memory pages and info !handle 0 0 0 # displays all handles with owning process r # show registers. Can see function return value after call in eax. Last item (in time) placed onto the stack is in esi. See this for more: http://unixwiz.net/techtips/win32-callconv-asm.html s # can be used to search memory. s -u 0 L?4000000000 blabla # In a user mode (one process) dump it searches 4GB from start for the word blable as unicode string in memory
    !search # search memory in context only.
    dc <address> # will print the memory at that address dds, dps, and dqs # To display the contents of memory in the given range, treat the numbers as code addresses and display the corresponding symbols.
     
    Breakpoints
    bl # list
    bp,  bc, be, bd # (put, clear, enable, disable)
    bc * # clears all breakpoints
    bs 4 "dt varname;g;" # with bl you can list breakpoints. You can get any of them to run stuff by executing bs against its number. In this example we want to run "dt varname;g;" whenever breakpoint 4 is hit. bl will show the execute info as well.
      ----
    #If you do not have sources but you do have pdb files, you can still set breakpoints to functions. Here is how it goes.
    #Use the X command to search for symbols. E.g.
    X mymodule!functionname
    #You can use asterisks (*) for partial match. For all matches you will see addresses of functions. You can use those addresses with the bp function.
    ----

    Modules and symbols

     
    lm                                            # show loaded modules
    lm v m <modulename>*             # show info about a module
    !lmi  modulename                       # dump the info for module modulename               
    X < module>!*mynameportion*    # search for every symbol (variable name) that matches *mynameportion*
    x <module>!*                            # list all symbols exported by <module>
    .symopt +0x40                         # this will make windbg load any pdb file that it can find for a dll, based on name matching. If you do not have the proper pdb can be useful as a last resort.
    .reload /f <dll/exe name>           # this forces a symbol (re)load for the module
    !symfix                                     # probably this will fix windows symbols?
    !sym noisy                               # display detailed info about loading (or failing to load) symbols
    List of symbol options: https://msdn.microsoft.com/en-us/library/windows/hardware/ff558827%28v=vs.85%29.aspx#symopt_load_lines

    Others

    b # if a debug assertion is shown, break into the code
    .reboot # reboots the pc windbg is currently connected to (useful if we connect to the PC remotely)
    .dump # creates a dump file from what's in windbg at the moment
     
    # Combining windbg with unixtools like grep (https://sourceforge.net/projects/unxutils/) (or any other shell commands but grep is the one that I miss a lot from windbg)
    .shell -ci "!threads" grep 236 # this will grep 236 from the text output printed by !threads
    Scripting, .foreach https://msdn.microsoft.com/en-us/library/windows/hardware/ff552157%28v=vs.85%29.aspx
    # Here is to list threads with !threads, get the address of the teb for each (3rd column in printout) then call !teb on each address and filter out the TEB address and the LastErrorValue for each. .foreach (obj {.shell -ci "!threads" egrep -v Total | gawk "{print $3}" }) {.shell -ci "!teb obj" egrep "LastErrorValue|TEB at"}
    # Will print sg. like
    TEB at 0000000000270000     LastErrorValue:       0 .shell: Process exited TEB at 00000000002d6000     LastErrorValue:       997 .shell: Process exited TEB at 00000000002da000     LastErrorValue:       0 .shell: Process exited TEB at 00000000002e2000     LastErrorValue:       0 .shell: Process exited TEB at 00000000002e4000     LastErrorValue:       0 .shell: Process exited
    # This approach is not perfect as there is some garbage returned at the end of the first command execution and we run !teb against the garbage which times out slowly. To improve that we'll try an if statement. # .if ($spat("${obj}","0x[0-9]+*")) - spat is a pattern matching thing. The pattern is a weak regular-expression-like syntax (https://msdn.microsoft.com/en-us/library/windows/hardware/ff558819%28v=vs.85%29.aspx).
    # What we're saying here is that the string has to start with 0x and then it should have one or more digits and anything after that (we have extra spaces...) and then the .if condition is true.
    # so this command works well (we can print what is passed in object with .echo):
    .foreach (obj {.shell -ci "!threads" egrep -v TEB | gawk "{print $3}" }) { .if ($spat("${obj}","0x[0-9]+*")) { .echo "${obj}" ; .shell -ci "!teb obj" egrep "LastErrorValue" } }
     
    # here is another similar example: .foreach (obj {.shell -ci "!heap -flt s 178" egrep -v HEAP | egrep -v ole32 | gawk "{print $5}" }) {!heap -p -a obj}
    # (needs gflags.exe /i MemoryLeak.exe +ust executed from the command prompt for your exe for it to make any sense, see http://www.codeproject.com/Articles/31382/Memory-Leak-Detection-Using-Windbg )
    If you have executed gflags and run your process etc. here is how to dump all the callstacks for all memory reservations

    # change filename as needed… .foreach (obj2 { .foreach (obj {.shell -ci "!heap -s" egrep "^0.*" | gawk "{print $1}"}) { .if ($spat("${obj}","[0-9]+*")) { .shell -ci "!heap -stat -h obj" grep .*-.*(.*) | gawk "{print $1}" } } } ) { .if ($spat("${obj2}","[0-9]+*")) { .shell -ci "!heap -flt s obj2" grep (busy) >> d:lmhost (2).1.txt } }

    # from cmd prompt run this: cat "lmhost (2).1.txt" | gawk "{print $5}" | sort | uniq > "lmhost (2).2.txt"

    # back to windbg, again change filenames as needed .foreach /f ( obj "d: lmhost (2).2.txt" ) { .shell -ci "!heap -p -a obj" grep .* >> "d:lmhost (2).3.txt" }
    #This last step can run for days if not weeks, be prepared...

    Usual way to investigate calls in a process

    !dml_proc  - to see all processes, pick the one you're interested in

                      then click "select user mode state" (this will load pdb files and set up the process context)
                      then click Full Details (this will list all threads with thread addresses and ids and callstacks)
    .thread <thread address> - this will switch the context to this thread. Now you can use all the callstack commands.
                                           Also you can use alt+2 and alt+6 to walk up & down on the call stack and see the values
                                           of variables. At this stage if you have the source file open the current line of execution
                                           will be shown as you click in the calls window.
     

    MANAGED

     
    You need to load the sos.dll for these to work (see above)
     
    List the threads that have managed code:
    !threads
     
    Select current thread:
    ~[n]s, e.g. ~11s
     
    Get the callstack in the active thread of a managed app:
    !clrstack
     
    The same for all threads:
    ~*e !clrstack
     
    Print exception in current thread
    !pe
     
    Print exceptions in all threads
    ~*e !pe
     
    Get the whole stack of a managed app (current thread):
    !dumpstack
     

    Dump all the managed objects that are currently on the stack:

    !dumpstackobjects
     

    Dump all the managed objects that are currently in the heap:

    !dumpheap -stat

    Getting the value of a member variable

    find the address of the container object using !dumpstackobjects, e.g.

    0:008> !dumpstackobjects

    Thread 8
    ESP/REG    Object     Name
    edx        0x44a7d98 System.Single[]
    0x6bafaec 0x444c4c0 System.Threading.Thread
    0x6bafaf4 0x444c4c0 System.Threading.Thread
    0x6bafafc 0x444c46c Project1.ThreadManager
    0x6bafb00 0x44a7d80 Project1.USBHardware/DownloadPointsToUsbParams
    0x6bafd6c 0x444c4a4 System.Threading.ThreadStart

    and then type

    !dumpobj 0x444c46c

    (for the ThreadManager object)

     

    print the help for the sos extension

    !clr10sos.help  
     

    Setting a managed breakpoint to a function call:

    !bpmd <modulename> <functionname> 

    Memory leaks

    C++: http://www.codeproject.com/Articles/31382/Memory-Leak-Detection-Using-Windbg

    Here is a similar approach:

    gflags.exe /i <leaking exe> +ust gflags.exe /i <leaking exe> /tracedb 50

    (re)start your exe, let it run and leak and then create a dump (i.e. through task manager), open the dump in windbg, load all symbols and then

    !heap -l

    will print the callstack for memory addresses that are potentially results of a leak.

    C#: http://blogs.msdn.com/b/ricom/archive/2004/12/10/279612.aspx

    Creating and debugging a memory dump

     
    The usual method of debugging a problem that occurs on a machine in production is that in case an error happens a dump file is generated and this dump file is debugged on the developer's desktop, where the code and the debugging environment is installed. This technology is built up on exceptions.
     
    In general, windows shows two types of error dialogs.
     
    The first type error dialog says:
     
    Runtime Error! This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information.
     
    This dialog just has an OK button.
     
    The second type error dialog can be configured in Control Panel -> System -> Advanced -> Error reporting, and it lets the user select to debug the application.
     
    These dialogs appear if an exception is not handled in the application, in other words in the case of a second chance exception. (A first chance exception is one that the application can/should catch; if the application does not catch, it becomes a second chance exception that the operating system catches, and in response to that the OS shows the above described dialogs.)
     
    Most automatic memory dump generators will start when the debug button is pushed on the second type error dialog, it means that they do not do anything for the first type error dialog. Our aim now is to create a dump file for any second chance exception that happens no matter which dialog is shown.
     
    Also keep in mind that these tools can produce a dump file in case a first chance exception happens, and they can be configured for which first case exceptions to produce the memory dump, but that is not our aim now in general.
      
    Creating a memory dump
     
    The most typical ways to achieve this are the followings:
     
    1. Configure Dr. Watson or WER so that if a crash happens, and the user selects to debug the problem, instead of the debugger Dr. Watson starts and it creates a dump file. Dr. Watson is part of the XP pro install. http://support.microsoft.com/kb/308538 and WER replaces it in Vista.
       
    2. (Use NTSD.exe. http://support.citrix.com/article/CTX105888)
    3. Install the "Debugging tools for Windows" package and use ADPlus.vbs. http://support.microsoft.com/kb/286350
    4. Install the "Debugging tools for Windows" package and use Userdump.exe. http://support.microsoft.com/kb/241215
    5. Start WinDbg, attach to the process and execute this command: .dump
    6. Use Sysinternals Procdump utility
       

    I have tested #3 and #4 with two scenarios: I have created a small Application that generated a division by zero exception and could also throw a user defined exception. I have also tested what throwing and catching an exception would cause in any of the above tools (that is part of the normal operation in our software). The result of this investigation is that if the application does not just die, but displays a dialog before, it is best to start ADPlus.vbs when the dialog is displayed and generate a snapshot of the process memory (for details see the comments just at the end of this subchapter). This is explained step by step in "Using windbg to create a crash dump" below. Nr. 1 is described in "Using built-in windows tools to generate crash reports".

     

    * Using windbg to create a crash dump

     
    For setting up the environment:
     
      1. Install the "Debugging tools for Windows" package
      2. Create a batch file with the following content:
     
            pushd "c:Program FilesWinDbg"
            cscript ADPlus.vbs -hang -NoDumpOnFirst -pn CrashDump.exe -o c: empdump -quiet
     
    Of course c:Program FilesWinDbg should be changed to the folder where you installed the "Debugging tools for Windows" package, and CrashDump.exe should be the process you are investigating. When any error dialog appears, execute the batch file. You will not be able to access the application until the error reporting process finishes.
     
    If the process dies without any error dialogs, there is a way to attach ADPlus.vbs from the start of the execution, this however slows down the process a noticeable amount. This is only useful if we want to investigate silent finish of the process, then we do not have any other choice (we could also use Userdump.exe for this purpose, but it has the same disadvantages).
     
    Comments:
     
    Userdump.exe produces a report on all (configured) first chance exceptions that is not useful for us and slows down the software a lot. With this tool there is no way to catch a second chance exception but not catch a first chance exception of the same type.
     
    For debugging a silent crash ADPlus.vbs would be executed just after the application starts, and it would be called like this: ADPlus.vbs -crash -NoDumpOnFirst -pn CrashDump.exe -o c: empdump -quiet
     
    There is a good description of ADPlus.vbs in debugger.chm (in the same folder where ADPlus.vbs is)

    * The firefox way

    https://developer.mozilla.org/en/How_to_get_a_stacktrace_with_WinDbg

     

    * Using built-in windows tools to generate crash reports

     
    Dr Watson on XP
     
    http://msdn.microsoft.com/en-us/library/cc266343.aspx
     
    To specify that you want dr watson to be the post-mortem debugger (auto invoke it if something crashes) execute this from console:
    drwtsn32.exe -I
     
    Once you set it up, if a crash happens, you will get the well known dialog which says:
     
    yourappprocess.exe has encountered a problem and needs to close....
     
    Here you should select "Don't Send."
     
    Then open a console and type drwtsn32.exe, this will open up a dialog with your latest crashes. Here you can see an editbox on the top, it says "Crash Dump" and it specifies the path to a file, this is the file that developers will need to see what happened. Just attach the user.dmp file somehow to the bug you're reporting (it is big so probably you'll need to share it from your computer or something?).
     
    Windows Error Reporting on Vista
     
    http://msdn.microsoft.com/en-us/library/bb513638%28VS.85%29.aspx
     
     
    Add/modify this registry entry:
     
    under
    HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsWindows Error Reporting
    add
    ForceQueue=1 (this value shouls be of type REG_DWORD)
     
    Once you put this change to your registry in case a crash happens the process will die silently. In this case go to Control Panel->Problem Reports and Solutions, select "View problem history" and double-click the crash that you want to report to the developers. Here you should select "View a temporary copy of these files" under the section "Files that help describe the problem". Attach minidump.mdmp to your bug report (and any other files that you think might be helpful).
     

    Creating a system memory dump (for drivers)

    Forcing a System Crash from the Keyboard

     

    Configure Windows to save full dump on BSOD

    page file settings

     
    Go to computer properties (Advanced system properties on Vista+) (WindowsKey + Break)
    Advanced Tab
    Settings Button in Performance section
    Advanced Tab
    Change button in Virtual memory section
    Paging file should be either system managed or have the maximum size specified by the page file size guidelines above.  For Vista+, you can just check the 'Automatically manage page file size for all drives' checkbox
     
    Log settings
     
    Go to computer properties (Advanced system properties on Vista+) (WindowsKey + Break)
    Advanced Tab
    Settings Button in Startup and Recovery section
    System failure section
    Check - Write an event to the system log
    Check - Send an administrative alert
    Automatically restart (tester's discretion)
    Set the 'Write debugging information' drop down to 'Complete memory dump'
    Set the Dump file to: %SystemDrive%MEMORY.DMP
    Check - Overwrite any existing file
     

    Configure Windows so that we can trigger a system crash via the keyboard

    Install the appropriate hotfix. E.g. look at these for the corresponding OSs:

    2003 SP1, XP x64 SP1: kb244139
    Vista SP1, 2008 SP1:  kb971284
     
    Create these registry keys
     
    In the registry key HKEY_LOCAL_MACHINESystemCurrentControlSetServicesi8042prtParameters, create a value named CrashOnCtrlScroll, and set it equal to a REG_DWORD value of 0x01.
    In the registry key HKEY_LOCAL_MACHINESystemCurrentControlSetServiceskbdhidParameters, create a value named CrashOnCtrlScroll, and set it equal to a REG_DWORD value of 0x01.
     
    Reboot
     

    Real-time checks

     

    A common problem with C type code is that you can very easily overindex, meaning you can write into memory that you really shouldn't, and the code won't complain (unlike in c++ or C# or java where if you overindex a vector or an arraylist you'll get a well define exception). This problem might not cause a crash when it happens, until another area of the code tries to read and use the memory. Even worse, the code might not crash even in that case, maybe some data has just changed mysteriously, but the process will survive. So now you have a strange number in one of your structures or your application crashes for no apparent reason, how do you find the offending code? There are tools and techniques that can be used to detect such problems but they slow down the processes, so these are not enabled by default. However it can be extremely useful for error analysis or dev testing; when they are turned on, the process or code will crash/bluescreen when it performs the overindexing, so the dump file that you get will contain the offending code and not the victim. I have heard of 2 such tools that can achieve this, they are verifier (for drivers) and gflags. gflags is part of windbg, verifier is installed on all recent windows versions by default.

     

    Debugging a memory dump

     
    If you want to debug a memory dump, these are the steps:
     
      1. Start WinDbg
      2. Push ctrl+D, open the dump file
      3. Push alt+6 for callstack, and select the more button for seeing the whole list (required in most cases)
      4. You can also jump around threads, as if you were debugging, but of course you cannot go forward or backward in time - this is just a snapshot

    Here is a good series of blogs about how complex crash-dump analysis can become: http://www.dumpanalysis.org/blog/index.php/2006/10/30/crash-dump-analysis-patterns-part-1/

     
    Further readings
      

     

     

    Apparently there is an extension that makes it possible to find out extra info about COM calls. Need to dig into this later. Here is the name of the dll: sieextpub.dll

     
     
  • 相关阅读:
    查询程序,统计文件每个单词出现几次,对应的出现在哪一行
    适配器 STL
    非关联容器|hash|unordered_map/multimap,unordered_set/multiset
    函数对象
    ACM-ICPC 2018 徐州赛区网络预赛 J. Maze Designer (最大生成树+LCA求节点距离)
    D. Connected Components Croc Champ 2013
    [Educational Codeforces Round 63 ] D. Beautiful Array (思维+DP)
    C. Vasily the Bear and Sequence Codeforces 336C(枚举,思维)
    Vasya and Beautiful Arrays CodeForces
    D. Happy Tree Party CodeForces 593D【树链剖分,树边权转点权】
  • 原文地址:https://www.cnblogs.com/WCFGROUP/p/5735136.html
Copyright © 2020-2023  润新知