http://www.oracle.com/technetwork/server-storage/solarisstudio/documentation/amd64-dbx-364568.html
AMD64 Instruction-Level Debugging With dbx
By Nasser Nouri, May 2008, Revised April 2011
Debugging at the machine-instruction level in the Solaris Studio dbx command-line debugger environment becomes very handy when a software bug cannot be found easily. Usually, programs are written in high-level languages such as C, C++, or Fortran, and most of the software defects can be debugged in the dbx environment at the same high level.
However, having some knowledge of the machine-instruction level of the system on which the program is running, and using the right tool, such as dbx, can shorten the time to identify the culprit and come up with the optimal solution to fix the defect.
This article describes how to use the dbx debugger efficiently on the AMD64 architecture. It describes how to display the contents of memory at specified addresses, and how to display machine instructions. Use the regs command to print out the contents of machine registers or the print command to print out individual registers. Use the nexti, stepi, stopi, and tracei commands to debug at AMD64 machine-instruction level.
AMD64 Architecture
dbx Commands
The Problem Statement
The dbx Failure
The Debugging Session
In Conclusion
AMD64 Architecture
First let's review briefly the AMD64 architecture and see how it is different from the 32-bit x86 architecture. I describe only the materials that are relevant to this article. For an in-depth understanding of AMD64 architecture, please refer to AMD64 manuals (http://developer.amd.com) and AMD64 Application Binary Interface (ABI) (http://www.x86-64.org).
The AMD64 architecture has sixteen 64-bit general purpose registers (GPRs): RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, R8, R9, R10, R11, R12, R13, R14, and R15. Compared to the x86 architecture, the AMD64 architecture has eight new GPRs.
The RAX, RBX, RCX, RDX, RBP, RSI, RDI, and RSP registers are used by both 32-bit and 64-bit binaries. However, in 32-bit mode, only the low 32 bits of these registers are accessible by 32-bit binaries. In the x86 architecture, these registers are EAX, EBX, ECX, EDX, EBP, ESI, EDI, and ESP, respectively.
The AMD64 architecture provides sixteen 128-bit XMM registers XMM0 through XMM15. Registers XMM0 through XMM7 are used for passing float and double parameters. The long double type is passed in memory. A long double in AMD64 architecture is 16 bytes long compared to 12 bytes in the x86 architecture. The long double type is implemented based on 80-bit extended (IEEE) standard.
The AMD64 architecture also provides eight x87 floating point registers, MMX0/FPR0 through MMX7/FPR7, each 80 bits wide.
In contrast to the 32-bit architecture in which the function parameters are passed on the stack, the 64-bit architecture has six registers available for integer parameter passing. If the number of integer parameters is more than six, the remaining parameters are passed on the stack.
The bool, char, short, int, long, long long, and pointer types are classified as integer class. For passing parameters of the integer class, the next available register of the sequence RDI, RSI, RDX, RCX, R8, and R9 is used.
Registers RBP, RBX, and R12 through R15 belong to the calling function, and the called function is required to preserve their values.
The RIP register is the instruction pointer register. In 64-bit mode, the RIP register is extended to 64 bits to support 64-bit offsets. In 32-bit x86 architecture, the instruction pointer register is the EIP register.
The return value of a function is classified based on the rules that are specified in AMD64 ABI. For instance, if the return value needs to be passed in memory, then the caller provides space for the return value and passes the address of this storage in the RDI register as if it were the first argument to the function. On return, the RAX register contains the address that has been passed by the caller in the RDI register.
Similarly, if the return type is integer, the next available register of the sequence RAX, RDX is used.
In addition to registers, each function has a frame on the run-time stack. The run-time stack grows downwards from a high address. Table 1 shows the stack organization.
Table 1 Stack Frame With Base Pointer
Position
Contents
Frame
8n+16 (%rbp)
...
32 (%rbp)
24 (%rbp)
16 (%rbp)
argument #n
...
argument #2
argument #1
argument #0
High address
Previous frame
8 (%rbp)
return address
Current frame
0 (%rbp)
previous %rbp value
Current frame
-8 (%rbp)
-16 (%rbp)
...
0 (%rsp)
local variable #1
local variable #2
...
local variable #n
Current frame
Low address
-128 (%rsp)
red zone
The RSP register is the stack pointer register and the RBP register is the frame pointer register. Stack operations make implicit use of the RSP register, and in some cases, the RBP register. The RSP register is decremented when items are pushed onto the stack, and incremented when they are popped off the stack. The RBP register points to the lowest address of the data structure that is passed from one function to another.
The 128-byte area beyond the location pointed to by the RSP register is known as red zone and is considered to be reserved. Functions can use this area for temporary data that is not needed across function calls. In particular, leaf functions can use this area for their entire stack frames, rather than adjusting the stack pointer in the prologue and the epilogue.
prologue:
pushq %rbp / save frame pointer
movq %rsp,%rbp / set new frame pointer
subq $48,%rsp / allocate stack space
movq %rbx,-16(%rbp) / save callee-saved registers
movq %r12,-24(%rbp)
movq %r13,-32(%rbp)
movq %r14,-40(%rbp)
movq %r15,-48(%rbp)
There is no need to adjust the RSP stack pointer register if the red zone area is used. In other words, the subq $48,%rsp instruction is not needed in function prologue if the red zone area is used.
epilogue:
movl -4(%rbp), %eax / set up return value
movq -16(%rbp),%rbx / restore callee-saved registers
movq -24(%rbp),%r12
movq -32(%rbp),%r13
movq -40(%rbp),%r14
movq -48(%rbp),%r15
leave
retThe C++ language has its own Application Binary Interface (ABI). The C++ ABI has well-defined rules for function parameter passing and return values. The C++ ABI rules supplement the AMD64 ABI rules; the C++ compiler has to use the C++ ABI rules for function parameter passing in addition to the AMD64 ABI rules.
dbx Commands
The following commands are documented in Solaris Studio 12.2: Debugging a Program With dbx for machine-instruction level debugging.
examine [ address ] [ / [ count ] [format ] ]
Display the contents of memory starting at address for count items in format format
stepi
Single step one machine instruction (step into calls)
nexti
Step one machine instruction (step over calls)
listi
Intermix source lines and assembly code
tracei
Tracing at the machine-instruction level
stopi
Set breakpoints at the machine-instruction level
dis
Disassemble 10 instructions, starting at the value of `+'
print expression, ...
Print the value of one or more expressions expression, ...
regs [-f] [F]
Print value of registers
-f: include floating-point registers (single precision)
-F: Include floating-point registers (double precision)
The Problem Statement
To demonstrate machine-instruction level debugging, let's use a real bug report that was filed against the 64-bit dbx on the AMD64 platform, including a test case. The bug report says “
On AMD64 dbx prints hex values instead of letters after strchr call:
(dbx) print strchr("hello", 'l') = 0xfffffd7fffdff742 "xdfxff^?xfdxffxff^D"Here is the test case:
char *b = "hello";
printf("%s
", b);
printf("%s
", strchr("hello", 'l'));
}
There is nothing wrong with the program. The bug is in dbx.
The dbx Failure
First let's observe the normal flow of the program in the dbx environment by just stepping through the test case code.
% dbx a.out
Reading a.out
Reading ld.so.1
Reading libc.so.1
(dbx) stop in main
(2) stop in main
(dbx) run
Running: a.out
(process id 16245)
stopped in main at line 3 in file "1.c"
3 char *b = "hello";
(dbx) next
stopped in main at line 4 in file "1.c"
4 printf("%s
", b);
(dbx) next
hello
stopped in main at line 5 in file "1.c"
5 printf("%s
", strchr("hello", 'l'));
(dbx) next
llo
stopped in main at line 6 in file "1.c"
6 }
(dbx) next
execution completed, exit code is 4
The print statement at line 5 calls the strchr() function with two parameters. The strchr() function searches through the first parameter hello and returns a pointer to the first occurrence of the l character. Hence, the llo character string is displayed correctly by the printf statement.
Now let's reproduce the failure by calling the strchr() function directly from the dbx command line using the print command. The call command in dbx can also be used to call the strchr() function from the command line.
% dbx a.out
Reading a.out
Reading ld.so.1
Reading libc.so.1
(dbx) stop in main
(2) stop in main
(dbx) run
Running: a.out (process id 14772)
stopped in main at line 3 in file "1.c"<
3 char *b = "hello";
(dbx) next stopped in main at line 4 in file "1.c"
4 printf("%s
", b);
(dbx) next hello
stopped in main at line 5 in file "1.c"
5 printf("%s
", strchr("hello", 'l'));
(dbx) print strchr("hello", 'l')strchr("hello", 'l') = 0xfffffd7fffdff742 "xdfxff^?xfdxffxff^D"
dbx prints incorrect output when the strchr() function is called by the print command. dbx should display the llo string instead of hex characters, since the call to the strchr() function is supposed to return a pointer to the first occurrence of the l character in the string hello.
The Debugging Session
Let's run the debugger with the a.out executable and stop right before the printf statement. The strchr() function is defined in the libc library and most likely is not compiled with the -g option. So there is no debugging information and we have to rely on the assembly code only.
The stopi command is used to set a breakpoint at the first machine instruction of the strchr() function.
% dbx a.out
Reading a.out
Reading ld.so.1
Reading libc.so.1
(dbx) stop in main
(2) stop in main
(dbx) run
Running: a.out (process id 15045)
stopped in main at line 3 in file "1.c"
3 char *b = "hello";
(dbx) next
stopped in main at line 4 in file "1.c"
4 printf("%s
", b);
(dbx) next
hello
stopped in main at line 5 in file "1.c"
5 printf("%s
", strchr("hello", 'l'));
(dbx) stopi at strchr
(3) stopi at &strchr
(dbx) print strchr("hello", 'l')stopped in strchr at 0xfffffd7fff307910
0xfffffd7fff307910: strchr : movb& (%rdi),%dl
dbx stops at the first instruction of the strchr() function after the strchr() function is called from the dbx command line using the print command.
The dis command can be used to display the first portion of machine instructions for the strchr() function.
(dbx) dis strchr
0xfffffd7fff307910: strchr : movb (%rdi),%dl
0xfffffd7fff307912: strchr+0x0002: cmpb %dh,%dl
0xfffffd7fff307915: strchr+0x0005: je strchr+0x3f [0xfffffd7fff30794f, .+0x3a ]
0xfffffd7fff307917: strchr+0x0007: testb %dl,%dl
0xfffffd7fff307919: strchr+0x0009: je strchr+0x33 [0xfffffd7fff307943, .+0x2a ]
0xfffffd7fff30791b: strchr+0x000b: movb 0x0000000000000001(%rdi),%dl
0xfffffd7fff30791e: strchr+0x000e: mpb %dh,%dl
0xfffffd7fff307921: strchr+0x0011: je strchr+0x3c [0xfffffd7fff30794c, .+0x2b ]
0xfffffd7fff307923: strchr+0x0013: testb %dl,%dl
0xfffffd7fff307925: strchr+0x0015: je strchr+0x33 [0xfffffd7fff307943, .+0x1e ]
The first instruction of the strchr() function is movb (%rdi),%dl, which moves the contents of the memory location pointed to by the %rdi register to the low eight bits of the %rdi register itself. The first instruction is not the pushq %rbp instruction, which means the strchr() function has no prologue. It is not a defect that the function does not have a prologue.
The debugger is stopped at the first instruction, which is the right place in the program to verify whether the input parameters are being passed correctly to the strchr() function. The strchr() function has two parameters. The first parameter is a pointer to the memory location that contains the hello character string and the second parameter is the character l. Based on the AMD64 ABI, the first and second parameters are assigned to the %rdi and %rsi registers in sequence. There are two ways to display the content of the %rdi and %rsi registers.
You can use the print command to print the contents of the individual registers. The -flx options force dbx to display the contents of the %rdi and %rsi registers in long-hex format.
(dbx) print -flx $rdi
$rdi = 0xfffffd7fffdff740
(dbx) print -flx $rsi
$rsi = 0x6cYou can use the regs command to display the contents of all of the AMD64 registers.
(dbx) regs
current frame: [1]
r15 0x0000000000000000
r14 0x0000000000000000
r13 0x0000000000000000
r12 0x0000000000000000
r11 0xfffffd7fff307910
r10 0x0000000000000000
r9 0x0000000000010000
r8 0xfefeff6e6b6b6467
rdi 0xfffffd7fffdff740
rsi 0x000000000000006c
rbp 0xfffffd7fffdff810
rbx 0xfffffd7fff3fb190
rdx 0x0000000000000000
rcx 0x000000003f570d87
rax 0x0000000000000000
trapno 0x0000000000000003
err 0x0000000000000000
rip 0xfffffd7fff307910:strchr movb (%rdi),%dl
cs 0x000000000000004b
eflags 0x0000000000000282
rsp 0xfffffd7fffdff738
ss 0x0000000000000043
fs 0x00000000000001bb
gs 0x0000000000000000
es 0x0000000000000000
ds 0x0000000000000000
fsbase 0xfffffd7fff3a2000
gsbase& 0xffffffff80000000
The %rdi register contains a pointer to the memory location 0xfffffd7fffdff740, which is allocated on the stack. In the normal program flow, the %rdi register contains a pointer to the memory location in the data segment. However, when dbx is asked to call a function (strchr()), dbx copies the memory location in the data segment onto the stack and passes the stack address to the %rdi register.
The contents of the memory location 0xfffffd7fffdff740 can be verified by using the examine command. The memory location should contain the hello character string.
(dbx) examine 0xfffffd7fffdff740 / 20xfffffd7fffdff740: 0x6c6c6568 0x0000006fBy looking up the ASCII table, we can verify that indeed the memory location 0xfffffd7fffdff740 contains the hello character string. The hex number 68 stands for the character h, 65 stands for the character e, 6c stands for the character l, and 6f stands for the character o.
You can use the examine command directly to display the contents of the memory location 0xfffffd7fffdff740 as a character string without referring to the ASCII table .
(dbx) examine 0xfffffd7fffdff740 / 6c0xfffffd7fffdff740: 'h' 'e' 'l' 'l' 'o' '