用户态调试
原文地址：[optimize]使用systemtap调试用户态程序
摘要
systemtap既能调试内核，也能调试用户态. 然后有if有判断有循环．可编程就代表什么都行，那么给我一个用gdb的理由？ 另外，世界之大，请不要忘记DTrace，linux 4.9之后支持．
新手扫盲文档：https://sourceware.org/systemtap/SystemTap_Beginners_Guide/index.html
由于我非常懒，所以只写了些例子在下面．建议先读一下上面的文档知道一下基本概念，之后再读我下面的例子，便基本可以进行简单的实践了．
用户态调试的依赖条件
3.5之前的linux需要打开内核选项CONFIG_UTRACE
grep CONFIG_UTRACE /boot/config-`uname -r`
3.5之后的linux需要打开内核选项CONFIG_UPROBES, CONFIG_UTRACE从这个版本开始被废弃．
grep CONFIG_UPROBES /boot/config-`uname -r`
其他
需要特别强调的是，systemtap除了跟踪函数以外，还可以跟踪行噢，使用：
process("a.out").statement("*@main.c:100")
例子１：最简单的调试用户态程序
stap -d /usr/sbin/ls --ldd -e 'probe process("ls").function("xmalloc") {print_usyms(ubacktrace())}' -c "ls /"
例子２：查看可以用来调试的函数
stap -L 'process("./nginx").function("*")' // 查看ｎｇｉｎｘ里的所有函数．并带着参数 stap -l 'process("./nginx").function("*")' // 同上，但是不带参数．
例子３：第一个简单的跟踪例子
[root@T9 sbin]# stap -e 'probe process("./nginx").function("ngx_process_events_and_timers"){print("test: xxx ")}' test: xxx test: xxx ．．．
下面的每次运行，会触发上面的打印信息．
[root@T9 sbin]# curl 127.0.0.1:5000 <html> <head><title>400 The plain HTTP request was sent to HTTPS port</title></head> <body> <center><h1>400 Bad Request</h1></center> <center>The plain HTTP request was sent to HTTPS port</center> <hr><center>JDCLOUD_ALB</center> </body> </html>
例子４：打印变量
找到这个变量的引用
[root@T9 sbin]# stap -L 'process("./nginx").function("ngx_http_process_request_header") ' process("/root/OUTPUT_alb/sbin/nginx").function("ngx_http_process_request_header@src/http/ngx_http_request.c:2184") $r:ngx_http_request_t* [root@T9 sbin]# stap -L 'process("./nginx").function("ngx_http_process_request_header").return ' process("/root/OUTPUT_alb/sbin/nginx").function("ngx_http_process_request_header@src/http/ngx_http_request.c:2184").return $return:ngx_int_t $r:ngx_http_request_t*
自动扩展打印，上面的变量
[root@T9 sbin]# stap -e 'probe process("./nginx").function("ngx_http_process_request_headers").return {print($$vars, " ");}' WARNING: confusing usage, consider @entry($$vars) in .return probe: identifier '$$vars' at <input>:1:85 source: probe process("./nginx").function("ngx_http_process_request_headers").return {print($$vars, " ");} ^ rev=0x55714398b950 p=0x30 len=0x0 n=0x14 rc=0x7ffea5393130 rv=0x557142aab232 h=0x5571439283d0 c=0x557142a3d91a hh=0x557143928488 r=0x3c0 cscf=0x5571439283d0 cmcf=0x7ffea5393160 ^C[root@T9 sbin]# stap -e 'probe process("./nginx").function("ngx_http_process_request_headers").return {print($n, " ");}' WARNING: confusing usage, consider @entry($n) in .return probe: identifier '$n' at <input>:1:85 source: probe process("./nginx").function("ngx_http_process_request_headers").return {print($n, " ");} ^ 20 20
例子５：使用脚本
[root@T9 sbin]# cat nginx.stp //probe process("./nginx").function("ngx_http_process_request_headers") { probe process("./nginx").function("ngx_http_parse_header_line") { print($$vars, " "); print(user_string($b->pos), " "); exit(); } [root@T9 sbin]# stap -v nginx.stp Pass 1: parsed user script and 474 library scripts using 245204virt/42624res/3472shr/39252data kb, in 440usr/40sys/491real ms. Pass 2: analyzed script: 1 probe, 13 functions, 1 embed, 0 globals using 248112virt/46600res/4496shr/42160data kb, in 40usr/10sys/39real ms. Pass 3: using cached /root/.systemtap/cache/cd/stap_cd35c88067ad0752258f5c08e9a79aa9_4346.c Pass 4: using cached /root/.systemtap/cache/cd/stap_cd35c88067ad0752258f5c08e9a79aa9_4346.ko Pass 5: starting run. r=0x557143928420 b=0x557143915520 allow_underscores=0x0 c=0x0 ch=0x0 p=0x557143969d60 hash=0x55714398b950 i=0x557142a3da19 state=0x0 lowcase=[...] User-Agent: curl/7.29.0 Host: 127.0.0.1:5000 Accept: */* Pass 5: run completed in 20usr/60sys/6613real ms. [root@T9 sbin]#
例子６：复杂的变量打印
[root@T9 sbin]# cat nginx.conn.stp //probe process("./nginx").function("ngx_http_process_request_headers") { probe process("./nginx").function("ngx_http_init_connection").return { print(@entry($$vars), " "); print("sa_family: ", @entry($c->local_sockaddr->sa_family), " "); print("conn: ", @entry($c$$), " "); print("conn.local_sockaddr: ", @cast(@entry($c->local_sockaddr), "struct sockaddr_in")$, " "); print("local port: ", @cast(@entry($c->local_sockaddr), "struct sockaddr_in")->sin_port, " "); print("port: ", @cast(@entry($c->sockaddr), "struct sockaddr_in")->sin_port, " "); // exit(); } [root@T9 sbin]# stap -v nginx.conn.stp Pass 1: parsed user script and 474 library scripts using 245204virt/42624res/3472shr/39252data kb, in 440usr/20sys/473real ms. Pass 2: analyzed script: 2 probes, 56 functions, 3 embeds, 12 globals using 373612virt/172304res/4824shr/167660data kb, in 1190usr/420sys/1612real ms. Pass 3: using cached /root/.systemtap/cache/d6/stap_d6e87115e317f5b8522d63cd053dc232_19996.c Pass 4: using cached /root/.systemtap/cache/d6/stap_d6e87115e317f5b8522d63cd053dc232_19996.ko Pass 5: starting run. c=0x557143969e70 i=0x0 rev=0xf sin=0x0 port=0x557143969e70 addr=0x5571439154c1 ctx=0x557143915450 hc=0xf sin6=0x557143919ee8 addr6=0x557143915450 cscf=0x557143919f40 sa_family: 2 conn: {.data=0x0, .read=0x55714398b9b0, .write=0x5571439979c0, .fd=3, .recv=0x557142a7d60a, .send=0x557142a7df72, .recv_chain=0x557142a7d8d4, .send_chain=0x557142a88625, .listening=0x557143917960, .sent=0, .revt=0, .log=0x557143915460, .pool=0x557143915400, .type=1, .sockaddr=0x557143915450, .socklen=16, .addr_text={.len=9, .data="127.0.0.1377377377"}, .proxy_protocol_addr={.len=0, .data=0x0}, .proxy_protocol_port=0, .ssl=0x0, .asynch=0, .async_fd=0, .udp=0x0, .local_sockaddr=0x5571439581f0, .local_socklen=16 conn.local_sockaddr: {.sin_family=2, .sin_port=35091, .sin_addr={...}, .sin_zero=[...]} local port: 35091 port: 6311
例子７：条件判断的例子
使用如下命令操作，分别访问端口５０００，和５００１．会有不同的打印日志：
[root@T9 sbin]# curl -q 127.0.0.1:5000 >/dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 254 100 254 0 0 240k 0 --:--:-- --:--:-- --:--:-- 248k [root@T9 sbin]# curl -q 127.0.0.1:5001 >/dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 612 100 612 0 0 386k 0 --:--:-- --:--:-- --:--:-- 597k
打印输出：
[root@T9 sbin]# cat nginx.conn.stp //probe process("./nginx").function("ngx_http_process_request_headers") { global server_port = 0 probe process("./nginx").function("ngx_http_init_connection").return { print(@entry($$vars), " "); print("sa_family: ", @entry($c->local_sockaddr->sa_family), " "); print("conn: ", @entry($c$$), " "); print("conn.local_sockaddr: ", @cast(@entry($c->local_sockaddr), "struct sockaddr_in")$, " "); print("local port: ", @cast(@entry($c->local_sockaddr), "struct sockaddr_in")->sin_port, " "); print("port: ", @cast(@entry($c->sockaddr), "struct sockaddr_in")->sin_port, " "); server_port = @cast(@entry($c->local_sockaddr), "struct sockaddr_in")->sin_port; // 34835 35091 if (server_port == 34835) { printf("hahahahaha 5000: %d ", server_port); } printf("ennnnnnnnnnnnnnnnnd "); // exit(); } [root@T9 sbin]# stap -v nginx.conn.stp Pass 1: parsed user script and 474 library scripts using 245204virt/42628res/3472shr/39252data kb, in 400usr/20sys/424real ms. Pass 2: analyzed script: 2 probes, 57 functions, 3 embeds, 15 globals using 373624virt/172320res/4828shr/167672data kb, in 1120usr/350sys/1509real ms. Pass 3: using cached /root/.systemtap/cache/b3/stap_b3bb3ceb4342deea4b0c8d77121e806b_21075.c Pass 4: using cached /root/.systemtap/cache/b3/stap_b3bb3ceb4342deea4b0c8d77121e806b_21075.ko Pass 5: starting run. c=0x557143969d60 i=0x0 rev=0xf sin=0x0 port=0x557143969d60 addr=0x5571439154c1 ctx=0x557143915450 hc=0xf sin6=0x557143919ee8 addr6=0x557143915450 cscf=0x557143919f40 sa_family: 2 conn: {.data=0x0, .read=0x55714398b950, .write=0x557143997960, .fd=3, .recv=0x557142a7d60a, .send=0x557142a7df72, .recv_chain=0x557142a7d8d4, .send_chain=0x557142a88625, .listening=0x557143917828, .sent=0, .revt=0, .log=0x557143915460, .pool=0x557143915400, .type=1, .sockaddr=0x557143915450, .socklen=16, .addr_text={.len=9, .data="127.0.0.1377377377"}, .proxy_protocol_addr={.len=0, .data=0x0}, .proxy_protocol_port=0, .ssl=0x0, .asynch=0, .async_fd=0, .udp=0x0, .local_sockaddr=0x5571439581a0, .local_socklen=16 conn.local_sockaddr: {.sin_family=2, .sin_port=34835, .sin_addr={...}, .sin_zero=[...]} local port: 34835 port: 8864 hahahahaha 5000: 34835 ennnnnnnnnnnnnnnnnd c=0x557143969d60 i=0x0 rev=0xf sin=0x0 port=0x557143969d60 addr=0x5571439154c1 ctx=0x557143915450 hc=0xf sin6=0x557143919ee8 addr6=0x557143915450 cscf=0x557143919f40 sa_family: 2 conn: {.data=0x0, .read=0x55714398b950, .write=0x557143997960, .fd=3, .recv=0x557142a7d60a, .send=0x557142a7df72, .recv_chain=0x557142a7d8d4, .send_chain=0x557142a88625, .listening=0x557143917960, .sent=0, .revt=0, .log=0x557143915460, .pool=0x557143915400, .type=1, .sockaddr=0x557143915450, .socklen=16, .addr_text={.len=9, .data="127.0.0.1377377377"}, .proxy_protocol_addr={.len=0, .data=0x0}, .proxy_protocol_port=0, .ssl=0x0, .asynch=0, .async_fd=0, .udp=0x0, .local_sockaddr=0x5571439581f0, .local_socklen=16 conn.local_sockaddr: {.sin_family=2, .sin_port=35091, .sin_addr={...}, .sin_zero=[...]} local port: 35091 port: 9895 ennnnnnnnnnnnnnnnnd