kill一个pthread_test.bin测试程序主线程、子线程退出kernel flow

kill一个pthread_test.bin测试程序主线程、子线程退出kernel flow

以下blog基于andorid Q，kernel 4.19

pthread_test.bi是执行pthread_test.bin的主线程，在这个主线程里会创建pthread_test_name线程

执行kill -9将pthread_test.bin进程kill，将会执行kill系统调用。

kill系统调用会进入kernel space

这个系统调用会给pthread_test.bin主线程以及子线程发送kill信号，发送call stack如下：

主线程：

[   68.942573] pthread_test.bi wake_up_state: 0x100.
[   68.942578] CPU: 3 PID: 4606 Comm: sh Tainted: P           O      4.19.116+ #34
[   68.942581] Hardware name: test_mach (DT)
[   68.942583] Call trace:
[   68.942588] dump_backtrace+0x0/0x4
[   68.942593] dump_stack+0xf4/0x134
[   68.942599] signal_wake_up_state+0x110/0x114
[   68.942605] complete_signal+0x240/0x2b0
[   68.942610] __send_signal+0x3b4/0x514
[   68.942616] do_send_sig_info+0x11c/0x1c4
[   68.942620] kill_pid_info+0xb0/0x130
[   68.942625] __arm64_sys_kill+0x1a8/0x598
[   68.942630] el0_svc_common+0xb8/0x1b8
[   68.942635] el0_svc_handler+0x74/0x90
[   68.942639] el0_svc+0x8/0x340

子线程（pthread_test_na）:

[   68.942718] pthread_test_na wake_up_state: 0x100.
[   68.942724] CPU: 3 PID: 4606 Comm: sh Tainted: P           O      4.19.116+ #34
[   68.942726] Hardware name: test_mach (DT)
[   68.942729] Call trace:
[   68.942734] dump_backtrace+0x0/0x4
[   68.942739] dump_stack+0xf4/0x134
[   68.942745] signal_wake_up_state+0xf8/0x114
[   68.942750] complete_signal+0x240/0x2b0
[   68.942756] __send_signal+0x3b4/0x514
[   68.942761] do_send_sig_info+0x11c/0x1c4
[   68.942766] kill_pid_info+0xb0/0x130
[   68.942770] __arm64_sys_kill+0x1a8/0x598
[   68.942775] el0_svc_common+0xb8/0x1b8
[   68.942780] el0_svc_handler+0x74/0x90
[   68.942785] el0_svc+0x8/0x340

发送kill信号后，主线程以及子线程的task_struct.thread_info.flags的TIF_SIGPENDING bit将会置上，然后分别wake_up主线程、子线程（测试程序里主线程、子线程都call了sleep使它俩都进入了sleep状态）

为什么会给主线程以及子线程都发送kill信号呢？原因在如下kill_pid_info()里，将会拿到进程的所有线程依次调用group_send_sig_info()

kernel/signal.c

int kill_pid_info(int sig, struct siginfo *info, struct pid *pid)
{
    int error = -ESRCH;
    struct task_struct *p;

    for (;;) {
        rcu_read_lock();
        p = pid_task(pid, PIDTYPE_PID);
        if (p)
            error = group_send_sig_info(sig, info, p);
        rcu_read_unlock();
        if (likely(!p || error != -ESRCH))
            return error;

        /*
         * The task was unhashed in between, try again.  If it
         * is dead, pid_task() will return NULL, if we race with
         * de_thread() it will find the new leader.
         */
    }
}

唤醒后，主线程、子线程将分别各自去处理这个kill信号，处理call stack如下：

主线程：

[   68.943810] pthread_test process(ffffffc05ef80000) exit_files(ffffffc05844c840).
[   68.943821] files is not null.
[   68.943830] CPU: 1 PID: 4863 Comm: pthread_test.bi Tainted: P           O      4.19.116+ #34
[   68.943839] Hardware name: test_mach (DT)
[   68.943844] Call trace:
[   68.943851] dump_backtrace+0x0/0x4
[   68.943859] dump_stack+0xf4/0x134
[   68.943867] exit_files+0x110/0x13c
[   68.943872] do_exit+0x8ec/0x1420
[   68.943878] do_group_exit+0x11c/0x160
[   68.943882] do_signal_stop+0x0/0x420
[   68.943888] do_notify_resume+0x170/0x25e8
[   68.943894] work_pending+0x8/0x10

子线程：

[   68.942913] pthread_test_name thread(ffffffc0690d1000) exit_files(ffffffc05844c840).
[   68.942920] CPU: 0 PID: 4864 Comm: pthread_test_na Tainted: P           O      4.19.116+ #34
[   68.942923] Hardware name: test_mach (DT)
[   68.942925] Call trace:
[   68.942930] dump_backtrace+0x0/0x4
[   68.942935] dump_stack+0xf4/0x134
[   68.942943] exit_files+0xd8/0x13c
[   68.942947] do_exit+0x8ec/0x1420
[   68.942952] do_group_exit+0x11c/0x160
[   68.942957] do_signal_stop+0x0/0x420
[   68.942962] do_notify_resume+0x170/0x25e8
[   68.942966] work_pending+0x8/0x10

从sleep系统调用返回user space时检查pending signal，发现此kill信号，各自执行do_group_exit()

退出后，这两个线程的task_struct结构体将延迟被free，free callstack：

子线程：

[   68.958319] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P           O      4.19.116+ #34
[   68.958323] Hardware name: test_mach (DT)
[   68.958327] Call trace:
[   68.958339] dump_backtrace+0x0/0x4
[   68.958347] dump_stack+0xf4/0x134
[   68.958355] free_task+0x158/0x194
[   68.958361] __put_task_struct+0x208/0x2e0
[   68.958366] delayed_put_task_struct+0x138/0x168
[   68.958373] rcu_process_callbacks+0x670/0xa34
[   68.958378] __do_softirq+0x1f8/0x490
[   68.958384] irq_exit+0x1d4/0x244
[   68.958390] __handle_domain_irq+0x140/0x1e0
[   68.958394] gic_handle_irq+0x50/0xbc
[   68.958399] el1_irq+0xe8/0x190
[   68.958405] arch_cpu_idle+0x208/0x470
[   68.958411] do_idle+0x208/0x340
[   68.958415] cpu_startup_entry+0x94/0x98
[   68.958422] kernel_init+0x0/0x514
[   68.958430] setup_command_line+0x0/0xc8

主线程：

[   68.962342] pthread_test thread exit.
[   68.962357] CPU: 1 PID: 17 Comm: ksoftirqd/1 Tainted: P           O      4.19.116+ #34
[   68.962361] Hardware name: test_mach (DT)
[   68.962364] Call trace:
[   68.962377] dump_backtrace+0x0/0x4
[   68.962385] dump_stack+0xf4/0x134
[   68.962393] free_task+0x16c/0x194
[   68.962398] __put_task_struct+0x208/0x2e0
[   68.962403] delayed_put_task_struct+0x138/0x168
[   68.962410] rcu_process_callbacks+0x670/0xa34
[   68.962415] __do_softirq+0x1f8/0x490
[   68.962421] run_ksoftirqd+0x3c/0x54
[   68.962426] smpboot_thread_fn+0x1b0/0x498
[   68.962432] kthread+0x130/0x140
[   68.962437] ret_from_fork+0x10/0x18