容器中的进程对linux信号的处理方式
在一个容器启动的时候,CMD 或者 ENTRYPOINT 里定义的命令会作为容器的主进程(main process)启动,pid 为 1,一旦这个主进程退出了,容器也会被销毁,容器内其他进程会被 kernel 直接 kill。
shell 程序除了不转发 signals,还有个更可气的特性是不响应退出信号SIGTERM和SIGKILL
但是,我们直接在linux上执行一个shell脚本,是可以通过kill -SIGTERM pid来结束掉的,就是为什么这个时候sh又响应退出信号了?
这是因为kernel会为每个进程加上默认的 signal handler,例外的是 pid=1 的进程,被 kernel 当作一个 init 角色,不会给他加上默认的 handler
在容器中,如果pid为1的进程是shell进程,那么这个进程就真的不会响应或者转发信号了,使用docker stop来停止这样的容器,结果就是docker发送给容器pid为1主进程的SIGTERM信号被忽略,等到stop命令默认的10s graceperiod到了
之后,docker发出的SIGKILL会最终结束这个容器。同样的,这样的容器部署在k8s中,当这个pod被删除时(认为控制的或者是rc等控制的),Terminating的时长将达到30s以上(k8s默认的podGracePeriod为30s)
参考SIGINT SIGTERM SIGKILL区别
程序收到信号后,如果不对信号处理,就会导致程序退出,但如果程序捕获信号进行处理,按照它的逻辑,它是不一定会退出的。
在这三个信号中,sigkill是不能被捕获的,程序收到这个信号后,一定会退出。这就是kill -9一定能保证将程序杀死的原因。
sigterm与sigint的区别
信号 产生方式 对进程的影响
sigint 通过ctrl+c将会对当进程发送此信号 信号被当前进程树接收到,也就是说,不仅当前进程会收到信号,它的子进程也会收到
sigterm kill命令不加参数就是发送这个信号 只有当前进程收到信号,子进程不会收到。如果当前进程被kill了,那么它的子进程的父进程将会是init,也就是pid为1的进程
两个信号可以停止进程:SIGTERM和SIGKILL。 SIGTERM比较友好,进程能捕捉这个信号,根据您的需要来关闭程序。在关闭程序之前,您可以结束打开的记录文件和完成正在做的任务。在某些情况下,假 如进程正在进行作业而且不能中断,那么进程可以忽略这个SIGTERM信号。
k8s中Pod的终止过程
Kubernetes 在创建容器后立即发送 postStart 事件。但是,不能保证 postStart 处理程序 在容器的 entrypoint 调用之前被调用。相对于容器的代码,postStart 处理程序以异步方式运行,但 Kubernetes 对容器的管理 会阻塞直到 postStart 处理程序完成。容器的状态直到 postStart 处理程序完成后才会设置为 RUNNING 。
Kubernetes 在容器终止之前立即发送 preStop 事件。 Kubernetes 对容器的管理一直阻塞直到 preStop 处理程序完成, 除非 Pod 的宽限期过期。
Termination of Pods
Because Pods represent running processes on nodes in the cluster, it is important to allow those processes to gracefully terminate when they are no longer needed (vs being violently killed with a KILL signal and having no chance to clean up). Users should be able to request deletion and know when processes terminate, but also be able to ensure that deletes eventually complete. When a user requests deletion of a Pod, the system records the intended grace period before the Pod is allowed to be forcefully killed, and a TERM signal is sent to the main process in each container. Once the grace period has expired, the KILL signal is sent to those processes, and the Pod is then deleted from the API server. If the Kubelet or the container manager is restarted while waiting for processes to terminate, the termination will be retried with the full grace period.
An example flow:
User sends command to delete Pod, with default grace period (30s)
The Pod in the API server is updated with the time beyond which the Pod is considered “dead” along with the grace period.
Pod shows up as “Terminating” when listed in client commands
(simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process.
If one of the Pod’s containers has defined a preStop hook, it is invoked inside of the container. If the preStop hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) extended grace period.
The container is sent the TERM signal. Note that not all containers in the Pod will receive the TERM signal at the same time and may each require a preStop hook if the order in which they shut down matters.
(simultaneous with 3) Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers. Pods that shutdown slowly cannot continue to serve traffic as load balancers (like the service proxy) remove them from their rotations.
When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
The Kubelet will finish deleting the Pod on the API server by setting grace period 0 (immediate deletion). The Pod disappears from the API and is no longer visible from the client.
By default, all deletes are graceful within 30 seconds. The kubectl delete command supports the --grace-period=<seconds> option which allows a user to override the default and specify their own value. The value 0 force deletes the Pod. You must specify an additional flag --force along with --grace-period=0 in order to perform force deletions.
Force deletion of pods
Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately. When a force deletion is performed, the API server does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It removes the Pod in the API immediately so a new Pod can be created with the same name. On the node, Pods that are set to terminate immediately will still be given a small grace period before being force killed.
Force deletions can be potentially dangerous for some Pods and should be performed with caution. In case of StatefulSet Pods, please refer to the task documentation for deleting Pods from a StatefulSet.
参考文档:
https://www.cnblogs.com/alexyuyu/articles/3853583.html
https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods
https://dzone.com/articles/gracefully-shutting-down-java-in-containers
https://www.jianshu.com/p/6d393cbb694ahttps://mp.weixin.qq.com/s?__biz=MzA5OTAyNzQ2OA==&mid=2649700264&idx=1&sn=f987b01eaffd5a9d6b0e899977053111&chksm=889308cbbfe481dde7228f3ce4a4088c22676b8c9db79b12948f9e5494ce829f5707cbc848e6&mpshare=1&scene=1&srcid=1015dM3AYEe6GhbKHLvAbkrV&sharer_sharetime=1571126439719&sharer_shareid=bc0f20da14de543a8c3c8d489c80ea9f&pass_ticket=VAE837i63Ws%2BsH38kXkOzNwIoYxC5%2BmfGevNFUitY%2F6DfvKJX7AkkyhyvFM0eJS%2B#rd