这里我们用虚拟机进行模拟问题。
有一天突然发现网站页面打不开了，报500错误，赶紧查看日志，发现报错如下
[Fri Jul 24 11:02:20.686132 2020] [:error] [pid 2322] [remote 10.0.0.253:8] IOError: [Errno 24] Too many open files: '/usr/share/openstack-dashboard/openstack_dashboard/themes/default/templates/500.html'
看到"Too many open files"这个报错，大家一般都能很快的反应到是哪里的问题，很明显是最大打开文件数量限制了。
我们首先进行确认：1. 检查系统可以打开的最大文件 2. httpd进程可以打开的最大文件数据
[root@controller ~]# ulimit -n 1024 [root@controller ~]# [root@controller ~]# [root@controller ~]# ps -ef|grep -v grep|grep httpd root 1319 1 0 02:55 ? 00:00:07 /usr/sbin/httpd -DFOREGROUND nova 1646 1319 0 02:55 ? 00:01:03 /usr/sbin/httpd -DFOREGROUND nova 1647 1319 0 02:55 ? 00:01:01 /usr/sbin/httpd -DFOREGROUND nova 1648 1319 0 02:55 ? 00:00:59 /usr/sbin/httpd -DFOREGROUND apache 1655 1319 0 02:55 ? 00:00:01 /usr/sbin/httpd -DFOREGROUND apache 2866 1319 0 03:01 ? 00:00:01 /usr/sbin/httpd -DFOREGROUND apache 8363 1319 0 05:19 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 14653 1319 0 07:56 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 17966 1319 0 09:14 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 17989 1319 0 09:15 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 18618 1319 0 09:30 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 22737 1319 0 11:08 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 27765 1319 0 13:14 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 28522 1319 0 13:34 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND [root@controller ~]# [root@controller ~]# cat /proc/1319/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 19032 19032 processes Max open files 1024 4096 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 19032 19032 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us
由输出可以看出系统默认最大只能打开文件数量为1024，httpd进程Max open files也为1024，难怪会报错了。
赶紧修改配置文件/etc/security/limits.conf，在修改这个文件之前首先查看系统支持最大打开多少文件数量，注意不同的系统下面的这个值可能会不一致，我的系统为CentOS7的虚拟机。
[root@controller ~]# cat /proc/sys/fs/file-max 480850
修改好的配置如下
[root@controller ~]# egrep -v '^#|^$' /etc/security/limits.conf * - nofile 480850
修改完了之后不需要重启系统，只需要退出当前session，重新登录修改即可生效。现在我们进行验证
[root@controller ~]# ulimit -n
480850
[root@controller ~]#
[root@controller ~]# systemctl restart httpd
[root@controller ~]#
[root@controller ~]# ss -tlnp|grep 80
LISTEN 0 128 *:4369 *:* users:(("epmd",pid=1806,fd=3),("systemd",pid=1,fd=42))
LISTEN 0 100 *:6080 *:* users:(("nova-novncproxy",pid=1300,fd=4))
LISTEN 0 128 [::]:80 [::]:* users:(("httpd",pid=31526,fd=4),("httpd",pid=31455,fd=4),("httpd",pid=31428,fd=4),("httpd",pid=31427,fd=4),("httpd",pid=31426,fd=4),("httpd",pid=31425,fd=4),("httpd",pid=31424,fd=4),("httpd",pid=31415,fd=4))
[root@controller ~]# ps -ef|grep httpd
root 31415 1 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
nova 31416 31415 4 14:36 ? 00:00:04 /usr/sbin/httpd -DFOREGROUND
nova 31417 31415 2 14:36 ? 00:00:02 /usr/sbin/httpd -DFOREGROUND
nova 31418 31415 2 14:36 ? 00:00:02 /usr/sbin/httpd -DFOREGROUND
apache 31424 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31425 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31426 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31427 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31428 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31455 31415 0 14:36 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 31526 31415 0 14:37 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
root 31603 31322 0 14:38 pts/1 00:00:00 grep --color=auto httpd
[root@controller ~]#
[root@controller ~]# cat /proc/31415/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 19032 19032 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 19032 19032 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
系统文件数量修改生效了，但是我们发现重启完httpd服务之后，httpd服务进程所能打开的最大文件数量依然是1024，怎么回事儿？
费了九牛二虎之力终于找到了原因，因为我们的httpd服务使用systemd来启动的，而systemd我们并没有配置最大可以打开的文件数量，默认为1024，所以必须要修改它的配置文件
vim /etc/systemd/system.conf，增加一行
DefaultLimitNOFILE=480850
这里这一步完事之后只需要systemctl daemon-reexec，即可使配置文件生效，不需要重新操作系统。
daemon-reexec
Reexecute the systemd manager. This will serialize the manager state, reexecute the process and deserialize the state
again. This command is of little use except for debugging and package upgrades. Sometimes, it might be helpful as a
heavy-weight daemon-reload. While the daemon is being reexecuted, all sockets systemd listening on behalf of user
configuration will stay accessible.
然后我们验证
[root@controller ~]# systemctl daemon-reexec [root@controller ~]# [root@controller ~]# [root@controller ~]# [root@controller ~]# systemctl restart httpd [root@controller ~]# [root@controller ~]# [root@controller ~]# ps -ef|grep httpd root 34213 1 1 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND nova 34214 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND nova 34215 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND nova 34216 34213 1 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 34222 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 34223 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 34224 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 34225 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 34226 34213 0 14:57 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND root 34260 31322 0 14:57 pts/1 00:00:00 grep --color=auto httpd [root@controller ~]# [root@controller ~]# [root@controller ~]# cat /proc/34213/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 19032 19032 processes Max open files 480850 480850 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 19032 19032 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us
我们可以看到httpd进程所能打开的最大文件数据量已经为480850了，已经设置成功了。