background writer进程是什么
background writer进程是将共享内存中的脏页写入磁盘的进程。
在对数据进行处理之前都需要将数据从磁盘读到内存中,更新完毕后再将修改的数据写回磁盘,在postgresql中由backgroud writer与server process负责将“脏“数据写回磁盘。
大致过程是这样的:
先由backgroud writer进程进行写,剩余的数据再由server process进程完成
background writer进程的相关参数
background writer在postgresql.conf中有一个专门的部分来配置它的行为
# - Background Writer - #bgwriter_delay = 200ms # 10-10000ms between rounds #bgwriter_lru_maxpages = 100 # max buffers written/round, 0 disables #bgwriter_lru_multiplier = 2.0 # 0-10.0 multiplier on buffers scanned/round #bgwriter_flush_after = 512kB # measured in pages, 0 disables
bgwriter_delay
backgroud writer进程连续两次flush数据之间的时间的间隔。默认值是200,单位是毫秒。
bgwriter_lru_maxpages
backgroud writer进程每次写的最大数据量,默认值是100,单位buffers。如果脏数据量小于该数值,写操作全部由backgroud writer进程完成;反之,大于该值时,大于的部分将由server process进程完成。设置该值为0时表示禁用backgroud writer写进程,完全由server process来完成;配置为-1时表示所有脏数据都由backgroud writer来完成。(这里不包括checkpoint操作)
bgwriter_lru_multiplier
此参数指示了每次写入磁盘的数据块数,当然,该值必须小于bgwriter_lru_maxpages。如果设置过小,需要写入的脏数据量大于每次写入的数据量,剩下的需要写入磁盘的工作,就需要server processes来完成,这将会降低性能。如果值配置太大,此时写入的脏数据量就会超过所需的缓冲区数量,这虽然便于以后再次应用缓冲区工作,但同时可能会出现IO浪费。这个参数的默认值是2.0。bgwriter的最大数据量计算方式:
1000/bgwriter_delay*bgwriter_lru_maxpages*8K=最大数据量
上述参数对于数据文件所在磁盘的带宽也很有参考意义
参数值依据系统状态而设置,通过监控系统的运行状况,进行分析总结,为参数配置合适的值。参考数据字典pg_stat_bgwriter。
postgres=# d pg_stat_bgwriter View "pg_catalog.pg_stat_bgwriter" Column | Type | Collation | Nullable | Default -----------------------+--------------------------+-----------+----------+--------- checkpoints_timed | bigint | | | checkpoints_req | bigint | | | checkpoint_write_time | double precision | | | checkpoint_sync_time | double precision | | | buffers_checkpoint | bigint | | | buffers_clean | bigint | | | maxwritten_clean | bigint | | | buffers_backend | bigint | | | buffers_backend_fsync | bigint | | | buffers_alloc | bigint | | | stats_reset | timestamp with time zone | | | postgres=# SELECT pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints_timed, pg_stat_get_bgwriter_requested_checkpoints() AS checkpoints_req, pg_stat_get_bgwriter_buf_written_checkpoints() AS buffers_checkpoint, pg_stat_get_bgwriter_buf_written_clean() AS buffers_clean, pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean, pg_stat_get_buf_written_backend() AS buffers_backend, pg_stat_get_buf_alloc() AS buffers_alloc; checkpoints_timed | checkpoints_req | buffers_checkpoint | buffers_clean | maxwritten_clean | buffers_backend | buffers_alloc -------------------+-----------------+--------------------+---------------+------------------+-----------------+--------------- 590 | 3 | 3 | 0 | 0 | 0 | 226 (1 row)
bgwriter_flush_after
当数据页大小达到bgwriter_flush_after时触发BgWriter,默认值为512KB。
为什么需要background writer
postgresql中脏数据的写入不仅仅是由backgroud writer参数决定的,checkpoint也对脏数据的写入进行了控制。
backgroud writer进程的间隔是以毫秒计算,而checkpoint的间隔时间相对要很长。所以bgwriter更像是在checkpoint的周期内去完成脏数据的写入。所以bgwriter进程如果能够实现周期均匀数据量合适的写入数据,减少IO的次数,对checkpoint也能够减轻压力,不至于发生集中的写入操作,而造成IO使用的高值,可以说backgroud writer间接使系统的IO负载趋于稳定。