[Linux性能调优] 磁盘I/O队列调度策略

这两天的一个小任务是MongoDB服务器的调优，恰好这段时间对Linux的各种性能诊断、调优感兴趣，就顺着这个任务多翻了些书和文章。

新学到的一个东西是 Linux磁盘的I/O队列调度策略，至少MySQL和PostgreSQL都推荐调整这个：

• http://www.mysqlperformanceblog.com/2009/01/30/linux-schedulers-in-tpcc-like-benchmark/
• http://www.cybertec.at/postgresql-linux-kernel-io-tuning/

傻瓜化说明

简单地说，对于磁盘I/O，Linux提供了cfq, deadline和noop三种调度策略

• cfq: 这个名字是Complete Fairness Queueing的缩写，它是一个复杂的调度策略，按进程创建多个队列，试图保持对多个进程的公平（这就没考虑读操作和写操作的不同耗时）
• deadline: 这个策略比较简单，只分了读和写两个队列（这显然会加速读取量比较大的系统），叫这个名字是内核为每个I/O操作都给出了一个超时时间
• noop: 这个策略最简单，只有单个队列，只有一些简单合并操作

考虑到硬件配置、实际应用场景（读写比例、顺序还是随机读写）的差异，上面的简单解释对于实际选择没有太大帮助，实际该选择哪个基本还是要实测来验证。不过下面几条说明供参考：

• 根据多篇文章的说法，deadline和noop差异不是太大，但它们俩与cfq差异就比较大
• MySQL这类数据存储系统不要使用cfq（时序数据库可能会有所不同。不过也有说从来没见过deadline比cfq差的情况）
• 对于虚拟机上面的磁盘，建议采用比较简单的noop，毕竟数据实际上怎么落盘取决于虚拟化那一层
• 我手边几个vm的默认值是：centos6是cfq，ubuntu12.04是xxxx，centos7和ubuntu14.04是deadline （不过这只代表这几台，我不知道是否具有代表性）

用如下命令可以查到每个磁盘的当前设置

# cat /sys/block/sda/queue/scheduler 
[noop] deadline cfq

（方括号里面的是当前选定的调度策略）

用如下方法 即时 可以修改设置

echo deadline > /sys/block/sda/queue/scheduler
# or
echo deadline | sudo tee /sys/block/sda/queue/scheduler

附：《高性能MySQL》里面的相关说明

在《高性能MySQL》第九章 Operating System and Hardware Optimization（英文版的第434页）有如下内容（注意要第三版，在第二版里面还没有这一节）

Choosing a Disk Queue Scheduler

On GNU/Linux, the queue scheduler determines the order in which requests to a block
device are actually sent to the underlying device.

The default is Completely Fair Queueing, or cfq. It’s okay for casual use on laptops and desktops, where it helps prevent
I/O starvation, but it’s terrible for servers. It causes very poor response times under the types of workload that MySQL generates, because it stalls some requests in the queue
needlessly.

You can see which schedulers are available, and which one is active, with the following
command:

$ cat /sys/block/sda/queue/scheduler
noop deadline [cfq]

You should replace sda with the device name of the disk you’re interested in. In our
example, the square brackets indicate which scheduler is in use for this device.

The
other two choices are suitable for server-class hardware, and in most cases they work
about equally well. The noop scheduler is appropriate for devices that do their own
scheduling behind the scenes, such as hardware RAID controllers and SANs, and deadline is fine both for RAID controllers and disks that are directly attached. Our benchmarks show very little difference between these two. The main thing is to use anything
but cfq, which can cause severe performance problems.

Take this advice with a grain of salt, though, because the disk schedulers actually come
in many variations in different kernels, and there is no indication of that in their names.

细节参考

• https://blog.codeship.com/linux-io-scheduler-tuning/
• http://www.mysqlperformanceblog.com/2009/01/30/linux-schedulers-in-tpcc-like-benchmark