前言
sysbench是一个开源的、模块化的、跨平台的多线程性能测试工具,可以用来进行CPU、内存、磁盘I/O、线程、数据库的性能测试。sysbench是基于LuaJIT的可编写脚本的多线程基准测试工具。
sysbench附带以下捆绑的基准测试:
oltp_*.lua
:类似OLTP的数据库基准测试的集合fileio
:文件系统级基准cpu
:简单的CPU基准测试memory
:内存访问基准threads
:基于线程的调度程序基准mutex
:POSIX互斥基准测试
特征
- 提供有关速率和延迟的大量统计信息,包括延迟百分位数和直方图;
- 即使有数千个并发线程,开销也很低。sysbench每秒能够生成和跟踪数亿个事件。
- 通过在用户提供的Lua脚本中实现预定义的挂钩,可以轻松创建新的基准测试;
- 可以也可以作为一个通用的Lua解释,只需更换
#!/usr/bin/lua
用#!/usr/bin/sysbench
在你的脚本。
安装
安装以Centos为例,其他版本或者mac请参考sysbench说明
1.下载安转
wget https://github.com/akopytov/sysbench/archive/1.0.zip -O "sysbench-1.0.zip" unzip sysbench-1.0.zip cd sysbench-1.0
//安装依赖
yum install -y automake libtool
2.编译安装
./autogen.sh ./configure export LD_LIBRARY_PATH=/usr/include/mysql make make install
3.监测是否安装成功
[root@guanbin-k8s-master ~]# sysbench --version sysbench 1.0.19
测试
1. cpu测试
cpu测试主要是进行素数的加法运算,我们的例子中,指定了最大的质数发生器数量为 20000
[root@guanbin-k8s-master ~]# sysbench --test=cpu --cpu-max-prime=20000 run WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options. sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Prime numbers limit: 20000 Initializing worker threads... Threads started! CPU speed: events per second: 168.59 General statistics: total time: 10.0027s total number of events: 1687 Latency (ms): min: 5.63 avg: 5.93 max: 66.09 95th percentile: 6.43 sum: 9999.10 Threads fairness: events (avg/stddev): 1687.0000/0.00 execution time (avg/stddev): 9.9991/0.00
执行时间 大约为10秒(这个测试最好是两台机器进行对比测试,才能发现那个机器有问题)
2.对内存(memory)进行测试
测试是在内存中传输 10G 的数据量,每个 block 大小为 8K
[root@guanbin-k8s-master ~]# sysbench --test=memory --memory-block-size=8k --memory-total-size=10G run WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options. sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Running memory speed test with the following options: block size: 8KiB total size: 10240MiB operation: write scope: global Initializing worker threads... Threads started! Total operations: 1310720 (724931.70 per second) 10240.00 MiB transferred (5663.53 MiB/sec) General statistics: total time: 1.8045s total number of events: 1310720 Latency (ms): min: 0.00 avg: 0.00 max: 12.44 95th percentile: 0.00 sum: 1441.61 Threads fairness: events (avg/stddev): 1310720.0000/0.00 execution time (avg/stddev): 1.4416/0.00
totaltime为1.8045s
3.对disk 的IO测试
新建一个临时目录,并确保磁盘剩余空间足够(--file-total-size不要超过剩余容量)
测试场景:16个线程,创建128个文件,总共占用磁盘空间10GB,测试磁盘读写效率
运行结束后要删除临时测试文件: rm -rf test_file.*
root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
128 files, 8192Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
Creating file test_file.5
Creating file test_file.6
Creating file test_file.7
Creating file test_file.8
Creating file test_file.9
Creating file test_file.10
Creating file test_file.11
Creating file test_file.12
Creating file test_file.13
Creating file test_file.14
Creating file test_file.15
Creating file test_file.16
Creating file test_file.17
Creating file test_file.18
Creating file test_file.19
Creating file test_file.20
Creating file test_file.21
Creating file test_file.22
Creating file test_file.23
Creating file test_file.24
Creating file test_file.25
Creating file test_file.26
Creating file test_file.27
Creating file test_file.28
Creating file test_file.29
Creating file test_file.30
Creating file test_file.31
Creating file test_file.32
Creating file test_file.33
Creating file test_file.34
Creating file test_file.35
Creating file test_file.36
Creating file test_file.37
Creating file test_file.38
Creating file test_file.39
Creating file test_file.40
Creating file test_file.41
Creating file test_file.42
Creating file test_file.43
Creating file test_file.44
Creating file test_file.45
Creating file test_file.46
Creating file test_file.47
Creating file test_file.48
Creating file test_file.49
Creating file test_file.50
Creating file test_file.51
Creating file test_file.52
Creating file test_file.53
Creating file test_file.54
Creating file test_file.55
Creating file test_file.56
Creating file test_file.57
Creating file test_file.58
Creating file test_file.59
Creating file test_file.60
Creating file test_file.61
Creating file test_file.62
Creating file test_file.63
Creating file test_file.64
Creating file test_file.65
Creating file test_file.66
Creating file test_file.67
Creating file test_file.68
Creating file test_file.69
Creating file test_file.70
Creating file test_file.71
Creating file test_file.72
Creating file test_file.73
Creating file test_file.74
Creating file test_file.75
Creating file test_file.76
Creating file test_file.77
Creating file test_file.78
Creating file test_file.79
Creating file test_file.80
Creating file test_file.81
Creating file test_file.82
Creating file test_file.83
Creating file test_file.84
Creating file test_file.85
Creating file test_file.86
Creating file test_file.87
Creating file test_file.88
Creating file test_file.89
Creating file test_file.90
Creating file test_file.91
Creating file test_file.92
Creating file test_file.93
Creating file test_file.94
Creating file test_file.95
Creating file test_file.96
Creating file test_file.97
Creating file test_file.98
Creating file test_file.99
Creating file test_file.100
Creating file test_file.101
Creating file test_file.102
Creating file test_file.103
Creating file test_file.104
Creating file test_file.105
Creating file test_file.106
Creating file test_file.107
Creating file test_file.108
Creating file test_file.109
Creating file test_file.110
Creating file test_file.111
Creating file test_file.112
Creating file test_file.113
Creating file test_file.114
Creating file test_file.115
Creating file test_file.116
Creating file test_file.117
Creating file test_file.118
Creating file test_file.119
Creating file test_file.120
Creating file test_file.121
Creating file test_file.122
Creating file test_file.123
Creating file test_file.124
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
1073741824 bytes written in 110.77 seconds (9.24 MiB/sec).
[root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
Running the test with following options:
Number of threads: 16
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 8MiB each
1GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...
Threads started!
File operations:
reads/s: 157.44
writes/s: 104.96
fsyncs/s: 480.79
Throughput:
read, MiB/s: 2.46
written, MiB/s: 1.64
General statistics:
total time: 13.3355s
total number of events: 7865
Latency (ms):
min: 0.00
avg: 20.61
max: 1115.85
95th percentile: 142.39
sum: 162071.40
Threads fairness:
events (avg/stddev): 491.5625/141.20
execution time (avg/stddev): 10.1295/0.03
吞吐量:读2.46MiB/s,写1.64MiB/s; 95%延迟 142.39ms
4.测试mysql性能
测试时使用的脚本为lua脚本,可以使用sysbench自带脚本,也可以自己开发。对于大多数应用,使用sysbench自带的脚本就足够了。不同版本的sysbench中,lua脚本的位置可能不同,可以自己在sysbench路径下使用find命令搜索oltp.lua。
P.S.:大多数数据服务都是oltp类型的,如果你不了解什么是oltp,那么大概率你的数据服务就是oltp类型的。MySQL OLTP(On-Line Transaction Processing联机事务处理过程);
- (1)尽量不要在MySQL服务器运行的机器上进行测试,一方面可能无法体现网络(哪怕是局域网)的影响,另一方面,sysbench的运行(尤其是设置的并发数较高时)会影响MySQL服务器的表现。
- (2)可以逐步增加客户端的并发连接数(--thread参数),观察在连接数不同情况下,MySQL服务器的表现;如分别设置为10,20,50,100等。
- (3)一般执行模式选择complex即可,如果需要特别测试服务器只读性能,或不使用事务时的性能,可以选择simple模式或nontrx模式。
- (4)如果连续进行多次测试,注意确保之前测试的数据已经被清理干净。
下面是sysbench使用的一个例子:
注意:先准备数据库
mysql> create database sbtest;
其中,执行模式为complex,使用了10个表,每个表有10万条数据,客户端的并发线程数为10,执行时间为120秒,每10秒生成一次报告。
sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --mysql-table-engine=innodb --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 prepare
其中的hostip、username、password请自行切换为自己真实的环境数据
查看执行结果
sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 run
Running the test with following options: Number of threads: 10 Report intermediate results every 10 second(s) Initializing random number generator from current time Initializing worker threads... Threads started! [ 10s ] thds: 10 tps: 17.19 qps: 362.78 (r/w/o: 254.64/72.76/35.38) lat (ms,95%): 1589.90 err/s: 0.00 reconn/s: 0.00 [ 20s ] thds: 10 tps: 33.50 qps: 670.08 (r/w/o: 469.06/134.02/67.01) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00 [ 30s ] thds: 10 tps: 39.30 qps: 786.00 (r/w/o: 550.20/157.20/78.60) lat (ms,95%): 458.96 err/s: 0.00 reconn/s: 0.00 [ 40s ] thds: 10 tps: 38.80 qps: 775.45 (r/w/o: 543.16/154.69/77.59) lat (ms,95%): 634.66 err/s: 0.00 reconn/s: 0.00 [ 50s ] thds: 10 tps: 30.10 qps: 602.54 (r/w/o: 421.43/120.91/60.20) lat (ms,95%): 590.56 err/s: 0.00 reconn/s: 0.00 [ 60s ] thds: 10 tps: 33.90 qps: 678.00 (r/w/o: 474.60/135.60/67.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00 [ 70s ] thds: 10 tps: 29.50 qps: 589.98 (r/w/o: 412.99/118.00/59.00) lat (ms,95%): 787.74 err/s: 0.00 reconn/s: 0.00 [ 80s ] thds: 10 tps: 39.90 qps: 798.02 (r/w/o: 558.62/159.60/79.80) lat (ms,95%): 569.67 err/s: 0.00 reconn/s: 0.00 [ 90s ] thds: 10 tps: 38.10 qps: 761.99 (r/w/o: 533.40/152.40/76.20) lat (ms,95%): 484.44 err/s: 0.00 reconn/s: 0.00 [ 100s ] thds: 10 tps: 43.90 qps: 877.99 (r/w/o: 614.60/175.60/87.80) lat (ms,95%): 383.33 err/s: 0.00 reconn/s: 0.00 [ 110s ] thds: 10 tps: 37.80 qps: 756.00 (r/w/o: 529.20/151.20/75.60) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00 [ 120s ] thds: 10 tps: 44.40 qps: 888.01 (r/w/o: 621.60/177.60/88.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00 SQL statistics: queries performed: read: 59836 write: 17096 other: 8548 total: 85480 transactions: 4274 (35.59 per sec.) queries: 85480 (711.77 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) General statistics: total time: 120.0919s total number of events: 4274 Latency (ms): min: 50.07 avg: 280.96 max: 2486.17 95th percentile: 559.50 sum: 1200810.10 Threads fairness: events (avg/stddev): 427.4000/4.20 execution time (avg/stddev): 120.0810/0.01
其中,对于我们比较重要的信息包括:
queries:查询总数及qps
transactions:事务总数及tps
Latency-95th percentile:前95%的请求的最大响应时间,本例中是344毫秒,这个延迟非常大,是因为我用的MySQL服务器性能很差;在正式环境中这个数值是绝对不能接受的。
最后清理数据:
sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-tables-count=10 cleanup
清理的日志输出
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2) Dropping table 'sbtest1'... Dropping table 'sbtest2'... Dropping table 'sbtest3'... Dropping table 'sbtest4'... Dropping table 'sbtest5'... Dropping table 'sbtest6'... Dropping table 'sbtest7'... Dropping table 'sbtest8'... Dropping table 'sbtest9'... Dropping table 'sbtest10'...