• 【MySQL】sysbench压测服务器及结果解读


    主要压测范围包括CPU测试、磁盘IO测试、线程测试、OLTP测试等,那么sysbench就可以满足我们的压测需求。下面我们简单来看下sysbench的安装使用以及压测结果的解读。

    一、sysbench安装

    sysbench安装其实非常简单,可以参考下git,这里我们讲下源码编译安装。
    上传sysbench到服务器、解压、安装:

    [root@localhost~] yum -y install make automake libtool pkgconfig libaio-devel
    [root@localhost~] git clone
     

    [root@localhost sysbench]# ls -l
    total 108
    -rwxr-xr-x 1 root root 26 Nov 23 15:39 autogen.sh
    -rw-r--r-- 1 root root 14511 Nov 23 15:39 ChangeLog
    drwxr-xr-x 2 root root 26 Nov 23 15:39 config
    -rw-r--r-- 1 root root 11396 Nov 23 15:39 configure.ac
    -rw-r--r-- 1 root root 18092 Nov 23 15:39 COPYING
    drwxr-xr-x 3 root root 135 Nov 23 15:39 debian
    -rwxr-xr-x 1 root root 7122 Nov 23 15:39 install-sh
    drwxr-xr-x 2 root root 4096 Nov 23 15:39 m4
    -rw-r--r-- 1 root root 1547 Nov 23 15:39 Makefile.am
    -rwxr-xr-x 1 root root 10266 Nov 23 15:39 missing
    -rwxr-xr-x 1 root root 1988 Nov 23 15:39 mkinstalldirs
    -rw-r--r-- 1 root root 18867 Nov 23 15:39 README.md
    drwxr-xr-x 2 root root 27 Nov 23 15:39 rpm
    drwxr-xr-x 2 root root 26 Nov 23 15:39 scripts
    drwxr-xr-x 2 root root 31 Nov 23 15:39 snap
    drwxr-xr-x 5 root root 4096 Nov 23 15:39 src
    drwxr-xr-x 4 root root 85 Nov 23 15:39 tests
    drwxr-xr-x 5 root root 55 Nov 23 15:39 third_party
    [root@localhost sysbench]#

     
    [root@localhost tmp]# unzip sysbench-master.zip
    [root@localhost tmp]# cd sysbench-master [root@localhost sysbench-master]# ./autogen.sh [root@localhost sysbench-master]# ./configure [root@localhost sysbench-master]# make -j [root@localhost sysbench-master]# make install

    其实可以查看下README.md,上面也写了如何安装

    ## Build and Install
     shell
        ./autogen.sh
        # Add --with-pgsql to build with PostgreSQL support
        ./configure
        make -j
        make install

    [root@localhost sysbench]# sysbench
    sysbench: error while loading shared libraries: libmysqlclient.so.20: cannot open shared object file: No such file or directory

    字面意思就是打不到共享库文件libmysqlclient.so.20.

    先查找一下本地有没有这个文件:

    [root@localhost sysbench]# find .  -name /libmysqlclient.so.20                                               ./usr/lib64/mysql/libmysqlclient.so.16
    ./usr/local/mysql/lib/libmysqlclient.so.20

    在/usr/loca/mysql/lib下面有这个文件,解决方法是先建立一个软链接到/usr/local/lib

    [root@localhost sysbench] ln -s /usr/local/mysql/lib/libmysqlclient.so.20 /usr/local/lib/libmysqlclient.so.20
    接着在/etc/ld.so.cnf中加入/usr/loca/lib这一行

    [root@localhost sysbench]# cat /etc/ld.so.conf
    include ld.so.conf.d/*.conf
    /usr/local/lib
    执行ldconfig -v更新下配置就可以了

    [root@localhost sysbench]# /sbin/ldconfig -v
    现在sysbench可以执行了!

    [root@localhost sysbench]# sysbench --version
    sysbench 1.1.0-5cd1244

    [root@localhost sysbench]# sysbench --help
    Usage:
    sysbench [options]... [testname] [command]

    Commands implemented by most tests: prepare run cleanup help

    General options:
    --threads=N number of threads to use [1]
    --events=N limit for total number of events [0]
    --time=N limit for total execution time in seconds [10]
    --warmup-time=N execute events for this many seconds with statistics disabled before the actual benchmark run with statistics enabled [0]
    --forced-shutdown=STRING number of seconds to wait after the --time limit before forcing shutdown, or 'off' to disable [off]
    --thread-stack-size=SIZE size of stack per thread [64K]
    --thread-init-timeout=N wait time in seconds for worker threads to initialize [30]
    --rate=N average transactions rate. 0 for unlimited rate [0]
    --report-interval=N periodically report intermediate statistics with a specified interval in seconds. 0 disables intermediate reports [0]
    --report-checkpoints=[LIST,...] dump full statistics and reset all counters at specified points in time. The argument is a list of comma-separated values representing the amount of time in seconds elapsed from start of test when report checkpoint(s) must be performed. Report checkpoints are off by default. []
    --debug[=on|off] print more debugging info [off]
    --validate[=on|off] perform validation checks where possible [off]
    --help[=on|off] print help and exit [off]
    --version[=on|off] print version and exit [off]
    --config-file=FILENAME File containing command line options
    --luajit-cmd=STRING perform LuaJIT control command. This option is equivalent to 'luajit -j'. See LuaJIT documentation for more information

    Pseudo-Random Numbers Generator options:
    --rand-type=STRING random numbers distribution {uniform, gaussian, special, pareto, zipfian} to use by default [special]
    --rand-seed=N seed for random number generator. When 0, the current time is used as an RNG seed. [0]
    --rand-spec-iter=N number of iterations for the special distribution [12]
    --rand-spec-pct=N percentage of the entire range where 'special' values will fall in the special distribution [1]
    --rand-spec-res=N percentage of 'special' values to use for the special distribution [75]
    --rand-pareto-h=N shape parameter for the Pareto distribution [0.2]
    --rand-zipfian-exp=N shape parameter (exponent, theta) for the Zipfian distribution [0.8]

    Log options:
    --verbosity=N verbosity level {5 - debug, 0 - only critical messages} [3]

    --percentile=N percentile to calculate in latency statistics (1-100). Use the special value of 0 to disable percentile calculations [95]
    --histogram[=on|off] print latency histogram in report [off]

    General database options:

    --db-driver=STRING specifies database driver to use ('help' to get list of available drivers) [mysql]
    --db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto]
    --db-debug[=on|off] print database-specific debug information [off]


    Compiled-in database drivers:
    mysql - MySQL driver

    mysql options:
    --mysql-host=[LIST,...] MySQL server host [localhost]
    --mysql-port=[LIST,...] MySQL server port [3306]
    --mysql-socket=[LIST,...] MySQL socket
    --mysql-user=STRING MySQL user [sbtest]
    --mysql-password=STRING MySQL password []
    --mysql-db=STRING MySQL database name [sbtest]
    --mysql-ssl[=on|off] use SSL connections, if available in the client library [off]
    --mysql-ssl-key=STRING path name of the client private key file
    --mysql-ssl-ca=STRING path name of the CA file
    --mysql-ssl-cert=STRING path name of the client public key certificate file
    --mysql-ssl-cipher=STRING use specific cipher for SSL connections []
    --mysql-compression[=on|off] use compression, if available in the client library [off]
    --mysql-debug[=on|off] trace all client library calls [off]
    --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205]
    --mysql-dry-run[=on|off] Dry run, pretend that all MySQL client API calls are successful without executing them [off]

    Compiled-in tests:
    fileio - File I/O test
    cpu - CPU performance test
    memory - Memory functions speed test
    threads - Threads subsystem performance test
    mutex - Mutex performance test

    See 'sysbench <testname> help' for a list of options for each test.

    二、压测过程及压测结果解读

    1.cpu测试

    [root@localhost storage]# sysbench --test=cpu --cpu-max-prime=2000000 run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from current time
    
    
    Prime numbers limit: 2000000
    
    Initializing worker threads...
    
    Threads started!
    
    CPU speed:
        events per second:     0.63
    
    Throughput:
        events/s (eps):                      0.6334
        time elapsed:                        11.0520s
        total number of events:              7
    
    Latency (ms):
             min:                                 1571.86
             avg:                                 1578.84
             max:                                 1620.53
             95th percentile:                     1618.78
             sum:                                11051.91
    
    Threads fairness:
        events (avg/stddev):           7.0000/0.00
        execution time (avg/stddev):   11.0519/0.00
    

    cpu测试主要是进行素数的加法运算,上面我们的例子中,指定了最大的质数发生器数量为 2000000,可以看出服务器此次测试 执行时间 大约为11.05秒
    2.磁盘IO测试

    [root@localhost storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw prepare
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    WARNING: --num-threads is deprecated, use --threads instead
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    128 files, 245760Kb each, 30720Mb total
    Creating files for the test...
    Extra file open flags: (none)
    Creating file test_file.0
    Creating file test_file.1
    Creating file test_file.2
    Creating file test_file.3
    Creating file test_file.4
    Creating file test_file.5
    Creating file test_file.6
    Creating file test_file.7
    Creating file test_file.8
    Creating file test_file.9
    Creating file test_file.10
    Creating file test_file.11
    Creating file test_file.12
    Creating file test_file.13
    Creating file test_file.14
    Creating file test_file.15
    Creating file test_file.16
    Creating file test_file.17
    Creating file test_file.18
    Creating file test_file.19
    Creating file test_file.20
    Creating file test_file.21
    Creating file test_file.22
    Creating file test_file.23
    Creating file test_file.24
    Creating file test_file.25
    Creating file test_file.26
    Creating file test_file.27
    Creating file test_file.28
    Creating file test_file.29
    Creating file test_file.30
    Creating file test_file.31
    Creating file test_file.32
    Creating file test_file.33
    Creating file test_file.34
    Creating file test_file.35
    Creating file test_file.36
    Creating file test_file.37
    Creating file test_file.38
    Creating file test_file.39
    Creating file test_file.40
    Creating file test_file.41
    Creating file test_file.42
    Creating file test_file.43
    Creating file test_file.44
    Creating file test_file.45
    Creating file test_file.46
    Creating file test_file.47
    Creating file test_file.48
    Creating file test_file.49
    Creating file test_file.50
    Creating file test_file.51
    Creating file test_file.52
    Creating file test_file.53
    Creating file test_file.54
    Creating file test_file.55
    Creating file test_file.56
    Creating file test_file.57
    Creating file test_file.58
    Creating file test_file.59
    Creating file test_file.60
    Creating file test_file.61
    Creating file test_file.62
    Creating file test_file.63
    Creating file test_file.64
    Creating file test_file.65
    Creating file test_file.66
    Creating file test_file.67
    Creating file test_file.68
    Creating file test_file.69
    Creating file test_file.70
    Creating file test_file.71
    Creating file test_file.72
    Creating file test_file.73
    Creating file test_file.74
    Creating file test_file.75
    Creating file test_file.76
    Creating file test_file.77
    Creating file test_file.78
    Creating file test_file.79
    Creating file test_file.80
    Creating file test_file.81
    Creating file test_file.82
    Creating file test_file.83
    Creating file test_file.84
    Creating file test_file.85
    Creating file test_file.86
    Creating file test_file.87
    Creating file test_file.88
    Creating file test_file.89
    Creating file test_file.90
    Creating file test_file.91
    Creating file test_file.92
    Creating file test_file.93
    Creating file test_file.94
    Creating file test_file.95
    Creating file test_file.96
    Creating file test_file.97
    Creating file test_file.98
    Creating file test_file.99
    Creating file test_file.100
    Creating file test_file.101
    Creating file test_file.102
    Creating file test_file.103
    Creating file test_file.104
    Creating file test_file.105
    Creating file test_file.106
    Creating file test_file.107
    Creating file test_file.108
    Creating file test_file.109
    Creating file test_file.110
    Creating file test_file.111
    Creating file test_file.112
    Creating file test_file.113
    Creating file test_file.114
    Creating file test_file.115
    Creating file test_file.116
    Creating file test_file.117
    Creating file test_file.118
    Creating file test_file.119
    Creating file test_file.120
    Creating file test_file.121
    Creating file test_file.122
    Creating file test_file.123
    Creating file test_file.124
    Creating file test_file.125
    Creating file test_file.126
    Creating file test_file.127
    32212254720 bytes written in 55.25 seconds (556.00 MiB/sec).
    
    [root@localhost storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw run
    
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    WARNING: --num-threads is deprecated, use --threads instead
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    Running the test with following options:
    Number of threads: 16
    Initializing random number generator from current time
    
    
    Extra file open flags: (none)
    128 files, 240MiB each
    30GiB total file size
    Block size 16KiB
    Number of IO requests: 0
    Read/Write ratio for combined random IO test: 1.50
    Periodic FSYNC enabled, calling fsync() each 100 requests.
    Calling fsync() at the end of test, Enabled.
    Using synchronous I/O mode
    Doing random r/w test
    Initializing worker threads...
    
    Threads started!
    
    
    Throughput:
             read:  IOPS=68491.05 1070.17 MiB/s (1122.16 MB/s)
             write: IOPS=45660.96 713.45 MiB/s (748.11 MB/s)
             fsync: IOPS=146103.57
    
    Latency (ms):
             min:                                  0.00
             avg:                                  0.06
             max:                                  3.61
             95th percentile:                      0.31
             sum:                             156128.82
    

    可以看到,磁盘的读IOPS可以达到68491.05,写IOPS可以达到45660.96,fsync可以达到146103.57
    3.线程测试

    [root@localhost storage]# sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    WARNING: --num-threads is deprecated, use --threads instead
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    Running the test with following options:
    Number of threads: 64
    Initializing random number generator from current time
    
    
    Initializing worker threads...
    
    Threads started!
    
    
    Throughput:
        events/s (eps):                      14851.1802
        time elapsed:                        10.0036s
        total number of events:              148565
    
    Latency (ms):
             min:                                    0.02
             avg:                                    4.31
             max:                                   24.49
             95th percentile:                       10.46
             sum:                               639889.93
    
    Threads fairness:
        events (avg/stddev):           2321.3281/42.94
        execution time (avg/stddev):   9.9983/0.00      
    

    (发送64次/个测试线程请求,每次/个线程请求产生/生成100个数量,每个线程的锁数量为2) ,测试执行时间为10秒
    4.内存测试

    [root@localhost storage]# sysbench --test=memory --memory-block-size=8k --memory-total-size=40G run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from current time
    
    
    Running memory speed test with the following options:
      block size: 8KiB
      total size: 40960MiB
      operation: write
      scope: global
    
    Initializing worker threads...
    
    Threads started!
    
    Total operations: 5242880 (1443403.42 per second)
    
    40960.00 MiB transferred (11276.59 MiB/sec)
    
    
    Throughput:
        events/s (eps):                      1443403.4239
        time elapsed:                        3.6323s
        total number of events:              5242880
    
    Latency (ms):
             min:                                    0.00
             avg:                                    0.00
             max:                                    0.02
             95th percentile:                        0.00
             sum:                                 2926.10
    
    Threads fairness:
        events (avg/stddev):           5242880.0000/0.00
        execution time (avg/stddev):   2.9261/0.00
    

    上述参数指定了本次测试整个过程是在内存中传输 40G 的数据量,每个 block 大小为 8K。测试结果显示:
    执行时间为3.6323秒,每秒传输速度为11276.59Mb每秒
    5.OLTP测试
    (1)100线程
    数据准备:

    [root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua 
    > --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock 
    > --tables=10 --table_size=20000000 --threads=100   --max-requests=0 prepare
    sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
    
    Initializing worker threads...
    
    Creating table 'sbtest1'...
    Creating table 'sbtest4'...
    Creating table 'sbtest2'...
    Creating table 'sbtest9'...
    Creating table 'sbtest6'...
    Creating table 'sbtest8'...
    Creating table 'sbtest7'...
    Creating table 'sbtest5'...
    Creating table 'sbtest10'...
    Creating table 'sbtest3'...
    Inserting 20000000 records into 'sbtest1'
    Inserting 20000000 records into 'sbtest7'
    Inserting 20000000 records into 'sbtest4'
    Inserting 20000000 records into 'sbtest9'
    Inserting 20000000 records into 'sbtest6'
    Inserting 20000000 records into 'sbtest8'
    Inserting 20000000 records into 'sbtest2'
    Inserting 20000000 records into 'sbtest5'
    Inserting 20000000 records into 'sbtest3'
    Inserting 20000000 records into 'sbtest10'
    Creating a secondary index on 'sbtest4'...
    Creating a secondary index on 'sbtest1'...
    Creating a secondary index on 'sbtest10'...
    Creating a secondary index on 'sbtest3'...
    Creating a secondary index on 'sbtest6'...
    Creating a secondary index on 'sbtest8'...
    Creating a secondary index on 'sbtest7'...
    Creating a secondary index on 'sbtest9'...
    Creating a secondary index on 'sbtest5'...
    Creating a secondary index on 'sbtest2'...
    

    开始测试:

    [root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua --mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=100 --time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_100.log
    

    测试过程中,可以tail -f /tmp/40_100.log查看输出
    测试结果:这里只展示统计部分

    SQL statistics:
        queries performed:
            read:                            143195290
            write:                           40912940
            other:                           20456470
            total:                           204564700
        transactions:                        10228235 (2841.15 per sec.)
        queries:                             204564700 (56823.07 per sec.)
        ignored errors:                      0      (0.00 per sec.)
        reconnects:                          0      (0.00 per sec.)
    
    Throughput:
        events/s (eps):                      2841.1536
        time elapsed:                        3600.0289s
        total number of events:              10228235
    
    Latency (ms):
             min:                                    2.49
             avg:                                   35.19
             max:                                 2017.54
             95th percentile:                       39.65
             sum:                            359973625.38
    
    Threads fairness:
        events (avg/stddev):           102282.3500/5194.20
        execution time (avg/stddev):   3599.7363/0.01
    

    (2)200线程
    数据准备:

    [root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua 
    > --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock 
    > --tables=10 --table_size=20000000 --threads=200   --max-requests=0 prepare
    

    开始测试:

    [root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua 
    --mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc 
    --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=200 --time=3600 
    --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_200.log
    

    测试结果:

    SQL statistics:
        queries performed:
            read:                            142876258
            write:                           40821788
            other:                           20410894
            total:                           204108940
        transactions:                        10205447 (2834.80 per sec.)
        queries:                             204108940 (56696.02 per sec.)
        ignored errors:                      0      (0.00 per sec.)
        reconnects:                          0      (0.00 per sec.)
    
    Throughput:
        events/s (eps):                      2834.8009
        time elapsed:                        3600.0577s
        total number of events:              10205447
    
    Latency (ms):
             min:                                    2.39
             avg:                                   70.55
             max:                                 2011.65
             95th percentile:                       80.03
             sum:                            719978372.22
    
    Threads fairness:
        events (avg/stddev):           51027.2350/3840.19
        execution time (avg/stddev):   3599.8919/0.02
    

    (3)300线程
    数据准备:

    [root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua 
    > --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock 
    > --tables=10 --table_size=20000000 --threads=300   --max-requests=0 prepare
    

    开始测试:

    sysbench /usr/local/share/sysbench/oltp_read_write.lua 
    --mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest 
    --mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=300 
    --time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_300.log
    

    测试结果:

    SQL statistics:
        queries performed:
            read:                            144281438
            write:                           41223268
            other:                           20611634
            total:                           206116340
        transactions:                        10305817 (2862.66 per sec.)
        queries:                             206116340 (57253.12 per sec.)
        ignored errors:                      0      (0.00 per sec.)
        reconnects:                          0      (0.00 per sec.)
    
    Throughput:
        events/s (eps):                      2862.6561
        time elapsed:                        3600.0891s
        total number of events:              10305817
    
    Latency (ms):
             min:                                    2.64
             avg:                                  104.79
             max:                                 4576.60
             95th percentile:                      123.28
             sum:                           1079986824.04
    
    Threads fairness:
        events (avg/stddev):           34352.7233/2459.96
        execution time (avg/stddev):   3599.9561/0.03
    

    生成压测图
    安装gnuplot

    plot 'D:40_100.log' using 9 with linepoint title '100 threads','D:40_200.log' using 9 with linepoint title '200 threads','D:40_300.log' using 9 with linepoint title '300 threads';
    
     
    多线程QPS对比图
     
    多线程TPS对比图

    三、压测注意事项

    1.在OLTP测试过程中,数据库初始化后,冷备data目录,每次测试完毕后删除当前数据目录,拷贝data目录重新启动,为的是每次开始测试前环境保持一致。
    2.测试完成后,需要再系统层做一些清理的工作
    shell>sync #刷新脏数据到磁盘
    shell>echo 3 >/proc/sys/vm/drop_cache #清除cache
    shell>swapoff -a && swapon -a 释放swap
    3.压测过程中需要修改参数max_prepared_stmt_count默认为16382,修改为65536
    4.如果是SSD硬盘的话,innodb_io_capacity和innodb_io_capacity_max两个参数也要相应调大,建议调整到50000以上。
    5.对于测试数据的准备,我们通过show engine innodb status观察Free buffers,尽量模拟生产环境,让测试数据填满整个buffer pool。我这里innodb_buffer_pool_size设置的40G,测试表数量10张,每张表数据2000万,差不多填满buffer pool。



  • 相关阅读:
    NOIP1998提高组——挖地雷
    模板——EXBSGS
    vue 知识点
    jq_js
    ruby 基础知识(一)
    idea maven pom配置文件
    .net core dapper (5)
    .net core dapper (4)
    .net core dapper (3)
    .net core dapper (2)
  • 原文地址:https://www.cnblogs.com/DataArt/p/10008334.html
Copyright © 2020-2023  润新知