• 服务器性能测试利器之sysbench


    前言

    sysbench是一个开源的、模块化的、跨平台的多线程性能测试工具,可以用来进行CPU、内存、磁盘I/O、线程、数据库的性能测试。sysbench是基于LuaJIT的可编写脚本的多线程基准测试工具。

    sysbench附带以下捆绑的基准测试:

    • oltp_*.lua:类似OLTP的数据库基准测试的集合
    • fileio:文件系统级基准
    • cpu:简单的CPU基准测试
    • memory:内存访问基准
    • threads:基于线程的调度程序基准
    • mutex:POSIX互斥基准测试

    特征

    • 提供有关速率和延迟的大量统计信息,包括延迟百分位数和直方图;
    • 即使有数千个并发线程,开销也很低。sysbench每秒能够生成和跟踪数亿个事件。
    • 通过在用户提供的Lua脚本中实现预定义的挂钩,可以轻松创建新的基准测试;
    • 可以也可以作为一个通用的Lua解释,只需更换#!/usr/bin/lua#!/usr/bin/sysbench在你的脚本。

    安装

        安装以Centos为例,其他版本或者mac请参考sysbench说明

    1.下载安转

    wget https://github.com/akopytov/sysbench/archive/1.0.zip -O "sysbench-1.0.zip"
      
    unzip sysbench-1.0.zip
     
     
    cd sysbench-1.0

    //安装依赖
    yum install -y automake libtool
     

    2.编译安装

    ./autogen.sh
    ./configure
    export LD_LIBRARY_PATH=/usr/include/mysql
    make
    make install
    

    3.监测是否安装成功

    [root@guanbin-k8s-master ~]# sysbench --version
    sysbench 1.0.19  

    测试

      1. cpu测试

       cpu测试主要是进行素数的加法运算,我们的例子中,指定了最大的质数发生器数量为 20000

    [root@guanbin-k8s-master ~]# sysbench --test=cpu --cpu-max-prime=20000 run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from current time
    
    
    Prime numbers limit: 20000
    
    Initializing worker threads...
    
    Threads started!
    
    CPU speed:
        events per second:   168.59
    
    General statistics:
        total time:                          10.0027s
        total number of events:              1687
    
    Latency (ms):
             min:                                    5.63
             avg:                                    5.93
             max:                                   66.09
             95th percentile:                        6.43
             sum:                                 9999.10
    
    Threads fairness:
        events (avg/stddev):           1687.0000/0.00
        execution time (avg/stddev):   9.9991/0.00
    

    执行时间 大约为10秒(这个测试最好是两台机器进行对比测试,才能发现那个机器有问题)

    2.对内存(memory)进行测试 
       测试是在内存中传输 10G 的数据量,每个 block 大小为 8K
    [root@guanbin-k8s-master ~]# sysbench --test=memory --memory-block-size=8k --memory-total-size=10G run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from current time
    
    
    Running memory speed test with the following options:
      block size: 8KiB
      total size: 10240MiB
      operation: write
      scope: global
    
    Initializing worker threads...
    
    Threads started!
    
    Total operations: 1310720 (724931.70 per second)
    
    10240.00 MiB transferred (5663.53 MiB/sec)
    
    
    General statistics:
        total time:                          1.8045s
        total number of events:              1310720
    
    Latency (ms):
             min:                                    0.00
             avg:                                    0.00
             max:                                   12.44
             95th percentile:                        0.00
             sum:                                 1441.61
    
    Threads fairness:
        events (avg/stddev):           1310720.0000/0.00
        execution time (avg/stddev):   1.4416/0.00
    

    totaltime为1.8045s

    3.对disk 的IO测试

    新建一个临时目录,并确保磁盘剩余空间足够(--file-total-size不要超过剩余容量)

    测试场景:16个线程,创建128个文件,总共占用磁盘空间10GB,测试磁盘读写效率

    运行结束后要删除临时测试文件: rm -rf test_file.*

    root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw prepare
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    WARNING: --num-threads is deprecated, use --threads instead
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    128 files, 8192Kb each, 1024Mb total
    Creating files for the test...
    Extra file open flags: (none)
    Creating file test_file.0
    Creating file test_file.1
    Creating file test_file.2
    Creating file test_file.3
    Creating file test_file.4
    Creating file test_file.5
    Creating file test_file.6
    Creating file test_file.7
    Creating file test_file.8
    Creating file test_file.9
    Creating file test_file.10
    Creating file test_file.11
    Creating file test_file.12
    Creating file test_file.13
    Creating file test_file.14
    Creating file test_file.15
    Creating file test_file.16
    Creating file test_file.17
    Creating file test_file.18
    Creating file test_file.19
    Creating file test_file.20
    Creating file test_file.21
    Creating file test_file.22
    Creating file test_file.23
    Creating file test_file.24
    Creating file test_file.25
    Creating file test_file.26
    Creating file test_file.27
    Creating file test_file.28
    Creating file test_file.29
    Creating file test_file.30
    Creating file test_file.31
    Creating file test_file.32
    Creating file test_file.33
    Creating file test_file.34
    Creating file test_file.35
    Creating file test_file.36
    Creating file test_file.37
    Creating file test_file.38
    Creating file test_file.39
    Creating file test_file.40
    Creating file test_file.41
    Creating file test_file.42
    Creating file test_file.43
    Creating file test_file.44
    Creating file test_file.45
    Creating file test_file.46
    Creating file test_file.47
    Creating file test_file.48
    Creating file test_file.49
    Creating file test_file.50
    Creating file test_file.51
    Creating file test_file.52
    Creating file test_file.53
    Creating file test_file.54
    Creating file test_file.55
    Creating file test_file.56
    Creating file test_file.57
    Creating file test_file.58
    Creating file test_file.59
    Creating file test_file.60
    Creating file test_file.61
    Creating file test_file.62
    Creating file test_file.63
    Creating file test_file.64
    Creating file test_file.65
    Creating file test_file.66
    Creating file test_file.67
    Creating file test_file.68
    Creating file test_file.69
    Creating file test_file.70
    Creating file test_file.71
    Creating file test_file.72
    Creating file test_file.73
    Creating file test_file.74
    Creating file test_file.75
    Creating file test_file.76
    Creating file test_file.77
    Creating file test_file.78
    Creating file test_file.79
    Creating file test_file.80
    Creating file test_file.81
    Creating file test_file.82
    Creating file test_file.83
    Creating file test_file.84
    Creating file test_file.85
    Creating file test_file.86
    Creating file test_file.87
    Creating file test_file.88
    Creating file test_file.89
    Creating file test_file.90
    Creating file test_file.91
    Creating file test_file.92
    Creating file test_file.93
    Creating file test_file.94
    Creating file test_file.95
    Creating file test_file.96
    Creating file test_file.97
    Creating file test_file.98
    Creating file test_file.99
    Creating file test_file.100
    Creating file test_file.101
    Creating file test_file.102
    Creating file test_file.103
    Creating file test_file.104
    Creating file test_file.105
    Creating file test_file.106
    Creating file test_file.107
    Creating file test_file.108
    Creating file test_file.109
    Creating file test_file.110
    Creating file test_file.111
    Creating file test_file.112
    Creating file test_file.113
    Creating file test_file.114
    Creating file test_file.115
    Creating file test_file.116
    Creating file test_file.117
    Creating file test_file.118
    Creating file test_file.119
    Creating file test_file.120
    Creating file test_file.121
    Creating file test_file.122
    Creating file test_file.123
    Creating file test_file.124
    Creating file test_file.125
    Creating file test_file.126
    Creating file test_file.127
    1073741824 bytes written in 110.77 seconds (9.24 MiB/sec).
    [root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw run
    WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
    WARNING: --num-threads is deprecated, use --threads instead
    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    Running the test with following options:
    Number of threads: 16
    Initializing random number generator from current time
    
    
    Extra file open flags: (none)
    128 files, 8MiB each
    1GiB total file size
    Block size 16KiB
    Number of IO requests: 0
    Read/Write ratio for combined random IO test: 1.50
    Periodic FSYNC enabled, calling fsync() each 100 requests.
    Calling fsync() at the end of test, Enabled.
    Using synchronous I/O mode
    Doing random r/w test
    Initializing worker threads...
    
    Threads started!
    
    
    File operations:
        reads/s:                      157.44
        writes/s:                     104.96
        fsyncs/s:                     480.79
    
    Throughput:
        read, MiB/s:                  2.46
        written, MiB/s:               1.64
    
    General statistics:
        total time:                          13.3355s
        total number of events:              7865
    
    Latency (ms):
             min:                                    0.00
             avg:                                   20.61
             max:                                 1115.85
             95th percentile:                      142.39
             sum:                               162071.40
    
    Threads fairness:
        events (avg/stddev):           491.5625/141.20
        execution time (avg/stddev):   10.1295/0.03  
    
    吞吐量:读2.46MiB/s,写1.64MiB/s; 95%延迟 142.39ms
     
    4.测试mysql性能
           测试时使用的脚本为lua脚本,可以使用sysbench自带脚本,也可以自己开发。对于大多数应用,使用sysbench自带的脚本就足够了。不同版本的sysbench中,lua脚本的位置可能不同,可以自己在sysbench路径下使用find命令搜索oltp.lua。
         P.S.:大多数数据服务都是oltp类型的,如果你不了解什么是oltp,那么大概率你的数据服务就是oltp类型的。MySQL OLTP(On-Line Transaction Processing联机事务处理过程);
    • (1)尽量不要在MySQL服务器运行的机器上进行测试,一方面可能无法体现网络(哪怕是局域网)的影响,另一方面,sysbench的运行(尤其是设置的并发数较高时)会影响MySQL服务器的表现。
    • (2)可以逐步增加客户端的并发连接数(--thread参数),观察在连接数不同情况下,MySQL服务器的表现;如分别设置为10,20,50,100等。
    • (3)一般执行模式选择complex即可,如果需要特别测试服务器只读性能,或不使用事务时的性能,可以选择simple模式或nontrx模式。
    • (4)如果连续进行多次测试,注意确保之前测试的数据已经被清理干净。

    下面是sysbench使用的一个例子:

     注意:先准备数据库

       mysql> create database sbtest;
    

     其中,执行模式为complex,使用了10个表,每个表有10万条数据,客户端的并发线程数为10,执行时间为120秒,每10秒生成一次报告。

    sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --mysql-table-engine=innodb --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 prepare
    

    其中的hostip、username、password请自行切换为自己真实的环境数据

    查看执行结果

    sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 run
    Running the test with following options:
    Number of threads: 10
    Report intermediate results every 10 second(s)
    Initializing random number generator from current time
    
    
    Initializing worker threads...
    
    Threads started!
    
    [ 10s ] thds: 10 tps: 17.19 qps: 362.78 (r/w/o: 254.64/72.76/35.38) lat (ms,95%): 1589.90 err/s: 0.00 reconn/s: 0.00
    [ 20s ] thds: 10 tps: 33.50 qps: 670.08 (r/w/o: 469.06/134.02/67.01) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
    [ 30s ] thds: 10 tps: 39.30 qps: 786.00 (r/w/o: 550.20/157.20/78.60) lat (ms,95%): 458.96 err/s: 0.00 reconn/s: 0.00
    [ 40s ] thds: 10 tps: 38.80 qps: 775.45 (r/w/o: 543.16/154.69/77.59) lat (ms,95%): 634.66 err/s: 0.00 reconn/s: 0.00
    [ 50s ] thds: 10 tps: 30.10 qps: 602.54 (r/w/o: 421.43/120.91/60.20) lat (ms,95%): 590.56 err/s: 0.00 reconn/s: 0.00
    [ 60s ] thds: 10 tps: 33.90 qps: 678.00 (r/w/o: 474.60/135.60/67.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
    [ 70s ] thds: 10 tps: 29.50 qps: 589.98 (r/w/o: 412.99/118.00/59.00) lat (ms,95%): 787.74 err/s: 0.00 reconn/s: 0.00
    [ 80s ] thds: 10 tps: 39.90 qps: 798.02 (r/w/o: 558.62/159.60/79.80) lat (ms,95%): 569.67 err/s: 0.00 reconn/s: 0.00
    [ 90s ] thds: 10 tps: 38.10 qps: 761.99 (r/w/o: 533.40/152.40/76.20) lat (ms,95%): 484.44 err/s: 0.00 reconn/s: 0.00
    [ 100s ] thds: 10 tps: 43.90 qps: 877.99 (r/w/o: 614.60/175.60/87.80) lat (ms,95%): 383.33 err/s: 0.00 reconn/s: 0.00
    [ 110s ] thds: 10 tps: 37.80 qps: 756.00 (r/w/o: 529.20/151.20/75.60) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
    [ 120s ] thds: 10 tps: 44.40 qps: 888.01 (r/w/o: 621.60/177.60/88.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
    SQL statistics:
        queries performed:
            read:                            59836
            write:                           17096
            other:                           8548
            total:                           85480
        transactions:                        4274   (35.59 per sec.)
        queries:                             85480  (711.77 per sec.)
        ignored errors:                      0      (0.00 per sec.)
        reconnects:                          0      (0.00 per sec.)
    
    General statistics:
        total time:                          120.0919s
        total number of events:              4274
    
    Latency (ms):
             min:                                   50.07
             avg:                                  280.96
             max:                                 2486.17
             95th percentile:                      559.50
             sum:                              1200810.10
    
    Threads fairness:
        events (avg/stddev):           427.4000/4.20
        execution time (avg/stddev):   120.0810/0.01

    其中,对于我们比较重要的信息包括:

    queries:查询总数及qps

    transactions:事务总数及tps

    Latency-95th percentile:前95%的请求的最大响应时间,本例中是344毫秒,这个延迟非常大,是因为我用的MySQL服务器性能很差;在正式环境中这个数值是绝对不能接受的。

    最后清理数据:

    sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-tables-count=10  cleanup

    清理的日志输出

    sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
    
    Dropping table 'sbtest1'...
    Dropping table 'sbtest2'...
    Dropping table 'sbtest3'...
    Dropping table 'sbtest4'...
    Dropping table 'sbtest5'...
    Dropping table 'sbtest6'...
    Dropping table 'sbtest7'...
    Dropping table 'sbtest8'...
    Dropping table 'sbtest9'...
    Dropping table 'sbtest10'...
  • 相关阅读:
    【JS】415- JS面向对象之 7 大基本原则
    【每周小回顾】14- 一起回顾上周精彩内容
    【全栈修炼】414- CORS和CSRF修炼宝典
    【全栈修炼】CORS和CSRF修炼宝典
    【JS】413- JavaScript中的位运算和权限设计
    【面试题】412- 35 道必须清楚的 React 面试题
    【CSS】410- 关于CSS盒子模型、BFC及其应用
    15.app后端怎么设计用户登录方案
    14.app后端如何设计api
    13.app后端为什么要用到消息队列
  • 原文地址:https://www.cnblogs.com/guanbin-529/p/12764888.html
Copyright © 2020-2023  润新知