root@rook-test:/# ceph osd status
+----+-----------------------------+-------+-------+--------+---------+--------+---------+----------------+
| id | host | used | avail | wr ops | wr data | rd ops | rd data | state |
+----+-----------------------------+-------+-------+--------+---------+--------+---------+----------------+
| 0 | rook-ceph-osd-k8s-m3-gprhn | 2055M | 3348G | 0 | 0 | 0 | 0 | exists,up |
| 1 | rook-ceph-osd-k8s-n2-k86cz | 0 | 0 | 0 | 0 | 0 | 0 | autoout,exists |
| 2 | rook-ceph-osd-ceph-t1-6c72x | 2055M | 7607M | 0 | 0 | 0 | 0 | exists,up |
| 3 | rook-ceph-osd-ceph-t2-jqr6g | 2055M | 7607M | 0 | 0 | 0 | 0 | exists,up |
| 4 | rook-ceph-osd-ceph-t3-rctg9 | 2055M | 7607M | 0 | 0 | 0 | 0 | exists,up |
+----+-----------------------------+-------+-------+--------+---------+--------+---------+----------------+
fio 测试容器在k8s-m3 上
fio -filename=/data/test_randread -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=16k -size=500m -numjobs=10 -runtime=60 -group_reporting -name=test1
test1: (g=0): rw=randread, bs=16K-16K/16K-16K/16K-16K, ioengine=psync, iodepth=1
...
fio-2.1.11
Starting 10 threads
test1: Laying out IO file(s) (1 file(s) / 500MB)
Jobs: 10 (f=10): [r(10)] [100.0% done] [21034KB/0KB/0KB /s] [1314/0/0 iops] [eta 00m:00s]
test1: (groupid=0, jobs=10): err= 0: pid=119: Tue May 8 05:28:19 2018
read : io=1910.2MB, bw=32575KB/s, iops=2035, runt= 60047msec
clat (usec): min=250, max=85801, avg=4907.11, stdev=15717.35
lat (usec): min=251, max=85813, avg=4907.44, stdev=15717.34
clat percentiles (usec):
| 1.00th=[ 820], 5.00th=[ 956], 10.00th=[ 1020], 20.00th=[ 1096],
| 30.00th=[ 1176], 40.00th=[ 1240], 50.00th=[ 1304], 60.00th=[ 1384],
| 70.00th=[ 1480], 80.00th=[ 1624], 90.00th=[ 1896], 95.00th=[ 2736],
| 99.00th=[76288], 99.50th=[78336], 99.90th=[81408], 99.95th=[83456],
| 99.99th=[85504]
bw (KB /s): min= 1842, max= 3904, per=10.03%, avg=3267.40, stdev=397.75
lat (usec) : 500=0.16%, 750=0.43%, 1000=7.72%
lat (msec) : 2=83.68%, 4=3.09%, 10=0.01%, 50=0.01%, 100=4.91%
cpu : usr=0.14%, sys=0.51%, ctx=122339, majf=0, minf=120
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=122251/w=0/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=1910.2MB, aggrb=32574KB/s, minb=32574KB/s, maxb=32574KB/s, mint=60047msec, maxt=60047msec
Disk stats (read/write):
rbd1: ios=122051/8, merge=0/6, ticks=593825/1688, in_queue=595645, util=99.98%