128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21274
  write: io=2622.4MB, bw=268504KB/s, iops=67125 , runt= 10001msec
    clat (usec): min=75 , max=58830 , avg=469.56, stdev=1163.30
     lat (usec): min=75 , max=58830 , avg=470.70, stdev=1163.56
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  422], 80.00th=[  486], 90.00th=[  612], 95.00th=[  804],
     | 99.00th=[ 2704], 99.50th=[ 5536], 99.90th=[16320], 99.95th=[22400],
     | 99.99th=[43264]
    bw (KB/s)  : min= 5632, max=12360, per=3.13%, avg=8410.40, stdev=1122.54
    lat (usec) : 100=0.09%, 250=23.23%, 500=58.29%, 750=12.62%, 1000=2.42%
    lat (msec) : 2=1.99%, 4=0.66%, 10=0.40%, 20=0.23%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.20%, sys=6.60%, ctx=684686, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=671327, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2622.4MB, aggrb=268503KB/s, minb=268503KB/s, maxb=268503KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=7/665377, merge=24/9, ticks=4/199055, in_queue=196701, util=92.89%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21348
  write: io=2555.3MB, bw=261628KB/s, iops=65406 , runt= 10001msec
    clat (usec): min=73 , max=61798 , avg=481.81, stdev=1229.81
     lat (usec): min=75 , max=61799 , avg=483.00, stdev=1230.55
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  197], 20.00th=[  237],
     | 30.00th=[  270], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  820],
     | 99.00th=[ 3184], 99.50th=[ 6688], 99.90th=[15936], 99.95th=[25472],
     | 99.99th=[42752]
    bw (KB/s)  : min= 4768, max=10816, per=3.14%, avg=8222.34, stdev=1045.57
    lat (usec) : 100=0.09%, 250=23.40%, 500=57.86%, 750=12.60%, 1000=2.51%
    lat (msec) : 2=2.04%, 4=0.67%, 10=0.51%, 20=0.25%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.18%, sys=6.47%, ctx=669045, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=654135, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2555.3MB, aggrb=261627KB/s, minb=261627KB/s, maxb=261627KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=28/648901, merge=5/4, ticks=8/196485, in_queue=194275, util=92.71%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21422
  write: io=2649.4MB, bw=271234KB/s, iops=67808 , runt= 10001msec
    clat (usec): min=76 , max=51660 , avg=464.78, stdev=1113.39
     lat (usec): min=77 , max=51661 , avg=465.98, stdev=1114.29
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  171], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  342], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  486], 90.00th=[  604], 95.00th=[  796],
     | 99.00th=[ 2544], 99.50th=[ 5408], 99.90th=[15040], 99.95th=[22144],
     | 99.99th=[42752]
    bw (KB/s)  : min= 4480, max=11512, per=3.12%, avg=8463.68, stdev=1194.15
    lat (usec) : 100=0.09%, 250=22.97%, 500=58.53%, 750=12.70%, 1000=2.44%
    lat (msec) : 2=1.98%, 4=0.64%, 10=0.45%, 20=0.16%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.17%, sys=6.61%, ctx=691982, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=678154, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2649.4MB, aggrb=271234KB/s, minb=271234KB/s, maxb=271234KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=3/671987, merge=0/7, ticks=2/203185, in_queue=200835, util=94.26%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21496
  write: io=2593.5MB, bw=265542KB/s, iops=66385 , runt= 10001msec
    clat (usec): min=70 , max=87634 , avg=474.64, stdev=1189.54
     lat (usec): min=71 , max=87635 , avg=475.81, stdev=1189.91
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  197], 20.00th=[  237],
     | 30.00th=[  270], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  812],
     | 99.00th=[ 2896], 99.50th=[ 6368], 99.90th=[15680], 99.95th=[25216],
     | 99.99th=[43264]
    bw (KB/s)  : min= 4031, max=10928, per=3.12%, avg=8272.53, stdev=1188.54
    lat (usec) : 100=0.09%, 250=23.37%, 500=57.77%, 750=12.84%, 1000=2.52%
    lat (msec) : 2=2.00%, 4=0.65%, 10=0.46%, 20=0.23%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.20%, sys=6.55%, ctx=678483, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=663922, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2593.5MB, aggrb=265542KB/s, minb=265542KB/s, maxb=265542KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=2/660459, merge=0/5, ticks=0/195613, in_queue=193250, util=91.24%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21570
  write: io=2566.1MB, bw=262800KB/s, iops=65699 , runt= 10002msec
    clat (usec): min=72 , max=64168 , avg=479.82, stdev=1182.95
     lat (usec): min=73 , max=64168 , avg=480.96, stdev=1183.26
    clat percentiles (usec):
     |  1.00th=[  129],  5.00th=[  169], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  342], 60.00th=[  382],
     | 70.00th=[  430], 80.00th=[  494], 90.00th=[  620], 95.00th=[  812],
     | 99.00th=[ 3024], 99.50th=[ 6688], 99.90th=[16192], 99.95th=[22400],
     | 99.99th=[42752]
    bw (KB/s)  : min= 4456, max=11184, per=3.11%, avg=8171.19, stdev=1120.96
    lat (usec) : 100=0.09%, 250=22.85%, 500=57.82%, 750=13.30%, 1000=2.53%
    lat (msec) : 2=1.96%, 4=0.66%, 10=0.47%, 20=0.27%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.15%, sys=6.42%, ctx=673198, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=657131, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2566.1MB, aggrb=262799KB/s, minb=262799KB/s, maxb=262799KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=3/649973, merge=0/4, ticks=2/198147, in_queue=195940, util=93.02%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21644
  write: io=2651.4MB, bw=271150KB/s, iops=67787 , runt= 10013msec
    clat (usec): min=68 , max=43675 , avg=464.53, stdev=1118.65
     lat (usec): min=69 , max=43676 , avg=465.71, stdev=1119.28
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  171], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  422], 80.00th=[  486], 90.00th=[  604], 95.00th=[  796],
     | 99.00th=[ 2672], 99.50th=[ 5472], 99.90th=[14016], 99.95th=[20352],
     | 99.99th=[42240]
    bw (KB/s)  : min= 5552, max=11544, per=3.14%, avg=8511.33, stdev=1058.50
    lat (usec) : 100=0.08%, 250=23.27%, 500=58.51%, 750=12.43%, 1000=2.42%
    lat (msec) : 2=1.95%, 4=0.62%, 10=0.46%, 20=0.20%, 50=0.05%
  cpu          : usr=1.19%, sys=6.60%, ctx=693382, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=678755, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2651.4MB, aggrb=271149KB/s, minb=271149KB/s, maxb=271149KB/s, mint=10013msec, maxt=10013msec

Disk stats (read/write):
  vda: ios=2/672473, merge=0/6, ticks=0/203892, in_queue=201671, util=94.10%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21718
  write: io=2577.4MB, bw=263819KB/s, iops=65954 , runt= 10004msec
    clat (usec): min=67 , max=45056 , avg=477.62, stdev=1153.77
     lat (usec): min=68 , max=45057 , avg=478.80, stdev=1154.59
    clat percentiles (usec):
     |  1.00th=[  129],  5.00th=[  169], 10.00th=[  197], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  342], 60.00th=[  378],
     | 70.00th=[  430], 80.00th=[  490], 90.00th=[  620], 95.00th=[  820],
     | 99.00th=[ 3024], 99.50th=[ 6560], 99.90th=[14912], 99.95th=[20096],
     | 99.99th=[42752]
    bw (KB/s)  : min= 5520, max=10864, per=3.13%, avg=8269.80, stdev=997.14
    lat (usec) : 100=0.10%, 250=23.31%, 500=57.61%, 750=12.90%, 1000=2.56%
    lat (msec) : 2=2.01%, 4=0.71%, 10=0.48%, 20=0.27%, 50=0.05%
  cpu          : usr=1.18%, sys=6.52%, ctx=674266, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=659812, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2577.4MB, aggrb=263819KB/s, minb=263819KB/s, maxb=263819KB/s, mint=10004msec, maxt=10004msec

Disk stats (read/write):
  vda: ios=5/654863, merge=21/7, ticks=3/197544, in_queue=195268, util=93.63%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21792
  write: io=2562.8MB, bw=262263KB/s, iops=65565 , runt= 10006msec
    clat (usec): min=69 , max=56318 , avg=480.48, stdev=1204.59
     lat (usec): min=70 , max=56319 , avg=481.71, stdev=1205.82
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  270], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  620], 95.00th=[  820],
     | 99.00th=[ 3024], 99.50th=[ 7072], 99.90th=[16768], 99.95th=[24448],
     | 99.99th=[42752]
    bw (KB/s)  : min= 4880, max=11048, per=3.14%, avg=8230.89, stdev=1014.37
    lat (usec) : 100=0.08%, 250=23.30%, 500=57.74%, 750=12.81%, 1000=2.50%
    lat (msec) : 2=2.09%, 4=0.68%, 10=0.47%, 20=0.26%, 50=0.07%
    lat (msec) : 100=0.01%
  cpu          : usr=1.16%, sys=6.52%, ctx=670197, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=656052, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2562.8MB, aggrb=262263KB/s, minb=262263KB/s, maxb=262263KB/s, mint=10006msec, maxt=10006msec

Disk stats (read/write):
  vda: ios=3/649871, merge=0/5, ticks=1/195314, in_queue=192909, util=92.58%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21866
  write: io=2633.3MB, bw=269483KB/s, iops=67370 , runt= 10006msec
    clat (usec): min=68 , max=44507 , avg=467.44, stdev=1099.28
     lat (usec): min=68 , max=44508 , avg=468.58, stdev=1099.54
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  486], 90.00th=[  612], 95.00th=[  804],
     | 99.00th=[ 2736], 99.50th=[ 6112], 99.90th=[12992], 99.95th=[18816],
     | 99.99th=[42752]
    bw (KB/s)  : min= 5064, max=12232, per=3.12%, avg=8398.64, stdev=1042.29
    lat (usec) : 100=0.09%, 250=23.20%, 500=58.34%, 750=12.53%, 1000=2.45%
    lat (msec) : 2=2.01%, 4=0.64%, 10=0.46%, 20=0.22%, 50=0.05%
  cpu          : usr=1.19%, sys=6.64%, ctx=688057, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=674111, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2633.3MB, aggrb=269482KB/s, minb=269482KB/s, maxb=269482KB/s, mint=10006msec, maxt=10006msec

Disk stats (read/write):
  vda: ios=2/667351, merge=0/4, ticks=0/199658, in_queue=197218, util=93.45%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=21940
  write: io=2575.5MB, bw=263700KB/s, iops=65925 , runt= 10001msec
    clat (usec): min=69 , max=98491 , avg=478.02, stdev=1245.34
     lat (usec): min=70 , max=98492 , avg=479.22, stdev=1245.97
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  197], 20.00th=[  237],
     | 30.00th=[  270], 40.00th=[  302], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  812],
     | 99.00th=[ 2992], 99.50th=[ 6816], 99.90th=[16512], 99.95th=[25984],
     | 99.99th=[43776]
    bw (KB/s)  : min= 3280, max=11912, per=3.11%, avg=8211.92, stdev=1315.07
    lat (usec) : 100=0.08%, 250=23.47%, 500=57.86%, 750=12.70%, 1000=2.51%
    lat (msec) : 2=1.95%, 4=0.66%, 10=0.45%, 20=0.25%, 50=0.07%
    lat (msec) : 100=0.01%
  cpu          : usr=1.20%, sys=6.51%, ctx=672259, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=659317, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2575.5MB, aggrb=263700KB/s, minb=263700KB/s, maxb=263700KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=5/655732, merge=28/3, ticks=4/194693, in_queue=192224, util=92.63%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=22014
  write: io=2651.4MB, bw=271441KB/s, iops=67860 , runt= 10002msec
    clat (usec): min=72 , max=58869 , avg=464.19, stdev=1105.51
     lat (usec): min=72 , max=58870 , avg=465.32, stdev=1105.75
    clat percentiles (usec):
     |  1.00th=[  129],  5.00th=[  169], 10.00th=[  197], 20.00th=[  237],
     | 30.00th=[  270], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  812],
     | 99.00th=[ 2640], 99.50th=[ 5216], 99.90th=[14144], 99.95th=[20608],
     | 99.99th=[42752]
    bw (KB/s)  : min= 4806, max=10792, per=3.13%, avg=8483.27, stdev=1085.17
    lat (usec) : 100=0.10%, 250=23.55%, 500=57.53%, 750=12.84%, 1000=2.55%
    lat (msec) : 2=2.07%, 4=0.68%, 10=0.46%, 20=0.16%, 50=0.05%
    lat (msec) : 100=0.01%
  cpu          : usr=1.24%, sys=6.70%, ctx=693958, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=678738, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2651.4MB, aggrb=271440KB/s, minb=271440KB/s, maxb=271440KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=28/672139, merge=5/7, ticks=17/200372, in_queue=198080, util=94.19%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=22088
  write: io=2697.3MB, bw=276114KB/s, iops=69028 , runt= 10003msec
    clat (usec): min=72 , max=43393 , avg=456.37, stdev=1044.30
     lat (usec): min=73 , max=43394 , avg=457.53, stdev=1044.73
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  197], 20.00th=[  237],
     | 30.00th=[  270], 40.00th=[  302], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  422], 80.00th=[  486], 90.00th=[  612], 95.00th=[  804],
     | 99.00th=[ 2480], 99.50th=[ 4896], 99.90th=[13120], 99.95th=[19840],
     | 99.99th=[42752]
    bw (KB/s)  : min= 6072, max=11936, per=3.10%, avg=8572.87, stdev=994.51
    lat (usec) : 100=0.09%, 250=23.62%, 500=57.97%, 750=12.53%, 1000=2.50%
    lat (msec) : 2=2.02%, 4=0.64%, 10=0.43%, 20=0.15%, 50=0.05%
  cpu          : usr=1.24%, sys=6.74%, ctx=703241, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=690491, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2697.3MB, aggrb=276113KB/s, minb=276113KB/s, maxb=276113KB/s, mint=10003msec, maxt=10003msec

Disk stats (read/write):
  vda: ios=7/683035, merge=26/3, ticks=4/200880, in_queue=198469, util=94.78%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=22162
  write: io=2606.6MB, bw=266882KB/s, iops=66720 , runt= 10001msec
    clat (usec): min=70 , max=58545 , avg=472.15, stdev=1159.40
     lat (usec): min=71 , max=58546 , avg=473.36, stdev=1160.02
    clat percentiles (usec):
     |  1.00th=[  129],  5.00th=[  169], 10.00th=[  197], 20.00th=[  239],
     | 30.00th=[  270], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  820],
     | 99.00th=[ 2832], 99.50th=[ 5728], 99.90th=[15680], 99.95th=[22912],
     | 99.99th=[43264]
    bw (KB/s)  : min= 4592, max=11648, per=3.12%, avg=8327.83, stdev=1224.43
    lat (usec) : 100=0.10%, 250=23.36%, 500=57.87%, 750=12.68%, 1000=2.56%
    lat (msec) : 2=2.01%, 4=0.68%, 10=0.45%, 20=0.22%, 50=0.07%
    lat (msec) : 100=0.01%
  cpu          : usr=1.24%, sys=6.54%, ctx=681635, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=667272, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2606.6MB, aggrb=266882KB/s, minb=266882KB/s, maxb=266882KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=2/662332, merge=0/5, ticks=0/197178, in_queue=194848, util=92.87%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=22236
  write: io=2602.9MB, bw=266503KB/s, iops=66625 , runt= 10001msec
    clat (usec): min=69 , max=92771 , avg=472.72, stdev=1184.38
     lat (usec): min=70 , max=108798 , avg=473.90, stdev=1188.17
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  171], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  338], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  804],
     | 99.00th=[ 2896], 99.50th=[ 5984], 99.90th=[14912], 99.95th=[22656],
     | 99.99th=[43264]
    bw (KB/s)  : min= 4620, max=11512, per=3.11%, avg=8296.55, stdev=1099.99
    lat (usec) : 100=0.08%, 250=22.96%, 500=58.20%, 750=12.95%, 1000=2.51%
    lat (msec) : 2=1.90%, 4=0.66%, 10=0.44%, 20=0.24%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.19%, sys=6.53%, ctx=681724, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=666325, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2602.9MB, aggrb=266503KB/s, minb=266503KB/s, maxb=266503KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=2/659487, merge=0/7, ticks=2/198479, in_queue=196249, util=92.19%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=22310
  write: io=2551.3MB, bw=261140KB/s, iops=65284 , runt= 10004msec
    clat (usec): min=72 , max=63536 , avg=482.76, stdev=1202.09
     lat (usec): min=73 , max=63537 , avg=483.92, stdev=1202.62
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  171], 10.00th=[  199], 20.00th=[  239],
     | 30.00th=[  274], 40.00th=[  306], 50.00th=[  342], 60.00th=[  378],
     | 70.00th=[  426], 80.00th=[  490], 90.00th=[  612], 95.00th=[  820],
     | 99.00th=[ 3280], 99.50th=[ 7264], 99.90th=[15680], 99.95th=[23168],
     | 99.99th=[43264]
    bw (KB/s)  : min= 4763, max=11536, per=3.12%, avg=8158.07, stdev=1197.91
    lat (usec) : 100=0.08%, 250=22.91%, 500=58.19%, 750=12.84%, 1000=2.50%
    lat (msec) : 2=1.97%, 4=0.65%, 10=0.54%, 20=0.26%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=1.17%, sys=6.45%, ctx=667877, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=653110, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2551.3MB, aggrb=261139KB/s, minb=261139KB/s, maxb=261139KB/s, mint=10004msec, maxt=10004msec

Disk stats (read/write):
  vda: ios=6/647223, merge=21/7, ticks=5/199227, in_queue=197023, util=92.54%
