128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1357
  write: io=868908KB, bw=506271KB/s, iops=126567 , runt= 10001msec
    clat (usec): min=21 , max=26119 , avg=247.64, stdev=348.95
     lat (usec): min=22 , max=26120 , avg=248.53, stdev=349.42
    clat percentiles (usec):
     |  1.00th=[   51],  5.00th=[   68], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  122], 40.00th=[  145], 50.00th=[  169], 60.00th=[  199],
     | 70.00th=[  241], 80.00th=[  306], 90.00th=[  454], 95.00th=[  636],
     | 99.00th=[ 1448], 99.50th=[ 2064], 99.90th=[ 4128], 99.95th=[ 5408],
     | 99.99th=[11968]
    bw (KB/s)  : min= 8424, max=21320, per=3.11%, avg=15759.73, stdev=2291.43
    lat (usec) : 50=0.80%, 100=18.09%, 250=52.75%, 500=20.15%, 750=4.65%
    lat (usec) : 1000=1.55%
    lat (msec) : 2=1.48%, 4=0.42%, 10=0.09%, 20=0.02%, 50=0.01%
  cpu          : usr=1.88%, sys=9.74%, ctx=1272740, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1265803, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4944.6MB, aggrb=506270KB/s, minb=506270KB/s, maxb=506270KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1250138, merge=0/4, ticks=0/96803, in_queue=96953, util=98.11%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1431
  write: io=846128KB, bw=503791KB/s, iops=125947 , runt= 10005msec
    clat (usec): min=22 , max=25537 , avg=248.88, stdev=343.44
     lat (usec): min=23 , max=25538 , avg=249.79, stdev=344.36
    clat percentiles (usec):
     |  1.00th=[   51],  5.00th=[   68], 10.00th=[   81], 20.00th=[  103],
     | 30.00th=[  125], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  247], 80.00th=[  314], 90.00th=[  454], 95.00th=[  628],
     | 99.00th=[ 1416], 99.50th=[ 2008], 99.90th=[ 3888], 99.95th=[ 5280],
     | 99.99th=[ 9792]
    bw (KB/s)  : min=11288, max=20184, per=3.11%, avg=15672.59, stdev=1557.11
    lat (usec) : 50=0.78%, 100=17.41%, 250=52.26%, 500=21.45%, 750=4.70%
    lat (usec) : 1000=1.51%
    lat (msec) : 2=1.40%, 4=0.41%, 10=0.08%, 20=0.01%, 50=0.01%
  cpu          : usr=1.80%, sys=9.59%, ctx=1266399, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1260108, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4922.4MB, aggrb=503791KB/s, minb=503791KB/s, maxb=503791KB/s, mint=10005msec, maxt=10005msec

Disk stats (read/write):
  vda: ios=53/1249652, merge=49/2, ticks=1/99765, in_queue=99929, util=98.37%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1505
  write: io=1197.6MB, bw=542007KB/s, iops=135501 , runt= 10001msec
    clat (usec): min=19 , max=22449 , avg=231.20, stdev=283.83
     lat (usec): min=19 , max=22450 , avg=232.07, stdev=284.23
    clat percentiles (usec):
     |  1.00th=[   52],  5.00th=[   69], 10.00th=[   83], 20.00th=[  104],
     | 30.00th=[  123], 40.00th=[  143], 50.00th=[  167], 60.00th=[  195],
     | 70.00th=[  231], 80.00th=[  290], 90.00th=[  410], 95.00th=[  572],
     | 99.00th=[ 1208], 99.50th=[ 1736], 99.90th=[ 3344], 99.95th=[ 4320],
     | 99.99th=[ 7904]
    bw (KB/s)  : min=12712, max=21560, per=3.09%, avg=16772.95, stdev=1294.80
    lat (usec) : 20=0.01%, 50=0.69%, 100=17.12%, 250=55.77%, 500=19.75%
    lat (usec) : 750=3.92%, 1000=1.29%
    lat (msec) : 2=1.09%, 4=0.31%, 10=0.06%, 20=0.01%, 50=0.01%
  cpu          : usr=1.94%, sys=10.00%, ctx=1360554, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1355153, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=5293.6MB, aggrb=542006KB/s, minb=542006KB/s, maxb=542006KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=1/1338020, merge=0/0, ticks=0/98598, in_queue=98691, util=98.41%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1579
  write: io=615788KB, bw=480961KB/s, iops=120240 , runt= 10001msec
    clat (usec): min=23 , max=27699 , avg=260.83, stdev=389.57
     lat (usec): min=24 , max=27700 , avg=261.75, stdev=390.12
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  203],
     | 70.00th=[  249], 80.00th=[  322], 90.00th=[  482], 95.00th=[  692],
     | 99.00th=[ 1624], 99.50th=[ 2256], 99.90th=[ 4448], 99.95th=[ 6624],
     | 99.99th=[11712]
    bw (KB/s)  : min=11440, max=18968, per=3.12%, avg=15029.08, stdev=1396.06
    lat (usec) : 50=0.88%, 100=18.03%, 250=51.27%, 500=20.43%, 750=5.09%
    lat (usec) : 1000=1.84%
    lat (msec) : 2=1.79%, 4=0.53%, 10=0.11%, 20=0.02%, 50=0.01%
  cpu          : usr=1.79%, sys=9.82%, ctx=1210452, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1202523, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4697.4MB, aggrb=480961KB/s, minb=480961KB/s, maxb=480961KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1190943, merge=0/1, ticks=0/86405, in_queue=86548, util=98.38%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1653
  write: io=672540KB, bw=486441KB/s, iops=121610 , runt= 10005msec
    clat (usec): min=22 , max=33285 , avg=257.85, stdev=373.88
     lat (usec): min=22 , max=33286 , avg=258.76, stdev=374.51
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  249], 80.00th=[  322], 90.00th=[  478], 95.00th=[  676],
     | 99.00th=[ 1576], 99.50th=[ 2192], 99.90th=[ 4128], 99.95th=[ 5600],
     | 99.99th=[10304]
    bw (KB/s)  : min= 9984, max=18256, per=3.12%, avg=15181.60, stdev=1188.29
    lat (usec) : 50=0.91%, 100=17.83%, 250=51.35%, 500=20.73%, 750=5.08%
    lat (usec) : 1000=1.78%
    lat (msec) : 2=1.70%, 4=0.51%, 10=0.10%, 20=0.01%, 50=0.01%
  cpu          : usr=1.78%, sys=9.85%, ctx=1223059, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1216711, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4752.8MB, aggrb=486441KB/s, minb=486441KB/s, maxb=486441KB/s, mint=10005msec, maxt=10005msec

Disk stats (read/write):
  vda: ios=24/1203591, merge=11/0, ticks=1/86454, in_queue=86451, util=98.37%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1727
  write: io=612324KB, bw=480615KB/s, iops=120153 , runt= 10001msec
    clat (usec): min=23 , max=42297 , avg=260.98, stdev=391.11
     lat (usec): min=23 , max=42297 , avg=261.90, stdev=391.57
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  124], 40.00th=[  147], 50.00th=[  175], 60.00th=[  207],
     | 70.00th=[  251], 80.00th=[  326], 90.00th=[  486], 95.00th=[  676],
     | 99.00th=[ 1576], 99.50th=[ 2192], 99.90th=[ 4576], 99.95th=[ 6560],
     | 99.99th=[11712]
    bw (KB/s)  : min=10291, max=18376, per=3.13%, avg=15044.86, stdev=1544.50
    lat (usec) : 50=0.88%, 100=17.87%, 250=50.96%, 500=20.86%, 750=5.34%
    lat (usec) : 1000=1.76%
    lat (msec) : 2=1.72%, 4=0.47%, 10=0.12%, 20=0.02%, 50=0.01%
  cpu          : usr=1.80%, sys=9.62%, ctx=1209612, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1201657, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4693.1MB, aggrb=480614KB/s, minb=480614KB/s, maxb=480614KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1187819, merge=0/2, ticks=0/90647, in_queue=90906, util=97.74%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1801
  write: io=589068KB, bw=478242KB/s, iops=119560 , runt= 10002msec
    clat (usec): min=21 , max=27856 , avg=262.22, stdev=415.61
     lat (usec): min=22 , max=27857 , avg=263.15, stdev=416.28
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  124], 40.00th=[  147], 50.00th=[  175], 60.00th=[  207],
     | 70.00th=[  251], 80.00th=[  326], 90.00th=[  486], 95.00th=[  676],
     | 99.00th=[ 1592], 99.50th=[ 2256], 99.90th=[ 4960], 99.95th=[ 7968],
     | 99.99th=[12864]
    bw (KB/s)  : min=10952, max=18104, per=3.12%, avg=14932.31, stdev=1402.53
    lat (usec) : 50=0.86%, 100=17.75%, 250=51.23%, 500=20.81%, 750=5.27%
    lat (usec) : 1000=1.75%
    lat (msec) : 2=1.68%, 4=0.50%, 10=0.11%, 20=0.03%, 50=0.01%
  cpu          : usr=1.80%, sys=9.53%, ctx=1202406, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1195843, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4671.3MB, aggrb=478241KB/s, minb=478241KB/s, maxb=478241KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=32/1186137, merge=3/3, ticks=2/91032, in_queue=91237, util=97.93%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1875
  write: io=690592KB, bw=488441KB/s, iops=122110 , runt= 10001msec
    clat (usec): min=21 , max=67685 , avg=256.77, stdev=359.41
     lat (usec): min=22 , max=67685 , avg=257.70, stdev=360.26
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   79], 20.00th=[  101],
     | 30.00th=[  122], 40.00th=[  145], 50.00th=[  171], 60.00th=[  203],
     | 70.00th=[  247], 80.00th=[  318], 90.00th=[  478], 95.00th=[  692],
     | 99.00th=[ 1608], 99.50th=[ 2256], 99.90th=[ 4048], 99.95th=[ 5408],
     | 99.99th=[ 9792]
    bw (KB/s)  : min=12183, max=18184, per=3.13%, avg=15270.72, stdev=1235.55
    lat (usec) : 50=0.97%, 100=18.45%, 250=51.09%, 500=20.26%, 750=5.00%
    lat (usec) : 1000=1.81%
    lat (msec) : 2=1.75%, 4=0.55%, 10=0.10%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=1.79%, sys=9.83%, ctx=1228842, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1221224, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4770.5MB, aggrb=488440KB/s, minb=488440KB/s, maxb=488440KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=1/1206762, merge=0/0, ticks=1/87258, in_queue=87316, util=98.31%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=1949
  write: io=701584KB, bw=489540KB/s, iops=122384 , runt= 10001msec
    clat (usec): min=22 , max=28739 , avg=256.23, stdev=339.71
     lat (usec): min=22 , max=28739 , avg=257.14, stdev=340.13
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  124], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  249], 80.00th=[  322], 90.00th=[  482], 95.00th=[  676],
     | 99.00th=[ 1528], 99.50th=[ 2160], 99.90th=[ 4016], 99.95th=[ 4896],
     | 99.99th=[ 9536]
    bw (KB/s)  : min=11864, max=18384, per=3.15%, avg=15403.92, stdev=1207.76
    lat (usec) : 50=0.89%, 100=17.71%, 250=51.44%, 500=20.76%, 750=5.16%
    lat (usec) : 1000=1.79%
    lat (msec) : 2=1.65%, 4=0.49%, 10=0.09%, 20=0.01%, 50=0.01%
  cpu          : usr=1.78%, sys=9.85%, ctx=1231054, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1223972, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4781.2MB, aggrb=489539KB/s, minb=489539KB/s, maxb=489539KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1212741, merge=0/1, ticks=0/88633, in_queue=88794, util=98.34%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2023
  write: io=707312KB, bw=490064KB/s, iops=122515 , runt= 10002msec
    clat (usec): min=16 , max=26112 , avg=255.91, stdev=364.62
     lat (usec): min=17 , max=26113 , avg=256.81, stdev=365.09
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   79], 20.00th=[  101],
     | 30.00th=[  122], 40.00th=[  145], 50.00th=[  171], 60.00th=[  203],
     | 70.00th=[  247], 80.00th=[  318], 90.00th=[  478], 95.00th=[  676],
     | 99.00th=[ 1576], 99.50th=[ 2192], 99.90th=[ 4192], 99.95th=[ 5536],
     | 99.99th=[10688]
    bw (KB/s)  : min=10880, max=19216, per=3.13%, avg=15323.33, stdev=1315.57
    lat (usec) : 20=0.01%, 50=0.92%, 100=18.27%, 250=51.38%, 500=20.32%
    lat (usec) : 750=5.05%, 1000=1.76%
    lat (msec) : 2=1.70%, 4=0.50%, 10=0.10%, 20=0.01%, 50=0.01%
  cpu          : usr=1.82%, sys=9.79%, ctx=1231597, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1225404, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4786.8MB, aggrb=490063KB/s, minb=490063KB/s, maxb=490063KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=55/1211276, merge=47/1, ticks=11/89170, in_queue=89285, util=98.34%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2098
  write: io=606100KB, bw=479944KB/s, iops=119986 , runt= 10002msec
    clat (usec): min=17 , max=29842 , avg=261.25, stdev=383.26
     lat (usec): min=18 , max=29843 , avg=262.19, stdev=384.12
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   79], 20.00th=[  101],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  251], 80.00th=[  326], 90.00th=[  486], 95.00th=[  692],
     | 99.00th=[ 1640], 99.50th=[ 2288], 99.90th=[ 4512], 99.95th=[ 5792],
     | 99.99th=[11712]
    bw (KB/s)  : min=10744, max=18544, per=3.12%, avg=14980.79, stdev=1220.63
    lat (usec) : 20=0.01%, 50=0.92%, 100=18.23%, 250=50.74%, 500=20.63%
    lat (usec) : 750=5.20%, 1000=1.82%
    lat (msec) : 2=1.80%, 4=0.53%, 10=0.11%, 20=0.01%, 50=0.01%
  cpu          : usr=1.74%, sys=9.80%, ctx=1207290, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1200101, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4687.1MB, aggrb=479944KB/s, minb=479944KB/s, maxb=479944KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=0/1185547, merge=0/1, ticks=0/86783, in_queue=86846, util=98.20%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2172
  write: io=684636KB, bw=487845KB/s, iops=121961 , runt= 10001msec
    clat (usec): min=20 , max=44374 , avg=257.16, stdev=344.86
     lat (usec): min=21 , max=44375 , avg=258.06, stdev=345.36
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  251], 80.00th=[  322], 90.00th=[  482], 95.00th=[  684],
     | 99.00th=[ 1576], 99.50th=[ 2192], 99.90th=[ 4016], 99.95th=[ 4960],
     | 99.99th=[ 9152]
    bw (KB/s)  : min=11032, max=18416, per=3.14%, avg=15296.09, stdev=1271.42
    lat (usec) : 50=0.95%, 100=17.93%, 250=51.08%, 500=20.78%, 750=5.15%
    lat (usec) : 1000=1.76%
    lat (msec) : 2=1.72%, 4=0.51%, 10=0.09%, 20=0.01%, 50=0.01%
  cpu          : usr=1.85%, sys=9.78%, ctx=1226479, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1219735, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4764.7MB, aggrb=487845KB/s, minb=487845KB/s, maxb=487845KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1203417, merge=0/1, ticks=0/88077, in_queue=88080, util=98.53%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2246
  write: io=585524KB, bw=477935KB/s, iops=119483 , runt= 10001msec
    clat (usec): min=23 , max=25169 , avg=262.41, stdev=374.14
     lat (usec): min=23 , max=25170 , avg=263.35, stdev=374.70
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   67], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  124], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  251], 80.00th=[  326], 90.00th=[  490], 95.00th=[  692],
     | 99.00th=[ 1640], 99.50th=[ 2288], 99.90th=[ 4576], 99.95th=[ 6496],
     | 99.99th=[ 9920]
    bw (KB/s)  : min= 9760, max=18224, per=3.13%, avg=14978.00, stdev=1340.61
    lat (usec) : 50=0.88%, 100=17.86%, 250=50.97%, 500=20.73%, 750=5.23%
    lat (usec) : 1000=1.84%
    lat (msec) : 2=1.81%, 4=0.54%, 10=0.13%, 20=0.01%, 50=0.01%
  cpu          : usr=1.80%, sys=9.75%, ctx=1203319, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1194957, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4667.9MB, aggrb=477935KB/s, minb=477935KB/s, maxb=477935KB/s, mint=10001msec, maxt=10001msec

Disk stats (read/write):
  vda: ios=0/1181494, merge=0/1, ticks=0/88837, in_queue=89043, util=98.03%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2320
  write: io=662440KB, bw=485577KB/s, iops=121394 , runt= 10002msec
    clat (usec): min=21 , max=20170 , avg=258.31, stdev=352.14
     lat (usec): min=21 , max=20171 , avg=259.24, stdev=353.00
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   79], 20.00th=[  101],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  249], 80.00th=[  322], 90.00th=[  486], 95.00th=[  692],
     | 99.00th=[ 1592], 99.50th=[ 2224], 99.90th=[ 4080], 99.95th=[ 5408],
     | 99.99th=[10048]
    bw (KB/s)  : min=11096, max=19200, per=3.15%, avg=15279.06, stdev=1303.29
    lat (usec) : 50=0.91%, 100=18.29%, 250=50.84%, 500=20.55%, 750=5.19%
    lat (usec) : 1000=1.80%
    lat (msec) : 2=1.78%, 4=0.52%, 10=0.10%, 20=0.01%, 50=0.01%
  cpu          : usr=1.79%, sys=9.74%, ctx=1221315, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1214186, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4742.1MB, aggrb=485577KB/s, minb=485577KB/s, maxb=485577KB/s, mint=10002msec, maxt=10002msec

Disk stats (read/write):
  vda: ios=0/1200525, merge=0/2, ticks=0/89294, in_queue=89377, util=98.32%
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=2394
  write: io=634160KB, bw=482509KB/s, iops=120627 , runt= 10007msec
    clat (usec): min=21 , max=32078 , avg=259.89, stdev=370.63
     lat (usec): min=22 , max=32079 , avg=260.80, stdev=371.09
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   66], 10.00th=[   80], 20.00th=[  102],
     | 30.00th=[  123], 40.00th=[  147], 50.00th=[  173], 60.00th=[  205],
     | 70.00th=[  251], 80.00th=[  326], 90.00th=[  482], 95.00th=[  684],
     | 99.00th=[ 1592], 99.50th=[ 2256], 99.90th=[ 4512], 99.95th=[ 6368],
     | 99.99th=[10304]
    bw (KB/s)  : min=11376, max=18336, per=3.14%, avg=15138.18, stdev=1399.59
    lat (usec) : 50=0.94%, 100=17.97%, 250=50.98%, 500=20.77%, 750=5.21%
    lat (usec) : 1000=1.78%
    lat (msec) : 2=1.71%, 4=0.51%, 10=0.12%, 20=0.01%, 50=0.01%
  cpu          : usr=1.73%, sys=9.73%, ctx=1214094, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=1207116, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=4715.4MB, aggrb=482508KB/s, minb=482508KB/s, maxb=482508KB/s, mint=10007msec, maxt=10007msec

Disk stats (read/write):
  vda: ios=0/1193792, merge=0/3, ticks=0/91248, in_queue=91538, util=98.43%
