128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=23664
  write: io=1025.2MB, bw=104936KB/s, iops=26234 , runt= 10004msec
    clat (usec): min=53 , max=16762 , avg=1215.97, stdev=888.84
     lat (usec): min=54 , max=16762 , avg=1216.37, stdev=888.85
    clat percentiles (usec):
     |  1.00th=[   85],  5.00th=[  197], 10.00th=[  298], 20.00th=[  478],
     | 30.00th=[  644], 40.00th=[  812], 50.00th=[ 1020], 60.00th=[ 1256],
     | 70.00th=[ 1528], 80.00th=[ 1880], 90.00th=[ 2384], 95.00th=[ 2832],
     | 99.00th=[ 3760], 99.50th=[ 4384], 99.90th=[ 7008], 99.95th=[ 7776],
     | 99.99th=[13632]
    bw (KB/s)  : min= 3003, max= 3648, per=3.12%, avg=3279.21, stdev=111.17
    lat (usec) : 100=1.57%, 250=5.93%, 500=13.94%, 750=14.95%, 1000=12.78%
    lat (msec) : 2=33.64%, 4=16.46%, 10=0.71%, 20=0.02%
  cpu          : usr=0.39%, sys=1.34%, ctx=269136, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=262446/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1025.2MB, aggrb=104936KB/s, minb=104936KB/s, maxb=104936KB/s, mint=10004msec, maxt=10004msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=23740
  write: io=1061.1MB, bw=108666KB/s, iops=27166 , runt= 10007msec
    clat (usec): min=53 , max=39035 , avg=1174.00, stdev=1365.49
     lat (usec): min=54 , max=39035 , avg=1174.41, stdev=1365.51
    clat percentiles (usec):
     |  1.00th=[   86],  5.00th=[  189], 10.00th=[  278], 20.00th=[  434],
     | 30.00th=[  572], 40.00th=[  716], 50.00th=[  884], 60.00th=[ 1080],
     | 70.00th=[ 1320], 80.00th=[ 1640], 90.00th=[ 2096], 95.00th=[ 2544],
     | 99.00th=[ 9408], 99.50th=[10048], 99.90th=[12096], 99.95th=[19072],
     | 99.99th=[27776]
    bw (KB/s)  : min= 2056, max= 4024, per=3.13%, avg=3397.58, stdev=565.62
    lat (usec) : 100=1.54%, 250=6.95%, 500=16.17%, 750=17.66%, 1000=13.80%
    lat (msec) : 2=32.04%, 4=9.97%, 10=1.31%, 20=0.52%, 50=0.04%
  cpu          : usr=0.42%, sys=1.34%, ctx=278943, majf=0, minf=902
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=271856/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1061.1MB, aggrb=108666KB/s, minb=108666KB/s, maxb=108666KB/s, mint=10007msec, maxt=10007msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=23814
  write: io=1045.8MB, bw=106994KB/s, iops=26748 , runt= 10002msec
    clat (usec): min=54 , max=39486 , avg=1192.79, stdev=1415.91
     lat (usec): min=54 , max=39487 , avg=1193.18, stdev=1415.92
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[  203], 10.00th=[  298], 20.00th=[  466],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1112],
     | 70.00th=[ 1352], 80.00th=[ 1640], 90.00th=[ 2040], 95.00th=[ 2416],
     | 99.00th=[ 9408], 99.50th=[10176], 99.90th=[14272], 99.95th=[17280],
     | 99.99th=[30848]
    bw (KB/s)  : min= 1904, max= 4048, per=3.12%, avg=3334.05, stdev=603.18
    lat (usec) : 100=1.39%, 250=5.97%, 500=14.89%, 750=17.89%, 1000=14.43%
    lat (msec) : 2=34.54%, 4=8.96%, 10=1.32%, 20=0.57%, 50=0.04%
  cpu          : usr=0.40%, sys=1.30%, ctx=275105, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=267539/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1045.8MB, aggrb=106994KB/s, minb=106994KB/s, maxb=106994KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=23888
  write: io=1141.8MB, bw=116886KB/s, iops=29221 , runt= 10002msec
    clat (usec): min=52 , max=12661 , avg=1091.56, stdev=834.77
     lat (usec): min=52 , max=12662 , avg=1091.96, stdev=834.78
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  217], 10.00th=[  314], 20.00th=[  474],
     | 30.00th=[  612], 40.00th=[  748], 50.00th=[  916], 60.00th=[ 1112],
     | 70.00th=[ 1352], 80.00th=[ 1640], 90.00th=[ 2024], 95.00th=[ 2320],
     | 99.00th=[ 3280], 99.50th=[ 5152], 99.90th=[10176], 99.95th=[11200],
     | 99.99th=[11840]
    bw (KB/s)  : min= 3345, max= 4016, per=3.15%, avg=3677.61, stdev=130.37
    lat (usec) : 100=1.19%, 250=5.40%, 500=15.12%, 750=18.18%, 1000=14.52%
    lat (msec) : 2=35.16%, 4=9.77%, 10=0.54%, 20=0.11%
  cpu          : usr=0.43%, sys=1.46%, ctx=299794, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=292274/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1141.8MB, aggrb=116886KB/s, minb=116886KB/s, maxb=116886KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=23962
  write: io=1055.4MB, bw=108048KB/s, iops=27012 , runt= 10002msec
    clat (usec): min=54 , max=30885 , avg=1181.08, stdev=1362.21
     lat (usec): min=55 , max=30886 , avg=1181.48, stdev=1362.22
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  199], 10.00th=[  294], 20.00th=[  454],
     | 30.00th=[  596], 40.00th=[  732], 50.00th=[  908], 60.00th=[ 1112],
     | 70.00th=[ 1336], 80.00th=[ 1640], 90.00th=[ 2064], 95.00th=[ 2448],
     | 99.00th=[ 9280], 99.50th=[10048], 99.90th=[12096], 99.95th=[16064],
     | 99.99th=[28544]
    bw (KB/s)  : min= 2003, max= 3984, per=3.12%, avg=3375.91, stdev=586.40
    lat (usec) : 100=1.30%, 250=6.34%, 500=15.47%, 750=17.87%, 1000=13.87%
    lat (msec) : 2=34.10%, 4=9.19%, 10=1.33%, 20=0.49%, 50=0.05%
  cpu          : usr=0.39%, sys=1.38%, ctx=277227, majf=0, minf=935
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=270175/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1055.4MB, aggrb=108048KB/s, minb=108048KB/s, maxb=108048KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24036
  write: io=1048.7MB, bw=107362KB/s, iops=26840 , runt= 10002msec
    clat (usec): min=52 , max=38509 , avg=1188.65, stdev=1400.29
     lat (usec): min=52 , max=38510 , avg=1189.05, stdev=1400.29
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[  205], 10.00th=[  302], 20.00th=[  466],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1096],
     | 70.00th=[ 1336], 80.00th=[ 1624], 90.00th=[ 2064], 95.00th=[ 2448],
     | 99.00th=[ 9408], 99.50th=[10048], 99.90th=[12608], 99.95th=[17536],
     | 99.99th=[30848]
    bw (KB/s)  : min= 1587, max= 4016, per=3.12%, avg=3346.05, stdev=592.66
    lat (usec) : 100=1.39%, 250=5.88%, 500=14.94%, 750=18.15%, 1000=14.54%
    lat (msec) : 2=34.12%, 4=9.00%, 10=1.46%, 20=0.47%, 50=0.05%
  cpu          : usr=0.36%, sys=1.37%, ctx=275655, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=268458/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1048.7MB, aggrb=107361KB/s, minb=107361KB/s, maxb=107361KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24110
  write: io=1151.8MB, bw=117904KB/s, iops=29475 , runt= 10003msec
    clat (usec): min=54 , max=13165 , avg=1082.08, stdev=806.71
     lat (usec): min=54 , max=13166 , avg=1082.48, stdev=806.71
    clat percentiles (usec):
     |  1.00th=[   98],  5.00th=[  223], 10.00th=[  322], 20.00th=[  482],
     | 30.00th=[  620], 40.00th=[  756], 50.00th=[  924], 60.00th=[ 1112],
     | 70.00th=[ 1320], 80.00th=[ 1608], 90.00th=[ 1992], 95.00th=[ 2320],
     | 99.00th=[ 3184], 99.50th=[ 4576], 99.90th=[ 9920], 99.95th=[10816],
     | 99.99th=[11968]
    bw (KB/s)  : min= 3256, max= 4007, per=3.14%, avg=3699.59, stdev=128.13
    lat (usec) : 100=1.04%, 250=5.30%, 500=14.87%, 750=18.21%, 1000=14.68%
    lat (msec) : 2=36.05%, 4=9.26%, 10=0.49%, 20=0.10%
  cpu          : usr=0.45%, sys=1.43%, ctx=302211, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=294848/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1151.8MB, aggrb=117903KB/s, minb=117903KB/s, maxb=117903KB/s, mint=10003msec, maxt=10003msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24184
  write: io=1049.7MB, bw=107452KB/s, iops=26863 , runt= 10003msec
    clat (usec): min=54 , max=30865 , avg=1187.64, stdev=1363.78
     lat (usec): min=54 , max=30866 , avg=1188.04, stdev=1363.78
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[  201], 10.00th=[  298], 20.00th=[  470],
     | 30.00th=[  612], 40.00th=[  748], 50.00th=[  916], 60.00th=[ 1112],
     | 70.00th=[ 1352], 80.00th=[ 1640], 90.00th=[ 2024], 95.00th=[ 2384],
     | 99.00th=[ 9408], 99.50th=[10048], 99.90th=[11584], 99.95th=[17024],
     | 99.99th=[28544]
    bw (KB/s)  : min= 1692, max= 4048, per=3.12%, avg=3353.59, stdev=578.25
    lat (usec) : 100=1.37%, 250=6.09%, 500=14.49%, 750=17.95%, 1000=14.37%
    lat (msec) : 2=35.09%, 4=8.68%, 10=1.42%, 20=0.49%, 50=0.05%
  cpu          : usr=0.38%, sys=1.33%, ctx=275504, majf=0, minf=935
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=268711/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1049.7MB, aggrb=107452KB/s, minb=107452KB/s, maxb=107452KB/s, mint=10003msec, maxt=10003msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24258
  write: io=1029.2MB, bw=105363KB/s, iops=26340 , runt= 10002msec
    clat (usec): min=55 , max=40216 , avg=1211.22, stdev=1434.95
     lat (usec): min=55 , max=40217 , avg=1211.62, stdev=1434.96
    clat percentiles (usec):
     |  1.00th=[   87],  5.00th=[  203], 10.00th=[  302], 20.00th=[  474],
     | 30.00th=[  620], 40.00th=[  764], 50.00th=[  924], 60.00th=[ 1112],
     | 70.00th=[ 1336], 80.00th=[ 1640], 90.00th=[ 2064], 95.00th=[ 2480],
     | 99.00th=[ 9024], 99.50th=[10048], 99.90th=[12992], 99.95th=[21376],
     | 99.99th=[30848]
    bw (KB/s)  : min= 1392, max= 3936, per=3.11%, avg=3274.58, stdev=713.14
    lat (usec) : 100=1.44%, 250=5.75%, 500=14.55%, 750=17.53%, 1000=14.62%
    lat (msec) : 2=35.11%, 4=8.61%, 10=1.87%, 20=0.45%, 50=0.06%
  cpu          : usr=0.39%, sys=1.31%, ctx=270450, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=263461/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1029.2MB, aggrb=105363KB/s, minb=105363KB/s, maxb=105363KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24334
  write: io=1134.8MB, bw=116171KB/s, iops=29042 , runt= 10002msec
    clat (usec): min=53 , max=15873 , avg=1098.30, stdev=875.61
     lat (usec): min=54 , max=15874 , avg=1098.69, stdev=875.61
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  217], 10.00th=[  318], 20.00th=[  486],
     | 30.00th=[  628], 40.00th=[  772], 50.00th=[  932], 60.00th=[ 1128],
     | 70.00th=[ 1336], 80.00th=[ 1608], 90.00th=[ 2008], 95.00th=[ 2320],
     | 99.00th=[ 3248], 99.50th=[ 5536], 99.90th=[10944], 99.95th=[11584],
     | 99.99th=[12736]
    bw (KB/s)  : min= 3240, max= 4000, per=3.13%, avg=3640.83, stdev=139.60
    lat (usec) : 100=1.15%, 250=5.47%, 500=14.41%, 750=17.73%, 1000=14.92%
    lat (msec) : 2=36.21%, 4=9.41%, 10=0.53%, 20=0.18%
  cpu          : usr=0.40%, sys=1.44%, ctx=298196, majf=0, minf=935
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=290485/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1134.8MB, aggrb=116170KB/s, minb=116170KB/s, maxb=116170KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24408
  write: io=1064.5MB, bw=108975KB/s, iops=27243 , runt= 10002msec
    clat (usec): min=53 , max=38463 , avg=1171.02, stdev=1319.80
     lat (usec): min=53 , max=38464 , avg=1171.43, stdev=1319.81
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  211], 10.00th=[  306], 20.00th=[  466],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1096],
     | 70.00th=[ 1336], 80.00th=[ 1624], 90.00th=[ 2040], 95.00th=[ 2416],
     | 99.00th=[ 8896], 99.50th=[ 9920], 99.90th=[11968], 99.95th=[17792],
     | 99.99th=[29312]
    bw (KB/s)  : min= 1424, max= 4080, per=3.12%, avg=3400.39, stdev=642.48
    lat (usec) : 100=1.21%, 250=5.79%, 500=15.19%, 750=18.22%, 1000=14.51%
    lat (msec) : 2=34.38%, 4=8.91%, 10=1.34%, 20=0.41%, 50=0.05%
  cpu          : usr=0.41%, sys=1.35%, ctx=279875, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=272492/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1064.5MB, aggrb=108975KB/s, minb=108975KB/s, maxb=108975KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24482
  write: io=1046.7MB, bw=107159KB/s, iops=26789 , runt= 10002msec
    clat (usec): min=54 , max=39119 , avg=1190.92, stdev=1407.63
     lat (usec): min=54 , max=39119 , avg=1191.33, stdev=1407.64
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[  205], 10.00th=[  302], 20.00th=[  470],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1096],
     | 70.00th=[ 1336], 80.00th=[ 1624], 90.00th=[ 2064], 95.00th=[ 2448],
     | 99.00th=[ 9408], 99.50th=[10048], 99.90th=[13120], 99.95th=[19072],
     | 99.99th=[28800]
    bw (KB/s)  : min= 1370, max= 3984, per=3.14%, avg=3366.22, stdev=691.51
    lat (usec) : 100=1.30%, 250=5.90%, 500=15.01%, 750=18.14%, 1000=14.35%
    lat (msec) : 2=34.26%, 4=9.13%, 10=1.37%, 20=0.50%, 50=0.05%
  cpu          : usr=0.39%, sys=1.30%, ctx=274923, majf=0, minf=935
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=267951/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1046.7MB, aggrb=107158KB/s, minb=107158KB/s, maxb=107158KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24556
  write: io=1147.3MB, bw=117421KB/s, iops=29355 , runt= 10005msec
    clat (usec): min=54 , max=12578 , avg=1086.29, stdev=821.01
     lat (usec): min=54 , max=12579 , avg=1086.69, stdev=821.02
    clat percentiles (usec):
     |  1.00th=[   97],  5.00th=[  223], 10.00th=[  326], 20.00th=[  486],
     | 30.00th=[  628], 40.00th=[  764], 50.00th=[  932], 60.00th=[ 1112],
     | 70.00th=[ 1336], 80.00th=[ 1608], 90.00th=[ 1992], 95.00th=[ 2320],
     | 99.00th=[ 3088], 99.50th=[ 4640], 99.90th=[10560], 99.95th=[11200],
     | 99.99th=[12096]
    bw (KB/s)  : min= 3296, max= 3976, per=3.14%, avg=3688.96, stdev=107.66
    lat (usec) : 100=1.05%, 250=5.20%, 500=14.61%, 750=18.13%, 1000=14.95%
    lat (msec) : 2=36.15%, 4=9.32%, 10=0.46%, 20=0.14%
  cpu          : usr=0.43%, sys=1.45%, ctx=301175, majf=0, minf=903
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=293700/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1147.3MB, aggrb=117421KB/s, minb=117421KB/s, maxb=117421KB/s, mint=10005msec, maxt=10005msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24630
  write: io=1064.2MB, bw=108943KB/s, iops=27235 , runt= 10002msec
    clat (usec): min=53 , max=38811 , avg=1171.34, stdev=1321.36
     lat (usec): min=53 , max=38812 , avg=1171.75, stdev=1321.36
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[  205], 10.00th=[  302], 20.00th=[  466],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1112],
     | 70.00th=[ 1336], 80.00th=[ 1624], 90.00th=[ 2040], 95.00th=[ 2448],
     | 99.00th=[ 8896], 99.50th=[ 9920], 99.90th=[12224], 99.95th=[16512],
     | 99.99th=[28288]
    bw (KB/s)  : min= 1384, max= 4008, per=3.12%, avg=3399.46, stdev=617.17
    lat (usec) : 100=1.33%, 250=5.93%, 500=15.19%, 750=17.92%, 1000=14.41%
    lat (msec) : 2=34.36%, 4=9.11%, 10=1.25%, 20=0.45%, 50=0.04%
  cpu          : usr=0.39%, sys=1.38%, ctx=279675, majf=0, minf=904
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=272413/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1064.2MB, aggrb=108943KB/s, minb=108943KB/s, maxb=108943KB/s, mint=10002msec, maxt=10002msec
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
...
128m: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.7
Starting 32 processes

128m: (groupid=0, jobs=32): err= 0: pid=24704
  write: io=1072.4MB, bw=109782KB/s, iops=27445 , runt= 10002msec
    clat (usec): min=54 , max=37510 , avg=1162.39, stdev=1279.82
     lat (usec): min=54 , max=37510 , avg=1162.79, stdev=1279.82
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  207], 10.00th=[  302], 20.00th=[  466],
     | 30.00th=[  604], 40.00th=[  748], 50.00th=[  908], 60.00th=[ 1112],
     | 70.00th=[ 1336], 80.00th=[ 1624], 90.00th=[ 2064], 95.00th=[ 2448],
     | 99.00th=[ 8160], 99.50th=[ 9792], 99.90th=[11712], 99.95th=[16512],
     | 99.99th=[28800]
    bw (KB/s)  : min= 1232, max= 4008, per=3.13%, avg=3440.79, stdev=619.79
    lat (usec) : 100=1.21%, 250=6.04%, 500=14.95%, 750=18.13%, 1000=14.45%
    lat (msec) : 2=34.28%, 4=9.32%, 10=1.24%, 20=0.33%, 50=0.04%
  cpu          : usr=0.41%, sys=1.37%, ctx=281813, majf=0, minf=902
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=274509/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=1072.4MB, aggrb=109781KB/s, minb=109781KB/s, maxb=109781KB/s, mint=10002msec, maxt=10002msec
