ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Π’ ΠΎΠ΄Π½ΠΎΠΉ ΠΈΠ· ΠΏΡ€Π΅Π΄Ρ‹Π΄ΡƒΡ‰ΠΈΡ… статСй я ΠΏΡ€ΠΈΠ²Π΅Π» Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ тСстирования Π΄Π΅ΡˆΠ΅Π²Ρ‹Ρ… Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ Ρ€Π°Π·Π»ΠΈΡ‡Π½Ρ‹Ρ… хостСров Ρ€ΡƒΠ½Π΅Ρ‚Π°. Бпасибо всСм ΠΊΠΎΠΌΠΌΠ΅Π½Ρ‚Π°Ρ‚ΠΎΡ€Π°ΠΌ ΠΈ людям, писавшим Π² Π»ΠΈΡ‡Π½Ρ‹Π΅ сообщСния, Π·Π° ΠΎΠ±Ρ€Π°Ρ‚Π½ΡƒΡŽ связь.

Π’ этот Ρ€Π°Π· я Ρ…ΠΎΡ‡Ρƒ ΠΏΡ€Π΅Π΄ΡΡ‚Π°Π²ΠΈΡ‚ΡŒ Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ тСстирования Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ извСстных ΠΈ ΠΊΡ€ΡƒΠΏΠ½Ρ‹Ρ… ΠΊΠΎΠΌΠΏΠ°Π½ΠΈΠΉ: DigitalOcean, Vultr, Linode ΠΈ Hetzner. Π‘Π΄Π΅Π»Π°Π½ΠΎ 38 тСстов для всСх доступных Π»ΠΎΠΊΠ°Ρ†ΠΈΠΉ.

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Π₯отя характСристики Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров Π½Π° этот Ρ€Π°Π· ΠΎΡ‡Π΅Π½ΡŒ Π±Π»ΠΈΠ·ΠΊΠΈ, Π½ΠΎ всС ΠΆΠ΅ стоит Π½Π°ΠΏΠΎΠΌΠ½ΠΈΡ‚ΡŒ, Ρ‡Ρ‚ΠΎ Ρ„ΠΎΡ€ΠΌΠ°Ρ‚ тСстирования Π½Π΅ слишком ΠΎΠ±ΡŠΠ΅ΠΊΡ‚ΠΈΠ²Π΅Π½, ΠΈ Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ стоит Ρ€Π°ΡΡΠΌΠ°Ρ‚Ρ€ΠΈΠ²Π°Ρ‚ΡŒ с ΠΎΠΏΡ€Π΅Π΄Π΅Π»Π΅Π½Π½ΠΎΠΉ Π΄ΠΎΠ»Π΅ΠΉ ΠΈΡ€ΠΎΠ½ΠΈΠΈ, Ρ‚Π°ΠΊ ΠΊΠ°ΠΊ ΠΈΠ·ΠΌΠ΅Ρ€Π΅Π½Π½Ρ‹Π΅ Π²Π΅Π»ΠΈΡ‡ΠΈΠ½Ρ‹ вСсьма ΠΎΡ‚Π½ΠΎΡΠΈΡ‚Π΅Π»ΡŒΠ½Ρ‹.

Как ΠΈ Π² ΠΏΡ€ΠΎΡˆΠ»Ρ‹ΠΉ Ρ€Π°Π· тСстированиС ΠΏΡ€ΠΎΠ²ΠΎΠ΄ΠΈΠ»ΠΎΡΡŒ ΠΏΡ€ΠΈ ΠΏΠΎΠΌΠΎΡ‰ΠΈ unixbench, dd, fio, speedtest, Ρ€Π°Π±ΠΎΡ‚Π°ΡŽΡ‰ΠΈΡ… Π½Π° Debian 9, ΠΏΠ°Ρ€Π°ΠΌΠ΅Ρ‚Ρ€Ρ‹ Π½Π΅ мСнялись (Π΄Π²Π° тСста dd: Π±Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K ΠΈ 1M; Ρ‡Π΅Ρ‚Ρ‹Ρ€Π΅ тСста fio: Π² Ρ€Π΅ΠΆΠΈΠΌΠ°Ρ… random read, random direct read, random write ΠΈ random direct write).

ОписаниС Ρ‚Π°Ρ€ΠΈΡ„ΠΎΠ² ΠΈ Ρ†Π΅Π½Ρ‹:

Π₯остСр
ОбъСм диска / РСсурсы процСссора / ΠžΠΏΠ΅Ρ€Π°Ρ‚ΠΈΠ²Π½Π°Ρ ΠΏΠ°ΠΌΡΡ‚ΡŒ
Π¦Π΅Π½Π°

DigitalOcean
25 GB SSD
CA, DE, IN, NL, US (CA) β€” 1x 2.2 GHz (Intel Xeon E5-2650v4);
SG, UK, US (NY) β€” 1x 2.3 GHz (Intel Xeon Gold 6140)
1 GB (MemTotal: 1020380 kB)
$5 / мСсяц (~313 Ρ€ΡƒΠ±Π»Π΅ΠΉ)

Vultr
25 GB SSD
AU, DE, FR, JP, NL, SG, UK, US (GA, IL, TX, CA, FL, NJ, WA) β€” 1x 2.4 GHz (vCPU);
CA β€” 1x 2.6 GHz (vCPU)
1 GB (MemTotal: 1020268 kB)
$5 / мСсяц (~313 Ρ€ΡƒΠ±Π»Π΅ΠΉ)

Linode
25 GB SSD
AU, US (NJ) β€” 1x 2.2 GHz (AMD EPYC 7601);
CA, DE, IN, US (GA) β€” 1x 2 GHz (AMD EPYC 7501);
JP, SG, US (CA, TX) β€” 1x 2.3 GHz (Intel Xeon E5-2697v4);
UK β€” 1x 2.5 GHz (Intel Xeon E5-2680v3);
1 GB (MemTotal: 1020264 kB)
$5 / мСсяц (~313 Ρ€ΡƒΠ±Π»Π΅ΠΉ)

Hetzner
20 GB NVMe SSD
DE (Π€Π°Π»ΡŒΠΊΠ΅Π½ΡˆΡ‚Π΅ΠΉΠ½) β€” 1x 2.3 GHz (Intel Xeon Skylake);
DE (ΠΡŽΡ€Π½Π±Π΅Ρ€Π³), FI β€” 1x 2.1 GHz (Intel Xeon Skylake)
2 GB (MemTotal: 2003300 kB)
2.99€ / мСсяц (~208 Ρ€ΡƒΠ±Π»Π΅ΠΉ)

Π Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ тСстов находятся ΠΏΠΎΠ΄ спойлСрами, Π° Π³Ρ€Π°Ρ„ΠΈΠΊΠΈ β€” Π² ΠΊΠΎΠ½Ρ†Π΅ ΡΡ‚Π°Ρ‚ΡŒΠΈ.

Π’ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹ΠΉ сСрвСр ΠΎΡ‚ DigitalOcean:

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

  • Ρ‚Π°Ρ€ΠΈΡ„ Standard Droplet (25 GB SSD / 1x vCPU / 1 GB RAM / 1000 Mbit/s Π·Π° $5 Π² мСсяц);
  • Π΄Π°Ρ‚Π°-Ρ†Π΅Π½Ρ‚Ρ€Ρ‹ Π² Π’Π΅Π»ΠΈΠΊΠΎΠ±Ρ€ΠΈΡ‚Π°Π½ΠΈΠΈ, Π“Π΅Ρ€ΠΌΠ°Π½ΠΈΠΈ, КанадС, Индии, НидСрландах, Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€Π΅, БША.

ВСсты:

Unixbench (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-tor1-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (4400.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   15:52:08 up 6 min,  1 user,  load average: 0.86, 0.44, 0.19; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 15:52:08 - 16:19:36
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       17067275.6 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3581.7 MWIPS (6.4 s, 7 samples)
Execl Throughput                               2166.6 lps   (29.7 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        244701.5 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           71178.2 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        663117.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              334042.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  58850.2 lps   (10.0 s, 7 samples)
Process Creation                               4662.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   2937.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    393.6 lpm   (60.1 s, 2 samples)
System Call Overhead                         230331.4 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   17067275.6   1462.5
Double-Precision Whetstone                       55.0       3581.7    651.2
Execl Throughput                                 43.0       2166.6    503.9
File Copy 1024 bufsize 2000 maxblocks          3960.0     244701.5    617.9
File Copy 256 bufsize 500 maxblocks            1655.0      71178.2    430.1
File Copy 4096 bufsize 8000 maxblocks          5800.0     663117.5   1143.3
Pipe Throughput                               12440.0     334042.9    268.5
Pipe-based Context Switching                   4000.0      58850.2    147.1
Process Creation                                126.0       4662.5    370.0
Shell Scripts (1 concurrent)                     42.4       2937.6    692.8
Shell Scripts (8 concurrent)                      6.0        393.6    656.0
System Call Overhead                          15000.0     230331.4    153.6
                                                                   ========
System Benchmarks Index Score                                         479.8

Unixbench (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-fra1-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (4400.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   23:37:04 up 3 min,  1 user,  load average: 0.66, 0.24, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Fri Dec 13 2019 23:37:04 - 00:05:10
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26912014.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4483.5 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3672.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        463711.6 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          126002.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1201991.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              630949.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  99295.5 lps   (10.0 s, 7 samples)
Process Creation                               8599.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6028.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    780.4 lpm   (60.0 s, 2 samples)
System Call Overhead                         355362.1 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26912014.0   2306.1
Double-Precision Whetstone                       55.0       4483.5    815.2
Execl Throughput                                 43.0       3672.5    854.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     463711.6   1171.0
File Copy 256 bufsize 500 maxblocks            1655.0     126002.9    761.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1201991.6   2072.4
Pipe Throughput                               12440.0     630949.6    507.2
Pipe-based Context Switching                   4000.0      99295.5    248.2
Process Creation                                126.0       8599.9    682.5
Shell Scripts (1 concurrent)                     42.4       6028.7   1421.9
Shell Scripts (8 concurrent)                      6.0        780.4   1300.6
System Call Overhead                          15000.0     355362.1    236.9
                                                                   ========
System Benchmarks Index Score                                         833.0

Unixbench (Индия, Π‘Π°Π½Π³Π°Π»ΠΎΡ€)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-blr1-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (4400.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   18:52:38 up 3 min,  1 user,  load average: 0.66, 0.24, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 18:52:38 - 19:20:46
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       23572479.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4128.6 MWIPS (10.0 s, 7 samples)
Execl Throughput                               3005.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        350138.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           95265.1 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        982227.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              532053.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  85820.9 lps   (10.0 s, 7 samples)
Process Creation                               7340.8 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   4636.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    597.8 lpm   (60.1 s, 2 samples)
System Call Overhead                         308446.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   23572479.8   2019.9
Double-Precision Whetstone                       55.0       4128.6    750.6
Execl Throughput                                 43.0       3005.2    698.9
File Copy 1024 bufsize 2000 maxblocks          3960.0     350138.9    884.2
File Copy 256 bufsize 500 maxblocks            1655.0      95265.1    575.6
File Copy 4096 bufsize 8000 maxblocks          5800.0     982227.6   1693.5
Pipe Throughput                               12440.0     532053.3    427.7
Pipe-based Context Switching                   4000.0      85820.9    214.6
Process Creation                                126.0       7340.8    582.6
Shell Scripts (1 concurrent)                     42.4       4636.3   1093.5
Shell Scripts (8 concurrent)                      6.0        597.8    996.3
System Call Overhead                          15000.0     308446.2    205.6
                                                                   ========
System Benchmarks Index Score                                         686.4

Unixbench (НидСрланды, АмстСрдам)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-ams3-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (4400.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   14:12:24 up 6 min,  1 user,  load average: 0.90, 0.34, 0.12; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 14:12:24 - 14:40:37
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       18016981.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3729.2 MWIPS (10.1 s, 7 samples)
Execl Throughput                               2416.3 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        283772.5 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           76113.2 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        826905.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                              461320.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  71653.1 lps   (10.0 s, 7 samples)
Process Creation                               5727.0 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   3489.5 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    445.3 lpm   (60.2 s, 2 samples)
System Call Overhead                         276916.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   18016981.9   1543.9
Double-Precision Whetstone                       55.0       3729.2    678.0
Execl Throughput                                 43.0       2416.3    561.9
File Copy 1024 bufsize 2000 maxblocks          3960.0     283772.5    716.6
File Copy 256 bufsize 500 maxblocks            1655.0      76113.2    459.9
File Copy 4096 bufsize 8000 maxblocks          5800.0     826905.7   1425.7
Pipe Throughput                               12440.0     461320.6    370.8
Pipe-based Context Switching                   4000.0      71653.1    179.1
Process Creation                                126.0       5727.0    454.5
Shell Scripts (1 concurrent)                     42.4       3489.5    823.0
Shell Scripts (8 concurrent)                      6.0        445.3    742.2
System Call Overhead                          15000.0     276916.2    184.6
                                                                   ========
System Benchmarks Index Score                                         559.5

Unixbench (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-sgp1-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (4589.2 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   18:08:59 up 3 min,  1 user,  load average: 0.75, 0.27, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 18:08:59 - 18:37:08
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26210094.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3688.2 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3280.2 lps   (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        395957.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          105354.4 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1118194.8 KBps  (30.0 s, 2 samples)
Pipe Throughput                              627972.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  86900.0 lps   (10.0 s, 7 samples)
Process Creation                               7904.4 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   4941.1 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    631.0 lpm   (60.1 s, 2 samples)
System Call Overhead                         368192.7 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26210094.9   2245.9
Double-Precision Whetstone                       55.0       3688.2    670.6
Execl Throughput                                 43.0       3280.2    762.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     395957.3    999.9
File Copy 256 bufsize 500 maxblocks            1655.0     105354.4    636.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1118194.8   1927.9
Pipe Throughput                               12440.0     627972.2    504.8
Pipe-based Context Switching                   4000.0      86900.0    217.3
Process Creation                                126.0       7904.4    627.3
Shell Scripts (1 concurrent)                     42.4       4941.1   1165.3
Shell Scripts (8 concurrent)                      6.0        631.0   1051.6
System Call Overhead                          15000.0     368192.7    245.5
                                                                   ========
System Benchmarks Index Score                                         744.9

Unixbench (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-lon1-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (4589.2 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   12:30:15 up 8 min,  1 user,  load average: 0.71, 0.24, 0.08; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 12 2019 12:30:15 - 12:58:24
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       31577703.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4164.5 MWIPS (9.8 s, 7 samples)
Execl Throughput                               3746.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        449769.6 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          127650.7 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1238103.8 KBps  (30.0 s, 2 samples)
Pipe Throughput                              684221.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  97428.3 lps   (10.0 s, 7 samples)
Process Creation                               9107.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   5976.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    746.2 lpm   (60.0 s, 2 samples)
System Call Overhead                         405652.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   31577703.9   2705.9
Double-Precision Whetstone                       55.0       4164.5    757.2
Execl Throughput                                 43.0       3746.2    871.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     449769.6   1135.8
File Copy 256 bufsize 500 maxblocks            1655.0     127650.7    771.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1238103.8   2134.7
Pipe Throughput                               12440.0     684221.5    550.0
Pipe-based Context Switching                   4000.0      97428.3    243.6
Process Creation                                126.0       9107.5    722.8
Shell Scripts (1 concurrent)                     42.4       5976.2   1409.5
Shell Scripts (8 concurrent)                      6.0        746.2   1243.6
System Call Overhead                          15000.0     405652.2    270.4
                                                                   ========
System Benchmarks Index Score                                         855.3

Unixbench (БША, Нью-Π™ΠΎΡ€ΠΊ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-nyc3-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (4589.2 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   13:10:48 up 4 min,  1 user,  load average: 0.61, 0.24, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 04 2019 13:10:48 - 13:38:54
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       31830632.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4067.9 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3892.2 lps   (29.5 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        484700.4 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          143546.1 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1306057.4 KBps  (30.0 s, 2 samples)
Pipe Throughput                              782935.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 109443.7 lps   (10.0 s, 7 samples)
Process Creation                              10168.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6770.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    864.3 lpm   (60.0 s, 2 samples)
System Call Overhead                         470032.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   31830632.4   2727.6
Double-Precision Whetstone                       55.0       4067.9    739.6
Execl Throughput                                 43.0       3892.2    905.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     484700.4   1224.0
File Copy 256 bufsize 500 maxblocks            1655.0     143546.1    867.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1306057.4   2251.8
Pipe Throughput                               12440.0     782935.5    629.4
Pipe-based Context Switching                   4000.0     109443.7    273.6
Process Creation                                126.0      10168.3    807.0
Shell Scripts (1 concurrent)                     42.4       6770.7   1596.9
Shell Scripts (8 concurrent)                      6.0        864.3   1440.5
System Call Overhead                          15000.0     470032.5    313.4
                                                                   ========
System Benchmarks Index Score                                         933.4

Unixbench (БША, Π‘Π°Π½-Ѐранциско)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-s-1vcpu-1gb-sfo2-01: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (4400.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   07:32:35 up 4 min,  1 user,  load average: 0.85, 0.38, 0.15; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 05 2019 07:32:35 - 08:00:35
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       18731484.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3866.3 MWIPS (9.2 s, 7 samples)
Execl Throughput                               2473.3 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        289169.4 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           81647.3 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        839008.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              464945.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  70296.3 lps   (10.0 s, 7 samples)
Process Creation                               6041.0 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   3583.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    458.9 lpm   (60.2 s, 2 samples)
System Call Overhead                         272373.3 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   18731484.4   1605.1
Double-Precision Whetstone                       55.0       3866.3    703.0
Execl Throughput                                 43.0       2473.3    575.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     289169.4    730.2
File Copy 256 bufsize 500 maxblocks            1655.0      81647.3    493.3
File Copy 4096 bufsize 8000 maxblocks          5800.0     839008.9   1446.6
Pipe Throughput                               12440.0     464945.7    373.8
Pipe-based Context Switching                   4000.0      70296.3    175.7
Process Creation                                126.0       6041.0    479.4
Shell Scripts (1 concurrent)                     42.4       3583.0    845.0
Shell Scripts (8 concurrent)                      6.0        458.9    764.9
System Call Overhead                          15000.0     272373.3    181.6
                                                                   ========
System Benchmarks Index Score                                         572.9

dd (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.32407 s, 340 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 8.07427 s, 266 MB/s

dd (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.45694 s, 482 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.40443 s, 631 MB/s

dd (Индия, Π‘Π°Π½Π³Π°Π»ΠΎΡ€)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.84399 s, 755 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.98814 s, 538 MB/s

dd (НидСрланды, АмстСрдам)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.25153 s, 505 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.97376 s, 432 MB/s

dd (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.79863 s, 565 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.49007 s, 615 MB/s

dd (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.75593 s, 373 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.54156 s, 388 MB/s

dd (БША, Нью-Π™ΠΎΡ€ΠΊ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.72736 s, 576 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.41343 s, 629 MB/s

dd (БША, Π‘Π°Π½-Ѐранциско)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.30885 s, 649 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.53563 s, 607 MB/s

fio (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28463: Sun Dec 15 16:21:05 2019
  read : io=810904KB, bw=13512KB/s, iops=3378, runt= 60013msec
    clat (usec): min=170, max=562215, avg=2329.21, stdev=7399.56
     lat (usec): min=171, max=562216, avg=2334.20, stdev=7405.70
    clat percentiles (usec):
     |  1.00th=[  245],  5.00th=[  314], 10.00th=[  402], 20.00th=[  540],
     | 30.00th=[  644], 40.00th=[  764], 50.00th=[  924], 60.00th=[ 1160],
     | 70.00th=[ 1544], 80.00th=[ 2288], 90.00th=[ 4320], 95.00th=[ 7904],
     | 99.00th=[29568], 99.50th=[37632], 99.90th=[50944], 99.95th=[67072],
     | 99.99th=[382976]
    bw (KB  /s): min=   20, max= 3280, per=12.65%, avg=1709.08, stdev=752.64
    lat (usec) : 250=1.18%, 500=15.24%, 750=22.38%, 1000=14.99%
    lat (msec) : 2=23.20%, 4=12.09%, 10=7.21%, 20=2.10%, 50=1.50%
    lat (msec) : 100=0.07%, 250=0.02%, 500=0.01%, 750=0.01%
  cpu          : usr=1.30%, sys=5.68%, ctx=197641, majf=0, minf=85
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=202726/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=810904KB, aggrb=13512KB/s, minb=13512KB/s, maxb=13512KB/s, mint=60013msec, maxt=60013msec

Disk stats (read/write):
  vda: ios=203045/12, merge=0/12, ticks=250920/612, in_queue=251312, util=74.66%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28501: Sun Dec 15 16:23:17 2019
  write: io=6566.3MB, bw=111997KB/s, iops=27999, runt= 60036msec
    clat (usec): min=2, max=508874, avg=164.65, stdev=3238.47
     lat (usec): min=3, max=508875, avg=181.01, stdev=3480.05
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    4], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    5], 40.00th=[    5], 50.00th=[    5], 60.00th=[    5],
     | 70.00th=[    5], 80.00th=[    6], 90.00th=[    7], 95.00th=[    8],
     | 99.00th=[   48], 99.50th=[ 7392], 99.90th=[40192], 99.95th=[55040],
     | 99.99th=[117248]
    bw (KB  /s): min=    0, max=139593, per=14.20%, avg=15900.11, stdev=8999.85
    lat (usec) : 4=0.41%, 10=95.61%, 20=2.47%, 50=0.52%, 100=0.13%
    lat (usec) : 250=0.16%, 500=0.05%, 750=0.03%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.03%, 10=0.10%, 20=0.16%, 50=0.23%
    lat (msec) : 100=0.05%, 250=0.01%, 500=0.01%, 750=0.01%
  cpu          : usr=3.51%, sys=4.83%, ctx=29270, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1680965/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=6566.3MB, aggrb=111997KB/s, minb=111997KB/s, maxb=111997KB/s, mint=60036msec, maxt=60036msec

Disk stats (read/write):
  vda: ios=653/312108, merge=0/1864, ticks=5244/3962028, in_queue=3970048, util=85.40%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28488: Sun Dec 15 16:22:17 2019
  read : io=926632KB, bw=15443KB/s, iops=3860, runt= 60003msec
    clat (usec): min=158, max=342030, avg=2041.48, stdev=5221.60
     lat (usec): min=159, max=342031, avg=2044.90, stdev=5224.66
    clat percentiles (usec):
     |  1.00th=[  247],  5.00th=[  322], 10.00th=[  406], 20.00th=[  524],
     | 30.00th=[  612], 40.00th=[  732], 50.00th=[  868], 60.00th=[ 1080],
     | 70.00th=[ 1416], 80.00th=[ 2096], 90.00th=[ 3888], 95.00th=[ 6752],
     | 99.00th=[22656], 99.50th=[34048], 99.90th=[47872], 99.95th=[52992],
     | 99.99th=[199680]
    bw (KB  /s): min=   81, max= 3704, per=12.60%, avg=1946.40, stdev=778.52
    lat (usec) : 250=1.16%, 500=16.28%, 750=24.32%, 1000=15.06%
    lat (msec) : 2=22.14%, 4=11.33%, 10=6.79%, 20=1.75%, 50=1.11%
    lat (msec) : 100=0.05%, 250=0.01%, 500=0.01%
  cpu          : usr=1.31%, sys=5.34%, ctx=227680, majf=0, minf=84
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=231658/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=926632KB, aggrb=15443KB/s, minb=15443KB/s, maxb=15443KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=230719/8, merge=0/7, ticks=268276/96, in_queue=268104, util=76.22%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28524: Sun Dec 15 16:24:19 2019
  write: io=124888KB, bw=2080.2KB/s, iops=520, runt= 60037msec
    clat (usec): min=653, max=401729, avg=15355.31, stdev=39324.23
     lat (usec): min=655, max=401731, avg=15358.38, stdev=39325.85
    clat percentiles (usec):
     |  1.00th=[  764],  5.00th=[  884], 10.00th=[  988], 20.00th=[ 1128],
     | 30.00th=[ 1288], 40.00th=[ 1448], 50.00th=[ 1640], 60.00th=[ 1848],
     | 70.00th=[ 2160], 80.00th=[ 2896], 90.00th=[94720], 95.00th=[124416],
     | 99.00th=[152576], 99.50th=[166912], 99.90th=[207872], 99.95th=[268288],
     | 99.99th=[395264]
    bw (KB  /s): min=   17, max=  584, per=12.52%, avg=260.41, stdev=87.52
    lat (usec) : 750=0.73%, 1000=10.17%
    lat (msec) : 2=54.34%, 4=19.08%, 10=3.99%, 20=0.49%, 50=0.27%
    lat (msec) : 100=1.36%, 250=9.51%, 500=0.06%
  cpu          : usr=0.23%, sys=0.89%, ctx=66162, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=31222/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=124888KB, aggrb=2080KB/s, minb=2080KB/s, maxb=2080KB/s, mint=60037msec, maxt=60037msec

Disk stats (read/write):
  vda: ios=569/31224, merge=0/1484, ticks=648/54976, in_queue=55592, util=91.32%

fio (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22841: Sat Dec 14 00:06:23 2019
  read : io=5529.7MB, bw=94371KB/s, iops=23592, runt= 60001msec
    clat (usec): min=2, max=208020, avg=332.04, stdev=793.12
     lat (usec): min=3, max=208021, avg=333.14, stdev=793.14
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    5], 10.00th=[  187], 20.00th=[  213],
     | 30.00th=[  229], 40.00th=[  245], 50.00th=[  262], 60.00th=[  286],
     | 70.00th=[  314], 80.00th=[  366], 90.00th=[  490], 95.00th=[  644],
     | 99.00th=[ 2096], 99.50th=[ 2960], 99.90th=[ 4128], 99.95th=[ 4896],
     | 99.99th=[10176]
    bw (KB  /s): min= 5640, max=19344, per=12.53%, avg=11821.28, stdev=2384.67
    lat (usec) : 4=2.92%, 10=2.71%, 20=0.28%, 50=0.05%, 100=0.01%
    lat (usec) : 250=36.74%, 500=47.83%, 750=5.95%, 1000=1.29%
    lat (msec) : 2=1.16%, 4=0.94%, 10=0.11%, 20=0.01%, 50=0.01%
    lat (msec) : 250=0.01%
  cpu          : usr=2.40%, sys=5.64%, ctx=1335613, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1415595/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=5529.7MB, aggrb=94371KB/s, minb=94371KB/s, maxb=94371KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1330026/16, merge=31/12, ticks=381688/32, in_queue=381284, util=98.86%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22875: Sat Dec 14 00:08:28 2019
  write: io=23387MB, bw=399054KB/s, iops=99763, runt= 60013msec
    clat (usec): min=2, max=347823, avg=53.51, stdev=1396.60
     lat (usec): min=2, max=348279, avg=58.09, stdev=1444.65
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    5], 95.00th=[    5],
     | 99.00th=[   13], 99.50th=[   16], 99.90th=[16320], 99.95th=[22656],
     | 99.99th=[36096]
    bw (KB  /s): min=    0, max=69737, per=13.03%, avg=52000.63, stdev=13148.08
    lat (usec) : 4=46.65%, 10=51.87%, 20=1.04%, 50=0.08%, 100=0.02%
    lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.09%, 20=0.13%, 50=0.06%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=4.38%, sys=5.58%, ctx=48485, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5987113/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=23387MB, aggrb=399054KB/s, minb=399054KB/s, maxb=399054KB/s, mint=60013msec, maxt=60013msec

Disk stats (read/write):
  vda: ios=269/703950, merge=0/5166, ticks=376/6209928, in_queue=6210116, util=97.19%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22862: Sat Dec 14 00:07:28 2019
  read : io=5254.4MB, bw=89673KB/s, iops=22418, runt= 60001msec
    clat (usec): min=128, max=189254, avg=349.55, stdev=876.11
     lat (usec): min=129, max=189255, avg=350.71, stdev=876.13
    clat percentiles (usec):
     |  1.00th=[  169],  5.00th=[  187], 10.00th=[  199], 20.00th=[  219],
     | 30.00th=[  233], 40.00th=[  249], 50.00th=[  266], 60.00th=[  290],
     | 70.00th=[  322], 80.00th=[  374], 90.00th=[  494], 95.00th=[  644],
     | 99.00th=[ 2064], 99.50th=[ 2992], 99.90th=[ 4192], 99.95th=[ 5472],
     | 99.99th=[12992]
    bw (KB  /s): min= 4986, max=14872, per=12.50%, avg=11210.81, stdev=1832.67
    lat (usec) : 250=40.19%, 500=50.04%, 750=6.37%, 1000=1.22%
    lat (msec) : 2=1.15%, 4=0.90%, 10=0.11%, 20=0.02%, 50=0.01%
    lat (msec) : 250=0.01%
  cpu          : usr=2.30%, sys=5.08%, ctx=1349121, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1345114/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=5254.4MB, aggrb=89672KB/s, minb=89672KB/s, maxb=89672KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1340578/8, merge=0/7, ticks=396628/20, in_queue=396256, util=99.82%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22896: Sat Dec 14 00:09:29 2019
  write: io=181408KB, bw=3022.6KB/s, iops=755, runt= 60018msec
    clat (usec): min=606, max=535365, avg=10574.01, stdev=38305.93
     lat (usec): min=607, max=535367, avg=10575.65, stdev=38305.97
    clat percentiles (usec):
     |  1.00th=[  660],  5.00th=[  700], 10.00th=[  732], 20.00th=[  812],
     | 30.00th=[  900], 40.00th=[  980], 50.00th=[ 1048], 60.00th=[ 1144],
     | 70.00th=[ 1304], 80.00th=[ 1624], 90.00th=[ 3152], 95.00th=[123392],
     | 99.00th=[183296], 99.50th=[218112], 99.90th=[305152], 99.95th=[358400],
     | 99.99th=[460800]
    bw (KB  /s): min=   14, max=  736, per=12.61%, avg=381.11, stdev=155.64
    lat (usec) : 750=12.63%, 1000=30.54%
    lat (msec) : 2=41.81%, 4=6.44%, 10=2.36%, 20=0.17%, 50=0.04%
    lat (msec) : 100=0.01%, 250=5.74%, 500=0.26%, 750=0.01%
  cpu          : usr=0.15%, sys=0.62%, ctx=93571, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=45352/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=181408KB, aggrb=3022KB/s, minb=3022KB/s, maxb=3022KB/s, mint=60018msec, maxt=60018msec

Disk stats (read/write):
  vda: ios=254/45392, merge=0/1838, ticks=272/56992, in_queue=57236, util=94.58%

fio (Индия, Π‘Π°Π½Π³Π°Π»ΠΎΡ€)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16212: Sun Dec 15 19:21:57 2019
  read : io=6139.3MB, bw=104771KB/s, iops=26192, runt= 60003msec
    clat (usec): min=2, max=168209, avg=297.31, stdev=431.72
     lat (usec): min=3, max=168210, avg=298.56, stdev=431.78
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[  189], 20.00th=[  213],
     | 30.00th=[  229], 40.00th=[  245], 50.00th=[  262], 60.00th=[  282],
     | 70.00th=[  310], 80.00th=[  354], 90.00th=[  442], 95.00th=[  540],
     | 99.00th=[  980], 99.50th=[ 1592], 99.90th=[ 3504], 99.95th=[ 4128],
     | 99.99th=[ 6752]
    bw (KB  /s): min= 9088, max=17808, per=12.49%, avg=13085.74, stdev=1494.25
    lat (usec) : 4=0.94%, 10=5.00%, 20=0.42%, 50=0.14%, 100=0.01%
    lat (usec) : 250=37.12%, 500=49.91%, 750=4.81%, 1000=0.69%
    lat (msec) : 2=0.60%, 4=0.30%, 10=0.05%, 20=0.01%, 50=0.01%
    lat (msec) : 250=0.01%
  cpu          : usr=3.02%, sys=7.08%, ctx=1475390, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1571645/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6139.3MB, aggrb=104771KB/s, minb=104771KB/s, maxb=104771KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=1464677/18, merge=0/9, ticks=324452/40, in_queue=324132, util=97.85%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16261: Sun Dec 15 19:24:02 2019
  write: io=18895MB, bw=322434KB/s, iops=80608, runt= 60009msec
    clat (usec): min=2, max=242956, avg=66.55, stdev=1165.23
     lat (usec): min=2, max=242958, avg=72.17, stdev=1229.79
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    5], 60.00th=[    5],
     | 70.00th=[    5], 80.00th=[    5], 90.00th=[    6], 95.00th=[    6],
     | 99.00th=[   16], 99.50th=[  159], 99.90th=[18304], 99.95th=[23424],
     | 99.99th=[36096]
    bw (KB  /s): min=    0, max=63440, per=12.78%, avg=41198.18, stdev=6458.96
    lat (usec) : 4=8.56%, 10=89.24%, 20=1.36%, 50=0.29%, 100=0.03%
    lat (usec) : 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.04%, 10=0.15%, 20=0.19%, 50=0.07%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.18%, sys=5.82%, ctx=54734, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=4837234/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=18895MB, aggrb=322433KB/s, minb=322433KB/s, maxb=322433KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  vda: ios=689/1218295, merge=0/4077, ticks=448/5951228, in_queue=5952404, util=90.57%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16234: Sun Dec 15 19:23:02 2019
  read : io=6596.6MB, bw=112569KB/s, iops=28142, runt= 60002msec
    clat (usec): min=121, max=184890, avg=276.48, stdev=539.90
     lat (usec): min=123, max=184891, avg=277.66, stdev=539.94
    clat percentiles (usec):
     |  1.00th=[  169],  5.00th=[  185], 10.00th=[  197], 20.00th=[  213],
     | 30.00th=[  225], 40.00th=[  237], 50.00th=[  251], 60.00th=[  266],
     | 70.00th=[  278], 80.00th=[  306], 90.00th=[  366], 95.00th=[  442],
     | 99.00th=[  660], 99.50th=[  868], 99.90th=[ 1832], 99.95th=[ 2224],
     | 99.99th=[ 3632]
    bw (KB  /s): min= 9216, max=15784, per=12.51%, avg=14079.06, stdev=854.30
    lat (usec) : 250=49.89%, 500=47.05%, 750=2.39%, 1000=0.27%
    lat (msec) : 2=0.32%, 4=0.07%, 10=0.01%, 20=0.01%, 250=0.01%
  cpu          : usr=3.13%, sys=6.76%, ctx=1693666, majf=0, minf=68
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1688590/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6596.6MB, aggrb=112568KB/s, minb=112568KB/s, maxb=112568KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1684306/17, merge=0/12, ticks=354396/0, in_queue=354000, util=99.71%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16285: Sun Dec 15 19:25:03 2019
  write: io=254516KB, bw=4240.1KB/s, iops=1060, runt= 60014msec
    clat (usec): min=600, max=238513, avg=7533.21, stdev=25823.98
     lat (usec): min=601, max=238514, avg=7534.78, stdev=25824.02
    clat percentiles (usec):
     |  1.00th=[  644],  5.00th=[  668], 10.00th=[  684], 20.00th=[  708],
     | 30.00th=[  732], 40.00th=[  764], 50.00th=[  844], 60.00th=[  940],
     | 70.00th=[ 1004], 80.00th=[ 1096], 90.00th=[ 1608], 95.00th=[98816],
     | 99.00th=[114176], 99.50th=[120320], 99.90th=[150528], 99.95th=[179200],
     | 99.99th=[211968]
    bw (KB  /s): min=  161, max=  944, per=12.61%, avg=534.54, stdev=151.72
    lat (usec) : 750=36.66%, 1000=32.68%
    lat (msec) : 2=22.13%, 4=2.00%, 10=0.26%, 20=0.03%, 50=0.01%
    lat (msec) : 100=1.48%, 250=4.75%
  cpu          : usr=0.22%, sys=0.89%, ctx=131469, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=63629/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=254516KB, aggrb=4240KB/s, minb=4240KB/s, maxb=4240KB/s, mint=60014msec, maxt=60014msec

Disk stats (read/write):
  vda: ios=688/63671, merge=0/1910, ticks=548/55720, in_queue=56248, util=92.49%

fio (НидСрланды, АмстСрдам)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5714: Wed Dec 11 14:41:53 2019
  read : io=1601.9MB, bw=27335KB/s, iops=6833, runt= 60009msec
    clat (usec): min=147, max=100526, avg=1153.46, stdev=1994.16
     lat (usec): min=148, max=100527, avg=1155.92, stdev=1997.60
    clat percentiles (usec):
     |  1.00th=[  209],  5.00th=[  245], 10.00th=[  270], 20.00th=[  338],
     | 30.00th=[  446], 40.00th=[  540], 50.00th=[  628], 60.00th=[  748],
     | 70.00th=[  940], 80.00th=[ 1272], 90.00th=[ 2192], 95.00th=[ 3728],
     | 99.00th=[10176], 99.50th=[13888], 99.90th=[23680], 99.95th=[27776],
     | 99.99th=[35584]
    bw (KB  /s): min= 1928, max= 7392, per=12.49%, avg=3415.22, stdev=835.60
    lat (usec) : 250=5.96%, 500=28.91%, 750=25.29%, 1000=12.39%
    lat (msec) : 2=16.24%, 4=6.70%, 10=3.48%, 20=0.85%, 50=0.18%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.45%, sys=4.77%, ctx=415144, majf=0, minf=90
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=410080/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1601.9MB, aggrb=27334KB/s, minb=27334KB/s, maxb=27334KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  vda: ios=409241/8, merge=0/6, ticks=311748/96, in_queue=311500, util=86.72%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5746: Wed Dec 11 14:43:59 2019
  write: io=10688MB, bw=182382KB/s, iops=45595, runt= 60009msec
    clat (usec): min=2, max=384094, avg=108.20, stdev=2173.94
     lat (usec): min=3, max=384096, avg=118.33, stdev=2308.57
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    4], 40.00th=[    5], 50.00th=[    5], 60.00th=[    5],
     | 70.00th=[    5], 80.00th=[    5], 90.00th=[    6], 95.00th=[    8],
     | 99.00th=[   34], 99.50th=[ 2608], 99.90th=[28032], 99.95th=[37632],
     | 99.99th=[76288]
    bw (KB  /s): min=    0, max=45740, per=13.54%, avg=24689.24, stdev=8568.82
    lat (usec) : 4=1.12%, 10=95.58%, 20=1.91%, 50=0.51%, 100=0.12%
    lat (usec) : 250=0.13%, 500=0.06%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.04%, 10=0.13%, 20=0.17%, 50=0.15%
    lat (msec) : 100=0.02%, 250=0.01%, 500=0.01%
  cpu          : usr=3.78%, sys=5.14%, ctx=39912, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2736147/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=10688MB, aggrb=182382KB/s, minb=182382KB/s, maxb=182382KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  vda: ios=170/452541, merge=0/4470, ticks=592/4649176, in_queue=4650320, util=90.11%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5733: Wed Dec 11 14:42:59 2019
  read : io=1587.8MB, bw=27097KB/s, iops=6774, runt= 60003msec
    clat (usec): min=146, max=129531, avg=1163.40, stdev=1924.84
     lat (usec): min=147, max=129532, avg=1165.57, stdev=1926.44
    clat percentiles (usec):
     |  1.00th=[  215],  5.00th=[  258], 10.00th=[  298], 20.00th=[  402],
     | 30.00th=[  502], 40.00th=[  572], 50.00th=[  652], 60.00th=[  772],
     | 70.00th=[  956], 80.00th=[ 1288], 90.00th=[ 2192], 95.00th=[ 3600],
     | 99.00th=[ 9664], 99.50th=[12992], 99.90th=[22400], 99.95th=[26496],
     | 99.99th=[38656]
    bw (KB  /s): min= 1720, max= 5008, per=12.52%, avg=3391.80, stdev=509.87
    lat (usec) : 250=4.06%, 500=25.57%, 750=28.59%, 1000=13.68%
    lat (msec) : 2=16.84%, 4=6.99%, 10=3.33%, 20=0.78%, 50=0.15%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.64%, sys=4.62%, ctx=408046, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=406469/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1587.8MB, aggrb=27096KB/s, minb=27096KB/s, maxb=27096KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=405647/8, merge=0/7, ticks=300840/196, in_queue=300772, util=84.98%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5759: Wed Dec 11 14:45:00 2019
  write: io=188084KB, bw=3133.5KB/s, iops=783, runt= 60024msec
    clat (usec): min=639, max=249688, avg=10195.17, stdev=29490.37
     lat (usec): min=640, max=249690, avg=10197.12, stdev=29490.47
    clat percentiles (usec):
     |  1.00th=[  692],  5.00th=[  724], 10.00th=[  748], 20.00th=[  804],
     | 30.00th=[  892], 40.00th=[ 1004], 50.00th=[ 1096], 60.00th=[ 1192],
     | 70.00th=[ 1368], 80.00th=[ 1688], 90.00th=[ 4320], 95.00th=[102912],
     | 99.00th=[119296], 99.50th=[126464], 99.90th=[146432], 99.95th=[150528],
     | 99.99th=[246784]
    bw (KB  /s): min=  135, max=  728, per=12.57%, avg=393.66, stdev=111.76
    lat (usec) : 750=10.98%, 1000=28.18%
    lat (msec) : 2=44.97%, 4=5.64%, 10=1.47%, 20=0.11%, 50=0.13%
    lat (msec) : 100=2.51%, 250=6.02%
  cpu          : usr=0.28%, sys=0.93%, ctx=98320, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=47021/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=188084KB, aggrb=3133KB/s, minb=3133KB/s, maxb=3133KB/s, mint=60024msec, maxt=60024msec

Disk stats (read/write):
  vda: ios=660/47082, merge=0/1723, ticks=944/54796, in_queue=55720, util=90.97%

fio (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=26021: Wed Dec 11 18:38:21 2019
  read : io=4308.6MB, bw=73532KB/s, iops=18382, runt= 60001msec
    clat (usec): min=2, max=75156, avg=428.64, stdev=674.69
     lat (usec): min=2, max=75156, avg=429.66, stdev=674.74
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[  153], 10.00th=[  187], 20.00th=[  215],
     | 30.00th=[  237], 40.00th=[  262], 50.00th=[  298], 60.00th=[  350],
     | 70.00th=[  414], 80.00th=[  502], 90.00th=[  652], 95.00th=[  900],
     | 99.00th=[ 3056], 99.50th=[ 4320], 99.90th=[ 7648], 99.95th=[ 9792],
     | 99.99th=[14784]
    bw (KB  /s): min= 3133, max=13504, per=12.49%, avg=9181.87, stdev=1826.22
    lat (usec) : 4=1.88%, 10=2.65%, 20=0.22%, 50=0.03%, 100=0.01%
    lat (usec) : 250=30.28%, 500=44.58%, 750=13.23%, 1000=2.86%
    lat (msec) : 2=2.33%, 4=1.35%, 10=0.54%, 20=0.04%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=1.72%, sys=4.35%, ctx=1055602, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1102992/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4308.6MB, aggrb=73531KB/s, minb=73531KB/s, maxb=73531KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1047795/8, merge=0/6, ticks=390052/52, in_queue=389732, util=97.89%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=26051: Wed Dec 11 18:40:26 2019
  write: io=22990MB, bw=392328KB/s, iops=98081, runt= 60006msec
    clat (usec): min=1, max=369457, avg=55.42, stdev=1456.41
     lat (usec): min=2, max=369458, avg=59.90, stdev=1504.16
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    4], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    5], 95.00th=[    5],
     | 99.00th=[    9], 99.50th=[   22], 99.90th=[16768], 99.95th=[22656],
     | 99.99th=[37120]
    bw (KB  /s): min=    0, max=70700, per=13.09%, avg=51347.13, stdev=13217.67
    lat (usec) : 2=0.01%, 4=39.98%, 10=59.15%, 20=0.32%, 50=0.18%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.09%, 20=0.13%, 50=0.07%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=3.92%, sys=5.85%, ctx=47216, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5885501/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=22990MB, aggrb=392327KB/s, minb=392327KB/s, maxb=392327KB/s, mint=60006msec, maxt=60006msec

Disk stats (read/write):
  vda: ios=253/695603, merge=0/4848, ticks=460/6040768, in_queue=6041620, util=94.92%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=26034: Wed Dec 11 18:39:26 2019
  read : io=4060.6MB, bw=69297KB/s, iops=17324, runt= 60002msec
    clat (usec): min=126, max=73471, avg=454.91, stdev=816.66
     lat (usec): min=127, max=73472, avg=455.93, stdev=816.68
    clat percentiles (usec):
     |  1.00th=[  161],  5.00th=[  179], 10.00th=[  191], 20.00th=[  211],
     | 30.00th=[  229], 40.00th=[  249], 50.00th=[  274], 60.00th=[  314],
     | 70.00th=[  374], 80.00th=[  466], 90.00th=[  636], 95.00th=[  996],
     | 99.00th=[ 4320], 99.50th=[ 5664], 99.90th=[ 8384], 99.95th=[ 9920],
     | 99.99th=[16320]
    bw (KB  /s): min= 3722, max=14064, per=12.47%, avg=8642.03, stdev=2176.97
    lat (usec) : 250=40.65%, 500=42.13%, 750=10.05%, 1000=2.20%
    lat (msec) : 2=2.13%, 4=1.68%, 10=1.11%, 20=0.04%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=1.66%, sys=3.86%, ctx=1045614, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1039489/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4060.6MB, aggrb=69296KB/s, minb=69296KB/s, maxb=69296KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1035654/8, merge=0/7, ticks=408452/1488, in_queue=409680, util=98.90%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=26068: Wed Dec 11 18:41:27 2019
  write: io=131456KB, bw=2190.6KB/s, iops=547, runt= 60012msec
    clat (usec): min=613, max=1435.9K, avg=14593.78, stdev=50405.98
     lat (usec): min=614, max=1435.9K, avg=14595.36, stdev=50406.03
    clat percentiles (usec):
     |  1.00th=[  684],  5.00th=[  732], 10.00th=[  812], 20.00th=[  964],
     | 30.00th=[ 1064], 40.00th=[ 1192], 50.00th=[ 1336], 60.00th=[ 1560],
     | 70.00th=[ 1880], 80.00th=[ 2448], 90.00th=[ 6112], 95.00th=[132096],
     | 99.00th=[214016], 99.50th=[268288], 99.90th=[411648], 99.95th=[468992],
     | 99.99th=[1400832]
    bw (KB  /s): min=    2, max=  672, per=12.89%, avg=282.21, stdev=143.92
    lat (usec) : 750=6.13%, 1000=16.97%
    lat (msec) : 2=49.80%, 4=14.23%, 10=4.16%, 20=0.69%, 50=0.21%
    lat (msec) : 100=0.05%, 250=7.07%, 500=0.67%, 750=0.01%, 2000=0.02%
  cpu          : usr=0.13%, sys=0.58%, ctx=68429, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=32864/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=131456KB, aggrb=2190KB/s, minb=2190KB/s, maxb=2190KB/s, mint=60012msec, maxt=60012msec

Disk stats (read/write):
  vda: ios=261/32874, merge=0/1477, ticks=160/56864, in_queue=57012, util=94.50%

fio (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31964: Thu Dec 12 12:59:43 2019
  read : io=4469.1MB, bw=76286KB/s, iops=19071, runt= 60001msec
    clat (usec): min=1, max=57633, avg=412.24, stdev=553.46
     lat (usec): min=2, max=57635, avg=413.42, stdev=553.53
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    8], 10.00th=[  187], 20.00th=[  221],
     | 30.00th=[  249], 40.00th=[  278], 50.00th=[  318], 60.00th=[  362],
     | 70.00th=[  422], 80.00th=[  506], 90.00th=[  652], 95.00th=[  852],
     | 99.00th=[ 2480], 99.50th=[ 3152], 99.90th=[ 5472], 99.95th=[ 7712],
     | 99.99th=[17280]
    bw (KB  /s): min= 4344, max=15384, per=12.49%, avg=9524.23, stdev=1621.11
    lat (usec) : 2=0.01%, 4=2.21%, 10=2.89%, 20=0.24%, 50=0.08%
    lat (usec) : 100=0.01%, 250=25.20%, 500=48.91%, 750=13.74%, 1000=3.08%
    lat (msec) : 2=2.17%, 4=1.24%, 10=0.19%, 20=0.03%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=1.86%, sys=4.84%, ctx=1086469, majf=1, minf=88
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1144306/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4469.1MB, aggrb=76285KB/s, minb=76285KB/s, maxb=76285KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1078551/8, merge=0/6, ticks=391976/24, in_queue=391736, util=98.85%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=32007: Thu Dec 12 13:01:50 2019
  write: io=17383MB, bw=296475KB/s, iops=74118, runt= 60038msec
    clat (usec): min=1, max=634520, avg=80.43, stdev=2797.83
     lat (usec): min=2, max=634522, avg=84.20, stdev=2824.23
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    5], 95.00th=[    5],
     | 99.00th=[    9], 99.50th=[   18], 99.90th=[20096], 99.95th=[35072],
     | 99.99th=[87552]
    bw (KB  /s): min=    0, max=87481, per=13.99%, avg=41472.38, stdev=18398.46
    lat (usec) : 2=0.01%, 4=55.86%, 10=43.23%, 20=0.45%, 50=0.13%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.07%, 20=0.10%, 50=0.08%
    lat (msec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%
  cpu          : usr=2.82%, sys=4.31%, ctx=36944, majf=0, minf=76
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=4449946/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=17383MB, aggrb=296475KB/s, minb=296475KB/s, maxb=296475KB/s, mint=60038msec, maxt=60038msec

Disk stats (read/write):
  vda: ios=262/323640, merge=0/6202, ticks=144/5185996, in_queue=5186620, util=96.41%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31989: Thu Dec 12 13:00:50 2019
  read : io=4223.6MB, bw=72081KB/s, iops=18020, runt= 60001msec
    clat (usec): min=109, max=106784, avg=436.41, stdev=534.14
     lat (usec): min=110, max=106785, avg=437.62, stdev=534.20
    clat percentiles (usec):
     |  1.00th=[  165],  5.00th=[  189], 10.00th=[  207], 20.00th=[  235],
     | 30.00th=[  262], 40.00th=[  294], 50.00th=[  334], 60.00th=[  382],
     | 70.00th=[  442], 80.00th=[  524], 90.00th=[  676], 95.00th=[  884],
     | 99.00th=[ 2416], 99.50th=[ 2992], 99.90th=[ 4512], 99.95th=[ 5856],
     | 99.99th=[14528]
    bw (KB  /s): min= 5672, max=11072, per=12.50%, avg=9011.87, stdev=833.30
    lat (usec) : 250=25.62%, 500=51.60%, 750=15.32%, 1000=3.50%
    lat (msec) : 2=2.49%, 4=1.31%, 10=0.13%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.82%, sys=4.50%, ctx=1084188, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1081230/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4223.6MB, aggrb=72080KB/s, minb=72080KB/s, maxb=72080KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1078880/8, merge=0/7, ticks=402188/24, in_queue=401836, util=99.65%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=32033: Thu Dec 12 13:02:51 2019
  write: io=91328KB, bw=1521.7KB/s, iops=380, runt= 60019msec
    clat (usec): min=629, max=1370.1K, avg=21010.33, stdev=67213.73
     lat (usec): min=631, max=1370.1K, avg=21012.43, stdev=67213.79
    clat percentiles (usec):
     |  1.00th=[  868],  5.00th=[ 1032], 10.00th=[ 1144], 20.00th=[ 1304],
     | 30.00th=[ 1464], 40.00th=[ 1656], 50.00th=[ 1880], 60.00th=[ 2192],
     | 70.00th=[ 2672], 80.00th=[ 3888], 90.00th=[11072], 95.00th=[187392],
     | 99.00th=[272384], 99.50th=[366592], 99.90th=[569344], 99.95th=[643072],
     | 99.99th=[1318912]
    bw (KB  /s): min=   22, max=  360, per=12.77%, avg=194.30, stdev=56.49
    lat (usec) : 750=0.12%, 1000=3.99%
    lat (msec) : 2=50.22%, 4=26.11%, 10=9.22%, 20=1.34%, 50=0.39%
    lat (msec) : 100=0.14%, 250=6.92%, 500=1.36%, 750=0.18%, 1000=0.01%
    lat (msec) : 2000=0.03%
  cpu          : usr=0.10%, sys=0.45%, ctx=47653, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=22832/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=91328KB, aggrb=1521KB/s, minb=1521KB/s, maxb=1521KB/s, mint=60019msec, maxt=60019msec

Disk stats (read/write):
  vda: ios=699/22885, merge=0/1284, ticks=488/58240, in_queue=58728, util=96.24%

fio (БША, Нью-Π™ΠΎΡ€ΠΊ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8043: Wed Dec  4 13:40:06 2019
  read : io=8650.3MB, bw=147629KB/s, iops=36907, runt= 60001msec
    clat (usec): min=1, max=12457, avg=212.16, stdev=168.73
     lat (usec): min=2, max=12458, avg=212.93, stdev=168.79
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  131], 20.00th=[  171],
     | 30.00th=[  181], 40.00th=[  191], 50.00th=[  201], 60.00th=[  211],
     | 70.00th=[  227], 80.00th=[  253], 90.00th=[  302], 95.00th=[  358],
     | 99.00th=[  564], 99.50th=[  836], 99.90th=[ 2544], 99.95th=[ 2928],
     | 99.99th=[ 4640]
    bw (KB  /s): min=12872, max=25280, per=12.50%, avg=18457.17, stdev=2181.08
    lat (usec) : 2=0.01%, 4=7.82%, 10=1.21%, 20=0.37%, 50=0.02%
    lat (usec) : 100=0.04%, 250=69.71%, 500=19.41%, 750=0.83%, 1000=0.15%
    lat (msec) : 2=0.20%, 4=0.20%, 10=0.01%, 20=0.01%
  cpu          : usr=2.25%, sys=5.84%, ctx=2010544, majf=2, minf=96
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2214468/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8650.3MB, aggrb=147628KB/s, minb=147628KB/s, maxb=147628KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2002054/8, merge=0/6, ticks=374052/0, in_queue=373720, util=97.53%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8088: Wed Dec  4 13:42:12 2019
  write: io=29272MB, bw=499455KB/s, iops=124863, runt= 60014msec
    clat (usec): min=1, max=365561, avg=44.38, stdev=1109.33
     lat (usec): min=2, max=365726, avg=47.80, stdev=1149.56
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[   10], 99.50th=[   44], 99.90th=[14400], 99.95th=[19072],
     | 99.99th=[32128]
    bw (KB  /s): min=    0, max=80828, per=12.91%, avg=64495.24, stdev=14149.46
    lat (usec) : 2=0.01%, 4=80.00%, 10=18.95%, 20=0.40%, 50=0.16%
    lat (usec) : 100=0.12%, 250=0.03%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.03%, 10=0.11%, 20=0.13%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=3.87%, sys=6.39%, ctx=59719, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=7493570/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=29272MB, aggrb=499454KB/s, minb=499454KB/s, maxb=499454KB/s, mint=60014msec, maxt=60014msec

Disk stats (read/write):
  vda: ios=654/652218, merge=0/6105, ticks=1156/6405124, in_queue=6406188, util=97.12%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8064: Wed Dec  4 13:41:11 2019
  read : io=8391.1MB, bw=143220KB/s, iops=35804, runt= 60001msec
    clat (usec): min=70, max=14982, avg=218.74, stdev=90.67
     lat (usec): min=71, max=14983, avg=219.51, stdev=90.78
    clat percentiles (usec):
     |  1.00th=[  141],  5.00th=[  157], 10.00th=[  165], 20.00th=[  175],
     | 30.00th=[  183], 40.00th=[  191], 50.00th=[  201], 60.00th=[  211],
     | 70.00th=[  225], 80.00th=[  247], 90.00th=[  290], 95.00th=[  338],
     | 99.00th=[  474], 99.50th=[  564], 99.90th=[ 1064], 99.95th=[ 1352],
     | 99.99th=[ 3056]
    bw (KB  /s): min=15832, max=19784, per=12.50%, avg=17904.29, stdev=655.80
    lat (usec) : 100=0.06%, 250=80.95%, 500=18.20%, 750=0.59%, 1000=0.09%
    lat (msec) : 2=0.09%, 4=0.02%, 10=0.01%, 20=0.01%
  cpu          : usr=2.32%, sys=5.26%, ctx=2152129, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2148329/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8391.1MB, aggrb=143219KB/s, minb=143219KB/s, maxb=143219KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2145398/17, merge=0/11, ticks=392640/0, in_queue=392288, util=99.31%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8107: Wed Dec  4 13:43:12 2019
  write: io=340360KB, bw=5672.5KB/s, iops=1418, runt= 60007msec
    clat (usec): min=269, max=229762, avg=5632.93, stdev=22773.91
     lat (usec): min=270, max=229763, avg=5634.26, stdev=22773.96
    clat percentiles (usec):
     |  1.00th=[  314],  5.00th=[  354], 10.00th=[  402], 20.00th=[  466],
     | 30.00th=[  502], 40.00th=[  540], 50.00th=[  580], 60.00th=[  660],
     | 70.00th=[  820], 80.00th=[  988], 90.00th=[ 1240], 95.00th=[ 2448],
     | 99.00th=[115200], 99.50th=[121344], 99.90th=[140288], 99.95th=[162816],
     | 99.99th=[220160]
    bw (KB  /s): min=  155, max= 1314, per=12.53%, avg=710.70, stdev=210.32
    lat (usec) : 500=29.48%, 750=35.93%, 1000=14.99%
    lat (msec) : 2=14.39%, 4=0.50%, 10=0.11%, 20=0.02%, 50=0.01%
    lat (msec) : 100=1.05%, 250=3.52%
  cpu          : usr=0.23%, sys=0.85%, ctx=174273, majf=0, minf=65
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=85090/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=340360KB, aggrb=5672KB/s, minb=5672KB/s, maxb=5672KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  vda: ios=598/85062, merge=0/2038, ticks=200/55736, in_queue=55928, util=92.53%

fio (БША, Π‘Π°Π½-Ѐранциско)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29456: Thu Dec  5 08:01:48 2019
  read : io=1303.9MB, bw=22251KB/s, iops=5562, runt= 60003msec
    clat (usec): min=142, max=80183, avg=1420.21, stdev=2913.39
     lat (usec): min=142, max=80184, avg=1422.58, stdev=2915.85
    clat percentiles (usec):
     |  1.00th=[  213],  5.00th=[  243], 10.00th=[  270], 20.00th=[  338],
     | 30.00th=[  458], 40.00th=[  572], 50.00th=[  676], 60.00th=[  836],
     | 70.00th=[ 1048], 80.00th=[ 1496], 90.00th=[ 2736], 95.00th=[ 4832],
     | 99.00th=[13888], 99.50th=[19584], 99.90th=[36608], 99.95th=[43776],
     | 99.99th=[62720]
    bw (KB  /s): min= 1008, max= 7440, per=12.52%, avg=2785.28, stdev=1196.35
    lat (usec) : 250=6.47%, 500=26.73%, 750=22.17%, 1000=12.54%
    lat (msec) : 2=17.74%, 4=8.01%, 10=4.56%, 20=1.31%, 50=0.46%
    lat (msec) : 100=0.03%
  cpu          : usr=1.34%, sys=3.88%, ctx=338517, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=333779/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1303.9MB, aggrb=22250KB/s, minb=22250KB/s, maxb=22250KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=333758/8, merge=0/6, ticks=326896/48, in_queue=326612, util=86.22%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29490: Thu Dec  5 08:03:54 2019
  write: io=12984MB, bw=221303KB/s, iops=55325, runt= 60079msec
    clat (usec): min=2, max=363723, avg=94.71, stdev=2166.04
     lat (usec): min=2, max=363724, avg=102.66, stdev=2264.24
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    5], 60.00th=[    5],
     | 70.00th=[    5], 80.00th=[    5], 90.00th=[    6], 95.00th=[    7],
     | 99.00th=[   19], 99.50th=[  274], 99.90th=[24192], 99.95th=[32640],
     | 99.99th=[82432]
    bw (KB  /s): min=    0, max=52047, per=13.24%, avg=29293.56, stdev=11137.47
    lat (usec) : 4=2.76%, 10=94.87%, 20=1.39%, 50=0.32%, 100=0.06%
    lat (usec) : 250=0.09%, 500=0.03%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.03%, 10=0.11%, 20=0.16%, 50=0.12%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=3.89%, sys=5.30%, ctx=36713, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=3323911/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=12984MB, aggrb=221302KB/s, minb=221302KB/s, maxb=221302KB/s, mint=60079msec, maxt=60079msec

Disk stats (read/write):
  vda: ios=696/369874, merge=0/4081, ticks=1004/5449776, in_queue=5451104, util=93.26%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29473: Thu Dec  5 08:02:53 2019
  read : io=1645.1MB, bw=28089KB/s, iops=7022, runt= 60002msec
    clat (usec): min=116, max=80577, avg=1125.46, stdev=2401.59
     lat (usec): min=117, max=80578, avg=1127.23, stdev=2402.88
    clat percentiles (usec):
     |  1.00th=[  201],  5.00th=[  229], 10.00th=[  249], 20.00th=[  290],
     | 30.00th=[  362], 40.00th=[  466], 50.00th=[  564], 60.00th=[  668],
     | 70.00th=[  852], 80.00th=[ 1144], 90.00th=[ 1992], 95.00th=[ 3600],
     | 99.00th=[11456], 99.50th=[16768], 99.90th=[30080], 99.95th=[38144],
     | 99.99th=[50944]
    bw (KB  /s): min= 1464, max= 9208, per=12.50%, avg=3510.17, stdev=1352.72
    lat (usec) : 250=10.20%, 500=33.01%, 750=21.97%, 1000=10.86%
    lat (msec) : 2=14.03%, 4=5.53%, 10=3.17%, 20=0.91%, 50=0.32%
    lat (msec) : 100=0.01%
  cpu          : usr=1.37%, sys=3.08%, ctx=428535, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=421347/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1645.1MB, aggrb=28088KB/s, minb=28088KB/s, maxb=28088KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=420438/8, merge=0/7, ticks=371708/188, in_queue=371760, util=92.14%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29514: Thu Dec  5 08:04:55 2019
  write: io=112088KB, bw=1867.3KB/s, iops=466, runt= 60030msec
    clat (usec): min=644, max=311332, avg=17114.08, stdev=44927.08
     lat (usec): min=645, max=311334, avg=17116.41, stdev=44927.87
    clat percentiles (usec):
     |  1.00th=[  716],  5.00th=[  772], 10.00th=[  828], 20.00th=[  980],
     | 30.00th=[ 1112], 40.00th=[ 1272], 50.00th=[ 1512], 60.00th=[ 1816],
     | 70.00th=[ 2288], 80.00th=[ 3632], 90.00th=[98816], 95.00th=[140288],
     | 99.00th=[189440], 99.50th=[207872], 99.90th=[254976], 99.95th=[280576],
     | 99.99th=[301056]
    bw (KB  /s): min=   53, max=  551, per=12.55%, avg=234.27, stdev=83.53
    lat (usec) : 750=3.25%, 1000=18.30%
    lat (msec) : 2=43.22%, 4=16.58%, 10=6.32%, 20=1.32%, 50=0.39%
    lat (msec) : 100=0.67%, 250=9.84%, 500=0.12%
  cpu          : usr=0.19%, sys=0.70%, ctx=59173, majf=0, minf=58
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=28022/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=112088KB, aggrb=1867KB/s, minb=1867KB/s, maxb=1867KB/s, mint=60030msec, maxt=60030msec

Disk stats (read/write):
  vda: ios=631/28086, merge=0/1468, ticks=780/56516, in_queue=57260, util=93.38%

Speedtest (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (159.203.34.131)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by FibreStream (Toronto, ON) [3.45 km]: 2.749 ms
Testing download speed................................................................................
Download: 336.52 Mbit/s
Testing upload speed................................................................................................
Upload: 601.00 Mbit/s

Speedtest (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (104.248.45.25)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by fdcservers.net (Frankfurt) [0.89 km]: 1.731 ms
Testing download speed................................................................................
Download: 2537.59 Mbit/s
Testing upload speed................................................................................................
Upload: 2018.71 Mbit/s

Speedtest (Индия, Π‘Π°Π½Π³Π°Π»ΠΎΡ€)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (139.59.17.112)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Bharti Airtel Ltd (Bangalore) [1.65 km]: 2.194 ms
Testing download speed................................................................................
Download: 672.15 Mbit/s
Testing upload speed................................................................................................
Upload: 865.85 Mbit/s

Speedtest (НидСрланды, АмстСрдам)

Retrieving speedtest.net configuration...
Testing from DigitalOcean (188.166.46.233)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Vancis (Amsterdam) [3.12 km]: 2.61 ms
Testing download speed................................................................................
Download: 290.28 Mbit/s
Testing upload speed................................................................................................
Upload: 874.21 Mbit/s

Speedtest (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (68.183.190.84)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Viewqwest Pte Ltd (Singapore) [13.00 km]: 2.036 ms
Testing download speed................................................................................
Download: 3591.73 Mbit/s
Testing upload speed................................................................................................
Upload: 1560.91 Mbit/s

Speedtest (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (142.93.37.95)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Iomart (Maidenhead) [3.98 km]: 2.04 ms
Testing download speed................................................................................
Download: 858.28 Mbit/s
Testing upload speed................................................................................................
Upload: 876.08 Mbit/s

Speedtest (БША, Нью-Π™ΠΎΡ€ΠΊ)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (167.71.82.38)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Interserver, inc (Secaucus, NJ) [8.75 km]: 1.879 ms
Testing download speed................................................................................
Download: 3598.06 Mbit/s
Testing upload speed................................................................................................
Upload: 2021.05 Mbit/s

Speedtest (БША, Π‘Π°Π½-Ѐранциско)

Retrieving speedtest.net configuration...
Testing from Digital Ocean (138.68.27.151)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Next Level Networks (Santa Clara, CA) [2.25 km]: 3.875 ms
Testing download speed................................................................................
Download: 574.97 Mbit/s
Testing upload speed................................................................................................
Upload: 261.58 Mbit/s

Π’ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹ΠΉ сСрвСр ΠΎΡ‚ Vultr:

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

  • Ρ‚Π°Ρ€ΠΈΡ„ Cloud Compute (25 GB SSD / 1x vCPU / 1 GB RAM / 10000 Mbit/s Π·Π° $5 Π² мСсяц);
  • Π΄Π°Ρ‚Π°-Ρ†Π΅Π½Ρ‚Ρ€Ρ‹ Π² Австралии, Π’Π΅Π»ΠΈΠΊΠΎΠ±Ρ€ΠΈΡ‚Π°Π½ΠΈΠΈ, Π“Π΅Ρ€ΠΌΠ°Π½ΠΈΠΈ, КанадС, НидСрландах, Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€Π΅, БША, Π€Ρ€Π°Π½Ρ†ΠΈΠΈ, Π―ΠΏΠΎΠ½ΠΈΠΈ.

ВСсты:

Unixbench (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:46:36 up 2 min,  1 user,  load average: 0.58, 0.19, 0.06; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:46:36 - 03:14:41
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25102626.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4295.6 MWIPS (9.8 s, 7 samples)
Execl Throughput                               4230.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        604499.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          169295.0 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1277011.0 KBps  (30.0 s, 2 samples)
Pipe Throughput                              959022.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 140482.6 lps   (10.0 s, 7 samples)
Process Creation                               9784.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7107.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    906.8 lpm   (60.0 s, 2 samples)
System Call Overhead                         673720.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25102626.8   2151.0
Double-Precision Whetstone                       55.0       4295.6    781.0
Execl Throughput                                 43.0       4230.2    983.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     604499.7   1526.5
File Copy 256 bufsize 500 maxblocks            1655.0     169295.0   1022.9
File Copy 4096 bufsize 8000 maxblocks          5800.0    1277011.0   2201.7
Pipe Throughput                               12440.0     959022.0    770.9
Pipe-based Context Switching                   4000.0     140482.6    351.2
Process Creation                                126.0       9784.3    776.5
Shell Scripts (1 concurrent)                     42.4       7107.8   1676.4
Shell Scripts (8 concurrent)                      6.0        906.8   1511.4
System Call Overhead                          15000.0     673720.5    449.1
                                                                   ========
System Benchmarks Index Score                                        1025.9

Unixbench (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 82d9ed4018dd (5187.8 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:42:53 up 2 min,  1 user,  load average: 0.64, 0.17, 0.06; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:42:53 - 03:11:00
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       38699273.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4619.5 MWIPS (10.2 s, 7 samples)
Execl Throughput                               6768.4 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        705978.4 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          192749.4 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1751883.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1012385.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 179610.0 lps   (10.0 s, 7 samples)
Process Creation                              15081.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  10739.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1378.7 lpm   (60.0 s, 2 samples)
System Call Overhead                         570458.3 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   38699273.9   3316.1
Double-Precision Whetstone                       55.0       4619.5    839.9
Execl Throughput                                 43.0       6768.4   1574.0
File Copy 1024 bufsize 2000 maxblocks          3960.0     705978.4   1782.8
File Copy 256 bufsize 500 maxblocks            1655.0     192749.4   1164.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1751883.6   3020.5
Pipe Throughput                               12440.0    1012385.6    813.8
Pipe-based Context Switching                   4000.0     179610.0    449.0
Process Creation                                126.0      15081.7   1197.0
Shell Scripts (1 concurrent)                     42.4      10739.0   2532.8
Shell Scripts (8 concurrent)                      6.0       1378.7   2297.8
System Call Overhead                          15000.0     570458.3    380.3
                                                                   ========
System Benchmarks Index Score                                        1314.5

Unixbench (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 714389bda930 (4800.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   23:41:12 up 3 min,  1 user,  load average: 0.49, 0.17, 0.06; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Fri Dec 13 2019 23:41:12 - 00:09:21
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26077546.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3561.5 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4233.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        495359.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          130299.0 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1347940.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              657753.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 117057.6 lps   (10.0 s, 7 samples)
Process Creation                               9390.0 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6618.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    870.7 lpm   (60.1 s, 2 samples)
System Call Overhead                         359105.0 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26077546.4   2234.6
Double-Precision Whetstone                       55.0       3561.5    647.5
Execl Throughput                                 43.0       4233.5    984.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     495359.7   1250.9
File Copy 256 bufsize 500 maxblocks            1655.0     130299.0    787.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1347940.9   2324.0
Pipe Throughput                               12440.0     657753.9    528.7
Pipe-based Context Switching                   4000.0     117057.6    292.6
Process Creation                                126.0       9390.0    745.2
Shell Scripts (1 concurrent)                     42.4       6618.6   1561.0
Shell Scripts (8 concurrent)                      6.0        870.7   1451.2
System Call Overhead                          15000.0     359105.0    239.4
                                                                   ========
System Benchmarks Index Score                                         875.9

Unixbench (Ѐранция, ΠŸΠ°Ρ€ΠΈΠΆ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   12:37:38 up 5 min,  1 user,  load average: 0.85, 0.48, 0.21; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 12 2019 12:37:38 - 13:05:33
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables        8896034.1 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     1762.9 MWIPS (9.5 s, 7 samples)
Execl Throughput                               1488.3 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        170322.0 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           46997.0 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        499490.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              253632.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  47089.9 lps   (10.0 s, 7 samples)
Process Creation                               3571.1 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   2557.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    339.8 lpm   (60.1 s, 2 samples)
System Call Overhead                         149530.8 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0    8896034.1    762.3
Double-Precision Whetstone                       55.0       1762.9    320.5
Execl Throughput                                 43.0       1488.3    346.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     170322.0    430.1
File Copy 256 bufsize 500 maxblocks            1655.0      46997.0    284.0
File Copy 4096 bufsize 8000 maxblocks          5800.0     499490.5    861.2
Pipe Throughput                               12440.0     253632.7    203.9
Pipe-based Context Switching                   4000.0      47089.9    117.7
Process Creation                                126.0       3571.1    283.4
Shell Scripts (1 concurrent)                     42.4       2557.8    603.3
Shell Scripts (8 concurrent)                      6.0        339.8    566.3
System Call Overhead                          15000.0     149530.8     99.7
                                                                   ========
System Benchmarks Index Score                                         335.9

Unixbench (Япония, Вокио)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   07:37:39 up 3 min,  1 user,  load average: 0.65, 0.31, 0.12; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 05 2019 07:37:39 - 08:05:44
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26212655.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4429.9 MWIPS (10.0 s, 7 samples)
Execl Throughput                               4627.0 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        676411.8 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          181937.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1590192.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1008509.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 147408.3 lps   (10.0 s, 7 samples)
Process Creation                              10743.2 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7856.9 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1025.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         689942.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26212655.7   2246.2
Double-Precision Whetstone                       55.0       4429.9    805.4
Execl Throughput                                 43.0       4627.0   1076.0
File Copy 1024 bufsize 2000 maxblocks          3960.0     676411.8   1708.1
File Copy 256 bufsize 500 maxblocks            1655.0     181937.6   1099.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1590192.7   2741.7
Pipe Throughput                               12440.0    1008509.9    810.7
Pipe-based Context Switching                   4000.0     147408.3    368.5
Process Creation                                126.0      10743.2    852.6
Shell Scripts (1 concurrent)                     42.4       7856.9   1853.0
Shell Scripts (8 concurrent)                      6.0       1025.9   1709.8
System Call Overhead                          15000.0     689942.2    460.0
                                                                   ========
System Benchmarks Index Score                                        1115.7

Unixbench (НидСрланды, АмстСрдам)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   18:14:32 up 4 min,  1 user,  load average: 0.54, 0.21, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 18:14:32 - 18:42:36
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       24400966.1 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4307.7 MWIPS (10.0 s, 7 samples)
Execl Throughput                               4137.2 lps   (29.7 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        563366.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          166725.8 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1351284.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              947831.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 127169.9 lps   (10.0 s, 7 samples)
Process Creation                               9682.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6961.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    917.2 lpm   (60.0 s, 2 samples)
System Call Overhead                         660482.8 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   24400966.1   2090.9
Double-Precision Whetstone                       55.0       4307.7    783.2
Execl Throughput                                 43.0       4137.2    962.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     563366.3   1422.6
File Copy 256 bufsize 500 maxblocks            1655.0     166725.8   1007.4
File Copy 4096 bufsize 8000 maxblocks          5800.0    1351284.6   2329.8
Pipe Throughput                               12440.0     947831.0    761.9
Pipe-based Context Switching                   4000.0     127169.9    317.9
Process Creation                                126.0       9682.9    768.5
Shell Scripts (1 concurrent)                     42.4       6961.7   1641.9
Shell Scripts (8 concurrent)                      6.0        917.2   1528.7
System Call Overhead                          15000.0     660482.8    440.3
                                                                   ========
System Benchmarks Index Score                                        1006.6

Unixbench (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   14:17:01 up 4 min,  1 user,  load average: 0.68, 0.23, 0.08; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 14:17:01 - 14:45:06
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25144861.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4260.9 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4040.1 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        427185.8 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          116507.8 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1148414.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                              626572.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 115253.6 lps   (10.0 s, 7 samples)
Process Creation                               9620.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6846.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    881.5 lpm   (60.0 s, 2 samples)
System Call Overhead                         344955.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25144861.2   2154.7
Double-Precision Whetstone                       55.0       4260.9    774.7
Execl Throughput                                 43.0       4040.1    939.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     427185.8   1078.8
File Copy 256 bufsize 500 maxblocks            1655.0     116507.8    704.0
File Copy 4096 bufsize 8000 maxblocks          5800.0    1148414.3   1980.0
Pipe Throughput                               12440.0     626572.7    503.7
Pipe-based Context Switching                   4000.0     115253.6    288.1
Process Creation                                126.0       9620.9    763.6
Shell Scripts (1 concurrent)                     42.4       6846.7   1614.8
Shell Scripts (8 concurrent)                      6.0        881.5   1469.2
System Call Overhead                          15000.0     344955.6    230.0
                                                                   ========
System Benchmarks Index Score                                         850.2

Unixbench (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel Core Processor (Haswell, no TSX, IBRS) (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   15:54:53 up 2 min,  1 user,  load average: 0.62, 0.21, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 15:54:53 - 16:22:57
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26199695.8 lps   (10.1 s, 7 samples)
Double-Precision Whetstone                     4515.1 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4408.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        595220.1 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          166098.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1395372.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              993618.1 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 141759.0 lps   (10.0 s, 7 samples)
Process Creation                              10582.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7349.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    939.8 lpm   (60.1 s, 2 samples)
System Call Overhead                         689617.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26199695.8   2245.0
Double-Precision Whetstone                       55.0       4515.1    820.9
Execl Throughput                                 43.0       4408.5   1025.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     595220.1   1503.1
File Copy 256 bufsize 500 maxblocks            1655.0     166098.6   1003.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1395372.9   2405.8
Pipe Throughput                               12440.0     993618.1    798.7
Pipe-based Context Switching                   4000.0     141759.0    354.4
Process Creation                                126.0      10582.5    839.9
Shell Scripts (1 concurrent)                     42.4       7349.8   1733.4
Shell Scripts (8 concurrent)                      6.0        939.8   1566.4
System Call Overhead                          15000.0     689617.2    459.7
                                                                   ========
System Benchmarks Index Score                                        1061.0

Unixbench (БША, Атланта)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   18:59:49 up 2 min,  1 user,  load average: 0.65, 0.22, 0.08; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 18:59:49 - 19:27:54
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25425673.3 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4296.1 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4230.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        463467.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          126388.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1186982.0 KBps  (30.0 s, 2 samples)
Pipe Throughput                              646774.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 122358.8 lps   (10.0 s, 7 samples)
Process Creation                               9800.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7261.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    938.8 lpm   (60.0 s, 2 samples)
System Call Overhead                         357385.0 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25425673.3   2178.7
Double-Precision Whetstone                       55.0       4296.1    781.1
Execl Throughput                                 43.0       4230.5    983.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     463467.3   1170.4
File Copy 256 bufsize 500 maxblocks            1655.0     126388.6    763.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    1186982.0   2046.5
Pipe Throughput                               12440.0     646774.2    519.9
Pipe-based Context Switching                   4000.0     122358.8    305.9
Process Creation                                126.0       9800.5    777.8
Shell Scripts (1 concurrent)                     42.4       7261.6   1712.6
Shell Scripts (8 concurrent)                      6.0        938.8   1564.6
System Call Overhead                          15000.0     357385.0    238.3
                                                                   ========
System Benchmarks Index Score                                         888.4

Unixbench (БША, Чикаго)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   01:04:10 up 2 min,  1 user,  load average: 0.60, 0.20, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 01:04:10 - 01:32:14
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25523985.5 lps   (10.1 s, 7 samples)
Double-Precision Whetstone                     4315.5 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4148.4 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        452161.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          123919.4 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1196969.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              642700.8 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 124108.9 lps   (10.0 s, 7 samples)
Process Creation                               9849.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7299.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    941.0 lpm   (60.0 s, 2 samples)
System Call Overhead                         355273.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25523985.5   2187.1
Double-Precision Whetstone                       55.0       4315.5    784.6
Execl Throughput                                 43.0       4148.4    964.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     452161.7   1141.8
File Copy 256 bufsize 500 maxblocks            1655.0     123919.4    748.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1196969.9   2063.7
Pipe Throughput                               12440.0     642700.8    516.6
Pipe-based Context Switching                   4000.0     124108.9    310.3
Process Creation                                126.0       9849.9    781.7
Shell Scripts (1 concurrent)                     42.4       7299.6   1721.6
Shell Scripts (8 concurrent)                      6.0        941.0   1568.4
System Call Overhead                          15000.0     355273.2    236.8
                                                                   ========
System Benchmarks Index Score                                         886.0

Unixbench (БША, Даллас)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4800.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   01:07:22 up 2 min,  1 user,  load average: 0.62, 0.19, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 01:07:22 - 01:35:27
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26030426.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4443.2 MWIPS (10.0 s, 7 samples)
Execl Throughput                               4537.8 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        634719.5 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          176552.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1486392.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                              990514.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 144687.4 lps   (10.0 s, 7 samples)
Process Creation                              10880.8 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7575.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    992.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         688433.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26030426.9   2230.5
Double-Precision Whetstone                       55.0       4443.2    807.9
Execl Throughput                                 43.0       4537.8   1055.3
File Copy 1024 bufsize 2000 maxblocks          3960.0     634719.5   1602.8
File Copy 256 bufsize 500 maxblocks            1655.0     176552.9   1066.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1486392.3   2562.7
Pipe Throughput                               12440.0     990514.7    796.2
Pipe-based Context Switching                   4000.0     144687.4    361.7
Process Creation                                126.0      10880.8    863.6
Shell Scripts (1 concurrent)                     42.4       7575.3   1786.6
Shell Scripts (8 concurrent)                      6.0        992.9   1654.8
System Call Overhead                          15000.0     688433.6    459.0
                                                                   ========
System Benchmarks Index Score                                        1090.0

Unixbench (БША, Лос-АндТСлСс)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   01:11:21 up 3 min,  1 user,  load average: 0.71, 0.29, 0.11; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 01:11:21 - 01:39:26
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25267089.5 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4246.4 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4112.8 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        456196.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          122812.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1192809.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                              636446.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 116546.4 lps   (10.0 s, 7 samples)
Process Creation                               9696.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7097.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    919.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         352701.1 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25267089.5   2165.1
Double-Precision Whetstone                       55.0       4246.4    772.1
Execl Throughput                                 43.0       4112.8    956.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     456196.7   1152.0
File Copy 256 bufsize 500 maxblocks            1655.0     122812.5    742.1
File Copy 4096 bufsize 8000 maxblocks          5800.0    1192809.7   2056.6
Pipe Throughput                               12440.0     636446.5    511.6
Pipe-based Context Switching                   4000.0     116546.4    291.4
Process Creation                                126.0       9696.7    769.6
Shell Scripts (1 concurrent)                     42.4       7097.2   1673.9
Shell Scripts (8 concurrent)                      6.0        919.9   1533.2
System Call Overhead                          15000.0     352701.1    235.1
                                                                   ========
System Benchmarks Index Score                                         872.5

Unixbench (БША, Майами)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   01:15:03 up 2 min,  1 user,  load average: 0.60, 0.21, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 01:15:03 - 01:43:08
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26582764.1 lps   (10.1 s, 7 samples)
Double-Precision Whetstone                     4412.8 MWIPS (10.0 s, 7 samples)
Execl Throughput                               4634.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        669481.5 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          179719.1 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1533680.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              997113.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 145060.8 lps   (10.0 s, 7 samples)
Process Creation                              11030.6 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7915.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1025.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         686188.0 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26582764.1   2277.9
Double-Precision Whetstone                       55.0       4412.8    802.3
Execl Throughput                                 43.0       4634.7   1077.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     669481.5   1690.6
File Copy 256 bufsize 500 maxblocks            1655.0     179719.1   1085.9
File Copy 4096 bufsize 8000 maxblocks          5800.0    1533680.6   2644.3
Pipe Throughput                               12440.0     997113.7    801.5
Pipe-based Context Switching                   4000.0     145060.8    362.7
Process Creation                                126.0      11030.6    875.4
Shell Scripts (1 concurrent)                     42.4       7915.2   1866.8
Shell Scripts (8 concurrent)                      6.0       1025.9   1709.8
System Call Overhead                          15000.0     686188.0    457.5
                                                                   ========
System Benchmarks Index Score                                        1111.4

Unixbench (БША, Нью-Π™ΠΎΡ€ΠΊ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 714389bda930 (4800.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   00:55:26 up 2 min,  1 user,  load average: 0.56, 0.23, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 00:55:26 - 01:23:34
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25633639.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3533.8 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4070.4 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        474437.0 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          126743.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1199905.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              647341.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  88978.2 lps   (10.0 s, 7 samples)
Process Creation                               7295.0 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6352.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    826.7 lpm   (60.0 s, 2 samples)
System Call Overhead                         353376.1 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25633639.9   2196.5
Double-Precision Whetstone                       55.0       3533.8    642.5
Execl Throughput                                 43.0       4070.4    946.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     474437.0   1198.1
File Copy 256 bufsize 500 maxblocks            1655.0     126743.5    765.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1199905.9   2068.8
Pipe Throughput                               12440.0     647341.0    520.4
Pipe-based Context Switching                   4000.0      88978.2    222.4
Process Creation                                126.0       7295.0    579.0
Shell Scripts (1 concurrent)                     42.4       6352.0   1498.1
Shell Scripts (8 concurrent)                      6.0        826.7   1377.9
System Call Overhead                          15000.0     353376.1    235.6
                                                                   ========
System Benchmarks Index Score                                         812.4

Unixbench (БША, Биэтл)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 714389bda930 (4800.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   01:18:39 up 2 min,  1 user,  load average: 0.73, 0.25, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 01:18:39 - 01:46:47
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26700878.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3622.7 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3907.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        486945.0 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          130046.2 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1283902.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                              662209.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 113981.0 lps   (10.0 s, 7 samples)
Process Creation                               8765.1 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7075.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    905.1 lpm   (60.1 s, 2 samples)
System Call Overhead                         364946.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26700878.9   2288.0
Double-Precision Whetstone                       55.0       3622.7    658.7
Execl Throughput                                 43.0       3907.5    908.7
File Copy 1024 bufsize 2000 maxblocks          3960.0     486945.0   1229.7
File Copy 256 bufsize 500 maxblocks            1655.0     130046.2    785.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1283902.3   2213.6
Pipe Throughput                               12440.0     662209.3    532.3
Pipe-based Context Switching                   4000.0     113981.0    285.0
Process Creation                                126.0       8765.1    695.6
Shell Scripts (1 concurrent)                     42.4       7075.8   1668.8
Shell Scripts (8 concurrent)                      6.0        905.1   1508.5
System Call Overhead                          15000.0     364946.5    243.3
                                                                   ========
System Benchmarks Index Score                                         870.5

Unixbench (БША, Π‘Π°Π½-Π₯осС)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: vultr.guest: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Virtual CPU 523cbcdd6ca4 (4788.9 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:40:36 up 3 min,  1 user,  load average: 0.62, 0.21, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:40:36 - 03:08:40
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       26677842.8 lps   (10.1 s, 7 samples)
Double-Precision Whetstone                     4517.5 MWIPS (10.0 s, 7 samples)
Execl Throughput                               4679.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        658109.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          179078.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1513335.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1011954.1 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 147993.7 lps   (10.0 s, 7 samples)
Process Creation                              11034.6 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7761.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1007.2 lpm   (60.0 s, 2 samples)
System Call Overhead                         686496.1 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   26677842.8   2286.0
Double-Precision Whetstone                       55.0       4517.5    821.4
Execl Throughput                                 43.0       4679.7   1088.3
File Copy 1024 bufsize 2000 maxblocks          3960.0     658109.9   1661.9
File Copy 256 bufsize 500 maxblocks            1655.0     179078.5   1082.0
File Copy 4096 bufsize 8000 maxblocks          5800.0    1513335.5   2609.2
Pipe Throughput                               12440.0    1011954.1    813.5
Pipe-based Context Switching                   4000.0     147993.7    370.0
Process Creation                                126.0      11034.6    875.8
Shell Scripts (1 concurrent)                     42.4       7761.3   1830.5
Shell Scripts (8 concurrent)                      6.0       1007.2   1678.6
System Call Overhead                          15000.0     686496.1    457.7
                                                                   ========
System Benchmarks Index Score                                        1111.5

dd (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.00964 s, 357 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 12.2036 s, 176 MB/s

dd (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.20408 s, 511 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.97048 s, 432 MB/s

dd (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.49292 s, 478 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.66099 s, 461 MB/s

dd (Ѐранция, ΠŸΠ°Ρ€ΠΈΠΆ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.42477 s, 334 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.17672 s, 348 MB/s

dd (Япония, Π’ΠΎΠΊΠΈΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.98321 s, 359 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.74079 s, 374 MB/s

dd (НидСрланды, АмстСрдам)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.00055 s, 358 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 7.24659 s, 296 MB/s

dd (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.21242 s, 346 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.38259 s, 336 MB/s

dd (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 7.77476 s, 276 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 7.65941 s, 280 MB/s

dd (БША, Атланта)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.17663 s, 348 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.14213 s, 350 MB/s

dd (БША, Π§ΠΈΠΊΠ°Π³ΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.75075 s, 373 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.78569 s, 371 MB/s

dd (БША, Даллас)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.7069 s, 376 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.60498 s, 383 MB/s

dd (БША, Лос-АндТСлСс)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 12.9305 s, 166 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.29062 s, 341 MB/s

dd (БША, Майами)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.89201 s, 364 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.74615 s, 374 MB/s

dd (БША, Нью-Π™ΠΎΡ€ΠΊ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.52842 s, 474 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.94189 s, 435 MB/s

dd (БША, Биэтл)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.54416 s, 473 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 5.68025 s, 378 MB/s

dd (БША, Π‘Π°Π½-Π₯осС)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.92419 s, 310 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 8.11701 s, 265 MB/s

fio (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28589: Mon Dec 16 03:16:07 2019
  read : io=7674.1MB, bw=130983KB/s, iops=32745, runt= 60001msec
    clat (usec): min=1, max=13364, avg=240.17, stdev=183.78
     lat (usec): min=2, max=13365, avg=240.76, stdev=183.79
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  171], 20.00th=[  193],
     | 30.00th=[  207], 40.00th=[  221], 50.00th=[  233], 60.00th=[  247],
     | 70.00th=[  266], 80.00th=[  290], 90.00th=[  334], 95.00th=[  374],
     | 99.00th=[  470], 99.50th=[  540], 99.90th=[ 2864], 99.95th=[ 3408],
     | 99.99th=[ 7008]
    bw (KB  /s): min=10904, max=23080, per=12.50%, avg=16375.51, stdev=1940.14
    lat (usec) : 2=0.02%, 4=5.24%, 10=1.36%, 20=0.94%, 50=0.29%
    lat (usec) : 100=0.02%, 250=53.43%, 500=37.99%, 750=0.42%, 1000=0.04%
    lat (msec) : 2=0.05%, 4=0.18%, 10=0.03%, 20=0.01%
  cpu          : usr=1.87%, sys=5.83%, ctx=1816331, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1964785/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7674.1MB, aggrb=130983KB/s, minb=130983KB/s, maxb=130983KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1809446/9, merge=0/4, ticks=391072/0, in_queue=391128, util=98.07%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28618: Mon Dec 16 03:18:14 2019
  write: io=31535MB, bw=537932KB/s, iops=134482, runt= 60029msec
    clat (usec): min=1, max=512047, avg=44.70, stdev=1034.85
     lat (usec): min=1, max=512048, avg=47.33, stdev=1074.92
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    4], 95.00th=[    6],
     | 99.00th=[   18], 99.50th=[   28], 99.90th=[15296], 99.95th=[19840],
     | 99.99th=[30848]
    bw (KB  /s): min=    1, max=85640, per=12.68%, avg=68203.30, stdev=11017.32
    lat (usec) : 2=0.12%, 4=77.58%, 10=19.53%, 20=2.07%, 50=0.27%
    lat (usec) : 100=0.04%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.03%, 10=0.10%, 20=0.13%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  cpu          : usr=2.64%, sys=7.81%, ctx=76430, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=8072876/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=31535MB, aggrb=537931KB/s, minb=537931KB/s, maxb=537931KB/s, mint=60029msec, maxt=60029msec

Disk stats (read/write):
  vda: ios=169/782742, merge=0/8447, ticks=1264/6814944, in_queue=6818788, util=98.98%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28602: Mon Dec 16 03:17:14 2019
  read : io=7471.5MB, bw=127506KB/s, iops=31876, runt= 60003msec
    clat (usec): min=132, max=10029, avg=246.78, stdev=107.92
     lat (usec): min=133, max=10030, avg=247.38, stdev=107.95
    clat percentiles (usec):
     |  1.00th=[  159],  5.00th=[  173], 10.00th=[  183], 20.00th=[  197],
     | 30.00th=[  209], 40.00th=[  219], 50.00th=[  231], 60.00th=[  245],
     | 70.00th=[  262], 80.00th=[  286], 90.00th=[  326], 95.00th=[  362],
     | 99.00th=[  446], 99.50th=[  494], 99.90th=[  884], 99.95th=[ 1752],
     | 99.99th=[ 4960]
    bw (KB  /s): min=13352, max=17536, per=12.50%, avg=15943.78, stdev=782.58
    lat (usec) : 250=63.11%, 500=36.43%, 750=0.33%, 1000=0.04%
    lat (msec) : 2=0.04%, 4=0.02%, 10=0.03%, 20=0.01%
  cpu          : usr=1.84%, sys=4.97%, ctx=1916371, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1912681/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7471.5MB, aggrb=127505KB/s, minb=127505KB/s, maxb=127505KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=1911024/10, merge=0/14, ticks=387936/0, in_queue=387700, util=99.79%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28631: Mon Dec 16 03:19:15 2019
  write: io=2116.3MB, bw=36114KB/s, iops=9028, runt= 60005msec
    clat (usec): min=60, max=52275, avg=881.45, stdev=5188.98
     lat (usec): min=60, max=52275, avg=882.13, stdev=5189.03
    clat percentiles (usec):
     |  1.00th=[   67],  5.00th=[   71], 10.00th=[   74], 20.00th=[   80],
     | 30.00th=[   85], 40.00th=[   90], 50.00th=[   95], 60.00th=[  101],
     | 70.00th=[  109], 80.00th=[  121], 90.00th=[  161], 95.00th=[  215],
     | 99.00th=[36096], 99.50th=[38144], 99.90th=[41216], 99.95th=[42240],
     | 99.99th=[44800]
    bw (KB  /s): min= 3169, max= 5891, per=12.51%, avg=4516.12, stdev=452.68
    lat (usec) : 100=57.43%, 250=39.03%, 500=1.24%, 750=0.04%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.02%, 20=0.01%, 50=2.21%
    lat (msec) : 100=0.01%
  cpu          : usr=0.61%, sys=2.76%, ctx=1095749, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=541749/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2116.3MB, aggrb=36113KB/s, minb=36113KB/s, maxb=36113KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  vda: ios=182/541139, merge=0/1556, ticks=100/47652, in_queue=47640, util=79.23%

fio (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4579: Mon Dec 16 03:12:15 2019
  read : io=12498MB, bw=213300KB/s, iops=53324, runt= 60001msec
    clat (usec): min=1, max=12335, avg=146.68, stdev=110.17
     lat (usec): min=1, max=12335, avg=147.24, stdev=110.17
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    4], 20.00th=[  135],
     | 30.00th=[  141], 40.00th=[  147], 50.00th=[  151], 60.00th=[  155],
     | 70.00th=[  161], 80.00th=[  169], 90.00th=[  183], 95.00th=[  201],
     | 99.00th=[  318], 99.50th=[  422], 99.90th=[ 1832], 99.95th=[ 2024],
     | 99.99th=[ 2672]
    bw (KB  /s): min=17680, max=35552, per=12.47%, avg=26605.05, stdev=3053.39
    lat (usec) : 2=0.92%, 4=8.94%, 10=0.77%, 20=0.02%, 50=0.01%
    lat (usec) : 100=0.13%, 250=87.61%, 500=1.28%, 750=0.05%, 1000=0.02%
    lat (msec) : 2=0.21%, 4=0.05%, 10=0.01%, 20=0.01%
  cpu          : usr=2.33%, sys=5.24%, ctx=2863832, majf=2, minf=87
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=3199546/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=12498MB, aggrb=213299KB/s, minb=213299KB/s, maxb=213299KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2853531/9, merge=0/4, ticks=413816/0, in_queue=413848, util=97.93%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4606: Mon Dec 16 03:14:20 2019
  write: io=40568MB, bw=692123KB/s, iops=173030, runt= 60020msec
    clat (usec): min=1, max=168030, avg=30.86, stdev=724.13
     lat (usec): min=1, max=168030, avg=33.62, stdev=769.11
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[    4], 99.50th=[    8], 99.90th=[11712], 99.95th=[16192],
     | 99.99th=[27776]
    bw (KB  /s): min=    1, max=102488, per=12.60%, avg=87188.15, stdev=9025.40
    lat (usec) : 2=0.15%, 4=98.40%, 10=1.00%, 20=0.07%, 50=0.06%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.02%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.03%, 10=0.09%, 20=0.09%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.36%, sys=6.42%, ctx=83149, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=10385305/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=40568MB, aggrb=692122KB/s, minb=692122KB/s, maxb=692122KB/s, mint=60020msec, maxt=60020msec

Disk stats (read/write):
  vda: ios=625/1328807, merge=0/8697, ticks=3072/7005192, in_queue=7008748, util=99.27%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4593: Mon Dec 16 03:13:20 2019
  read : io=11311MB, bw=193037KB/s, iops=48259, runt= 60001msec
    clat (usec): min=75, max=8391, avg=162.37, stdev=140.17
     lat (usec): min=76, max=8391, avg=162.94, stdev=140.17
    clat percentiles (usec):
     |  1.00th=[  123],  5.00th=[  131], 10.00th=[  135], 20.00th=[  141],
     | 30.00th=[  145], 40.00th=[  149], 50.00th=[  151], 60.00th=[  157],
     | 70.00th=[  161], 80.00th=[  169], 90.00th=[  183], 95.00th=[  199],
     | 99.00th=[  290], 99.50th=[  406], 99.90th=[ 2192], 99.95th=[ 3120],
     | 99.99th=[ 6432]
    bw (KB  /s): min=17560, max=26000, per=12.50%, avg=24134.45, stdev=1483.61
    lat (usec) : 100=0.12%, 250=98.48%, 500=1.10%, 750=0.05%, 1000=0.02%
    lat (msec) : 2=0.11%, 4=0.08%, 10=0.04%
  cpu          : usr=2.16%, sys=4.65%, ctx=2903321, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2895610/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=11311MB, aggrb=193037KB/s, minb=193037KB/s, maxb=193037KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2892534/8, merge=0/5, ticks=434584/0, in_queue=434828, util=99.87%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4621: Mon Dec 16 03:15:21 2019
  write: io=2264.6MB, bw=38648KB/s, iops=9661, runt= 60002msec
    clat (usec): min=51, max=256774, avg=824.62, stdev=6716.44
     lat (usec): min=51, max=256775, avg=825.21, stdev=6716.44
    clat percentiles (usec):
     |  1.00th=[   56],  5.00th=[   57], 10.00th=[   58], 20.00th=[   60],
     | 30.00th=[   61], 40.00th=[   63], 50.00th=[   65], 60.00th=[   67],
     | 70.00th=[   71], 80.00th=[   89], 90.00th=[  139], 95.00th=[  203],
     | 99.00th=[43264], 99.50th=[55552], 99.90th=[81408], 99.95th=[96768],
     | 99.99th=[134144]
    bw (KB  /s): min= 1323, max= 8856, per=12.51%, avg=4835.55, stdev=1290.36
    lat (usec) : 100=81.77%, 250=14.40%, 500=1.52%, 750=0.17%, 1000=0.17%
    lat (msec) : 2=0.37%, 4=0.21%, 10=0.09%, 20=0.01%, 50=0.58%
    lat (msec) : 100=0.66%, 250=0.04%, 500=0.01%
  cpu          : usr=0.45%, sys=1.61%, ctx=1167262, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=579733/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2264.6MB, aggrb=38647KB/s, minb=38647KB/s, maxb=38647KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=173/578889, merge=0/1547, ticks=252/52532, in_queue=52724, util=87.45%

fio (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21959: Sat Dec 14 00:10:37 2019
  read : io=7915.7MB, bw=135091KB/s, iops=33772, runt= 60001msec
    clat (usec): min=1, max=15947, avg=231.15, stdev=202.63
     lat (usec): min=2, max=15947, avg=232.04, stdev=202.68
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    9], 10.00th=[  153], 20.00th=[  175],
     | 30.00th=[  189], 40.00th=[  201], 50.00th=[  215], 60.00th=[  229],
     | 70.00th=[  249], 80.00th=[  278], 90.00th=[  330], 95.00th=[  386],
     | 99.00th=[  588], 99.50th=[  940], 99.90th=[ 2704], 99.95th=[ 3152],
     | 99.99th=[ 8256]
    bw (KB  /s): min=13072, max=22224, per=12.50%, avg=16885.51, stdev=1416.82
    lat (usec) : 2=0.01%, 4=2.12%, 10=3.30%, 20=1.99%, 50=0.58%
    lat (usec) : 100=0.02%, 250=62.14%, 500=28.13%, 750=1.10%, 1000=0.15%
    lat (msec) : 2=0.21%, 4=0.23%, 10=0.02%, 20=0.01%
  cpu          : usr=2.90%, sys=7.72%, ctx=1870975, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2026398/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7915.7MB, aggrb=135090KB/s, minb=135090KB/s, maxb=135090KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1861991/9, merge=0/4, ticks=329328/0, in_queue=328932, util=98.13%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21985: Sat Dec 14 00:12:43 2019
  write: io=26172MB, bw=446593KB/s, iops=111648, runt= 60009msec
    clat (usec): min=1, max=104202, avg=46.04, stdev=817.78
     lat (usec): min=2, max=104203, avg=50.84, stdev=880.51
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    5], 95.00th=[    7],
     | 99.00th=[   18], 99.50th=[   95], 99.90th=[14272], 99.95th=[18816],
     | 99.99th=[28032]
    bw (KB  /s): min=    1, max=66344, per=12.64%, avg=56439.90, stdev=6246.70
    lat (usec) : 2=0.01%, 4=74.88%, 10=21.89%, 20=2.34%, 50=0.31%
    lat (usec) : 100=0.08%, 250=0.03%, 500=0.02%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.05%, 10=0.15%, 20=0.14%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.84%, sys=5.78%, ctx=92604, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6699905/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=26172MB, aggrb=446593KB/s, minb=446593KB/s, maxb=446593KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  vda: ios=169/1517807, merge=0/10554, ticks=652/6860812, in_queue=6861820, util=98.12%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21972: Sat Dec 14 00:11:43 2019
  read : io=8536.2MB, bw=145672KB/s, iops=36417, runt= 60004msec
    clat (usec): min=118, max=11287, avg=214.02, stdev=81.60
     lat (usec): min=118, max=11288, avg=214.87, stdev=81.67
    clat percentiles (usec):
     |  1.00th=[  143],  5.00th=[  155], 10.00th=[  163], 20.00th=[  173],
     | 30.00th=[  183], 40.00th=[  191], 50.00th=[  201], 60.00th=[  213],
     | 70.00th=[  227], 80.00th=[  247], 90.00th=[  278], 95.00th=[  310],
     | 99.00th=[  398], 99.50th=[  442], 99.90th=[  628], 99.95th=[  804],
     | 99.99th=[ 3664]
    bw (KB  /s): min=14304, max=19464, per=12.50%, avg=18209.07, stdev=907.94
    lat (usec) : 250=81.57%, 500=18.17%, 750=0.20%, 1000=0.03%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=3.16%, sys=6.72%, ctx=2190379, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2185219/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8536.2MB, aggrb=145671KB/s, minb=145671KB/s, maxb=145671KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=2182468/8, merge=0/5, ticks=399108/0, in_queue=398872, util=99.82%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21998: Sat Dec 14 00:13:43 2019
  write: io=2255.2MB, bw=38485KB/s, iops=9621, runt= 60005msec
    clat (usec): min=65, max=42951, avg=825.62, stdev=4606.39
     lat (usec): min=65, max=42951, avg=826.56, stdev=4606.42
    clat percentiles (usec):
     |  1.00th=[   71],  5.00th=[   74], 10.00th=[   75], 20.00th=[   79],
     | 30.00th=[   83], 40.00th=[   88], 50.00th=[   93], 60.00th=[   98],
     | 70.00th=[  105], 80.00th=[  114], 90.00th=[  133], 95.00th=[  161],
     | 99.00th=[30336], 99.50th=[31616], 99.90th=[34048], 99.95th=[35072],
     | 99.99th=[38144]
    bw (KB  /s): min= 3420, max= 6336, per=12.51%, avg=4815.62, stdev=460.55
    lat (usec) : 100=62.03%, 250=35.23%, 500=0.26%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.45%
  cpu          : usr=1.02%, sys=3.03%, ctx=1169418, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=577318/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2255.2MB, aggrb=38484KB/s, minb=38484KB/s, maxb=38484KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  vda: ios=177/576692, merge=0/1614, ticks=108/44140, in_queue=44096, util=73.32%

fio (Ѐранция, ΠŸΠ°Ρ€ΠΈΠΆ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21844: Thu Dec 12 13:06:55 2019
  read : io=3057.1MB, bw=52185KB/s, iops=13046, runt= 60004msec
    clat (usec): min=4, max=52368, avg=596.01, stdev=581.93
     lat (usec): min=5, max=52370, avg=598.59, stdev=582.58
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[  251], 10.00th=[  306], 20.00th=[  354],
     | 30.00th=[  398], 40.00th=[  446], 50.00th=[  510], 60.00th=[  572],
     | 70.00th=[  652], 80.00th=[  756], 90.00th=[  940], 95.00th=[ 1176],
     | 99.00th=[ 1992], 99.50th=[ 2928], 99.90th=[ 8032], 99.95th=[ 9536],
     | 99.99th=[18048]
    bw (KB  /s): min= 2432, max=10000, per=12.51%, avg=6526.19, stdev=974.23
    lat (usec) : 10=3.48%, 20=0.46%, 50=0.34%, 100=0.03%, 250=0.64%
    lat (usec) : 500=43.85%, 750=30.97%, 1000=12.02%
    lat (msec) : 2=7.22%, 4=0.67%, 10=0.28%, 20=0.03%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=3.03%, sys=6.50%, ctx=754863, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=782820/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=3057.1MB, aggrb=52184KB/s, minb=52184KB/s, maxb=52184KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=748009/9, merge=0/4, ticks=297144/0, in_queue=296676, util=97.38%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21880: Thu Dec 12 13:09:04 2019
  write: io=9695.7MB, bw=165439KB/s, iops=41359, runt= 60012msec
    clat (usec): min=4, max=272163, avg=120.45, stdev=1565.02
     lat (usec): min=5, max=272194, avg=133.49, stdev=1688.89
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[    8], 60.00th=[    8],
     | 70.00th=[    8], 80.00th=[    9], 90.00th=[   11], 95.00th=[   15],
     | 99.00th=[  129], 99.50th=[ 8768], 99.90th=[24192], 99.95th=[30336],
     | 99.99th=[41728]
    bw (KB  /s): min=    0, max=28015, per=12.73%, avg=21062.09, stdev=3164.09
    lat (usec) : 10=85.36%, 20=11.94%, 50=1.55%, 100=0.11%, 250=0.12%
    lat (usec) : 500=0.04%, 750=0.02%, 1000=0.02%
    lat (msec) : 2=0.06%, 4=0.09%, 10=0.23%, 20=0.30%, 50=0.17%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=4.62%, sys=5.28%, ctx=55211, majf=0, minf=91
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2482084/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=9695.7MB, aggrb=165439KB/s, minb=165439KB/s, maxb=165439KB/s, mint=60012msec, maxt=60012msec

Disk stats (read/write):
  vda: ios=570/734015, merge=0/6974, ticks=960/4050752, in_queue=4051320, util=67.62%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21859: Thu Dec 12 13:08:04 2019
  read : io=3183.3MB, bw=54325KB/s, iops=13581, runt= 60003msec
    clat (usec): min=177, max=31523, avg=571.81, stdev=426.10
     lat (usec): min=178, max=31525, avg=574.34, stdev=426.65
    clat percentiles (usec):
     |  1.00th=[  251],  5.00th=[  290], 10.00th=[  318], 20.00th=[  354],
     | 30.00th=[  394], 40.00th=[  438], 50.00th=[  494], 60.00th=[  556],
     | 70.00th=[  620], 80.00th=[  708], 90.00th=[  860], 95.00th=[ 1032],
     | 99.00th=[ 1704], 99.50th=[ 2192], 99.90th=[ 7456], 99.95th=[ 8768],
     | 99.99th=[10944]
    bw (KB  /s): min= 2080, max= 8136, per=12.52%, avg=6803.39, stdev=713.56
    lat (usec) : 250=0.91%, 500=49.97%, 750=32.72%, 1000=10.66%
    lat (msec) : 2=5.10%, 4=0.44%, 10=0.18%, 20=0.01%, 50=0.01%
  cpu          : usr=3.21%, sys=6.17%, ctx=821085, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=814921/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=3183.3MB, aggrb=54325KB/s, minb=54325KB/s, maxb=54325KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=814160/25, merge=0/21, ticks=309004/0, in_queue=308492, util=98.07%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=21905: Thu Dec 12 13:10:05 2019
  write: io=513932KB, bw=8564.1KB/s, iops=2141, runt= 60004msec
    clat (usec): min=124, max=75798, avg=3712.21, stdev=9522.14
     lat (usec): min=125, max=75801, avg=3715.65, stdev=9522.72
    clat percentiles (usec):
     |  1.00th=[  201],  5.00th=[  237], 10.00th=[  262], 20.00th=[  298],
     | 30.00th=[  330], 40.00th=[  366], 50.00th=[  418], 60.00th=[  482],
     | 70.00th=[  556], 80.00th=[  668], 90.00th=[25728], 95.00th=[30592],
     | 99.00th=[36096], 99.50th=[39168], 99.90th=[44800], 99.95th=[48896],
     | 99.99th=[62720]
    bw (KB  /s): min=  361, max= 1687, per=12.50%, avg=1070.72, stdev=161.85
    lat (usec) : 250=7.32%, 500=55.42%, 750=20.91%, 1000=3.87%
    lat (msec) : 2=1.37%, 4=0.23%, 10=0.16%, 20=0.05%, 50=10.64%
    lat (msec) : 100=0.04%
  cpu          : usr=0.84%, sys=3.45%, ctx=269617, majf=0, minf=64
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=128483/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=513932KB, aggrb=8564KB/s, minb=8564KB/s, maxb=8564KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=563/128541, merge=0/2127, ticks=392/44204, in_queue=44448, util=73.41%

fio (Япония, Вокио)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27596: Thu Dec  5 08:07:04 2019
  read : io=8536.6MB, bw=145688KB/s, iops=36421, runt= 60001msec
    clat (usec): min=1, max=8864, avg=215.91, stdev=140.30
     lat (usec): min=1, max=8865, avg=216.46, stdev=140.31
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[  159], 20.00th=[  185],
     | 30.00th=[  197], 40.00th=[  209], 50.00th=[  217], 60.00th=[  229],
     | 70.00th=[  241], 80.00th=[  258], 90.00th=[  286], 95.00th=[  314],
     | 99.00th=[  398], 99.50th=[  540], 99.90th=[ 2224], 99.95th=[ 2736],
     | 99.99th=[ 3632]
    bw (KB  /s): min=14880, max=25576, per=12.49%, avg=18199.00, stdev=2075.46
    lat (usec) : 2=0.29%, 4=7.93%, 10=0.72%, 20=0.12%, 50=0.01%
    lat (usec) : 100=0.01%, 250=66.79%, 500=23.59%, 750=0.13%, 1000=0.06%
    lat (msec) : 2=0.19%, 4=0.15%, 10=0.01%
  cpu          : usr=1.91%, sys=5.45%, ctx=1991593, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2185355/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8536.6MB, aggrb=145687KB/s, minb=145687KB/s, maxb=145687KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1983091/10, merge=0/3, ticks=408440/0, in_queue=408168, util=98.38%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27622: Thu Dec  5 08:09:12 2019
  write: io=35740MB, bw=609892KB/s, iops=152473, runt= 60007msec
    clat (usec): min=1, max=206966, avg=37.85, stdev=961.08
     lat (usec): min=1, max=206981, avg=40.42, stdev=1001.48
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    4],
     | 99.00th=[    9], 99.50th=[   14], 99.90th=[13248], 99.95th=[19584],
     | 99.99th=[33536]
    bw (KB  /s): min=    1, max=99528, per=12.64%, avg=77112.48, stdev=10362.96
    lat (usec) : 2=0.26%, 4=90.58%, 10=8.18%, 20=0.62%, 50=0.06%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.02%, 10=0.08%, 20=0.10%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.89%, sys=7.40%, ctx=68728, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9149450/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=35740MB, aggrb=609892KB/s, minb=609892KB/s, maxb=609892KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  vda: ios=180/643486, merge=0/8512, ticks=1204/6913368, in_queue=6916056, util=99.07%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27609: Thu Dec  5 08:08:11 2019
  read : io=7775.5MB, bw=132697KB/s, iops=33174, runt= 60002msec
    clat (usec): min=132, max=4782, avg=237.29, stdev=83.99
     lat (usec): min=133, max=4783, avg=237.83, stdev=84.00
    clat percentiles (usec):
     |  1.00th=[  161],  5.00th=[  175], 10.00th=[  185], 20.00th=[  197],
     | 30.00th=[  209], 40.00th=[  219], 50.00th=[  227], 60.00th=[  239],
     | 70.00th=[  251], 80.00th=[  266], 90.00th=[  294], 95.00th=[  318],
     | 99.00th=[  390], 99.50th=[  450], 99.90th=[ 1400], 99.95th=[ 1784],
     | 99.99th=[ 3184]
    bw (KB  /s): min=14368, max=18584, per=12.50%, avg=16593.13, stdev=701.89
    lat (usec) : 250=69.35%, 500=30.27%, 750=0.13%, 1000=0.07%
    lat (msec) : 2=0.15%, 4=0.03%, 10=0.01%
  cpu          : usr=1.77%, sys=4.65%, ctx=1994084, majf=0, minf=57
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1990521/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7775.5MB, aggrb=132696KB/s, minb=132696KB/s, maxb=132696KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1983437/8, merge=0/5, ticks=409560/0, in_queue=409268, util=99.87%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27635: Thu Dec  5 08:10:12 2019
  write: io=2355.4MB, bw=40196KB/s, iops=10048, runt= 60003msec
    clat (usec): min=59, max=50972, avg=792.04, stdev=4971.28
     lat (usec): min=60, max=50973, avg=792.63, stdev=4971.29
    clat percentiles (usec):
     |  1.00th=[   66],  5.00th=[   69], 10.00th=[   73], 20.00th=[   78],
     | 30.00th=[   83], 40.00th=[   87], 50.00th=[   92], 60.00th=[   96],
     | 70.00th=[  101], 80.00th=[  108], 90.00th=[  124], 95.00th=[  149],
     | 99.00th=[36096], 99.50th=[38144], 99.90th=[41216], 99.95th=[42240],
     | 99.99th=[44288]
    bw (KB  /s): min= 3600, max= 6696, per=12.51%, avg=5026.41, stdev=488.82
    lat (usec) : 100=67.89%, 250=29.95%, 500=0.18%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.94%
    lat (msec) : 100=0.01%
  cpu          : usr=0.59%, sys=2.72%, ctx=1218150, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=602963/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2355.4MB, aggrb=40195KB/s, minb=40195KB/s, maxb=40195KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=169/602207, merge=0/1593, ticks=12/48036, in_queue=47908, util=79.85%

fio (НидСрланды, АмстСрдам)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15909: Wed Dec 11 18:43:57 2019
  read : io=7981.3MB, bw=136211KB/s, iops=34052, runt= 60001msec
    clat (usec): min=1, max=5923, avg=230.65, stdev=151.37
     lat (usec): min=1, max=5924, avg=231.24, stdev=151.40
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  171], 20.00th=[  195],
     | 30.00th=[  209], 40.00th=[  219], 50.00th=[  229], 60.00th=[  241],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  306], 95.00th=[  338],
     | 99.00th=[  434], 99.50th=[  524], 99.90th=[ 2672], 99.95th=[ 3312],
     | 99.99th=[ 4320]
    bw (KB  /s): min=13288, max=23776, per=12.50%, avg=17025.53, stdev=1846.05
    lat (usec) : 2=0.08%, 4=7.03%, 10=0.82%, 20=0.18%, 50=0.03%
    lat (usec) : 100=0.01%, 250=57.74%, 500=33.54%, 750=0.26%, 1000=0.06%
    lat (msec) : 2=0.07%, 4=0.17%, 10=0.02%
  cpu          : usr=1.94%, sys=6.08%, ctx=1883534, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2043203/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7981.3MB, aggrb=136211KB/s, minb=136211KB/s, maxb=136211KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1873042/9, merge=0/4, ticks=388148/0, in_queue=387944, util=98.33%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15935: Wed Dec 11 18:46:05 2019
  write: io=32167MB, bw=548938KB/s, iops=137234, runt= 60005msec
    clat (usec): min=1, max=179157, avg=40.91, stdev=950.87
     lat (usec): min=1, max=179157, avg=43.74, stdev=995.56
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[    8], 99.50th=[   16], 99.90th=[14400], 99.95th=[20096],
     | 99.99th=[32384]
    bw (KB  /s): min=    0, max=94521, per=12.76%, avg=70066.25, stdev=10498.62
    lat (usec) : 2=0.09%, 4=80.81%, 10=18.19%, 20=0.47%, 50=0.08%
    lat (usec) : 100=0.03%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.03%, 10=0.09%, 20=0.11%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=3.05%, sys=7.35%, ctx=68257, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=8234759/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=32167MB, aggrb=548938KB/s, minb=548938KB/s, maxb=548938KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  vda: ios=180/704167, merge=0/7607, ticks=1196/6839992, in_queue=6841764, util=98.87%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15922: Wed Dec 11 18:45:04 2019
  read : io=7375.3MB, bw=125867KB/s, iops=31466, runt= 60002msec
    clat (usec): min=136, max=10916, avg=250.04, stdev=101.92
     lat (usec): min=136, max=10916, avg=250.63, stdev=101.96
    clat percentiles (usec):
     |  1.00th=[  165],  5.00th=[  179], 10.00th=[  189], 20.00th=[  203],
     | 30.00th=[  215], 40.00th=[  225], 50.00th=[  235], 60.00th=[  247],
     | 70.00th=[  262], 80.00th=[  278], 90.00th=[  314], 95.00th=[  354],
     | 99.00th=[  564], 99.50th=[  676], 99.90th=[ 1032], 99.95th=[ 1512],
     | 99.99th=[ 3920]
    bw (KB  /s): min= 8752, max=18208, per=12.51%, avg=15747.04, stdev=1769.43
    lat (usec) : 250=62.53%, 500=35.96%, 750=1.19%, 1000=0.21%
    lat (msec) : 2=0.08%, 4=0.03%, 10=0.01%, 20=0.01%
  cpu          : usr=1.77%, sys=5.13%, ctx=1891322, majf=0, minf=88
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1888072/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7375.3MB, aggrb=125867KB/s, minb=125867KB/s, maxb=125867KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1886265/8, merge=0/5, ticks=420800/0, in_queue=420836, util=99.87%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15948: Wed Dec 11 18:47:05 2019
  write: io=1885.3MB, bw=32172KB/s, iops=8043, runt= 60004msec
    clat (usec): min=63, max=64992, avg=989.65, stdev=5493.21
     lat (usec): min=64, max=64993, avg=990.38, stdev=5493.23
    clat percentiles (usec):
     |  1.00th=[   73],  5.00th=[   81], 10.00th=[   86], 20.00th=[   92],
     | 30.00th=[   97], 40.00th=[  102], 50.00th=[  107], 60.00th=[  114],
     | 70.00th=[  123], 80.00th=[  139], 90.00th=[  171], 95.00th=[  221],
     | 99.00th=[35584], 99.50th=[37632], 99.90th=[42752], 99.95th=[44800],
     | 99.99th=[55552]
    bw (KB  /s): min= 2203, max= 5426, per=12.52%, avg=4027.61, stdev=480.41
    lat (usec) : 100=34.99%, 250=61.01%, 500=1.28%, 750=0.09%, 1000=0.05%
    lat (msec) : 2=0.06%, 4=0.03%, 10=0.01%, 20=0.01%, 50=2.46%
    lat (msec) : 100=0.02%
  cpu          : usr=0.60%, sys=2.78%, ctx=977664, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=482614/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1885.3MB, aggrb=32172KB/s, minb=32172KB/s, maxb=32172KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=170/482245, merge=0/1678, ticks=132/47556, in_queue=47540, util=79.02%

fio (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30926: Wed Dec 11 14:46:27 2019
  read : io=7697.7MB, bw=131370KB/s, iops=32842, runt= 60001msec
    clat (usec): min=1, max=34126, avg=238.32, stdev=232.97
     lat (usec): min=2, max=34127, avg=239.17, stdev=232.98
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  165], 20.00th=[  189],
     | 30.00th=[  201], 40.00th=[  213], 50.00th=[  227], 60.00th=[  241],
     | 70.00th=[  262], 80.00th=[  290], 90.00th=[  334], 95.00th=[  378],
     | 99.00th=[  540], 99.50th=[  844], 99.90th=[ 2512], 99.95th=[ 3056],
     | 99.99th=[ 4640]
    bw (KB  /s): min=11272, max=22698, per=12.50%, avg=16416.34, stdev=1953.87
    lat (usec) : 2=0.01%, 4=6.34%, 10=1.30%, 20=0.20%, 50=0.02%
    lat (usec) : 100=0.01%, 250=56.94%, 500=33.94%, 750=0.68%, 1000=0.16%
    lat (msec) : 2=0.20%, 4=0.19%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=2.34%, sys=5.49%, ctx=1820233, majf=2, minf=95
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1970586/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7697.7MB, aggrb=131370KB/s, minb=131370KB/s, maxb=131370KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1811866/16, merge=0/8, ticks=385580/0, in_queue=385364, util=98.23%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30954: Wed Dec 11 14:48:35 2019
  write: io=26015MB, bw=443885KB/s, iops=110971, runt= 60015msec
    clat (usec): min=1, max=192632, avg=45.41, stdev=880.56
     lat (usec): min=2, max=192634, avg=50.34, stdev=950.31
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[   18], 99.50th=[   35], 99.90th=[14656], 99.95th=[20096],
     | 99.99th=[32128]
    bw (KB  /s): min=    1, max=66360, per=12.66%, avg=56217.23, stdev=5810.74
    lat (usec) : 2=0.01%, 4=66.86%, 10=30.89%, 20=1.55%, 50=0.21%
    lat (usec) : 100=0.05%, 250=0.03%, 500=0.01%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.04%, 10=0.12%, 20=0.13%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.75%, sys=5.94%, ctx=77362, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6659941/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=26015MB, aggrb=443885KB/s, minb=443885KB/s, maxb=443885KB/s, mint=60015msec, maxt=60015msec

Disk stats (read/write):
  vda: ios=171/1050982, merge=0/9077, ticks=640/6829808, in_queue=6830932, util=98.80%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30940: Wed Dec 11 14:47:34 2019
  read : io=7409.7MB, bw=126453KB/s, iops=31613, runt= 60002msec
    clat (usec): min=130, max=24402, avg=247.64, stdev=134.86
     lat (usec): min=131, max=24403, avg=248.50, stdev=134.89
    clat percentiles (usec):
     |  1.00th=[  157],  5.00th=[  171], 10.00th=[  181], 20.00th=[  193],
     | 30.00th=[  205], 40.00th=[  215], 50.00th=[  225], 60.00th=[  239],
     | 70.00th=[  258], 80.00th=[  282], 90.00th=[  326], 95.00th=[  370],
     | 99.00th=[  556], 99.50th=[  804], 99.90th=[ 1720], 99.95th=[ 2480],
     | 99.99th=[ 3952]
    bw (KB  /s): min= 4128, max=17392, per=12.50%, avg=15809.98, stdev=1368.84
    lat (usec) : 250=66.66%, 500=32.07%, 750=0.69%, 1000=0.28%
    lat (msec) : 2=0.23%, 4=0.06%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=2.51%, sys=4.77%, ctx=1900108, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1896859/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7409.7MB, aggrb=126453KB/s, minb=126453KB/s, maxb=126453KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1893026/8, merge=0/5, ticks=404240/0, in_queue=404112, util=99.88%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30967: Wed Dec 11 14:49:35 2019
  write: io=2091.3MB, bw=35686KB/s, iops=8921, runt= 60002msec
    clat (usec): min=60, max=76654, avg=890.95, stdev=5124.75
     lat (usec): min=61, max=76655, avg=891.89, stdev=5124.77
    clat percentiles (usec):
     |  1.00th=[   66],  5.00th=[   69], 10.00th=[   72], 20.00th=[   79],
     | 30.00th=[   84], 40.00th=[   89], 50.00th=[   95], 60.00th=[  100],
     | 70.00th=[  108], 80.00th=[  121], 90.00th=[  167], 95.00th=[  233],
     | 99.00th=[34560], 99.50th=[36608], 99.90th=[39680], 99.95th=[42240],
     | 99.99th=[56576]
    bw (KB  /s): min= 2831, max= 5484, per=12.50%, avg=4461.68, stdev=440.44
    lat (usec) : 100=58.61%, 250=37.16%, 500=1.74%, 750=0.08%, 1000=0.02%
    lat (msec) : 2=0.03%, 4=0.02%, 10=0.01%, 20=0.01%, 50=2.31%
    lat (msec) : 100=0.02%
  cpu          : usr=0.66%, sys=2.82%, ctx=1083156, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=535303/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2091.3MB, aggrb=35685KB/s, minb=35685KB/s, maxb=35685KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=170/534669, merge=0/1610, ticks=132/49496, in_queue=49572, util=82.41%

fio (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=20488: Sun Dec 15 16:24:24 2019
  read : io=743988KB, bw=12399KB/s, iops=3099, runt= 60004msec
    clat (usec): min=142, max=15787, avg=2574.70, stdev=1413.98
     lat (usec): min=143, max=15788, avg=2575.38, stdev=1414.09
    clat percentiles (usec):
     |  1.00th=[  185],  5.00th=[  221], 10.00th=[  266], 20.00th=[ 1256],
     | 30.00th=[ 2128], 40.00th=[ 2480], 50.00th=[ 2768], 60.00th=[ 3024],
     | 70.00th=[ 3280], 80.00th=[ 3600], 90.00th=[ 4128], 95.00th=[ 4640],
     | 99.00th=[ 6176], 99.50th=[ 6880], 99.90th=[ 8256], 99.95th=[ 8896],
     | 99.99th=[10304]
    bw (KB  /s): min= 1136, max= 4691, per=12.52%, avg=1551.89, stdev=544.41
    lat (usec) : 250=8.44%, 500=8.36%, 750=1.31%, 1000=0.96%
    lat (msec) : 2=8.13%, 4=61.25%, 10=11.53%, 20=0.01%
  cpu          : usr=0.30%, sys=0.92%, ctx=186568, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=185997/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=743988KB, aggrb=12398KB/s, minb=12398KB/s, maxb=12398KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=186162/8, merge=0/4, ticks=470968/44, in_queue=470988, util=99.87%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=20514: Sun Dec 15 16:26:34 2019
  write: io=3170.2MB, bw=54082KB/s, iops=13520, runt= 60040msec
    clat (usec): min=1, max=2811.6K, avg=487.71, stdev=17323.71
     lat (usec): min=1, max=2811.6K, avg=489.04, stdev=17329.86
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    4], 95.00th=[    6],
     | 99.00th=[   14], 99.50th=[   31], 99.90th=[126464], 99.95th=[189440],
     | 99.99th=[757760]
    bw (KB  /s): min=    0, max=40279, per=18.07%, avg=9773.52, stdev=7070.57
    lat (usec) : 2=0.05%, 4=78.16%, 10=19.65%, 20=1.52%, 50=0.15%
    lat (usec) : 100=0.05%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.03%, 20=0.04%, 50=0.09%
    lat (msec) : 100=0.07%, 250=0.11%, 500=0.02%, 750=0.01%, 1000=0.01%
    lat (msec) : 2000=0.01%, >=2000=0.01%
  cpu          : usr=0.28%, sys=0.87%, ctx=15613, majf=0, minf=85
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=811771/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=3170.2MB, aggrb=54082KB/s, minb=54082KB/s, maxb=54082KB/s, mint=60040msec, maxt=60040msec

Disk stats (read/write):
  vda: ios=170/84307, merge=0/6212, ticks=592/2584112, in_queue=2586072, util=99.40%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=20501: Sun Dec 15 16:25:34 2019
  read : io=1298.7MB, bw=22162KB/s, iops=5540, runt= 60004msec
    clat (usec): min=136, max=16560, avg=1438.74, stdev=1490.59
     lat (usec): min=137, max=16560, avg=1439.37, stdev=1490.64
    clat percentiles (usec):
     |  1.00th=[  171],  5.00th=[  189], 10.00th=[  203], 20.00th=[  225],
     | 30.00th=[  249], 40.00th=[  286], 50.00th=[  402], 60.00th=[ 1496],
     | 70.00th=[ 2448], 80.00th=[ 2992], 90.00th=[ 3568], 95.00th=[ 4048],
     | 99.00th=[ 5344], 99.50th=[ 6112], 99.90th=[ 7776], 99.95th=[ 8640],
     | 99.99th=[10176]
    bw (KB  /s): min= 1117, max= 8504, per=12.52%, avg=2775.28, stdev=1765.96
    lat (usec) : 250=30.63%, 500=21.52%, 750=3.15%, 1000=1.92%
    lat (msec) : 2=6.75%, 4=30.70%, 10=5.32%, 20=0.01%
  cpu          : usr=0.46%, sys=1.15%, ctx=333216, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=332448/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1298.7MB, aggrb=22161KB/s, minb=22161KB/s, maxb=22161KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=332167/8, merge=0/5, ticks=464228/760, in_queue=464876, util=99.86%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=20527: Sun Dec 15 16:27:36 2019
  write: io=339804KB, bw=5661.7KB/s, iops=1415, runt= 60019msec
    clat (usec): min=66, max=1766.6K, avg=5646.40, stdev=40055.68
     lat (usec): min=66, max=1766.6K, avg=5647.17, stdev=40055.71
    clat percentiles (usec):
     |  1.00th=[   73],  5.00th=[   81], 10.00th=[   86], 20.00th=[   94],
     | 30.00th=[  101], 40.00th=[  107], 50.00th=[  115], 60.00th=[  125],
     | 70.00th=[  143], 80.00th=[  179], 90.00th=[ 1012], 95.00th=[ 6496],
     | 99.00th=[177152], 99.50th=[242688], 99.90th=[528384], 99.95th=[724992],
     | 99.99th=[1105920]
    bw (KB  /s): min=    2, max= 2640, per=13.35%, avg=755.86, stdev=514.57
    lat (usec) : 100=28.39%, 250=57.85%, 500=2.51%, 750=0.63%, 1000=0.59%
    lat (msec) : 2=1.96%, 4=2.08%, 10=1.67%, 20=0.56%, 50=1.20%
    lat (msec) : 100=0.65%, 250=1.47%, 500=0.33%, 750=0.07%, 1000=0.02%
    lat (msec) : 2000=0.02%
  cpu          : usr=0.13%, sys=0.62%, ctx=172753, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=84951/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=339804KB, aggrb=5661KB/s, minb=5661KB/s, maxb=5661KB/s, mint=60019msec, maxt=60019msec

Disk stats (read/write):
  vda: ios=172/84970, merge=0/2006, ticks=544/57484, in_queue=58024, util=94.67%

fio (БША, Атланта)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15476: Sun Dec 15 19:29:14 2019
  read : io=7837.8MB, bw=133749KB/s, iops=33437, runt= 60001msec
    clat (usec): min=1, max=7324, avg=234.11, stdev=141.99
     lat (usec): min=2, max=7325, avg=234.93, stdev=142.02
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  167], 20.00th=[  193],
     | 30.00th=[  207], 40.00th=[  219], 50.00th=[  231], 60.00th=[  245],
     | 70.00th=[  262], 80.00th=[  286], 90.00th=[  322], 95.00th=[  358],
     | 99.00th=[  458], 99.50th=[  556], 99.90th=[ 2352], 99.95th=[ 2864],
     | 99.99th=[ 3888]
    bw (KB  /s): min=11944, max=23232, per=12.49%, avg=16709.43, stdev=1866.94
    lat (usec) : 2=0.01%, 4=6.74%, 10=1.06%, 20=0.17%, 50=0.03%
    lat (usec) : 100=0.01%, 250=55.01%, 500=36.31%, 750=0.36%, 1000=0.09%
    lat (msec) : 2=0.06%, 4=0.16%, 10=0.01%
  cpu          : usr=2.32%, sys=5.36%, ctx=1851110, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2006274/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7837.8MB, aggrb=133749KB/s, minb=133749KB/s, maxb=133749KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1845303/14, merge=0/7, ticks=397380/0, in_queue=397304, util=98.34%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15511: Sun Dec 15 19:31:22 2019
  write: io=26235MB, bw=447631KB/s, iops=111907, runt= 60014msec
    clat (usec): min=1, max=117339, avg=44.70, stdev=861.50
     lat (usec): min=2, max=136215, avg=49.63, stdev=930.64
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[   19], 99.50th=[   40], 99.90th=[14272], 99.95th=[19328],
     | 99.99th=[31104]
    bw (KB  /s): min=    1, max=65736, per=12.63%, avg=56547.13, stdev=5586.94
    lat (usec) : 2=0.01%, 4=68.30%, 10=29.61%, 20=1.32%, 50=0.29%
    lat (usec) : 100=0.05%, 250=0.02%, 500=0.01%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.04%, 10=0.12%, 20=0.13%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.95%, sys=5.87%, ctx=77785, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6716037/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=26235MB, aggrb=447631KB/s, minb=447631KB/s, maxb=447631KB/s, mint=60014msec, maxt=60014msec

Disk stats (read/write):
  vda: ios=192/1075724, merge=0/8974, ticks=768/6927672, in_queue=6933620, util=98.80%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15496: Sun Dec 15 19:30:21 2019
  read : io=7593.5MB, bw=129593KB/s, iops=32398, runt= 60001msec
    clat (usec): min=135, max=6091, avg=241.58, stdev=71.45
     lat (usec): min=135, max=6091, avg=242.43, stdev=71.49
    clat percentiles (usec):
     |  1.00th=[  161],  5.00th=[  175], 10.00th=[  185], 20.00th=[  199],
     | 30.00th=[  209], 40.00th=[  219], 50.00th=[  231], 60.00th=[  243],
     | 70.00th=[  258], 80.00th=[  278], 90.00th=[  310], 95.00th=[  342],
     | 99.00th=[  422], 99.50th=[  462], 99.90th=[  708], 99.95th=[  980],
     | 99.99th=[ 2448]
    bw (KB  /s): min=13104, max=17648, per=12.50%, avg=16200.46, stdev=780.38
    lat (usec) : 250=65.19%, 500=34.51%, 750=0.22%, 1000=0.04%
    lat (msec) : 2=0.03%, 4=0.01%, 10=0.01%
  cpu          : usr=2.51%, sys=4.83%, ctx=1947114, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1943932/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7593.5MB, aggrb=129593KB/s, minb=129593KB/s, maxb=129593KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1937903/10, merge=0/7, ticks=409816/0, in_queue=409684, util=99.82%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15524: Sun Dec 15 19:32:22 2019
  write: io=2278.9MB, bw=38891KB/s, iops=9722, runt= 60003msec
    clat (usec): min=60, max=51553, avg=817.32, stdev=4886.56
     lat (usec): min=60, max=51554, avg=818.22, stdev=4886.59
    clat percentiles (usec):
     |  1.00th=[   66],  5.00th=[   68], 10.00th=[   70], 20.00th=[   76],
     | 30.00th=[   81], 40.00th=[   85], 50.00th=[   91], 60.00th=[   96],
     | 70.00th=[  102], 80.00th=[  112], 90.00th=[  139], 95.00th=[  181],
     | 99.00th=[34048], 99.50th=[36096], 99.90th=[38656], 99.95th=[39680],
     | 99.99th=[41728]
    bw (KB  /s): min= 3460, max= 5968, per=12.51%, avg=4863.46, stdev=404.00
    lat (usec) : 100=65.85%, 250=31.55%, 500=0.42%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.14%
    lat (msec) : 100=0.01%
  cpu          : usr=0.78%, sys=2.71%, ctx=1179671, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=583392/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2278.9MB, aggrb=38890KB/s, minb=38890KB/s, maxb=38890KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=612/582769, merge=0/1520, ticks=392/47444, in_queue=47752, util=78.96%

fio (БША, Чикаго)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=14831: Mon Dec 16 01:33:33 2019
  read : io=8035.5MB, bw=137136KB/s, iops=34284, runt= 60001msec
    clat (usec): min=1, max=4262, avg=227.92, stdev=133.01
     lat (usec): min=2, max=4267, avg=228.79, stdev=133.05
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[  165], 20.00th=[  189],
     | 30.00th=[  201], 40.00th=[  213], 50.00th=[  225], 60.00th=[  239],
     | 70.00th=[  255], 80.00th=[  278], 90.00th=[  314], 95.00th=[  350],
     | 99.00th=[  458], 99.50th=[  548], 99.90th=[ 2256], 99.95th=[ 2768],
     | 99.99th=[ 3408]
    bw (KB  /s): min=13152, max=23400, per=12.49%, avg=17131.96, stdev=2002.66
    lat (usec) : 2=0.01%, 4=6.94%, 10=0.98%, 20=0.21%, 50=0.05%
    lat (usec) : 100=0.01%, 250=58.57%, 500=32.57%, 750=0.41%, 1000=0.06%
    lat (msec) : 2=0.05%, 4=0.15%, 10=0.01%
  cpu          : usr=2.58%, sys=5.57%, ctx=1895150, majf=2, minf=94
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2057075/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8035.5MB, aggrb=137136KB/s, minb=137136KB/s, maxb=137136KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1884570/9, merge=0/4, ticks=377920/0, in_queue=377704, util=98.11%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=14858: Mon Dec 16 01:35:41 2019
  write: io=26627MB, bw=454312KB/s, iops=113578, runt= 60016msec
    clat (usec): min=1, max=140433, avg=44.25, stdev=841.09
     lat (usec): min=2, max=153527, avg=48.93, stdev=907.33
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[   19], 99.50th=[   38], 99.90th=[14144], 99.95th=[19584],
     | 99.99th=[30592]
    bw (KB  /s): min=    1, max=67693, per=12.64%, avg=57424.28, stdev=5407.98
    lat (usec) : 2=0.01%, 4=70.71%, 10=26.77%, 20=1.74%, 50=0.30%
    lat (usec) : 100=0.04%, 250=0.02%, 500=0.02%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.04%, 10=0.12%, 20=0.13%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.94%, sys=5.93%, ctx=79995, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6816502/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=26627MB, aggrb=454312KB/s, minb=454312KB/s, maxb=454312KB/s, mint=60016msec, maxt=60016msec

Disk stats (read/write):
  vda: ios=499/1074165, merge=0/9300, ticks=2744/6945264, in_queue=6948456, util=98.75%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=14845: Mon Dec 16 01:34:41 2019
  read : io=7746.9MB, bw=132208KB/s, iops=33052, runt= 60002msec
    clat (usec): min=132, max=31092, avg=236.79, stdev=90.81
     lat (usec): min=132, max=31092, avg=237.63, stdev=90.84
    clat percentiles (usec):
     |  1.00th=[  157],  5.00th=[  171], 10.00th=[  181], 20.00th=[  193],
     | 30.00th=[  205], 40.00th=[  215], 50.00th=[  225], 60.00th=[  237],
     | 70.00th=[  253], 80.00th=[  274], 90.00th=[  306], 95.00th=[  338],
     | 99.00th=[  414], 99.50th=[  462], 99.90th=[  708], 99.95th=[  924],
     | 99.99th=[ 3376]
    bw (KB  /s): min=13992, max=17600, per=12.50%, avg=16531.33, stdev=620.44
    lat (usec) : 250=68.62%, 500=31.05%, 750=0.25%, 1000=0.04%
    lat (msec) : 2=0.03%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=2.35%, sys=4.85%, ctx=1987876, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1983189/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7746.9MB, aggrb=132208KB/s, minb=132208KB/s, maxb=132208KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1977990/8, merge=0/5, ticks=408236/0, in_queue=407928, util=99.82%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=14873: Mon Dec 16 01:36:42 2019
  write: io=2261.6MB, bw=38594KB/s, iops=9648, runt= 60004msec
    clat (usec): min=61, max=44709, avg=823.69, stdev=4921.76
     lat (usec): min=61, max=44709, avg=824.57, stdev=4921.78
    clat percentiles (usec):
     |  1.00th=[   67],  5.00th=[   69], 10.00th=[   72], 20.00th=[   77],
     | 30.00th=[   82], 40.00th=[   86], 50.00th=[   91], 60.00th=[   96],
     | 70.00th=[  102], 80.00th=[  112], 90.00th=[  141], 95.00th=[  185],
     | 99.00th=[34560], 99.50th=[36096], 99.90th=[38144], 99.95th=[39168],
     | 99.99th=[40704]
    bw (KB  /s): min= 3547, max= 5952, per=12.50%, avg=4824.65, stdev=445.14
    lat (usec) : 100=65.60%, 250=31.76%, 500=0.46%, 750=0.03%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.14%
  cpu          : usr=0.85%, sys=2.61%, ctx=1170628, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=578946/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2261.6MB, aggrb=38593KB/s, minb=38593KB/s, maxb=38593KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=169/578268, merge=0/1640, ticks=164/47368, in_queue=47436, util=78.80%

fio (БША, Даллас)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30640: Mon Dec 16 01:36:46 2019
  read : io=8101.4MB, bw=138255KB/s, iops=34563, runt= 60001msec
    clat (usec): min=1, max=6512, avg=227.73, stdev=127.95
     lat (usec): min=1, max=6513, avg=228.26, stdev=127.96
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[  171], 20.00th=[  197],
     | 30.00th=[  209], 40.00th=[  221], 50.00th=[  231], 60.00th=[  243],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  298], 95.00th=[  326],
     | 99.00th=[  402], 99.50th=[  478], 99.90th=[ 2160], 99.95th=[ 2576],
     | 99.99th=[ 3280]
    bw (KB  /s): min=13720, max=24296, per=12.49%, avg=17270.28, stdev=1840.54
    lat (usec) : 2=0.19%, 4=7.28%, 10=0.68%, 20=0.10%, 50=0.01%
    lat (usec) : 100=0.01%, 250=57.33%, 500=33.96%, 750=0.17%, 1000=0.03%
    lat (msec) : 2=0.10%, 4=0.14%, 10=0.01%
  cpu          : usr=1.64%, sys=5.37%, ctx=1906830, majf=3, minf=93
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2073857/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8101.4MB, aggrb=138254KB/s, minb=138254KB/s, maxb=138254KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1898815/11, merge=0/3, ticks=429144/0, in_queue=428844, util=98.21%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30667: Mon Dec 16 01:38:53 2019
  write: io=37120MB, bw=633470KB/s, iops=158367, runt= 60005msec
    clat (usec): min=1, max=268949, avg=35.49, stdev=825.71
     lat (usec): min=1, max=268950, avg=38.11, stdev=871.35
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    4],
     | 99.00th=[    7], 99.50th=[   14], 99.90th=[12992], 99.95th=[18304],
     | 99.99th=[30080]
    bw (KB  /s): min=    0, max=113093, per=12.76%, avg=80836.06, stdev=9692.35
    lat (usec) : 2=0.17%, 4=90.19%, 10=8.87%, 20=0.40%, 50=0.04%
    lat (usec) : 100=0.03%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.02%, 10=0.08%, 20=0.10%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=2.99%, sys=7.69%, ctx=71565, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9502839/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=37120MB, aggrb=633469KB/s, minb=633469KB/s, maxb=633469KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  vda: ios=178/767585, merge=0/7960, ticks=1388/6914052, in_queue=6915356, util=98.99%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30654: Mon Dec 16 01:37:53 2019
  read : io=7669.1MB, bw=130896KB/s, iops=32724, runt= 60002msec
    clat (usec): min=138, max=4498, avg=240.61, stdev=71.44
     lat (usec): min=139, max=4499, avg=241.16, stdev=71.45
    clat percentiles (usec):
     |  1.00th=[  165],  5.00th=[  179], 10.00th=[  189], 20.00th=[  203],
     | 30.00th=[  213], 40.00th=[  223], 50.00th=[  233], 60.00th=[  243],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  298], 95.00th=[  322],
     | 99.00th=[  386], 99.50th=[  422], 99.90th=[  668], 99.95th=[ 1240],
     | 99.99th=[ 3408]
    bw (KB  /s): min=14360, max=17592, per=12.51%, avg=16370.02, stdev=521.56
    lat (usec) : 250=65.23%, 500=34.55%, 750=0.13%, 1000=0.02%
    lat (msec) : 2=0.04%, 4=0.02%, 10=0.01%
  cpu          : usr=1.77%, sys=4.75%, ctx=1966567, majf=0, minf=72
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1963511/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7669.1MB, aggrb=130896KB/s, minb=130896KB/s, maxb=130896KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=1959684/10, merge=0/4, ticks=400752/0, in_queue=400488, util=99.80%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30680: Mon Dec 16 01:39:54 2019
  write: io=2169.8MB, bw=37028KB/s, iops=9256, runt= 60005msec
    clat (usec): min=64, max=65768, avg=859.93, stdev=5258.49
     lat (usec): min=64, max=65768, avg=860.55, stdev=5258.50
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   74], 10.00th=[   78], 20.00th=[   84],
     | 30.00th=[   89], 40.00th=[   94], 50.00th=[   99], 60.00th=[  104],
     | 70.00th=[  110], 80.00th=[  119], 90.00th=[  139], 95.00th=[  165],
     | 99.00th=[37632], 99.50th=[39680], 99.90th=[42240], 99.95th=[42752],
     | 99.99th=[44288]
    bw (KB  /s): min= 2715, max= 6448, per=12.51%, avg=4630.24, stdev=529.55
    lat (usec) : 100=51.63%, 250=45.89%, 500=0.36%, 750=0.03%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.04%
    lat (msec) : 100=0.01%
  cpu          : usr=0.56%, sys=2.64%, ctx=1122771, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=555463/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2169.8MB, aggrb=37027KB/s, minb=37027KB/s, maxb=37027KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  vda: ios=170/554923, merge=0/1544, ticks=80/48648, in_queue=48628, util=80.92%

fio (БША, Лос-АндТСлСс)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30302: Mon Dec 16 01:40:53 2019
  read : io=4527.3MB, bw=77216KB/s, iops=19304, runt= 60038msec
    clat (usec): min=2, max=1002.6K, avg=409.03, stdev=3749.04
     lat (usec): min=2, max=1002.6K, avg=409.87, stdev=3749.05
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    7], 10.00th=[  177], 20.00th=[  195],
     | 30.00th=[  209], 40.00th=[  219], 50.00th=[  233], 60.00th=[  247],
     | 70.00th=[  266], 80.00th=[  298], 90.00th=[  354], 95.00th=[  426],
     | 99.00th=[ 1736], 99.50th=[ 6176], 99.90th=[41728], 99.95th=[44288],
     | 99.99th=[55040]
    bw (KB  /s): min=    4, max=21176, per=12.78%, avg=9869.13, stdev=6950.72
    lat (usec) : 4=3.86%, 10=1.28%, 20=0.21%, 50=0.02%, 100=0.01%
    lat (usec) : 250=56.16%, 500=34.99%, 750=1.52%, 1000=0.64%
    lat (msec) : 2=0.37%, 4=0.23%, 10=0.38%, 20=0.04%, 50=0.27%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2000=0.01%
  cpu          : usr=1.49%, sys=3.24%, ctx=1100743, majf=0, minf=84
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1158981/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4527.3MB, aggrb=77216KB/s, minb=77216KB/s, maxb=77216KB/s, mint=60038msec, maxt=60038msec

Disk stats (read/write):
  vda: ios=1096289/9, merge=0/4, ticks=432752/56, in_queue=432560, util=99.17%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30328: Mon Dec 16 01:43:06 2019
  write: io=27193MB, bw=464019KB/s, iops=116004, runt= 60009msec
    clat (usec): min=1, max=176038, avg=43.66, stdev=914.60
     lat (usec): min=2, max=200012, avg=48.16, stdev=982.18
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[   15], 99.50th=[   23], 99.90th=[14912], 99.95th=[20096],
     | 99.99th=[32128]
    bw (KB  /s): min=    1, max=72232, per=12.65%, avg=58689.60, stdev=6954.65
    lat (usec) : 2=0.01%, 4=71.10%, 10=27.12%, 20=1.20%, 50=0.15%
    lat (usec) : 100=0.04%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.03%, 10=0.10%, 20=0.12%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.91%, sys=5.92%, ctx=71147, majf=0, minf=60
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6961325/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=27193MB, aggrb=464018KB/s, minb=464018KB/s, maxb=464018KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  vda: ios=174/804965, merge=0/9109, ticks=844/6904508, in_queue=6906288, util=98.78%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30315: Mon Dec 16 01:42:06 2019
  read : io=4759.4MB, bw=81225KB/s, iops=20306, runt= 60001msec
    clat (usec): min=134, max=83824, avg=388.59, stdev=2280.97
     lat (usec): min=134, max=83825, avg=389.43, stdev=2280.98
    clat percentiles (usec):
     |  1.00th=[  161],  5.00th=[  175], 10.00th=[  185], 20.00th=[  199],
     | 30.00th=[  211], 40.00th=[  221], 50.00th=[  233], 60.00th=[  247],
     | 70.00th=[  266], 80.00th=[  294], 90.00th=[  342], 95.00th=[  394],
     | 99.00th=[  708], 99.50th=[ 3728], 99.90th=[43776], 99.95th=[46848],
     | 99.99th=[55552]
    bw (KB  /s): min=   77, max=16936, per=12.65%, avg=10274.83, stdev=6866.05
    lat (usec) : 250=61.27%, 500=36.86%, 750=0.93%, 1000=0.16%
    lat (msec) : 2=0.20%, 4=0.10%, 10=0.17%, 20=0.03%, 50=0.27%
    lat (msec) : 100=0.02%
  cpu          : usr=1.63%, sys=3.04%, ctx=1220916, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1218402/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4759.4MB, aggrb=81225KB/s, minb=81225KB/s, maxb=81225KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1216358/8, merge=0/5, ticks=429456/96, in_queue=429404, util=99.85%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30341: Mon Dec 16 01:44:07 2019
  write: io=1957.9MB, bw=33412KB/s, iops=8352, runt= 60003msec
    clat (usec): min=60, max=282347, avg=952.06, stdev=5702.32
     lat (usec): min=61, max=282348, avg=952.98, stdev=5702.34
    clat percentiles (usec):
     |  1.00th=[   68],  5.00th=[   72], 10.00th=[   76], 20.00th=[   83],
     | 30.00th=[   88], 40.00th=[   93], 50.00th=[   98], 60.00th=[  104],
     | 70.00th=[  111], 80.00th=[  125], 90.00th=[  177], 95.00th=[  243],
     | 99.00th=[36096], 99.50th=[38144], 99.90th=[43264], 99.95th=[46848],
     | 99.99th=[112128]
    bw (KB  /s): min= 1125, max= 5928, per=12.51%, avg=4180.03, stdev=644.04
    lat (usec) : 100=52.16%, 250=43.18%, 500=2.04%, 750=0.12%, 1000=0.05%
    lat (msec) : 2=0.07%, 4=0.04%, 10=0.02%, 20=0.01%, 50=2.29%
    lat (msec) : 100=0.03%, 250=0.01%, 500=0.01%
  cpu          : usr=0.74%, sys=2.51%, ctx=1014281, majf=0, minf=68
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=501203/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1957.9MB, aggrb=33411KB/s, minb=33411KB/s, maxb=33411KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=173/500597, merge=0/1694, ticks=236/48308, in_queue=48456, util=80.38%

fio (БША, Майами)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29120: Mon Dec 16 01:44:27 2019
  read : io=9084.3MB, bw=155035KB/s, iops=38758, runt= 60001msec
    clat (usec): min=1, max=4963, avg=202.89, stdev=119.18
     lat (usec): min=1, max=4964, avg=203.40, stdev=119.20
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[  153], 20.00th=[  177],
     | 30.00th=[  189], 40.00th=[  199], 50.00th=[  209], 60.00th=[  217],
     | 70.00th=[  229], 80.00th=[  243], 90.00th=[  266], 95.00th=[  290],
     | 99.00th=[  350], 99.50th=[  394], 99.90th=[ 2064], 99.95th=[ 2416],
     | 99.99th=[ 3216]
    bw (KB  /s): min=15200, max=26664, per=12.50%, avg=19375.82, stdev=2121.46
    lat (usec) : 2=0.37%, 4=8.07%, 10=0.62%, 20=0.14%, 50=0.03%
    lat (usec) : 100=0.01%, 250=74.10%, 500=16.39%, 750=0.05%, 1000=0.02%
    lat (msec) : 2=0.10%, 4=0.12%, 10=0.01%
  cpu          : usr=1.75%, sys=5.59%, ctx=2120348, majf=2, minf=85
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2325570/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=9084.3MB, aggrb=155035KB/s, minb=155035KB/s, maxb=155035KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2106654/9, merge=0/4, ticks=407152/0, in_queue=406904, util=98.14%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29147: Mon Dec 16 01:46:35 2019
  write: io=36107MB, bw=616162KB/s, iops=154040, runt= 60007msec
    clat (usec): min=1, max=249081, avg=36.39, stdev=848.32
     lat (usec): min=1, max=249082, avg=39.14, stdev=895.29
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    4],
     | 99.00th=[   17], 99.50th=[   19], 99.90th=[13248], 99.95th=[19072],
     | 99.99th=[31872]
    bw (KB  /s): min=    1, max=116880, per=12.62%, avg=77787.91, stdev=9285.64
    lat (usec) : 2=0.15%, 4=92.02%, 10=6.23%, 20=1.17%, 50=0.11%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.02%, 10=0.08%, 20=0.10%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=3.13%, sys=7.64%, ctx=72362, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9243503/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=36107MB, aggrb=616161KB/s, minb=616161KB/s, maxb=616161KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  vda: ios=175/658814, merge=0/8609, ticks=1444/6946772, in_queue=6949828, util=99.24%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29134: Mon Dec 16 01:45:35 2019
  read : io=8421.3MB, bw=143720KB/s, iops=35929, runt= 60001msec
    clat (usec): min=131, max=5964, avg=218.95, stdev=56.93
     lat (usec): min=132, max=5964, avg=219.50, stdev=56.95
    clat percentiles (usec):
     |  1.00th=[  155],  5.00th=[  165], 10.00th=[  173], 20.00th=[  185],
     | 30.00th=[  195], 40.00th=[  203], 50.00th=[  211], 60.00th=[  221],
     | 70.00th=[  233], 80.00th=[  247], 90.00th=[  270], 95.00th=[  294],
     | 99.00th=[  350], 99.50th=[  378], 99.90th=[  596], 99.95th=[  948],
     | 99.99th=[ 2128]
    bw (KB  /s): min=14576, max=19592, per=12.50%, avg=17966.93, stdev=957.49
    lat (usec) : 250=81.46%, 500=18.40%, 750=0.07%, 1000=0.02%
    lat (msec) : 2=0.03%, 4=0.01%, 10=0.01%
  cpu          : usr=1.80%, sys=5.00%, ctx=2158860, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2155832/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8421.3MB, aggrb=143719KB/s, minb=143719KB/s, maxb=143719KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=2154518/8, merge=0/5, ticks=426160/0, in_queue=425888, util=99.86%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29160: Mon Dec 16 01:47:35 2019
  write: io=2524.6MB, bw=43084KB/s, iops=10770, runt= 60003msec
    clat (usec): min=59, max=66717, avg=738.95, stdev=4867.38
     lat (usec): min=59, max=66717, avg=739.53, stdev=4867.39
    clat percentiles (usec):
     |  1.00th=[   65],  5.00th=[   67], 10.00th=[   69], 20.00th=[   72],
     | 30.00th=[   76], 40.00th=[   80], 50.00th=[   84], 60.00th=[   89],
     | 70.00th=[   94], 80.00th=[  102], 90.00th=[  117], 95.00th=[  137],
     | 99.00th=[37120], 99.50th=[39168], 99.90th=[41728], 99.95th=[42752],
     | 99.99th=[44288]
    bw (KB  /s): min= 3867, max= 6800, per=12.50%, avg=5386.89, stdev=586.21
    lat (usec) : 100=77.20%, 250=20.84%, 500=0.17%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.77%
    lat (msec) : 100=0.01%
  cpu          : usr=0.60%, sys=2.64%, ctx=1304520, majf=0, minf=58
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=646286/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2524.6MB, aggrb=43083KB/s, minb=43083KB/s, maxb=43083KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=169/645485, merge=0/1412, ticks=124/48104, in_queue=48064, util=79.91%

fio (БША, Нью-Π™ΠΎΡ€ΠΊ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31460: Mon Dec 16 01:24:50 2019
  read : io=7908.6MB, bw=134971KB/s, iops=33742, runt= 60001msec
    clat (usec): min=2, max=12146, avg=231.04, stdev=161.17
     lat (usec): min=2, max=12147, avg=231.97, stdev=161.24
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    8], 10.00th=[  157], 20.00th=[  179],
     | 30.00th=[  193], 40.00th=[  207], 50.00th=[  221], 60.00th=[  237],
     | 70.00th=[  258], 80.00th=[  286], 90.00th=[  334], 95.00th=[  382],
     | 99.00th=[  524], 99.50th=[  620], 99.90th=[ 2384], 99.95th=[ 2960],
     | 99.99th=[ 4448]
    bw (KB  /s): min=13384, max=22712, per=12.51%, avg=16881.32, stdev=1712.99
    lat (usec) : 4=3.51%, 10=2.33%, 20=1.79%, 50=0.38%, 100=0.01%
    lat (usec) : 250=58.30%, 500=32.43%, 750=0.94%, 1000=0.08%
    lat (msec) : 2=0.05%, 4=0.16%, 10=0.01%, 20=0.01%
  cpu          : usr=2.97%, sys=7.95%, ctx=1868322, majf=2, minf=102
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2024594/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7908.6MB, aggrb=134970KB/s, minb=134970KB/s, maxb=134970KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1855108/9, merge=0/4, ticks=324384/0, in_queue=323904, util=98.33%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31487: Mon Dec 16 01:26:56 2019
  write: io=23284MB, bw=397335KB/s, iops=99333, runt= 60007msec
    clat (usec): min=1, max=128077, avg=54.48, stdev=877.46
     lat (usec): min=2, max=128078, avg=59.22, stdev=936.71
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    7], 95.00th=[   10],
     | 99.00th=[   25], 99.50th=[  716], 99.90th=[15680], 99.95th=[19840],
     | 99.99th=[28288]
    bw (KB  /s): min=    1, max=63112, per=12.66%, avg=50293.42, stdev=7068.36
    lat (usec) : 2=0.01%, 4=61.17%, 10=33.34%, 20=4.16%, 50=0.63%
    lat (usec) : 100=0.10%, 250=0.04%, 500=0.03%, 750=0.03%, 1000=0.01%
    lat (msec) : 2=0.04%, 4=0.06%, 10=0.17%, 20=0.16%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.45%, sys=6.10%, ctx=95599, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5960722/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=23284MB, aggrb=397335KB/s, minb=397335KB/s, maxb=397335KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  vda: ios=174/1557066, merge=0/10682, ticks=160/6476912, in_queue=6477100, util=94.01%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31474: Mon Dec 16 01:25:56 2019
  read : io=8180.5MB, bw=139606KB/s, iops=34901, runt= 60003msec
    clat (usec): min=118, max=10289, avg=223.40, stdev=102.27
     lat (usec): min=119, max=10289, avg=224.29, stdev=102.35
    clat percentiles (usec):
     |  1.00th=[  145],  5.00th=[  157], 10.00th=[  165], 20.00th=[  177],
     | 30.00th=[  187], 40.00th=[  197], 50.00th=[  209], 60.00th=[  221],
     | 70.00th=[  237], 80.00th=[  258], 90.00th=[  294], 95.00th=[  330],
     | 99.00th=[  422], 99.50th=[  478], 99.90th=[  748], 99.95th=[ 1176],
     | 99.99th=[ 5344]
    bw (KB  /s): min=14192, max=19104, per=12.50%, avg=17452.85, stdev=775.78
    lat (usec) : 250=76.22%, 500=23.39%, 750=0.30%, 1000=0.04%
    lat (msec) : 2=0.03%, 4=0.01%, 10=0.02%, 20=0.01%
  cpu          : usr=3.12%, sys=6.80%, ctx=2099536, majf=0, minf=65
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2094191/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8180.5MB, aggrb=139605KB/s, minb=139605KB/s, maxb=139605KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=2090729/8, merge=0/5, ticks=366884/0, in_queue=366540, util=99.80%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=31500: Mon Dec 16 01:27:57 2019
  write: io=2163.8MB, bw=36926KB/s, iops=9231, runt= 60004msec
    clat (usec): min=66, max=48157, avg=860.52, stdev=4642.86
     lat (usec): min=66, max=48158, avg=861.51, stdev=4642.89
    clat percentiles (usec):
     |  1.00th=[   72],  5.00th=[   75], 10.00th=[   77], 20.00th=[   82],
     | 30.00th=[   87], 40.00th=[   92], 50.00th=[   97], 60.00th=[  103],
     | 70.00th=[  110], 80.00th=[  119], 90.00th=[  141], 95.00th=[  173],
     | 99.00th=[30080], 99.50th=[31104], 99.90th=[33024], 99.95th=[34048],
     | 99.99th=[37120]
    bw (KB  /s): min= 3373, max= 5824, per=12.51%, avg=4618.34, stdev=393.71
    lat (usec) : 100=53.98%, 250=43.09%, 500=0.28%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.61%
  cpu          : usr=0.93%, sys=3.20%, ctx=1122973, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=553923/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2163.8MB, aggrb=36925KB/s, minb=36925KB/s, maxb=36925KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  vda: ios=171/553407, merge=0/1598, ticks=188/44808, in_queue=44824, util=74.39%

fio (БША, Биэтл)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30897: Mon Dec 16 01:48:04 2019
  read : io=8540.3MB, bw=145752KB/s, iops=36437, runt= 60001msec
    clat (usec): min=1, max=15745, avg=214.05, stdev=185.39
     lat (usec): min=2, max=15745, avg=214.92, stdev=185.76
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    8], 10.00th=[  153], 20.00th=[  175],
     | 30.00th=[  187], 40.00th=[  195], 50.00th=[  205], 60.00th=[  217],
     | 70.00th=[  231], 80.00th=[  249], 90.00th=[  286], 95.00th=[  334],
     | 99.00th=[  510], 99.50th=[ 1004], 99.90th=[ 2544], 99.95th=[ 2832],
     | 99.99th=[ 4256]
    bw (KB  /s): min=13048, max=24792, per=12.49%, avg=18199.83, stdev=1860.18
    lat (usec) : 2=0.01%, 4=2.96%, 10=3.83%, 20=2.03%, 50=0.28%
    lat (usec) : 100=0.01%, 250=71.01%, 500=18.85%, 750=0.43%, 1000=0.11%
    lat (msec) : 2=0.23%, 4=0.26%, 10=0.01%, 20=0.01%
  cpu          : usr=3.03%, sys=7.68%, ctx=1995422, majf=2, minf=87
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2186315/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8540.3MB, aggrb=145751KB/s, minb=145751KB/s, maxb=145751KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1985256/10, merge=0/3, ticks=350132/0, in_queue=350100, util=98.15%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30924: Mon Dec 16 01:50:10 2019
  write: io=25968MB, bw=443099KB/s, iops=110774, runt= 60012msec
    clat (usec): min=1, max=99789, avg=47.62, stdev=816.45
     lat (usec): min=2, max=108292, avg=52.13, stdev=875.35
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    5], 95.00th=[    7],
     | 99.00th=[   16], 99.50th=[  159], 99.90th=[14528], 99.95th=[18816],
     | 99.99th=[27520]
    bw (KB  /s): min=    1, max=66464, per=12.65%, avg=56037.47, stdev=5979.41
    lat (usec) : 2=0.01%, 4=71.13%, 10=25.78%, 20=2.32%, 50=0.18%
    lat (usec) : 100=0.07%, 250=0.03%, 500=0.02%, 750=0.03%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.06%, 10=0.15%, 20=0.15%, 50=0.04%
    lat (msec) : 100=0.01%
  cpu          : usr=4.66%, sys=5.89%, ctx=94569, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=6647820/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=25968MB, aggrb=443099KB/s, minb=443099KB/s, maxb=443099KB/s, mint=60012msec, maxt=60012msec

Disk stats (read/write):
  vda: ios=169/1692956, merge=0/10324, ticks=260/6620580, in_queue=6621932, util=96.39%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30911: Mon Dec 16 01:49:09 2019
  read : io=8739.6MB, bw=149150KB/s, iops=37287, runt= 60002msec
    clat (usec): min=121, max=13671, avg=209.06, stdev=70.91
     lat (usec): min=122, max=13672, avg=209.89, stdev=70.97
    clat percentiles (usec):
     |  1.00th=[  147],  5.00th=[  159], 10.00th=[  167], 20.00th=[  177],
     | 30.00th=[  185], 40.00th=[  191], 50.00th=[  199], 60.00th=[  209],
     | 70.00th=[  219], 80.00th=[  235], 90.00th=[  262], 95.00th=[  290],
     | 99.00th=[  362], 99.50th=[  398], 99.90th=[  524], 99.95th=[  700],
     | 99.99th=[ 2192]
    bw (KB  /s): min=16304, max=19560, per=12.50%, avg=18649.24, stdev=461.35
    lat (usec) : 250=86.74%, 500=13.13%, 750=0.08%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=3.19%, sys=6.68%, ctx=2243199, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2237318/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8739.6MB, aggrb=149149KB/s, minb=149149KB/s, maxb=149149KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=2229705/8, merge=0/5, ticks=387068/0, in_queue=387048, util=99.89%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30937: Mon Dec 16 01:51:10 2019
  write: io=2329.5MB, bw=39753KB/s, iops=9938, runt= 60006msec
    clat (usec): min=67, max=47499, avg=799.36, stdev=4594.63
     lat (usec): min=68, max=47500, avg=800.27, stdev=4594.65
    clat percentiles (usec):
     |  1.00th=[   74],  5.00th=[   76], 10.00th=[   77], 20.00th=[   80],
     | 30.00th=[   84], 40.00th=[   88], 50.00th=[   92], 60.00th=[   96],
     | 70.00th=[  101], 80.00th=[  109], 90.00th=[  121], 95.00th=[  137],
     | 99.00th=[31360], 99.50th=[32640], 99.90th=[34560], 99.95th=[35584],
     | 99.99th=[38144]
    bw (KB  /s): min= 3671, max= 5968, per=12.51%, avg=4972.47, stdev=355.85
    lat (usec) : 100=66.85%, 250=30.72%, 500=0.10%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=2.31%
  cpu          : usr=1.01%, sys=2.93%, ctx=1207149, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=596349/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2329.5MB, aggrb=39752KB/s, minb=39752KB/s, maxb=39752KB/s, mint=60006msec, maxt=60006msec

Disk stats (read/write):
  vda: ios=173/595676, merge=0/1579, ticks=128/45208, in_queue=45220, util=75.16%

fio (БША, Π‘Π°Π½-Π₯осС)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22097: Mon Dec 16 03:10:03 2019
  read : io=8602.3MB, bw=146810KB/s, iops=36702, runt= 60001msec
    clat (usec): min=1, max=37853, avg=214.28, stdev=166.22
     lat (usec): min=1, max=37853, avg=214.82, stdev=166.22
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[  157], 20.00th=[  181],
     | 30.00th=[  193], 40.00th=[  203], 50.00th=[  213], 60.00th=[  223],
     | 70.00th=[  237], 80.00th=[  255], 90.00th=[  286], 95.00th=[  318],
     | 99.00th=[  442], 99.50th=[  676], 99.90th=[ 2352], 99.95th=[ 2768],
     | 99.99th=[ 4048]
    bw (KB  /s): min=12072, max=25480, per=12.50%, avg=18347.99, stdev=2080.92
    lat (usec) : 2=0.21%, 4=7.02%, 10=1.23%, 20=0.58%, 50=0.04%
    lat (usec) : 100=0.01%, 250=68.73%, 500=21.43%, 750=0.29%, 1000=0.11%
    lat (msec) : 2=0.17%, 4=0.17%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=1.79%, sys=5.69%, ctx=2009851, majf=2, minf=95
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2202186/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8602.3MB, aggrb=146809KB/s, minb=146809KB/s, maxb=146809KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  vda: ios=1997992/9, merge=0/4, ticks=411600/0, in_queue=411320, util=98.32%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22123: Mon Dec 16 03:12:12 2019
  write: io=35829MB, bw=610865KB/s, iops=152716, runt= 60061msec
    clat (usec): min=1, max=192241, avg=38.25, stdev=985.47
     lat (usec): min=1, max=270211, avg=40.80, stdev=1029.68
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    4],
     | 99.00th=[   10], 99.50th=[   18], 99.90th=[13376], 99.95th=[19584],
     | 99.99th=[35584]
    bw (KB  /s): min=    1, max=97767, per=12.65%, avg=77295.31, stdev=10675.70
    lat (usec) : 2=0.22%, 4=92.18%, 10=6.54%, 20=0.68%, 50=0.08%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.02%, 4=0.02%, 10=0.07%, 20=0.10%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.80%, sys=7.39%, ctx=68251, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9172295/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=35829MB, aggrb=610865KB/s, minb=610865KB/s, maxb=610865KB/s, mint=60061msec, maxt=60061msec

Disk stats (read/write):
  vda: ios=172/642212, merge=0/8314, ticks=1048/6914268, in_queue=6915840, util=99.12%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22110: Mon Dec 16 03:11:12 2019
  read : io=8145.5MB, bw=139011KB/s, iops=34752, runt= 60002msec
    clat (usec): min=131, max=6165, avg=226.49, stdev=78.22
     lat (usec): min=132, max=6166, avg=227.00, stdev=78.23
    clat percentiles (usec):
     |  1.00th=[  155],  5.00th=[  167], 10.00th=[  177], 20.00th=[  189],
     | 30.00th=[  197], 40.00th=[  207], 50.00th=[  215], 60.00th=[  227],
     | 70.00th=[  239], 80.00th=[  255], 90.00th=[  282], 95.00th=[  310],
     | 99.00th=[  394], 99.50th=[  454], 99.90th=[ 1160], 99.95th=[ 1624],
     | 99.99th=[ 3152]
    bw (KB  /s): min=14064, max=18608, per=12.50%, avg=17378.33, stdev=844.92
    lat (usec) : 250=77.18%, 500=22.45%, 750=0.20%, 1000=0.05%
    lat (msec) : 2=0.10%, 4=0.02%, 10=0.01%
  cpu          : usr=1.73%, sys=4.97%, ctx=2088154, majf=0, minf=72
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2085239/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=8145.5MB, aggrb=139011KB/s, minb=139011KB/s, maxb=139011KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  vda: ios=2080813/8, merge=0/5, ticks=428548/0, in_queue=428396, util=99.87%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=22136: Mon Dec 16 03:13:12 2019
  write: io=2439.5MB, bw=41632KB/s, iops=10407, runt= 60003msec
    clat (usec): min=60, max=52245, avg=764.84, stdev=4993.14
     lat (usec): min=61, max=52245, avg=765.41, stdev=4993.15
    clat percentiles (usec):
     |  1.00th=[   66],  5.00th=[   68], 10.00th=[   70], 20.00th=[   74],
     | 30.00th=[   78], 40.00th=[   82], 50.00th=[   87], 60.00th=[   91],
     | 70.00th=[   97], 80.00th=[  105], 90.00th=[  123], 95.00th=[  149],
     | 99.00th=[37632], 99.50th=[40192], 99.90th=[42752], 99.95th=[43264],
     | 99.99th=[45312]
    bw (KB  /s): min= 3512, max= 6645, per=12.50%, avg=5204.98, stdev=587.61
    lat (usec) : 100=74.30%, 250=23.54%, 500=0.30%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.80%
    lat (msec) : 100=0.01%
  cpu          : usr=0.57%, sys=2.59%, ctx=1260736, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=624507/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2439.5MB, aggrb=41631KB/s, minb=41631KB/s, maxb=41631KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  vda: ios=174/623749, merge=0/1473, ticks=232/48604, in_queue=48728, util=80.83%

Speedtest (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (139.180.167.158)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Exetel (Brisbane) [0.59 km]: 15.529 ms
Testing download speed................................................................................
Download: 563.62 Mbit/s
Testing upload speed................................................................................................
Upload: 443.93 Mbit/s

Speedtest (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (155.138.134.23)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by FibreStream (Toronto, ON) [3.45 km]: 1.132 ms
Testing download speed................................................................................
Download: 5165.30 Mbit/s
Testing upload speed................................................................................................
Upload: 3611.44 Mbit/s

Speedtest (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (136.244.81.190)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Vodafone DE (Frankfurt) [2.33 km]: 2.37 ms
Testing download speed................................................................................
Download: 612.19 Mbit/s
Testing upload speed................................................................................................
Upload: 811.55 Mbit/s

Speedtest (Ѐранция, ΠŸΠ°Ρ€ΠΈΠΆ)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (199.247.12.8)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Naitways (Paris) [2.04 km]: 3.818 ms
Testing download speed................................................................................
Download: 1079.52 Mbit/s
Testing upload speed................................................................................................
Upload: 897.98 Mbit/s

Speedtest (Япония, Вокио)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (45.77.22.120)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by GLBB Japan (Tokyo) [12.49 km]: 1.346 ms
Testing download speed................................................................................
Download: 3699.98 Mbit/s
Testing upload speed................................................................................................
Upload: 1897.78 Mbit/s

Speedtest (НидСрланды, АмстСрдам)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (209.250.247.182)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by YISP B.V. (Haarlem) [1.45 km]: 1.08 ms
Testing download speed................................................................................
Download: 2836.05 Mbit/s
Testing upload speed................................................................................................
Upload: 2417.74 Mbit/s

Speedtest (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (66.42.49.136)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by NewMedia Express (Singapore) [6.13 km]: 15.508 ms
Testing download speed................................................................................
Download: 236.03 Mbit/s
Testing upload speed................................................................................................
Upload: 161.70 Mbit/s

Speedtest (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (95.179.238.22)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Redraw Internet (London) [2.97 km]: 1.057 ms
Testing download speed................................................................................
Download: 714.62 Mbit/s
Testing upload speed................................................................................................
Upload: 759.21 Mbit/s

Speedtest (БША, Атланта)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (155.138.200.236)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Whitesky Communications LLC (Atlanta, GA) [6.39 km]: 1.689 ms
Testing download speed................................................................................
Download: 4354.27 Mbit/s
Testing upload speed................................................................................................
Upload: 3473.80 Mbit/s

Speedtest (БША, Чикаго)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (207.148.12.189)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Windstream (Chicago, IL) [31.77 km]: 8.795 ms
Testing download speed................................................................................
Download: 1454.70 Mbit/s
Testing upload speed................................................................................................
Upload: 1008.58 Mbit/s

Speedtest (БША, Даллас)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (144.202.72.219)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Nitel (Dallas, TX) [1.45 km]: 1.099 ms
Testing download speed................................................................................
Download: 460.90 Mbit/s
Testing upload speed................................................................................................
Upload: 302.76 Mbit/s

Speedtest (БША, Лос-АндТСлСс)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (45.76.68.204)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Bel Air Internet LLC (Los Angeles, CA) [2.73 km]: 1.023 ms
Testing download speed................................................................................
Download: 3429.38 Mbit/s
Testing upload speed................................................................................................
Upload: 2450.90 Mbit/s

Speedtest (БША, Майами)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (140.82.28.193)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by TIGO (Miami, FL) [2.79 km]: 0.949 ms
Testing download speed................................................................................
Download: 938.03 Mbit/s
Testing upload speed................................................................................................
Upload: 835.08 Mbit/s

Speedtest (БША, Нью-Π™ΠΎΡ€ΠΊ)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (45.77.97.253)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Vultr (Matawan, NJ) [24.71 km]: 1.114 ms
Testing download speed................................................................................
Download: 3753.04 Mbit/s
Testing upload speed................................................................................................
Upload: 2846.02 Mbit/s

Speedtest (БША, Биэтл)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (149.28.15.136)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Wowrack (Seattle, WA) [1.31 km]: 10.518 ms
Testing download speed................................................................................
Download: 4249.55 Mbit/s
Testing upload speed................................................................................................
Upload: 2548.90 Mbit/s

Speedtest (БША, Π‘Π°Π½-Π₯осС)

Retrieving speedtest.net configuration...
Testing from Choopa, LLC (45.77.2.227)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Razzolink Inc (San Jose, CA) [0.45 km]: 1.203 ms
Testing download speed................................................................................
Download: 4801.60 Mbit/s
Testing upload speed................................................................................................
Upload: 3428.94 Mbit/s

Π’ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹ΠΉ сСрвСр ΠΎΡ‚ Linode:

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

  • Ρ‚Π°Ρ€ΠΈΡ„ Nanode (25 GB SSD / 1x vCPU / 1 GB RAM / 1000 Mbit/s Π·Π° $5 Π² мСсяц);
  • Π΄Π°Ρ‚Π°-Ρ†Π΅Π½Ρ‚Ρ€Ρ‹ Π² Австралии, Π’Π΅Π»ΠΈΠΊΠΎΠ±Ρ€ΠΈΡ‚Π°Π½ΠΈΠΈ, Π“Π΅Ρ€ΠΌΠ°Π½ΠΈΠΈ, Индии, КанадС, Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€Π΅, БША, Π―ΠΏΠΎΠ½ΠΈΠΈ.

ВСсты:

Unixbench (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7601 32-Core Processor (4400.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:55:11 up 2 min,  1 user,  load average: 0.66, 0.22, 0.08; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:55:11 - 03:23:14
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       30133063.3 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4446.8 MWIPS (9.7 s, 7 samples)
Execl Throughput                               5037.1 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        840751.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          232585.4 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       2355072.1 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1567758.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 218719.7 lps   (10.0 s, 7 samples)
Process Creation                              15426.4 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   9564.1 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1254.2 lpm   (60.0 s, 2 samples)
System Call Overhead                        2412004.0 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   30133063.3   2582.1
Double-Precision Whetstone                       55.0       4446.8    808.5
Execl Throughput                                 43.0       5037.1   1171.4
File Copy 1024 bufsize 2000 maxblocks          3960.0     840751.9   2123.1
File Copy 256 bufsize 500 maxblocks            1655.0     232585.4   1405.4
File Copy 4096 bufsize 8000 maxblocks          5800.0    2355072.1   4060.5
Pipe Throughput                               12440.0    1567758.3   1260.3
Pipe-based Context Switching                   4000.0     218719.7    546.8
Process Creation                                126.0      15426.4   1224.3
Shell Scripts (1 concurrent)                     42.4       9564.1   2255.7
Shell Scripts (8 concurrent)                      6.0       1254.2   2090.3
System Call Overhead                          15000.0    2412004.0   1608.0
                                                                   ========
System Benchmarks Index Score                                        1548.1

Unixbench (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7501 32-Core Processor (4000.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   23:43:33 up 2 min,  1 user,  load average: 0.67, 0.23, 0.08; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Fri Dec 13 2019 23:43:33 - 00:11:37
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       28218808.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4106.8 MWIPS (9.8 s, 7 samples)
Execl Throughput                               4001.1 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        776704.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          211047.3 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1886947.8 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1465079.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 204475.4 lps   (10.0 s, 7 samples)
Process Creation                              13209.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7985.9 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1067.9 lpm   (60.0 s, 2 samples)
System Call Overhead                        2245138.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   28218808.4   2418.1
Double-Precision Whetstone                       55.0       4106.8    746.7
Execl Throughput                                 43.0       4001.1    930.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     776704.7   1961.4
File Copy 256 bufsize 500 maxblocks            1655.0     211047.3   1275.2
File Copy 4096 bufsize 8000 maxblocks          5800.0    1886947.8   3253.4
Pipe Throughput                               12440.0    1465079.0   1177.7
Pipe-based Context Switching                   4000.0     204475.4    511.2
Process Creation                                126.0      13209.9   1048.4
Shell Scripts (1 concurrent)                     42.4       7985.9   1883.5
Shell Scripts (8 concurrent)                      6.0       1067.9   1779.8
System Call Overhead                          15000.0    2245138.6   1496.8
                                                                   ========
System Benchmarks Index Score                                        1368.9

Unixbench (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7501 32-Core Processor (4000.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   19:05:00 up 4 min,  1 user,  load average: 0.56, 0.27, 0.12; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 19:05:00 - 19:33:04
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       28555684.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4214.4 MWIPS (9.7 s, 7 samples)
Execl Throughput                               4731.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        799611.6 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          218261.4 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1772488.0 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1458871.8 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 205092.6 lps   (10.0 s, 7 samples)
Process Creation                              14474.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   8362.5 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1126.3 lpm   (60.0 s, 2 samples)
System Call Overhead                        2170470.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   28555684.8   2446.9
Double-Precision Whetstone                       55.0       4214.4    766.3
Execl Throughput                                 43.0       4731.7   1100.4
File Copy 1024 bufsize 2000 maxblocks          3960.0     799611.6   2019.2
File Copy 256 bufsize 500 maxblocks            1655.0     218261.4   1318.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1772488.0   3056.0
Pipe Throughput                               12440.0    1458871.8   1172.7
Pipe-based Context Switching                   4000.0     205092.6    512.7
Process Creation                                126.0      14474.3   1148.8
Shell Scripts (1 concurrent)                     42.4       8362.5   1972.3
Shell Scripts (8 concurrent)                      6.0       1126.3   1877.2
System Call Overhead                          15000.0    2170470.6   1447.0
                                                                   ========
System Benchmarks Index Score                                        1410.7

Unixbench (Индия, ΠœΡƒΠΌΠ±Π°ΠΈ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7501 32-Core Processor (4000.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:53:15 up 2 min,  1 user,  load average: 0.59, 0.23, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:53:15 - 03:21:18
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       28681152.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4207.2 MWIPS (9.8 s, 7 samples)
Execl Throughput                               4827.1 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        798915.0 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          218718.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       2211908.2 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1480545.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 205294.0 lps   (10.0 s, 7 samples)
Process Creation                              14957.8 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   8948.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1173.7 lpm   (60.0 s, 2 samples)
System Call Overhead                        2297535.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   28681152.2   2457.7
Double-Precision Whetstone                       55.0       4207.2    764.9
Execl Throughput                                 43.0       4827.1   1122.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     798915.0   2017.5
File Copy 256 bufsize 500 maxblocks            1655.0     218718.5   1321.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    2211908.2   3813.6
Pipe Throughput                               12440.0    1480545.2   1190.1
Pipe-based Context Switching                   4000.0     205294.0    513.2
Process Creation                                126.0      14957.8   1187.1
Shell Scripts (1 concurrent)                     42.4       8948.3   2110.5
Shell Scripts (8 concurrent)                      6.0       1173.7   1956.1
System Call Overhead                          15000.0    2297535.6   1531.7
                                                                   ========
System Benchmarks Index Score                                        1465.8

Unixbench (Япония, Вокио)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz (4600.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   02:50:20 up 2 min,  1 user,  load average: 0.80, 0.28, 0.10; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 02:50:20 - 03:18:26
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25781663.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4162.8 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3905.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        436438.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          117964.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1190983.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              600292.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 111777.7 lps   (10.0 s, 7 samples)
Process Creation                               9587.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6991.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    905.6 lpm   (60.0 s, 2 samples)
System Call Overhead                         341762.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25781663.0   2209.2
Double-Precision Whetstone                       55.0       4162.8    756.9
Execl Throughput                                 43.0       3905.5    908.3
File Copy 1024 bufsize 2000 maxblocks          3960.0     436438.9   1102.1
File Copy 256 bufsize 500 maxblocks            1655.0     117964.6    712.8
File Copy 4096 bufsize 8000 maxblocks          5800.0    1190983.5   2053.4
Pipe Throughput                               12440.0     600292.7    482.6
Pipe-based Context Switching                   4000.0     111777.7    279.4
Process Creation                                126.0       9587.3    760.9
Shell Scripts (1 concurrent)                     42.4       6991.2   1648.9
Shell Scripts (8 concurrent)                      6.0        905.6   1509.3
System Call Overhead                          15000.0     341762.2    227.8
                                                                   ========
System Benchmarks Index Score                                         850.1

Unixbench (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz (4600.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   00:59:58 up 3 min,  1 user,  load average: 0.68, 0.25, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Mon Dec 16 2019 00:59:58 - 01:28:03
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25946607.9 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4166.4 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3876.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        449371.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          118226.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1265344.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              603138.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 112800.7 lps   (10.0 s, 7 samples)
Process Creation                               9763.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6993.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    905.4 lpm   (60.0 s, 2 samples)
System Call Overhead                         341957.3 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25946607.9   2223.4
Double-Precision Whetstone                       55.0       4166.4    757.5
Execl Throughput                                 43.0       3876.2    901.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     449371.9   1134.8
File Copy 256 bufsize 500 maxblocks            1655.0     118226.6    714.4
File Copy 4096 bufsize 8000 maxblocks          5800.0    1265344.6   2181.6
Pipe Throughput                               12440.0     603138.5    484.8
Pipe-based Context Switching                   4000.0     112800.7    282.0
Process Creation                                126.0       9763.9    774.9
Shell Scripts (1 concurrent)                     42.4       6993.3   1649.4
Shell Scripts (8 concurrent)                      6.0        905.4   1508.9
System Call Overhead                          15000.0     341957.3    228.0
                                                                   ========
System Benchmarks Index Score                                         859.0

Unixbench (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (5000.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   15:58:44 up 2 min,  1 user,  load average: 0.81, 0.32, 0.12; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Sun Dec 15 2019 15:58:44 - 16:26:52
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       27229246.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3624.0 MWIPS (9.7 s, 7 samples)
Execl Throughput                               3869.2 lps   (29.6 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        463380.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          125234.3 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1075247.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                              650363.8 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 119004.4 lps   (10.0 s, 7 samples)
Process Creation                               9502.3 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7004.4 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    881.0 lpm   (60.0 s, 2 samples)
System Call Overhead                         356503.7 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   27229246.0   2333.3
Double-Precision Whetstone                       55.0       3624.0    658.9
Execl Throughput                                 43.0       3869.2    899.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     463380.7   1170.2
File Copy 256 bufsize 500 maxblocks            1655.0     125234.3    756.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    1075247.3   1853.9
Pipe Throughput                               12440.0     650363.8    522.8
Pipe-based Context Switching                   4000.0     119004.4    297.5
Process Creation                                126.0       9502.3    754.2
Shell Scripts (1 concurrent)                     42.4       7004.4   1652.0
Shell Scripts (8 concurrent)                      6.0        881.0   1468.3
System Call Overhead                          15000.0     356503.7    237.7
                                                                   ========
System Benchmarks Index Score                                         855.4

Unixbench (БША, Атланта)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7501 32-Core Processor (4000.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   07:44:55 up 3 min,  1 user,  load average: 0.86, 0.39, 0.15; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 05 2019 07:44:55 - 08:13:01
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       27076602.3 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4059.3 MWIPS (9.8 s, 7 samples)
Execl Throughput                               3779.0 lps   (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        648772.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          182118.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1695046.2 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1411027.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 186657.9 lps   (10.0 s, 7 samples)
Process Creation                              10081.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7214.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    909.5 lpm   (60.0 s, 2 samples)
System Call Overhead                        2183324.7 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   27076602.3   2320.2
Double-Precision Whetstone                       55.0       4059.3    738.1
Execl Throughput                                 43.0       3779.0    878.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     648772.3   1638.3
File Copy 256 bufsize 500 maxblocks            1655.0     182118.5   1100.4
File Copy 4096 bufsize 8000 maxblocks          5800.0    1695046.2   2922.5
Pipe Throughput                               12440.0    1411027.5   1134.3
Pipe-based Context Switching                   4000.0     186657.9    466.6
Process Creation                                126.0      10081.5    800.1
Shell Scripts (1 concurrent)                     42.4       7214.0   1701.4
Shell Scripts (8 concurrent)                      6.0        909.5   1515.9
System Call Overhead                          15000.0    2183324.7   1455.5
                                                                   ========
System Benchmarks Index Score                                        1235.1

Unixbench (БША, Даллас)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz (4600.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   14:18:56 up 4 min,  1 user,  load average: 0.72, 0.27, 0.10; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 14:18:56 - 14:47:01
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25639903.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4159.3 MWIPS (9.8 s, 7 samples)
Execl Throughput                               3906.9 lps   (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        450551.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          120930.2 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1241998.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              599891.8 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 117628.7 lps   (10.0 s, 7 samples)
Process Creation                               9526.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7065.9 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    909.3 lpm   (60.0 s, 2 samples)
System Call Overhead                         342067.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25639903.4   2197.1
Double-Precision Whetstone                       55.0       4159.3    756.2
Execl Throughput                                 43.0       3906.9    908.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     450551.9   1137.8
File Copy 256 bufsize 500 maxblocks            1655.0     120930.2    730.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    1241998.9   2141.4
Pipe Throughput                               12440.0     599891.8    482.2
Pipe-based Context Switching                   4000.0     117628.7    294.1
Process Creation                                126.0       9526.7    756.1
Shell Scripts (1 concurrent)                     42.4       7065.9   1666.5
Shell Scripts (8 concurrent)                      6.0        909.3   1515.4
System Call Overhead                          15000.0     342067.6    228.0
                                                                   ========
System Benchmarks Index Score                                         861.0

Unixbench (БША, Π€Ρ€ΠΈΠΌΠΎΠ½Ρ‚)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz (4600.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   18:18:58 up 3 min,  1 user,  load average: 0.64, 0.23, 0.09; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 11 2019 18:18:58 - 18:47:02
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       25968547.5 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4168.6 MWIPS (9.8 s, 7 samples)
Execl Throughput                               3912.3 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        467206.8 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          121416.0 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1441297.8 KBps  (30.0 s, 2 samples)
Pipe Throughput                              604580.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 117233.5 lps   (10.0 s, 7 samples)
Process Creation                               9613.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7118.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    936.4 lpm   (60.0 s, 2 samples)
System Call Overhead                         343482.8 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   25968547.5   2225.2
Double-Precision Whetstone                       55.0       4168.6    757.9
Execl Throughput                                 43.0       3912.3    909.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     467206.8   1179.8
File Copy 256 bufsize 500 maxblocks            1655.0     121416.0    733.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1441297.8   2485.0
Pipe Throughput                               12440.0     604580.9    486.0
Pipe-based Context Switching                   4000.0     117233.5    293.1
Process Creation                                126.0       9613.5    763.0
Shell Scripts (1 concurrent)                     42.4       7118.7   1678.9
Shell Scripts (8 concurrent)                      6.0        936.4   1560.7
System Call Overhead                          15000.0     343482.8    229.0
                                                                   ========
System Benchmarks Index Score                                         879.8

Unixbench (БША, ΠΡŒΡŽΠ°Ρ€ΠΊ)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: localhost: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3 (2019-09-02)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD EPYC 7601 32-Core Processor (4400.0 bogomips)
          x86-64, MMX, AMD MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   12:37:25 up 4 min,  1 user,  load average: 0.59, 0.26, 0.10; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Thu Dec 12 2019 12:37:25 - 13:05:28
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       28849705.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4265.4 MWIPS (9.7 s, 7 samples)
Execl Throughput                               3989.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        764588.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          214230.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1826179.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1533597.7 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 202033.0 lps   (10.0 s, 7 samples)
Process Creation                              11268.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7998.9 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1012.8 lpm   (60.0 s, 2 samples)
System Call Overhead                        2192009.9 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   28849705.0   2472.1
Double-Precision Whetstone                       55.0       4265.4    775.5
Execl Throughput                                 43.0       3989.7    927.8
File Copy 1024 bufsize 2000 maxblocks          3960.0     764588.9   1930.8
File Copy 256 bufsize 500 maxblocks            1655.0     214230.9   1294.4
File Copy 4096 bufsize 8000 maxblocks          5800.0    1826179.5   3148.6
Pipe Throughput                               12440.0    1533597.7   1232.8
Pipe-based Context Switching                   4000.0     202033.0    505.1
Process Creation                                126.0      11268.5    894.3
Shell Scripts (1 concurrent)                     42.4       7998.9   1886.5
Shell Scripts (8 concurrent)                      6.0       1012.8   1688.1
System Call Overhead                          15000.0    2192009.9   1461.3
                                                                   ========
System Benchmarks Index Score                                        1348.9

dd (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.84047 s, 1.2 GB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.70147 s, 1.3 GB/s

dd (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.06427 s, 1.0 GB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.03386 s, 1.1 GB/s

dd (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.23042 s, 963 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.02314 s, 1.1 GB/s

dd (Индия, ΠœΡƒΠΌΠ±Π°ΠΈ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.87866 s, 1.1 GB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.79076 s, 1.2 GB/s

dd (Япония, Π’ΠΎΠΊΠΈΠΎ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.3769 s, 903 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.4095 s, 891 MB/s

dd (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.31406 s, 928 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.32339 s, 924 MB/s

dd (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.40905 s, 891 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.72641 s, 788 MB/s

dd (БША, Атланта)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.98208 s, 720 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 24.2335 s, 88.6 MB/s

dd (БША, Даллас)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.42872 s, 884 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.36519 s, 638 MB/s

dd (БША, Π€Ρ€ΠΈΠΌΠΎΠ½Ρ‚)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.50491 s, 857 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.53728 s, 846 MB/s

dd (БША, ΠΡŒΡŽΠ°Ρ€ΠΊ)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.13085 s, 1.0 GB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.08892 s, 695 MB/s

fio (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8604: Mon Dec 16 03:24:20 2019
  read : io=6755.1MB, bw=115299KB/s, iops=28824, runt= 60001msec
    clat (usec): min=0, max=100545, avg=274.82, stdev=569.86
     lat (usec): min=1, max=100545, avg=275.21, stdev=569.86
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[  169], 20.00th=[  197],
     | 30.00th=[  211], 40.00th=[  223], 50.00th=[  237], 60.00th=[  255],
     | 70.00th=[  294], 80.00th=[  354], 90.00th=[  446], 95.00th=[  540],
     | 99.00th=[  740], 99.50th=[  804], 99.90th=[ 1944], 99.95th=[ 2352],
     | 99.99th=[13760]
    bw (KB  /s): min= 6472, max=21216, per=12.51%, avg=14418.69, stdev=2131.98
    lat (usec) : 2=2.23%, 4=5.07%, 10=0.30%, 20=0.16%, 50=0.03%
    lat (usec) : 100=0.02%, 250=49.90%, 500=35.60%, 750=5.76%, 1000=0.73%
    lat (msec) : 2=0.11%, 4=0.08%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=0.90%, sys=6.29%, ctx=1609714, majf=2, minf=104
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1729514/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6755.1MB, aggrb=115299KB/s, minb=115299KB/s, maxb=115299KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1593629/10, merge=136/5, ticks=403736/0, in_queue=403412, util=99.03%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8633: Mon Dec 16 03:26:23 2019
  write: io=37600MB, bw=641604KB/s, iops=160400, runt= 60009msec
    clat (usec): min=1, max=138933, avg=36.00, stdev=698.17
     lat (usec): min=1, max=138933, avg=38.40, stdev=740.72
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    1], 10.00th=[    1], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    2], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[    4], 99.50th=[  113], 99.90th=[11712], 99.95th=[16064],
     | 99.99th=[27008]
    bw (KB  /s): min=    1, max=103354, per=12.62%, avg=80945.26, stdev=7323.10
    lat (usec) : 2=10.82%, 4=88.07%, 10=0.29%, 20=0.13%, 50=0.13%
    lat (usec) : 100=0.05%, 250=0.03%, 500=0.04%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.04%, 4=0.05%, 10=0.15%, 20=0.10%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.46%, sys=7.39%, ctx=100753, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9625502/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=37600MB, aggrb=641603KB/s, minb=641603KB/s, maxb=641603KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  sda: ios=172/3092395, merge=31/1341, ticks=4/5409372, in_queue=5404936, util=70.18%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8617: Mon Dec 16 03:25:23 2019
  read : io=6485.6MB, bw=110683KB/s, iops=27670, runt= 60002msec
    clat (usec): min=61, max=579406, avg=286.37, stdev=809.14
     lat (usec): min=61, max=579406, avg=286.76, stdev=809.14
    clat percentiles (usec):
     |  1.00th=[  155],  5.00th=[  175], 10.00th=[  185], 20.00th=[  201],
     | 30.00th=[  213], 40.00th=[  223], 50.00th=[  237], 60.00th=[  255],
     | 70.00th=[  294], 80.00th=[  350], 90.00th=[  438], 95.00th=[  524],
     | 99.00th=[  708], 99.50th=[  772], 99.90th=[  900], 99.95th=[  964],
     | 99.99th=[22144]
    bw (KB  /s): min= 2889, max=15576, per=12.51%, avg=13846.49, stdev=1486.54
    lat (usec) : 100=0.03%, 250=57.83%, 500=36.15%, 750=5.33%, 1000=0.63%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  cpu          : usr=0.93%, sys=5.87%, ctx=1664165, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1660305/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6485.6MB, aggrb=110683KB/s, minb=110683KB/s, maxb=110683KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1659424/20, merge=317/15, ticks=421880/0, in_queue=421648, util=99.87%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8646: Mon Dec 16 03:27:23 2019
  write: io=1930.2MB, bw=32955KB/s, iops=8238, runt= 60001msec
    clat (usec): min=66, max=66656, avg=968.12, stdev=6325.23
     lat (usec): min=67, max=66656, avg=968.51, stdev=6325.24
    clat percentiles (usec):
     |  1.00th=[   77],  5.00th=[   79], 10.00th=[   80], 20.00th=[   83],
     | 30.00th=[   87], 40.00th=[   90], 50.00th=[   92], 60.00th=[   94],
     | 70.00th=[   96], 80.00th=[  103], 90.00th=[  306], 95.00th=[  346],
     | 99.00th=[48384], 99.50th=[49920], 99.90th=[50944], 99.95th=[50944],
     | 99.99th=[53504]
    bw (KB  /s): min= 2736, max= 5424, per=12.51%, avg=4121.09, stdev=523.74
    lat (usec) : 100=76.13%, 250=10.66%, 500=11.42%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.36%
    lat (msec) : 100=0.42%
  cpu          : usr=0.30%, sys=2.13%, ctx=997805, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=494335/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1930.2MB, aggrb=32955KB/s, minb=32955KB/s, maxb=32955KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=50/493789, merge=157/1755, ticks=40/49980, in_queue=49936, util=83.16%

fio (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15535: Sat Dec 14 00:12:44 2019
  read : io=7105.6MB, bw=121265KB/s, iops=30316, runt= 60001msec
    clat (usec): min=0, max=1003.9K, avg=260.89, stdev=940.50
     lat (usec): min=1, max=1003.9K, avg=261.29, stdev=940.50
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[  161], 20.00th=[  189],
     | 30.00th=[  203], 40.00th=[  215], 50.00th=[  227], 60.00th=[  245],
     | 70.00th=[  278], 80.00th=[  326], 90.00th=[  406], 95.00th=[  482],
     | 99.00th=[  708], 99.50th=[  828], 99.90th=[ 2288], 99.95th=[ 2704],
     | 99.99th=[14400]
    bw (KB  /s): min= 3942, max=21760, per=12.53%, avg=15190.66, stdev=1907.96
    lat (usec) : 2=3.64%, 4=3.76%, 10=0.34%, 20=0.12%, 50=0.01%
    lat (usec) : 100=0.05%, 250=53.79%, 500=34.02%, 750=3.49%, 1000=0.50%
    lat (msec) : 2=0.12%, 4=0.14%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 2000=0.01%
  cpu          : usr=1.05%, sys=7.07%, ctx=1681777, majf=2, minf=98
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1819009/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7105.6MB, aggrb=121265KB/s, minb=121265KB/s, maxb=121265KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1671621/10, merge=144/5, ticks=385432/0, in_queue=385092, util=98.64%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15561: Sat Dec 14 00:14:47 2019
  write: io=34626MB, bw=590888KB/s, iops=147722, runt= 60007msec
    clat (usec): min=1, max=110371, avg=38.56, stdev=740.93
     lat (usec): min=1, max=110371, avg=41.20, stdev=786.70
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    1], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    2], 90.00th=[    2], 95.00th=[    3],
     | 99.00th=[    4], 99.50th=[  155], 99.90th=[12608], 99.95th=[17280],
     | 99.99th=[28032]
    bw (KB  /s): min=    1, max=93867, per=12.62%, avg=74551.39, stdev=7277.00
    lat (usec) : 2=7.80%, 4=91.03%, 10=0.34%, 20=0.07%, 50=0.08%
    lat (usec) : 100=0.12%, 250=0.08%, 500=0.03%, 750=0.03%, 1000=0.03%
    lat (msec) : 2=0.04%, 4=0.05%, 10=0.14%, 20=0.11%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.88%, sys=6.88%, ctx=91744, majf=0, minf=76
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=8864358/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=34626MB, aggrb=590888KB/s, minb=590888KB/s, maxb=590888KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  sda: ios=172/2770808, merge=31/1461, ticks=4/5007316, in_queue=5002336, util=67.08%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15548: Sat Dec 14 00:13:47 2019
  read : io=6997.4MB, bw=119417KB/s, iops=29854, runt= 60002msec
    clat (usec): min=51, max=1008.4K, avg=264.70, stdev=1211.12
     lat (usec): min=52, max=1008.4K, avg=265.16, stdev=1211.12
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[  139], 10.00th=[  165], 20.00th=[  185],
     | 30.00th=[  199], 40.00th=[  211], 50.00th=[  223], 60.00th=[  241],
     | 70.00th=[  270], 80.00th=[  322], 90.00th=[  398], 95.00th=[  478],
     | 99.00th=[  716], 99.50th=[  788], 99.90th=[ 1032], 99.95th=[ 1320],
     | 99.99th=[ 9536]
    bw (KB  /s): min=  391, max=23312, per=12.54%, avg=14972.94, stdev=1899.15
    lat (usec) : 100=2.50%, 250=61.34%, 500=31.96%, 750=3.47%, 1000=0.61%
    lat (msec) : 2=0.09%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 2000=0.01%
  cpu          : usr=1.09%, sys=6.67%, ctx=1822378, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1791317/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6997.4MB, aggrb=119417KB/s, minb=119417KB/s, maxb=119417KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1787341/10, merge=136/5, ticks=397456/0, in_queue=397140, util=99.63%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=15574: Sat Dec 14 00:15:48 2019
  write: io=2134.7MB, bw=36430KB/s, iops=9107, runt= 60003msec
    clat (usec): min=64, max=143251, avg=875.05, stdev=5614.52
     lat (usec): min=65, max=143252, avg=875.49, stdev=5614.53
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   71], 10.00th=[   73], 20.00th=[   75],
     | 30.00th=[   77], 40.00th=[   80], 50.00th=[   82], 60.00th=[   86],
     | 70.00th=[   91], 80.00th=[  104], 90.00th=[  233], 95.00th=[  306],
     | 99.00th=[40192], 99.50th=[44288], 99.90th=[46848], 99.95th=[48384],
     | 99.99th=[61184]
    bw (KB  /s): min= 2576, max= 6080, per=12.51%, avg=4556.05, stdev=585.26
    lat (usec) : 100=78.06%, 250=13.06%, 500=6.91%, 750=0.05%, 1000=0.03%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.84%
    lat (msec) : 100=0.03%, 250=0.01%
  cpu          : usr=0.40%, sys=2.49%, ctx=1103528, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=546471/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2134.7MB, aggrb=36429KB/s, minb=36429KB/s, maxb=36429KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=179/545918, merge=322/1528, ticks=408/47584, in_queue=47876, util=79.18%

fio (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5708: Sun Dec 15 19:34:11 2019
  read : io=7256.2MB, bw=123836KB/s, iops=30959, runt= 60001msec
    clat (usec): min=1, max=928516, avg=255.21, stdev=866.57
     lat (usec): min=1, max=928516, avg=255.62, stdev=866.58
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[  145], 20.00th=[  191],
     | 30.00th=[  209], 40.00th=[  221], 50.00th=[  235], 60.00th=[  251],
     | 70.00th=[  274], 80.00th=[  310], 90.00th=[  366], 95.00th=[  426],
     | 99.00th=[  692], 99.50th=[  804], 99.90th=[ 2736], 99.95th=[ 3376],
     | 99.99th=[13376]
    bw (KB  /s): min= 3947, max=28312, per=12.53%, avg=15518.60, stdev=2109.24
    lat (usec) : 2=2.34%, 4=5.09%, 10=0.35%, 20=0.17%, 50=0.03%
    lat (usec) : 100=0.48%, 250=51.00%, 500=37.75%, 750=2.13%, 1000=0.36%
    lat (msec) : 2=0.09%, 4=0.19%, 10=0.02%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 1000=0.01%
  cpu          : usr=1.06%, sys=7.27%, ctx=1732810, majf=2, minf=96
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1857571/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7256.2MB, aggrb=123836KB/s, minb=123836KB/s, maxb=123836KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1707687/10, merge=136/5, ticks=380524/0, in_queue=380068, util=98.56%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5737: Sun Dec 15 19:36:15 2019
  write: io=32623MB, bw=556695KB/s, iops=139173, runt= 60007msec
    clat (usec): min=1, max=84101, avg=41.66, stdev=765.39
     lat (usec): min=1, max=84101, avg=44.37, stdev=811.18
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    2], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[    7], 99.50th=[  306], 99.90th=[12992], 99.95th=[17280],
     | 99.99th=[28288]
    bw (KB  /s): min=    1, max=95040, per=12.63%, avg=70283.71, stdev=7683.83
    lat (usec) : 2=4.00%, 4=94.50%, 10=0.53%, 20=0.11%, 50=0.19%
    lat (usec) : 100=0.10%, 250=0.05%, 500=0.03%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.04%, 4=0.05%, 10=0.15%, 20=0.12%, 50=0.03%
    lat (msec) : 100=0.01%
  cpu          : usr=1.62%, sys=6.95%, ctx=93483, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=8351405/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=32623MB, aggrb=556695KB/s, minb=556695KB/s, maxb=556695KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  sda: ios=172/2732266, merge=31/1355, ticks=144/5451696, in_queue=5446676, util=71.80%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5721: Sun Dec 15 19:35:14 2019
  read : io=6805.7MB, bw=116146KB/s, iops=29036, runt= 60002msec
    clat (usec): min=56, max=1006.1K, avg=272.36, stdev=1070.90
     lat (usec): min=56, max=1006.1K, avg=272.80, stdev=1070.90
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  169], 10.00th=[  181], 20.00th=[  199],
     | 30.00th=[  213], 40.00th=[  225], 50.00th=[  237], 60.00th=[  253],
     | 70.00th=[  278], 80.00th=[  310], 90.00th=[  362], 95.00th=[  418],
     | 99.00th=[  612], 99.50th=[  812], 99.90th=[ 3152], 99.95th=[ 4960],
     | 99.99th=[22400]
    bw (KB  /s): min=  315, max=22760, per=12.53%, avg=14554.98, stdev=2178.59
    lat (usec) : 100=0.40%, 250=57.47%, 500=40.06%, 750=1.49%, 1000=0.25%
    lat (msec) : 2=0.16%, 4=0.11%, 10=0.05%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 750=0.01%, 2000=0.01%
  cpu          : usr=1.02%, sys=6.78%, ctx=1754222, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1742254/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6805.7MB, aggrb=116146KB/s, minb=116146KB/s, maxb=116146KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1739426/12, merge=316/13, ticks=389336/0, in_queue=389096, util=99.74%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=5750: Sun Dec 15 19:37:15 2019
  write: io=2106.5MB, bw=35947KB/s, iops=8986, runt= 60003msec
    clat (usec): min=66, max=79631, avg=887.06, stdev=5832.79
     lat (usec): min=66, max=79632, avg=887.50, stdev=5832.80
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   73], 10.00th=[   74], 20.00th=[   76],
     | 30.00th=[   80], 40.00th=[   84], 50.00th=[   88], 60.00th=[   93],
     | 70.00th=[   99], 80.00th=[  113], 90.00th=[  209], 95.00th=[  270],
     | 99.00th=[43776], 99.50th=[46336], 99.90th=[49920], 99.95th=[52992],
     | 99.99th=[61184]
    bw (KB  /s): min= 2848, max= 5945, per=12.52%, avg=4499.10, stdev=664.94
    lat (usec) : 100=70.00%, 250=23.23%, 500=4.81%, 750=0.09%, 1000=0.04%
    lat (msec) : 2=0.03%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.67%
    lat (msec) : 100=0.10%
  cpu          : usr=0.38%, sys=2.28%, ctx=1088372, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=539238/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2106.5MB, aggrb=35947KB/s, minb=35947KB/s, maxb=35947KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=57/538803, merge=144/1594, ticks=28/47844, in_queue=47732, util=79.49%

fio (Индия, ΠœΡƒΠΌΠ±Π°ΠΈ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29999: Mon Dec 16 03:22:24 2019
  read : io=7680.4MB, bw=131076KB/s, iops=32769, runt= 60001msec
    clat (usec): min=1, max=1005.9K, avg=241.25, stdev=958.77
     lat (usec): min=1, max=1005.9K, avg=241.61, stdev=958.78
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[  161], 20.00th=[  185],
     | 30.00th=[  199], 40.00th=[  209], 50.00th=[  219], 60.00th=[  233],
     | 70.00th=[  255], 80.00th=[  294], 90.00th=[  342], 95.00th=[  398],
     | 99.00th=[  636], 99.50th=[  716], 99.90th=[ 2352], 99.95th=[ 2896],
     | 99.99th=[12736]
    bw (KB  /s): min= 3690, max=22984, per=12.52%, avg=16407.99, stdev=1870.64
    lat (usec) : 2=3.82%, 4=3.66%, 10=0.35%, 20=0.09%, 50=0.01%
    lat (usec) : 100=0.02%, 250=59.86%, 500=30.20%, 750=1.58%, 1000=0.16%
    lat (msec) : 2=0.09%, 4=0.12%, 10=0.02%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 500=0.01%, 2000=0.01%
  cpu          : usr=1.05%, sys=6.95%, ctx=1815281, majf=2, minf=89
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1966177/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7680.4MB, aggrb=131076KB/s, minb=131076KB/s, maxb=131076KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1805586/33, merge=187/30, ticks=392240/4, in_queue=392088, util=98.48%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30118: Mon Dec 16 03:24:27 2019
  write: io=36526MB, bw=623330KB/s, iops=155832, runt= 60004msec
    clat (usec): min=1, max=119319, avg=36.80, stdev=722.41
     lat (usec): min=1, max=119319, avg=39.29, stdev=765.26
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    1], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    2], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[    4], 99.50th=[   97], 99.90th=[11968], 99.95th=[16768],
     | 99.99th=[27776]
    bw (KB  /s): min=    1, max=97309, per=12.62%, avg=78634.24, stdev=7494.43
    lat (usec) : 2=6.84%, 4=92.00%, 10=0.33%, 20=0.07%, 50=0.15%
    lat (usec) : 100=0.12%, 250=0.04%, 500=0.03%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.04%, 4=0.05%, 10=0.14%, 20=0.10%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.89%, sys=6.90%, ctx=94592, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=9350572/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=36526MB, aggrb=623329KB/s, minb=623329KB/s, maxb=623329KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=276/2956177, merge=354/1423, ticks=184/5082644, in_queue=5078200, util=67.28%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30060: Mon Dec 16 03:23:27 2019
  read : io=7109.1MB, bw=121340KB/s, iops=30335, runt= 60001msec
    clat (usec): min=54, max=1005.3K, avg=260.81, stdev=980.86
     lat (usec): min=54, max=1005.3K, avg=261.23, stdev=980.88
    clat percentiles (usec):
     |  1.00th=[  149],  5.00th=[  167], 10.00th=[  177], 20.00th=[  193],
     | 30.00th=[  203], 40.00th=[  215], 50.00th=[  225], 60.00th=[  241],
     | 70.00th=[  266], 80.00th=[  302], 90.00th=[  354], 95.00th=[  414],
     | 99.00th=[  644], 99.50th=[  716], 99.90th=[  948], 99.95th=[ 3376],
     | 99.99th=[12352]
    bw (KB  /s): min= 3389, max=23968, per=12.52%, avg=15196.38, stdev=1379.78
    lat (usec) : 100=0.09%, 250=64.27%, 500=33.03%, 750=2.27%, 1000=0.24%
    lat (msec) : 2=0.04%, 4=0.01%, 10=0.03%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 2000=0.01%
  cpu          : usr=1.10%, sys=6.31%, ctx=1824541, majf=0, minf=84
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1820134/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7109.1MB, aggrb=121340KB/s, minb=121340KB/s, maxb=121340KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1814323/36, merge=54/29, ticks=399948/4, in_queue=399628, util=99.33%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30175: Mon Dec 16 03:25:28 2019
  write: io=2144.3MB, bw=36593KB/s, iops=9148, runt= 60003msec
    clat (usec): min=66, max=76722, avg=871.26, stdev=5688.89
     lat (usec): min=66, max=76722, avg=871.72, stdev=5688.97
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   73], 10.00th=[   75], 20.00th=[   78],
     | 30.00th=[   80], 40.00th=[   82], 50.00th=[   85], 60.00th=[   88],
     | 70.00th=[   92], 80.00th=[  102], 90.00th=[  225], 95.00th=[  290],
     | 99.00th=[42752], 99.50th=[44800], 99.90th=[46848], 99.95th=[48896],
     | 99.99th=[61184]
    bw (KB  /s): min= 2913, max= 6144, per=12.50%, avg=4575.83, stdev=573.66
    lat (usec) : 100=78.69%, 250=13.10%, 500=6.35%, 750=0.02%, 1000=0.02%
    lat (msec) : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.76%
    lat (msec) : 100=0.03%
  cpu          : usr=0.37%, sys=2.38%, ctx=1108100, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=548927/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2144.3MB, aggrb=36593KB/s, minb=36593KB/s, maxb=36593KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=351/548346, merge=381/1573, ticks=220/47296, in_queue=47364, util=78.53%

fio (Япония, Вокио)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=8994: Mon Dec 16 03:19:35 2019
  read : io=6104.3MB, bw=104172KB/s, iops=26042, runt= 60004msec
    clat (usec): min=2, max=175502, avg=301.62, stdev=709.96
     lat (usec): min=2, max=175503, avg=302.53, stdev=709.98
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[  199], 20.00th=[  215],
     | 30.00th=[  227], 40.00th=[  237], 50.00th=[  247], 60.00th=[  258],
     | 70.00th=[  278], 80.00th=[  310], 90.00th=[  382], 95.00th=[  516],
     | 99.00th=[ 1464], 99.50th=[ 2608], 99.90th=[ 5088], 99.95th=[ 7840],
     | 99.99th=[27776]
    bw (KB  /s): min= 4801, max=18768, per=12.52%, avg=13047.25, stdev=1979.51
    lat (usec) : 4=4.96%, 10=1.05%, 20=0.22%, 50=0.02%, 100=0.28%
    lat (usec) : 250=46.83%, 500=41.33%, 750=2.46%, 1000=1.19%
    lat (msec) : 2=0.93%, 4=0.54%, 10=0.16%, 20=0.02%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.92%, sys=7.58%, ctx=1471253, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1562682/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6104.3MB, aggrb=104171KB/s, minb=104171KB/s, maxb=104171KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=1464787/22, merge=242/13, ticks=373888/12, in_queue=373712, util=98.37%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=9020: Mon Dec 16 03:21:39 2019
  write: io=21704MB, bw=370364KB/s, iops=92590, runt= 60007msec
    clat (usec): min=2, max=119437, avg=53.10, stdev=808.22
     lat (usec): min=3, max=119438, avg=59.46, stdev=877.89
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[  104], 99.50th=[ 2672], 99.90th=[13504], 99.95th=[18816],
     | 99.99th=[28032]
    bw (KB  /s): min=    1, max=59442, per=12.66%, avg=46888.23, stdev=3647.44
    lat (usec) : 4=46.23%, 10=52.01%, 20=0.43%, 50=0.10%, 100=0.18%
    lat (usec) : 250=0.22%, 500=0.11%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.09%, 4=0.15%, 10=0.24%, 20=0.12%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.07%, sys=6.16%, ctx=115702, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5556102/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=21704MB, aggrb=370363KB/s, minb=370363KB/s, maxb=370363KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  sda: ios=146/1732737, merge=57/1467, ticks=140/5125380, in_queue=5123836, util=67.83%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=9007: Mon Dec 16 03:20:39 2019
  read : io=5844.9MB, bw=99749KB/s, iops=24937, runt= 60002msec
    clat (usec): min=70, max=100509, avg=314.94, stdev=758.21
     lat (usec): min=70, max=100509, avg=315.82, stdev=758.24
    clat percentiles (usec):
     |  1.00th=[  139],  5.00th=[  199], 10.00th=[  205], 20.00th=[  217],
     | 30.00th=[  227], 40.00th=[  233], 50.00th=[  241], 60.00th=[  253],
     | 70.00th=[  270], 80.00th=[  298], 90.00th=[  370], 95.00th=[  516],
     | 99.00th=[ 1384], 99.50th=[ 2864], 99.90th=[ 7136], 99.95th=[ 9408],
     | 99.99th=[40192]
    bw (KB  /s): min= 2952, max=15256, per=12.51%, avg=12474.81, stdev=2609.26
    lat (usec) : 100=0.35%, 250=57.08%, 500=37.25%, 750=2.41%, 1000=1.29%
    lat (msec) : 2=0.93%, 4=0.39%, 10=0.24%, 20=0.02%, 50=0.02%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.06%, sys=7.24%, ctx=1501769, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1496285/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=5844.9MB, aggrb=99749KB/s, minb=99749KB/s, maxb=99749KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1490329/10, merge=152/5, ticks=394100/0, in_queue=393836, util=99.70%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=9033: Mon Dec 16 03:22:39 2019
  write: io=1783.9MB, bw=30443KB/s, iops=7610, runt= 60004msec
    clat (usec): min=74, max=414357, avg=1045.25, stdev=6500.27
     lat (usec): min=75, max=414358, avg=1046.20, stdev=6500.29
    clat percentiles (usec):
     |  1.00th=[   77],  5.00th=[   79], 10.00th=[   80], 20.00th=[   83],
     | 30.00th=[   85], 40.00th=[   88], 50.00th=[   93], 60.00th=[   99],
     | 70.00th=[  110], 80.00th=[  126], 90.00th=[  163], 95.00th=[  247],
     | 99.00th=[37120], 99.50th=[42240], 99.90th=[59136], 99.95th=[69120],
     | 99.99th=[124416]
    bw (KB  /s): min=  354, max= 5447, per=12.53%, avg=3813.73, stdev=898.12
    lat (usec) : 100=60.09%, 250=34.95%, 500=1.55%, 750=0.33%, 1000=0.14%
    lat (msec) : 2=0.30%, 4=0.24%, 10=0.06%, 20=0.01%, 50=2.13%
    lat (msec) : 100=0.20%, 250=0.01%, 500=0.01%
  cpu          : usr=0.62%, sys=2.31%, ctx=924429, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=456672/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1783.9MB, aggrb=30442KB/s, minb=30442KB/s, maxb=30442KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=65/456148, merge=144/1721, ticks=8/47868, in_queue=47764, util=78.32%

fio (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29700: Mon Dec 16 01:29:12 2019
  read : io=7460.3MB, bw=127319KB/s, iops=31829, runt= 60001msec
    clat (usec): min=2, max=100538, avg=245.95, stdev=526.10
     lat (usec): min=2, max=100539, avg=246.84, stdev=526.11
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[  181], 20.00th=[  213],
     | 30.00th=[  223], 40.00th=[  229], 50.00th=[  235], 60.00th=[  243],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  310], 95.00th=[  354],
     | 99.00th=[  516], 99.50th=[  868], 99.90th=[ 2864], 99.95th=[ 3376],
     | 99.99th=[ 7648]
    bw (KB  /s): min= 7664, max=21344, per=12.51%, avg=15924.29, stdev=1653.02
    lat (usec) : 4=6.65%, 10=1.02%, 20=0.27%, 50=0.02%, 100=0.24%
    lat (usec) : 250=56.60%, 500=34.12%, 750=0.52%, 1000=0.11%
    lat (msec) : 2=0.18%, 4=0.25%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.37%, sys=8.65%, ctx=1767874, majf=2, minf=96
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1909821/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7460.3MB, aggrb=127319KB/s, minb=127319KB/s, maxb=127319KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1751526/10, merge=136/5, ticks=379692/0, in_queue=379548, util=98.45%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29726: Mon Dec 16 01:31:16 2019
  write: io=22301MB, bw=380447KB/s, iops=95111, runt= 60026msec
    clat (usec): min=2, max=99818, avg=50.79, stdev=815.47
     lat (usec): min=2, max=99819, avg=56.89, stdev=884.08
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[   97], 99.50th=[ 1992], 99.90th=[13248], 99.95th=[18304],
     | 99.99th=[29824]
    bw (KB  /s): min=    1, max=60142, per=12.65%, avg=48143.86, stdev=4034.99
    lat (usec) : 4=50.77%, 10=47.42%, 20=0.47%, 50=0.18%, 100=0.18%
    lat (usec) : 250=0.22%, 500=0.10%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.09%, 4=0.14%, 10=0.21%, 20=0.12%, 50=0.04%
    lat (msec) : 100=0.01%
  cpu          : usr=4.24%, sys=6.15%, ctx=116463, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5709171/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=22301MB, aggrb=380446KB/s, minb=380446KB/s, maxb=380446KB/s, mint=60026msec, maxt=60026msec

Disk stats (read/write):
  sda: ios=181/1565641, merge=31/1692, ticks=840/7100268, in_queue=7103928, util=90.92%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29713: Mon Dec 16 01:30:15 2019
  read : io=6622.7MB, bw=113024KB/s, iops=28255, runt= 60001msec
    clat (usec): min=64, max=1003.4K, avg=277.49, stdev=1028.50
     lat (usec): min=65, max=1003.4K, avg=278.34, stdev=1028.52
    clat percentiles (usec):
     |  1.00th=[  113],  5.00th=[  195], 10.00th=[  207], 20.00th=[  213],
     | 30.00th=[  219], 40.00th=[  225], 50.00th=[  233], 60.00th=[  241],
     | 70.00th=[  253], 80.00th=[  274], 90.00th=[  314], 95.00th=[  378],
     | 99.00th=[  868], 99.50th=[ 1688], 99.90th=[ 4960], 99.95th=[ 8032],
     | 99.99th=[43264]
    bw (KB  /s): min=  234, max=19920, per=12.52%, avg=14153.85, stdev=3251.43
    lat (usec) : 100=0.60%, 250=66.91%, 500=30.22%, 750=1.10%, 1000=0.31%
    lat (msec) : 2=0.42%, 4=0.27%, 10=0.14%, 20=0.03%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 2000=0.01%
  cpu          : usr=2.03%, sys=7.84%, ctx=1722385, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1695388/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6622.7MB, aggrb=113023KB/s, minb=113023KB/s, maxb=113023KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1689128/10, merge=156/5, ticks=386848/0, in_queue=386776, util=99.63%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=29739: Mon Dec 16 01:32:16 2019
  write: io=2204.8MB, bw=37628KB/s, iops=9406, runt= 60001msec
    clat (usec): min=71, max=106108, avg=844.73, stdev=4994.03
     lat (usec): min=72, max=106109, avg=845.63, stdev=4994.04
    clat percentiles (usec):
     |  1.00th=[   77],  5.00th=[   78], 10.00th=[   79], 20.00th=[   81],
     | 30.00th=[   82], 40.00th=[   84], 50.00th=[   86], 60.00th=[   89],
     | 70.00th=[   95], 80.00th=[  105], 90.00th=[  126], 95.00th=[  161],
     | 99.00th=[33536], 99.50th=[35584], 99.90th=[41216], 99.95th=[45824],
     | 99.99th=[54016]
    bw (KB  /s): min= 2250, max= 5573, per=12.51%, avg=4707.66, stdev=501.18
    lat (usec) : 100=75.26%, 250=21.53%, 500=0.56%, 750=0.15%, 1000=0.06%
    lat (msec) : 2=0.12%, 4=0.10%, 10=0.01%, 20=0.01%, 50=2.19%
    lat (msec) : 100=0.02%, 250=0.01%
  cpu          : usr=0.78%, sys=2.67%, ctx=1141856, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=564422/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2204.8MB, aggrb=37627KB/s, minb=37627KB/s, maxb=37627KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=55/563927, merge=144/1523, ticks=8/44340, in_queue=44224, util=73.69%

fio (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28829: Sun Dec 15 16:28:01 2019
  read : io=3631.9MB, bw=61959KB/s, iops=15489, runt= 60023msec
    clat (usec): min=2, max=134462, avg=509.94, stdev=1233.42
     lat (usec): min=2, max=134463, avg=510.93, stdev=1233.79
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[  189], 10.00th=[  207], 20.00th=[  223],
     | 30.00th=[  235], 40.00th=[  247], 50.00th=[  270], 60.00th=[  298],
     | 70.00th=[  346], 80.00th=[  430], 90.00th=[  652], 95.00th=[ 1320],
     | 99.00th=[ 5408], 99.50th=[ 7840], 99.90th=[14528], 99.95th=[17792],
     | 99.99th=[34048]
    bw (KB  /s): min= 1440, max=16184, per=12.55%, avg=7778.51, stdev=3012.08
    lat (usec) : 4=2.91%, 10=0.57%, 20=0.18%, 50=0.02%, 100=0.14%
    lat (usec) : 250=37.38%, 500=43.43%, 750=7.02%, 1000=1.99%
    lat (msec) : 2=2.60%, 4=1.95%, 10=1.55%, 20=0.24%, 50=0.02%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.38%, sys=5.15%, ctx=905144, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=929741/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=3631.9MB, aggrb=61958KB/s, minb=61958KB/s, maxb=61958KB/s, mint=60023msec, maxt=60023msec

Disk stats (read/write):
  sda: ios=895279/10, merge=144/5, ticks=396812/12, in_queue=396516, util=97.96%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28855: Sun Dec 15 16:30:07 2019
  write: io=19601MB, bw=334417KB/s, iops=83604, runt= 60019msec
    clat (usec): min=2, max=260881, avg=60.05, stdev=1166.81
     lat (usec): min=2, max=260882, avg=66.48, stdev=1253.78
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[   48], 99.50th=[ 2128], 99.90th=[15680], 99.95th=[21376],
     | 99.99th=[40192]
    bw (KB  /s): min=    0, max=73357, per=12.86%, avg=43019.84, stdev=11843.30
    lat (usec) : 4=60.71%, 10=37.73%, 20=0.44%, 50=0.13%, 100=0.16%
    lat (usec) : 250=0.13%, 500=0.08%, 750=0.03%, 1000=0.02%
    lat (msec) : 2=0.06%, 4=0.11%, 10=0.21%, 20=0.12%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=3.95%, sys=5.77%, ctx=81576, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5017839/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=19601MB, aggrb=334416KB/s, minb=334416KB/s, maxb=334416KB/s, mint=60019msec, maxt=60019msec

Disk stats (read/write):
  sda: ios=699/1536338, merge=60/1385, ticks=532/5432372, in_queue=5430380, util=77.08%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28842: Sun Dec 15 16:29:06 2019
  read : io=4286.8MB, bw=73146KB/s, iops=18286, runt= 60011msec
    clat (usec): min=66, max=157991, avg=431.19, stdev=1243.53
     lat (usec): min=67, max=157992, avg=432.10, stdev=1243.78
    clat percentiles (usec):
     |  1.00th=[  113],  5.00th=[  191], 10.00th=[  203], 20.00th=[  215],
     | 30.00th=[  225], 40.00th=[  235], 50.00th=[  249], 60.00th=[  266],
     | 70.00th=[  290], 80.00th=[  338], 90.00th=[  478], 95.00th=[  788],
     | 99.00th=[ 4640], 99.50th=[ 7456], 99.90th=[14912], 99.95th=[18304],
     | 99.99th=[48384]
    bw (KB  /s): min=  924, max=15136, per=12.56%, avg=9184.53, stdev=3293.69
    lat (usec) : 100=0.62%, 250=50.61%, 500=39.45%, 750=4.09%, 1000=1.23%
    lat (msec) : 2=1.62%, 4=1.06%, 10=1.06%, 20=0.24%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.62%, sys=5.73%, ctx=1112738, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1097395/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4286.8MB, aggrb=73146KB/s, minb=73146KB/s, maxb=73146KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  sda: ios=1096684/10, merge=159/5, ticks=395396/4, in_queue=395132, util=99.05%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28870: Sun Dec 15 16:31:07 2019
  write: io=750168KB, bw=12501KB/s, iops=3125, runt= 60008msec
    clat (usec): min=76, max=396545, avg=2550.32, stdev=14063.14
     lat (usec): min=77, max=396546, avg=2551.50, stdev=14063.56
    clat percentiles (usec):
     |  1.00th=[   81],  5.00th=[   84], 10.00th=[   87], 20.00th=[   94],
     | 30.00th=[  101], 40.00th=[  112], 50.00th=[  129], 60.00th=[  151],
     | 70.00th=[  179], 80.00th=[  243], 90.00th=[  548], 95.00th=[ 3376],
     | 99.00th=[74240], 99.50th=[98816], 99.90th=[164864], 99.95th=[189440],
     | 99.99th=[276480]
    bw (KB  /s): min=  103, max= 4120, per=12.70%, avg=1587.53, stdev=808.38
    lat (usec) : 100=28.22%, 250=52.47%, 500=8.76%, 750=1.96%, 1000=0.91%
    lat (msec) : 2=1.63%, 4=1.65%, 10=0.97%, 20=0.19%, 50=1.17%
    lat (msec) : 100=1.60%, 250=0.47%, 500=0.01%
  cpu          : usr=0.36%, sys=1.58%, ctx=383096, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=187542/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=750168KB, aggrb=12501KB/s, minb=12501KB/s, maxb=12501KB/s, mint=60008msec, maxt=60008msec

Disk stats (read/write):
  sda: ios=232/187357, merge=433/2112, ticks=520/52288, in_queue=52668, util=85.28%

fio (БША, Атланта)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1353: Thu Dec  5 08:14:33 2019
  read : io=5863.2MB, bw=100061KB/s, iops=25015, runt= 60002msec
    clat (usec): min=1, max=109953, avg=316.23, stdev=905.11
     lat (usec): min=1, max=109954, avg=316.70, stdev=905.29
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[   10], 10.00th=[  169], 20.00th=[  195],
     | 30.00th=[  213], 40.00th=[  227], 50.00th=[  245], 60.00th=[  266],
     | 70.00th=[  298], 80.00th=[  342], 90.00th=[  418], 95.00th=[  510],
     | 99.00th=[ 1288], 99.50th=[ 2576], 99.90th=[12480], 99.95th=[17792],
     | 99.99th=[43264]
    bw (KB  /s): min= 4007, max=19840, per=12.49%, avg=12499.98, stdev=3205.40
    lat (usec) : 2=0.12%, 4=1.95%, 10=2.75%, 20=1.10%, 50=0.12%
    lat (usec) : 100=0.27%, 250=46.38%, 500=41.94%, 750=3.30%, 1000=0.66%
    lat (msec) : 2=0.73%, 4=0.38%, 10=0.16%, 20=0.10%, 50=0.03%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.12%, sys=7.37%, ctx=1424468, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1500958/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=5863.2MB, aggrb=100060KB/s, minb=100060KB/s, maxb=100060KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1407476/10, merge=144/5, ticks=367876/0, in_queue=367508, util=98.27%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1386: Thu Dec  5 08:16:37 2019
  write: io=30002MB, bw=511974KB/s, iops=127993, runt= 60007msec
    clat (usec): min=1, max=168979, avg=47.39, stdev=843.23
     lat (usec): min=1, max=270942, avg=50.03, stdev=898.26
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    3], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[   16], 99.50th=[  382], 99.90th=[14912], 99.95th=[19328],
     | 99.99th=[29056]
    bw (KB  /s): min=    1, max=89640, per=12.64%, avg=64734.89, stdev=7943.05
    lat (usec) : 2=0.09%, 4=96.74%, 10=2.01%, 20=0.19%, 50=0.16%
    lat (usec) : 100=0.19%, 250=0.10%, 500=0.04%, 750=0.03%, 1000=0.02%
    lat (msec) : 2=0.05%, 4=0.05%, 10=0.15%, 20=0.14%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.71%, sys=7.21%, ctx=89916, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=7680511/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=30002MB, aggrb=511974KB/s, minb=511974KB/s, maxb=511974KB/s, mint=60007msec, maxt=60007msec

Disk stats (read/write):
  sda: ios=149/2401249, merge=59/1413, ticks=224/4709572, in_queue=4705400, util=65.68%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1366: Thu Dec  5 08:15:37 2019
  read : io=4400.3MB, bw=75092KB/s, iops=18773, runt= 60004msec
    clat (usec): min=54, max=146994, avg=421.38, stdev=1591.75
     lat (usec): min=55, max=146995, avg=422.04, stdev=1592.67
    clat percentiles (usec):
     |  1.00th=[   97],  5.00th=[  159], 10.00th=[  173], 20.00th=[  193],
     | 30.00th=[  209], 40.00th=[  223], 50.00th=[  241], 60.00th=[  266],
     | 70.00th=[  302], 80.00th=[  354], 90.00th=[  462], 95.00th=[  676],
     | 99.00th=[ 4048], 99.50th=[ 9280], 99.90th=[19584], 99.95th=[28032],
     | 99.99th=[60672]
    bw (KB  /s): min= 1760, max=19184, per=12.53%, avg=9406.77, stdev=3435.47
    lat (usec) : 100=1.10%, 250=52.98%, 500=37.72%, 750=3.93%, 1000=1.36%
    lat (msec) : 2=1.36%, 4=0.55%, 10=0.55%, 20=0.37%, 50=0.08%
    lat (msec) : 100=0.02%, 250=0.01%
  cpu          : usr=1.07%, sys=5.29%, ctx=1151545, majf=0, minf=67
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1126460/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4400.3MB, aggrb=75092KB/s, minb=75092KB/s, maxb=75092KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=1124490/19, merge=311/14, ticks=392884/24, in_queue=392608, util=98.99%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1399: Thu Dec  5 08:17:38 2019
  write: io=1424.9MB, bw=24315KB/s, iops=6078, runt= 60003msec
    clat (usec): min=66, max=168724, avg=1311.68, stdev=7949.99
     lat (usec): min=66, max=168724, avg=1312.29, stdev=7950.64
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   73], 10.00th=[   75], 20.00th=[   78],
     | 30.00th=[   81], 40.00th=[   86], 50.00th=[   91], 60.00th=[  100],
     | 70.00th=[  118], 80.00th=[  167], 90.00th=[  286], 95.00th=[  390],
     | 99.00th=[43776], 99.50th=[58112], 99.90th=[91648], 99.95th=[103936],
     | 99.99th=[125440]
    bw (KB  /s): min=  600, max= 5231, per=12.54%, avg=3048.58, stdev=962.56
    lat (usec) : 100=59.56%, 250=28.19%, 500=8.00%, 750=0.69%, 1000=0.39%
    lat (msec) : 2=0.43%, 4=0.14%, 10=0.14%, 20=0.08%, 50=1.67%
    lat (msec) : 100=0.63%, 250=0.06%
  cpu          : usr=0.39%, sys=2.24%, ctx=738676, majf=0, minf=59
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=364746/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1424.9MB, aggrb=24315KB/s, minb=24315KB/s, maxb=24315KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=152/363129, merge=338/1944, ticks=264/48728, in_queue=48864, util=79.69%

fio (БША, Даллас)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1700: Wed Dec 11 14:48:11 2019
  read : io=6200.4MB, bw=105812KB/s, iops=26452, runt= 60001msec
    clat (usec): min=2, max=102104, avg=296.47, stdev=729.41
     lat (usec): min=2, max=102105, avg=297.48, stdev=729.66
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[  201], 20.00th=[  223],
     | 30.00th=[  233], 40.00th=[  245], 50.00th=[  258], 60.00th=[  274],
     | 70.00th=[  290], 80.00th=[  314], 90.00th=[  358], 95.00th=[  422],
     | 99.00th=[ 1064], 99.50th=[ 2064], 99.90th=[ 6816], 99.95th=[10048],
     | 99.99th=[36608]
    bw (KB  /s): min= 7008, max=19088, per=12.50%, avg=13230.61, stdev=1790.79
    lat (usec) : 4=4.58%, 10=1.55%, 20=0.51%, 50=0.02%, 100=0.14%
    lat (usec) : 250=36.05%, 500=54.16%, 750=1.36%, 1000=0.53%
    lat (msec) : 2=0.56%, 4=0.36%, 10=0.13%, 20=0.03%, 50=0.02%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.03%, sys=8.96%, ctx=1487519, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1587201/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6200.4MB, aggrb=105811KB/s, minb=105811KB/s, maxb=105811KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1477598/10, merge=134/5, ticks=337536/0, in_queue=337040, util=98.37%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1726: Wed Dec 11 14:50:15 2019
  write: io=22194MB, bw=378745KB/s, iops=94686, runt= 60006msec
    clat (usec): min=2, max=116106, avg=52.63, stdev=827.76
     lat (usec): min=2, max=116107, avg=58.41, stdev=892.15
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[   97], 99.50th=[ 2320], 99.90th=[13888], 99.95th=[19328],
     | 99.99th=[29056]
    bw (KB  /s): min=    1, max=60735, per=12.65%, avg=47903.56, stdev=4565.88
    lat (usec) : 4=45.50%, 10=52.63%, 20=0.53%, 50=0.15%, 100=0.21%
    lat (usec) : 250=0.19%, 500=0.11%, 750=0.05%, 1000=0.03%
    lat (msec) : 2=0.09%, 4=0.14%, 10=0.22%, 20=0.12%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.18%, sys=6.17%, ctx=119947, majf=0, minf=74
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5681736/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=22194MB, aggrb=378744KB/s, minb=378744KB/s, maxb=378744KB/s, mint=60006msec, maxt=60006msec

Disk stats (read/write):
  sda: ios=204/1655764, merge=0/1663, ticks=432/5708944, in_queue=5711996, util=75.42%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1713: Wed Dec 11 14:49:15 2019
  read : io=6961.6MB, bw=118808KB/s, iops=29702, runt= 60001msec
    clat (usec): min=67, max=100512, avg=263.94, stdev=572.86
     lat (usec): min=68, max=100513, avg=264.80, stdev=572.88
    clat percentiles (usec):
     |  1.00th=[  149],  5.00th=[  199], 10.00th=[  205], 20.00th=[  213],
     | 30.00th=[  225], 40.00th=[  231], 50.00th=[  237], 60.00th=[  243],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  306], 95.00th=[  346],
     | 99.00th=[  516], 99.50th=[  932], 99.90th=[ 3088], 99.95th=[ 6304],
     | 99.99th=[14784]
    bw (KB  /s): min= 7356, max=17056, per=12.51%, avg=14865.38, stdev=1547.41
    lat (usec) : 100=0.36%, 250=65.31%, 500=33.25%, 750=0.45%, 1000=0.17%
    lat (msec) : 2=0.26%, 4=0.12%, 10=0.07%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.30%, sys=9.10%, ctx=1787510, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1782151/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6961.6MB, aggrb=118808KB/s, minb=118808KB/s, maxb=118808KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1775430/12, merge=142/6, ticks=374772/0, in_queue=374608, util=99.88%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=1739: Wed Dec 11 14:51:16 2019
  write: io=2319.8MB, bw=39588KB/s, iops=9897, runt= 60003msec
    clat (usec): min=72, max=129405, avg=802.66, stdev=4689.80
     lat (usec): min=73, max=129406, avg=803.62, stdev=4689.82
    clat percentiles (usec):
     |  1.00th=[   76],  5.00th=[   78], 10.00th=[   79], 20.00th=[   81],
     | 30.00th=[   82], 40.00th=[   84], 50.00th=[   86], 60.00th=[   88],
     | 70.00th=[   91], 80.00th=[   97], 90.00th=[  111], 95.00th=[  137],
     | 99.00th=[30848], 99.50th=[33024], 99.90th=[37632], 99.95th=[41216],
     | 99.99th=[62720]
    bw (KB  /s): min= 2833, max= 6248, per=12.52%, avg=4956.34, stdev=397.19
    lat (usec) : 100=83.08%, 250=14.10%, 500=0.21%, 750=0.09%, 1000=0.05%
    lat (msec) : 2=0.10%, 4=0.08%, 10=0.01%, 20=0.01%, 50=2.27%
    lat (msec) : 100=0.02%, 250=0.01%
  cpu          : usr=0.88%, sys=2.91%, ctx=1201789, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=593856/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2319.8MB, aggrb=39588KB/s, minb=39588KB/s, maxb=39588KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=56/593337, merge=143/1500, ticks=12/43260, in_queue=43144, util=71.87%

fio (БША, Π€Ρ€ΠΈΠΌΠΎΠ½Ρ‚)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30282: Wed Dec 11 18:48:11 2019
  read : io=7310.9MB, bw=124764KB/s, iops=31190, runt= 60004msec
    clat (usec): min=2, max=108489, avg=250.77, stdev=583.00
     lat (usec): min=2, max=108489, avg=251.65, stdev=583.02
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[  159], 20.00th=[  211],
     | 30.00th=[  223], 40.00th=[  231], 50.00th=[  239], 60.00th=[  247],
     | 70.00th=[  258], 80.00th=[  274], 90.00th=[  306], 95.00th=[  342],
     | 99.00th=[  540], 99.50th=[ 1368], 99.90th=[ 3504], 99.95th=[ 6496],
     | 99.99th=[13888]
    bw (KB  /s): min= 7848, max=20600, per=12.51%, avg=15607.82, stdev=1622.54
    lat (usec) : 4=6.90%, 10=0.84%, 20=0.19%, 50=0.01%, 100=0.37%
    lat (usec) : 250=53.78%, 500=36.77%, 750=0.37%, 1000=0.14%
    lat (msec) : 2=0.23%, 4=0.30%, 10=0.07%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.17%, sys=8.32%, ctx=1751322, majf=3, minf=88
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1871578/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7310.9MB, aggrb=124763KB/s, minb=124763KB/s, maxb=124763KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=1719411/10, merge=136/5, ticks=367604/0, in_queue=367292, util=98.42%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30308: Wed Dec 11 18:50:15 2019
  write: io=22228MB, bw=379297KB/s, iops=94824, runt= 60010msec
    clat (usec): min=2, max=71934, avg=51.74, stdev=781.09
     lat (usec): min=3, max=76027, avg=57.84, stdev=846.45
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[  105], 99.50th=[ 2640], 99.90th=[12864], 99.95th=[18304],
     | 99.99th=[28032]
    bw (KB  /s): min=    1, max=57849, per=12.64%, avg=47960.37, stdev=3393.26
    lat (usec) : 4=52.13%, 10=46.07%, 20=0.42%, 50=0.11%, 100=0.22%
    lat (usec) : 250=0.21%, 500=0.11%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.09%, 4=0.16%, 10=0.24%, 20=0.11%, 50=0.04%
    lat (msec) : 100=0.01%
  cpu          : usr=4.16%, sys=6.13%, ctx=122895, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5690402/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=22228MB, aggrb=379296KB/s, minb=379296KB/s, maxb=379296KB/s, mint=60010msec, maxt=60010msec

Disk stats (read/write):
  sda: ios=172/1796795, merge=31/1518, ticks=196/5151300, in_queue=5149896, util=69.28%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30295: Wed Dec 11 18:49:15 2019
  read : io=7053.2MB, bw=120372KB/s, iops=30093, runt= 60001msec
    clat (usec): min=67, max=149994, avg=259.65, stdev=564.00
     lat (usec): min=68, max=149995, avg=260.50, stdev=564.01
    clat percentiles (usec):
     |  1.00th=[  118],  5.00th=[  193], 10.00th=[  205], 20.00th=[  217],
     | 30.00th=[  225], 40.00th=[  231], 50.00th=[  239], 60.00th=[  247],
     | 70.00th=[  258], 80.00th=[  270], 90.00th=[  298], 95.00th=[  326],
     | 99.00th=[  442], 99.50th=[  732], 99.90th=[ 3536], 99.95th=[ 6752],
     | 99.99th=[14016]
    bw (KB  /s): min= 7751, max=16512, per=12.51%, avg=15054.93, stdev=1315.85
    lat (usec) : 100=0.47%, 250=63.33%, 500=35.47%, 750=0.24%, 1000=0.10%
    lat (msec) : 2=0.20%, 4=0.09%, 10=0.08%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.33%, sys=8.06%, ctx=1846033, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1805615/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7053.2MB, aggrb=120372KB/s, minb=120372KB/s, maxb=120372KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1803762/10, merge=144/5, ticks=382228/0, in_queue=382160, util=99.88%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=30321: Wed Dec 11 18:51:16 2019
  write: io=2204.4MB, bw=37617KB/s, iops=9404, runt= 60005msec
    clat (usec): min=75, max=155277, avg=845.18, stdev=5068.01
     lat (usec): min=76, max=155278, avg=846.06, stdev=5068.02
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   81], 10.00th=[   82], 20.00th=[   84],
     | 30.00th=[   85], 40.00th=[   87], 50.00th=[   90], 60.00th=[   92],
     | 70.00th=[   97], 80.00th=[  104], 90.00th=[  122], 95.00th=[  147],
     | 99.00th=[34048], 99.50th=[36096], 99.90th=[40704], 99.95th=[43776],
     | 99.99th=[55040]
    bw (KB  /s): min= 2514, max= 5664, per=12.51%, avg=4707.09, stdev=458.93
    lat (usec) : 100=74.69%, 250=22.43%, 500=0.32%, 750=0.13%, 1000=0.06%
    lat (msec) : 2=0.13%, 4=0.09%, 10=0.01%, 20=0.01%, 50=2.14%
    lat (msec) : 100=0.02%, 250=0.01%
  cpu          : usr=0.73%, sys=2.64%, ctx=1141286, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=564308/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2204.4MB, aggrb=37617KB/s, minb=37617KB/s, maxb=37617KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  sda: ios=63/563824, merge=144/1549, ticks=68/44880, in_queue=44828, util=74.54%

fio (БША, ΠΡŒΡŽΠ°Ρ€ΠΊ)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=10448: Thu Dec 12 13:06:37 2019
  read : io=7558.4MB, bw=128994KB/s, iops=32248, runt= 60001msec
    clat (usec): min=1, max=1002.2K, avg=244.98, stdev=946.83
     lat (usec): min=1, max=1002.2K, avg=245.40, stdev=946.83
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[  153], 20.00th=[  183],
     | 30.00th=[  199], 40.00th=[  211], 50.00th=[  223], 60.00th=[  239],
     | 70.00th=[  262], 80.00th=[  298], 90.00th=[  354], 95.00th=[  410],
     | 99.00th=[  596], 99.50th=[  844], 99.90th=[ 2544], 99.95th=[ 2960],
     | 99.99th=[13888]
    bw (KB  /s): min= 4981, max=22968, per=12.52%, avg=16154.17, stdev=1818.71
    lat (usec) : 2=1.77%, 4=3.76%, 10=1.80%, 20=0.40%, 50=0.18%
    lat (usec) : 100=0.47%, 250=57.15%, 500=32.56%, 750=1.32%, 1000=0.19%
    lat (msec) : 2=0.19%, 4=0.20%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 2000=0.01%
  cpu          : usr=1.24%, sys=7.51%, ctx=1795177, majf=2, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1934945/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7558.4MB, aggrb=128994KB/s, minb=128994KB/s, maxb=128994KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1775630/10, merge=144/5, ticks=400756/0, in_queue=400528, util=98.21%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=10474: Thu Dec 12 13:08:40 2019
  write: io=32935MB, bw=561990KB/s, iops=140497, runt= 60011msec
    clat (usec): min=1, max=71142, avg=42.34, stdev=755.46
     lat (usec): min=1, max=71142, avg=44.95, stdev=799.24
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    3], 90.00th=[    3], 95.00th=[    3],
     | 99.00th=[   14], 99.50th=[  378], 99.90th=[13120], 99.95th=[17536],
     | 99.99th=[26752]
    bw (KB  /s): min=    1, max=90912, per=12.64%, avg=71015.53, stdev=7060.81
    lat (usec) : 2=1.50%, 4=96.15%, 10=1.26%, 20=0.13%, 50=0.11%
    lat (usec) : 100=0.22%, 250=0.10%, 500=0.04%, 750=0.04%, 1000=0.03%
    lat (msec) : 2=0.05%, 4=0.05%, 10=0.16%, 20=0.12%, 50=0.03%
    lat (msec) : 100=0.01%
  cpu          : usr=1.88%, sys=7.08%, ctx=102574, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=8431394/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=32935MB, aggrb=561989KB/s, minb=561989KB/s, maxb=561989KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  sda: ios=204/2664201, merge=0/1530, ticks=96/5015780, in_queue=5011968, util=66.88%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=10461: Thu Dec 12 13:07:39 2019
  read : io=7272.5MB, bw=124105KB/s, iops=31026, runt= 60002msec
    clat (usec): min=52, max=150788, avg=254.66, stdev=558.22
     lat (usec): min=52, max=150788, avg=255.12, stdev=558.23
    clat percentiles (usec):
     |  1.00th=[  129],  5.00th=[  159], 10.00th=[  169], 20.00th=[  185],
     | 30.00th=[  199], 40.00th=[  209], 50.00th=[  221], 60.00th=[  237],
     | 70.00th=[  262], 80.00th=[  302], 90.00th=[  358], 95.00th=[  418],
     | 99.00th=[  564], 99.50th=[  708], 99.90th=[ 1928], 99.95th=[ 2832],
     | 99.99th=[10816]
    bw (KB  /s): min= 8656, max=22752, per=12.51%, avg=15520.89, stdev=1889.80
    lat (usec) : 100=0.52%, 250=65.45%, 500=32.26%, 750=1.34%, 1000=0.19%
    lat (msec) : 2=0.16%, 4=0.07%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=1.28%, sys=6.40%, ctx=1876818, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1861643/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7272.5MB, aggrb=124105KB/s, minb=124105KB/s, maxb=124105KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1861380/10, merge=144/5, ticks=390012/0, in_queue=389716, util=99.03%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=10487: Thu Dec 12 13:09:40 2019
  write: io=2191.1MB, bw=37407KB/s, iops=9351, runt= 60004msec
    clat (usec): min=64, max=485184, avg=852.18, stdev=5707.61
     lat (usec): min=65, max=485184, avg=852.64, stdev=5707.61
    clat percentiles (usec):
     |  1.00th=[   69],  5.00th=[   70], 10.00th=[   72], 20.00th=[   73],
     | 30.00th=[   75], 40.00th=[   76], 50.00th=[   79], 60.00th=[   82],
     | 70.00th=[   87], 80.00th=[  104], 90.00th=[  187], 95.00th=[  266],
     | 99.00th=[38656], 99.50th=[40704], 99.90th=[45824], 99.95th=[52480],
     | 99.99th=[88576]
    bw (KB  /s): min=  311, max= 6184, per=12.54%, avg=4691.88, stdev=732.96
    lat (usec) : 100=78.71%, 250=15.08%, 500=3.93%, 750=0.17%, 1000=0.08%
    lat (msec) : 2=0.08%, 4=0.02%, 10=0.01%, 20=0.01%, 50=1.83%
    lat (msec) : 100=0.06%, 250=0.01%, 500=0.01%
  cpu          : usr=0.42%, sys=2.55%, ctx=1133358, majf=0, minf=75
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=561138/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=2191.1MB, aggrb=37406KB/s, minb=37406KB/s, maxb=37406KB/s, mint=60004msec, maxt=60004msec

Disk stats (read/write):
  sda: ios=55/560849, merge=144/1569, ticks=48/45804, in_queue=45712, util=76.12%

Speedtest (Австралия, Π‘ΠΈΠ΄Π½Π΅ΠΉ)

Retrieving speedtest.net configuration...
Testing from Linode (172.105.166.138)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by 'Yes' Optus (Sydney) [1.01 km]: 5.273 ms
Testing download speed................................................................................
Download: 3748.73 Mbit/s
Testing upload speed................................................................................................
Upload: 1914.90 Mbit/s

Speedtest (Канада, Π’ΠΎΡ€ΠΎΠ½Ρ‚ΠΎ)

Retrieving speedtest.net configuration...
Testing from Linode (172.105.27.90)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Bell Mobility (Toronto, ON) [3.45 km]: 4.353 ms
Testing download speed................................................................................
Download: 2291.58 Mbit/s
Testing upload speed................................................................................................
Upload: 1898.46 Mbit/s

Speedtest (ГСрмания, Π€Ρ€Π°Π½ΠΊΡ„ΡƒΡ€Ρ‚)

Retrieving speedtest.net configuration...
Testing from Linode (172.105.66.241)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Vodafone DE (Frankfurt) [0.89 km]: 2.5 ms
Testing download speed................................................................................
Download: 334.21 Mbit/s
Testing upload speed................................................................................................
Upload: 778.76 Mbit/s

Speedtest (Индия, ΠœΡƒΠΌΠ±Π°ΠΈ)

Retrieving speedtest.net configuration...
Testing from Linode (172.105.53.168)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by I-ON (Mumbai) [13.76 km]: 41.212 ms
Testing download speed................................................................................
Download: 2668.93 Mbit/s
Testing upload speed................................................................................................
Upload: 2456.87 Mbit/s

Speedtest (Япония, Вокио)

Retrieving speedtest.net configuration...
Testing from Linode (172.104.99.120)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by OPEN Project (via 20G SINET) (Tokyo) [6.34 km]: 2.983 ms
Testing download speed................................................................................
Download: 1514.59 Mbit/s
Testing upload speed................................................................................................
Upload: 742.07 Mbit/s

Speedtest (Π‘ΠΈΠ½Π³Π°ΠΏΡƒΡ€)

Retrieving speedtest.net configuration...
Testing from Linode (139.162.42.89)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by StarHub Mobile Pte Ltd (Singapore) [7.42 km]: 4.323 ms
Testing download speed................................................................................
Download: 2594.09 Mbit/s
Testing upload speed................................................................................................
Upload: 2845.87 Mbit/s

Speedtest (ВСликобритания, Π›ΠΎΠ½Π΄ΠΎΠ½)

Retrieving speedtest.net configuration...
Testing from Linode (85.159.213.129)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Spitfire Network Services (London) [0.92 km]: 3.546 ms
Testing download speed................................................................................
Download: 1292.02 Mbit/s
Testing upload speed................................................................................................
Upload: 1262.76 Mbit/s

Speedtest (БША, Атланта)

Retrieving speedtest.net configuration...
Testing from Linode (45.79.223.127)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Hotwire Fision (Atlanta, GA) [0.37 km]: 1.599 ms
Testing download speed................................................................................
Download: 623.66 Mbit/s
Testing upload speed................................................................................................
Upload: 899.49 Mbit/s

Speedtest (БША, Даллас)

Retrieving speedtest.net configuration...
Testing from Linode (45.33.18.235)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Windstream (Dallas, TX) [2.33 km]: 6.618 ms
Testing download speed................................................................................
Download: 1750.90 Mbit/s
Testing upload speed................................................................................................
Upload: 2639.56 Mbit/s

Speedtest (БША, Π€Ρ€ΠΈΠΌΠΎΠ½Ρ‚)

Retrieving speedtest.net configuration...
Testing from Linode (45.79.226.251)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Converse in Code Networks (Fremont, CA) [1.89 km]: 4.107 ms
Testing download speed................................................................................
Download: 683.26 Mbit/s
Testing upload speed................................................................................................
Upload: 746.06 Mbit/s

Speedtest (БША, ΠΡŒΡŽΠ°Ρ€ΠΊ)

Retrieving speedtest.net configuration...
Testing from Linode (45.79.181.215)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Interserver, inc (Secaucus, NJ) [11.03 km]: 5.212 ms
Testing download speed................................................................................
Download: 1773.85 Mbit/s
Testing upload speed................................................................................................
Upload: 2021.52 Mbit/s

Π’ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹ΠΉ сСрвСр ΠΎΡ‚ Hetzner:

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

  • Ρ‚Π°Ρ€ΠΈΡ„ CX11 (20 GB SSD / 1x vCPU / 2 GB RAM / 10000 Mbit/s Π·Π° 2.99€ Π² мСсяц);
  • Π΄Π°Ρ‚Π°-Ρ†Π΅Π½Ρ‚Ρ€Ρ‹ Π² Π“Π΅Ρ€ΠΌΠ°Π½ΠΈΠΈ, Ѐинляндии.

ВСсты:

Unixbench (ГСрмания, Π€Π°Π»ΡŒΠΊΠ΅Π½ΡˆΡ‚Π΅ΠΉΠ½)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-2gb-fsn1-1: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel Xeon Processor (Skylake, IBRS) (4589.2 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   17:44:12 up  1:02,  1 user,  load average: 0.45, 0.12, 0.04; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 04 2019 17:44:12 - 18:12:22
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       31999500.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4041.0 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4467.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        522204.8 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          132016.7 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1293513.2 KBps  (30.0 s, 2 samples)
Pipe Throughput                              703333.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 105055.3 lps   (10.0 s, 7 samples)
Process Creation                               9959.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7888.1 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1021.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         420618.3 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   31999500.4   2742.0
Double-Precision Whetstone                       55.0       4041.0    734.7
Execl Throughput                                 43.0       4467.7   1039.0
File Copy 1024 bufsize 2000 maxblocks          3960.0     522204.8   1318.7
File Copy 256 bufsize 500 maxblocks            1655.0     132016.7    797.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    1293513.2   2230.2
Pipe Throughput                               12440.0     703333.3    565.4
Pipe-based Context Switching                   4000.0     105055.3    262.6
Process Creation                                126.0       9959.9    790.5
Shell Scripts (1 concurrent)                     42.4       7888.1   1860.4
Shell Scripts (8 concurrent)                      6.0       1021.9   1703.2
System Call Overhead                          15000.0     420618.3    280.4
                                                                   ========
System Benchmarks Index Score                                         945.8

Unixbench (ГСрмания, ΠΡŽΡ€Π½Π±Π΅Ρ€Π³)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-2gb-nbg1-1: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel Xeon Processor (Skylake, IBRS) (4200.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   17:43:43 up  1:01,  1 user,  load average: 0.68, 0.20, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 04 2019 17:43:43 - 18:11:50
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       30981175.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3970.6 MWIPS (9.9 s, 7 samples)
Execl Throughput                               4214.2 lps   (29.6 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        455583.2 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          134162.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1327033.8 KBps  (30.0 s, 2 samples)
Pipe Throughput                              709339.4 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 102866.3 lps   (10.0 s, 7 samples)
Process Creation                               8830.0 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   7109.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    888.5 lpm   (60.1 s, 2 samples)
System Call Overhead                         424010.8 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   30981175.0   2654.8
Double-Precision Whetstone                       55.0       3970.6    721.9
Execl Throughput                                 43.0       4214.2    980.0
File Copy 1024 bufsize 2000 maxblocks          3960.0     455583.2   1150.5
File Copy 256 bufsize 500 maxblocks            1655.0     134162.5    810.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1327033.8   2288.0
Pipe Throughput                               12440.0     709339.4    570.2
Pipe-based Context Switching                   4000.0     102866.3    257.2
Process Creation                                126.0       8830.0    700.8
Shell Scripts (1 concurrent)                     42.4       7109.0   1676.6
Shell Scripts (8 concurrent)                      6.0        888.5   1480.9
System Call Overhead                          15000.0     424010.8    282.7
                                                                   ========
System Benchmarks Index Score                                         901.7

Unixbench (Ѐинляндия, Π₯Сльсинки)

   BYTE UNIX Benchmarks (Version 5.1.3)

   System: debian-2gb-hel1-1: GNU/Linux
   OS: GNU/Linux -- 4.9.0-11-amd64 -- #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel Xeon Processor (Skylake, IBRS) (4200.0 bogomips)
          x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET
   17:45:00 up  1:02,  1 user,  load average: 0.67, 0.22, 0.07; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Wed Dec 04 2019 17:45:00 - 18:13:09
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       29256935.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3758.3 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3536.8 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        416729.8 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          116120.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1211197.2 KBps  (30.0 s, 2 samples)
Pipe Throughput                              636599.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  92831.7 lps   (10.0 s, 7 samples)
Process Creation                               8632.2 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6427.6 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    784.8 lpm   (60.1 s, 2 samples)
System Call Overhead                         378836.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   29256935.7   2507.0
Double-Precision Whetstone                       55.0       3758.3    683.3
Execl Throughput                                 43.0       3536.8    822.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     416729.8   1052.3
File Copy 256 bufsize 500 maxblocks            1655.0     116120.6    701.6
File Copy 4096 bufsize 8000 maxblocks          5800.0    1211197.2   2088.3
Pipe Throughput                               12440.0     636599.3    511.7
Pipe-based Context Switching                   4000.0      92831.7    232.1
Process Creation                                126.0       8632.2    685.1
Shell Scripts (1 concurrent)                     42.4       6427.6   1515.9
Shell Scripts (8 concurrent)                      6.0        784.8   1308.0
System Call Overhead                          15000.0     378836.2    252.6
                                                                   ========
System Benchmarks Index Score                                         817.1

dd (ГСрмания, Π€Π°Π»ΡŒΠΊΠ΅Π½ΡˆΡ‚Π΅ΠΉΠ½)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.1357 s, 685 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 2.41475 s, 889 MB/s

dd (ГСрмания, ΠΡŽΡ€Π½Π±Π΅Ρ€Π³)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.34572 s, 494 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 61.8634 s, 34.7 MB/s

dd (Ѐинляндия, Π₯Сльсинки)Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 1M

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.91158 s, 549 MB/s

Π‘Π»ΠΎΠΊΠ°ΠΌΠΈ ΠΏΠΎ 64K

32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 561.943 s, 3.8 MB/s

fio (ГСрмания, Π€Π°Π»ΡŒΠΊΠ΅Π½ΡˆΡ‚Π΅ΠΉΠ½)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27954: Wed Dec  4 18:13:32 2019
  read : io=12935MB, bw=220758KB/s, iops=55189, runt= 60001msec
    clat (usec): min=1, max=15276, avg=138.82, stdev=194.23
     lat (usec): min=2, max=15277, avg=139.89, stdev=194.55
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[   68], 60.00th=[  189],
     | 70.00th=[  215], 80.00th=[  251], 90.00th=[  326], 95.00th=[  414],
     | 99.00th=[  660], 99.50th=[  812], 99.90th=[ 2192], 99.95th=[ 3056],
     | 99.99th=[ 4448]
    bw (KB  /s): min=13160, max=52192, per=12.50%, avg=27592.76, stdev=9637.34
    lat (usec) : 2=0.13%, 4=41.65%, 10=6.64%, 20=1.32%, 50=0.21%
    lat (usec) : 100=0.19%, 250=29.76%, 500=17.46%, 750=1.99%, 1000=0.37%
    lat (msec) : 2=0.18%, 4=0.09%, 10=0.02%, 20=0.01%
  cpu          : usr=2.75%, sys=7.24%, ctx=1708991, majf=2, minf=110
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=3311429/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=12935MB, aggrb=220758KB/s, minb=220758KB/s, maxb=220758KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1641059/31, merge=0/26, ticks=326344/444, in_queue=326668, util=97.95%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27993: Wed Dec  4 18:15:39 2019
  write: io=30828MB, bw=526046KB/s, iops=131511, runt= 60009msec
    clat (usec): min=1, max=72109, avg=40.58, stdev=834.02
     lat (usec): min=2, max=72109, avg=44.54, stdev=898.63
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
     | 99.00th=[    8], 99.50th=[   12], 99.90th=[15936], 99.95th=[20096],
     | 99.99th=[31104]
    bw (KB  /s): min=    1, max=80631, per=12.58%, avg=66166.54, stdev=7286.63
    lat (usec) : 2=0.01%, 4=74.79%, 10=24.56%, 20=0.28%, 50=0.05%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.08%, 20=0.11%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=4.73%, sys=6.73%, ctx=50571, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=7891869/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=30828MB, aggrb=526045KB/s, minb=526045KB/s, maxb=526045KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  sda: ios=527/424265, merge=89/1974, ticks=692/2002184, in_queue=2002388, util=27.50%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=27977: Wed Dec  4 18:14:37 2019
  read : io=7711.1MB, bw=131613KB/s, iops=32903, runt= 60002msec
    clat (usec): min=61, max=10698, avg=237.50, stdev=110.59
     lat (usec): min=62, max=10706, avg=238.41, stdev=110.73
    clat percentiles (usec):
     |  1.00th=[  117],  5.00th=[  141], 10.00th=[  157], 20.00th=[  177],
     | 30.00th=[  191], 40.00th=[  201], 50.00th=[  213], 60.00th=[  227],
     | 70.00th=[  245], 80.00th=[  274], 90.00th=[  342], 95.00th=[  418],
     | 99.00th=[  612], 99.50th=[  724], 99.90th=[ 1144], 99.95th=[ 1608],
     | 99.99th=[ 2928]
    bw (KB  /s): min= 8408, max=19592, per=12.50%, avg=16453.29, stdev=2036.10
    lat (usec) : 100=0.10%, 250=71.70%, 500=25.83%, 750=1.95%, 1000=0.28%
    lat (msec) : 2=0.11%, 4=0.03%, 10=0.01%, 20=0.01%
  cpu          : usr=2.15%, sys=6.16%, ctx=1978828, majf=0, minf=80
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1974259/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7711.1MB, aggrb=131612KB/s, minb=131612KB/s, maxb=131612KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
  sda: ios=1969675/13, merge=0/7, ticks=379064/4, in_queue=378660, util=99.83%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=28016: Wed Dec  4 18:16:39 2019
  write: io=1513.9MB, bw=25834KB/s, iops=6458, runt= 60005msec
    clat (usec): min=71, max=55895, avg=1232.03, stdev=6253.29
     lat (usec): min=71, max=55898, avg=1233.17, stdev=6253.36
    clat percentiles (usec):
     |  1.00th=[   83],  5.00th=[   87], 10.00th=[   92], 20.00th=[   99],
     | 30.00th=[  106], 40.00th=[  114], 50.00th=[  125], 60.00th=[  143],
     | 70.00th=[  163], 80.00th=[  191], 90.00th=[  249], 95.00th=[  362],
     | 99.00th=[39168], 99.50th=[42240], 99.90th=[46336], 99.95th=[47872],
     | 99.99th=[51456]
    bw (KB  /s): min=  660, max= 4936, per=12.51%, avg=3231.59, stdev=699.18
    lat (usec) : 100=20.10%, 250=70.08%, 500=6.16%, 750=0.58%, 1000=0.05%
    lat (msec) : 2=0.03%, 4=0.02%, 10=0.01%, 20=0.01%, 50=2.97%
    lat (msec) : 100=0.02%
  cpu          : usr=0.64%, sys=2.58%, ctx=787097, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=387547/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1513.9MB, aggrb=25834KB/s, minb=25834KB/s, maxb=25834KB/s, mint=60005msec, maxt=60005msec

Disk stats (read/write):
  sda: ios=680/387270, merge=0/1946, ticks=836/48344, in_queue=49052, util=80.31%

fio (ГСрмания, ΠΡŽΡ€Π½Π±Π΅Ρ€Π³)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4283: Wed Dec  4 18:14:01 2019
  read : io=10029MB, bw=171161KB/s, iops=42790, runt= 60001msec
    clat (usec): min=1, max=50938, avg=181.05, stdev=348.78
     lat (usec): min=2, max=50938, avg=182.04, stdev=349.01
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    4], 10.00th=[    5], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    9], 50.00th=[  157], 60.00th=[  203],
     | 70.00th=[  243], 80.00th=[  302], 90.00th=[  426], 95.00th=[  564],
     | 99.00th=[  988], 99.50th=[ 1272], 99.90th=[ 2928], 99.95th=[ 3888],
     | 99.99th=[12608]
    bw (KB  /s): min=12264, max=34672, per=12.51%, avg=21413.37, stdev=6618.44
    lat (usec) : 2=0.01%, 4=4.65%, 10=35.58%, 20=6.15%, 50=1.07%
    lat (usec) : 100=0.10%, 250=24.16%, 500=21.47%, 750=4.58%, 1000=1.27%
    lat (msec) : 2=0.78%, 4=0.14%, 10=0.03%, 20=0.02%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=2.19%, sys=8.10%, ctx=1364825, majf=2, minf=95
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2567457/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=10029MB, aggrb=171160KB/s, minb=171160KB/s, maxb=171160KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1337530/8, merge=0/4, ticks=280012/0, in_queue=279880, util=97.89%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4309: Wed Dec  4 18:16:07 2019
  write: io=30705MB, bw=523948KB/s, iops=130987, runt= 60010msec
    clat (usec): min=1, max=76630, avg=41.59, stdev=835.90
     lat (usec): min=2, max=82084, avg=45.14, stdev=893.25
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    2], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[    8], 99.50th=[   13], 99.90th=[16064], 99.95th=[20096],
     | 99.99th=[30080]
    bw (KB  /s): min=    1, max=83288, per=12.63%, avg=66191.01, stdev=7467.72
    lat (usec) : 2=0.01%, 4=69.29%, 10=29.98%, 20=0.36%, 50=0.04%
    lat (usec) : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.08%, 20=0.11%, 50=0.06%
    lat (msec) : 100=0.01%
  cpu          : usr=4.42%, sys=7.05%, ctx=56571, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=7860536/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=30705MB, aggrb=523948KB/s, minb=523948KB/s, maxb=523948KB/s, mint=60010msec, maxt=60010msec

Disk stats (read/write):
  sda: ios=138/439786, merge=0/1844, ticks=132/2046436, in_queue=2045992, util=28.66%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4296: Wed Dec  4 18:15:06 2019
  read : io=7458.3MB, bw=127281KB/s, iops=31820, runt= 60003msec
    clat (usec): min=62, max=19424, avg=245.43, stdev=172.33
     lat (usec): min=62, max=19425, avg=246.39, stdev=172.54
    clat percentiles (usec):
     |  1.00th=[  122],  5.00th=[  143], 10.00th=[  155], 20.00th=[  171],
     | 30.00th=[  183], 40.00th=[  197], 50.00th=[  211], 60.00th=[  229],
     | 70.00th=[  251], 80.00th=[  290], 90.00th=[  370], 95.00th=[  458],
     | 99.00th=[  700], 99.50th=[  836], 99.90th=[ 1528], 99.95th=[ 2192],
     | 99.99th=[ 6304]
    bw (KB  /s): min=12104, max=19336, per=12.50%, avg=15914.07, stdev=1323.40
    lat (usec) : 100=0.04%, 250=69.70%, 500=26.60%, 750=2.89%, 1000=0.50%
    lat (msec) : 2=0.21%, 4=0.03%, 10=0.02%, 20=0.01%
  cpu          : usr=2.11%, sys=6.34%, ctx=1914023, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1909305/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=7458.3MB, aggrb=127280KB/s, minb=127280KB/s, maxb=127280KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=1904100/8, merge=0/5, ticks=362412/0, in_queue=362124, util=99.79%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=4323: Wed Dec  4 18:17:08 2019
  write: io=1437.8MB, bw=24535KB/s, iops=6133, runt= 60006msec
    clat (usec): min=59, max=78562, avg=1296.69, stdev=6139.93
     lat (usec): min=60, max=78563, avg=1297.96, stdev=6140.01
    clat percentiles (usec):
     |  1.00th=[   71],  5.00th=[   75], 10.00th=[   79], 20.00th=[   88],
     | 30.00th=[  100], 40.00th=[  114], 50.00th=[  131], 60.00th=[  151],
     | 70.00th=[  173], 80.00th=[  207], 90.00th=[  290], 95.00th=[  470],
     | 99.00th=[35584], 99.50th=[39168], 99.90th=[45824], 99.95th=[48896],
     | 99.99th=[55552]
    bw (KB  /s): min=  718, max= 5150, per=12.50%, avg=3067.24, stdev=665.86
    lat (usec) : 100=29.91%, 250=56.93%, 500=8.50%, 750=0.97%, 1000=0.13%
    lat (msec) : 2=0.07%, 4=0.02%, 10=0.02%, 20=0.02%, 50=3.39%
    lat (msec) : 100=0.04%
  cpu          : usr=0.72%, sys=2.84%, ctx=749031, majf=0, minf=73
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=368068/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1437.8MB, aggrb=24535KB/s, minb=24535KB/s, maxb=24535KB/s, mint=60006msec, maxt=60006msec

Disk stats (read/write):
  sda: ios=418/367714, merge=0/2000, ticks=376/46720, in_queue=46952, util=77.54%

fio (Ѐинляндия, Π₯Сльсинки)Read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16484: Wed Dec  4 18:23:39 2019
  read : io=9472.7MB, bw=161663KB/s, iops=40415, runt= 60001msec
    clat (usec): min=1, max=134988, avg=191.81, stdev=499.59
     lat (usec): min=2, max=134989, avg=192.84, stdev=499.98
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    4], 20.00th=[    6],
     | 30.00th=[    8], 40.00th=[   10], 50.00th=[  157], 60.00th=[  207],
     | 70.00th=[  249], 80.00th=[  318], 90.00th=[  458], 95.00th=[  612],
     | 99.00th=[ 1064], 99.50th=[ 1368], 99.90th=[ 3280], 99.95th=[ 4320],
     | 99.99th=[15424]
    bw (KB  /s): min= 2974, max=37808, per=12.49%, avg=20198.33, stdev=6446.07
    lat (usec) : 2=0.01%, 4=8.45%, 10=31.44%, 20=6.28%, 50=1.26%
    lat (usec) : 100=0.06%, 250=22.59%, 500=21.67%, 750=5.44%, 1000=1.60%
    lat (msec) : 2=0.99%, 4=0.14%, 10=0.05%, 20=0.02%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=2.26%, sys=8.06%, ctx=1284108, majf=2, minf=113
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2424986/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=9472.7MB, aggrb=161663KB/s, minb=161663KB/s, maxb=161663KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sda: ios=1264059/8, merge=0/4, ticks=269492/0, in_queue=269092, util=97.51%

Write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16510: Wed Dec  4 18:25:44 2019
  write: io=27890MB, bw=475947KB/s, iops=118986, runt= 60006msec
    clat (usec): min=1, max=124945, avg=46.69, stdev=899.56
     lat (usec): min=2, max=124946, avg=50.31, stdev=954.85
    clat percentiles (usec):
     |  1.00th=[    2],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    5],
     | 99.00th=[   10], 99.50th=[   18], 99.90th=[16064], 99.95th=[20608],
     | 99.99th=[31104]
    bw (KB  /s): min=    1, max=75300, per=12.61%, avg=59998.47, stdev=7600.10
    lat (usec) : 2=0.01%, 4=49.88%, 10=48.95%, 20=0.69%, 50=0.11%
    lat (usec) : 100=0.03%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.02%, 10=0.09%, 20=0.13%, 50=0.07%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=4.33%, sys=7.01%, ctx=51459, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=7139920/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=27890MB, aggrb=475947KB/s, minb=475947KB/s, maxb=475947KB/s, mint=60006msec, maxt=60006msec

Disk stats (read/write):
  sda: ios=76/415614, merge=61/1682, ticks=120/1827512, in_queue=1827116, util=26.02%

Direct read

benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16497: Wed Dec  4 18:24:44 2019
  read : io=6984.9MB, bw=119201KB/s, iops=29800, runt= 60003msec
    clat (usec): min=67, max=20518, avg=261.82, stdev=203.48
     lat (usec): min=67, max=20518, avg=262.86, stdev=203.69
    clat percentiles (usec):
     |  1.00th=[  119],  5.00th=[  141], 10.00th=[  153], 20.00th=[  173],
     | 30.00th=[  187], 40.00th=[  203], 50.00th=[  219], 60.00th=[  239],
     | 70.00th=[  266], 80.00th=[  314], 90.00th=[  418], 95.00th=[  524],
     | 99.00th=[  772], 99.50th=[  916], 99.90th=[ 1816], 99.95th=[ 3088],
     | 99.99th=[ 7904]
    bw (KB  /s): min=10992, max=18064, per=12.50%, avg=14901.36, stdev=1197.05
    lat (usec) : 100=0.05%, 250=64.41%, 500=29.73%, 750=4.67%, 1000=0.78%
    lat (msec) : 2=0.28%, 4=0.05%, 10=0.03%, 20=0.01%, 50=0.01%
  cpu          : usr=2.08%, sys=6.62%, ctx=1793721, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1788109/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=6984.9MB, aggrb=119201KB/s, minb=119201KB/s, maxb=119201KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=1785701/10, merge=0/6, ticks=352024/0, in_queue=351460, util=99.73%

Direct write

benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=16523: Wed Dec  4 18:26:45 2019
  write: io=1400.5MB, bw=23899KB/s, iops=5974, runt= 60003msec
    clat (usec): min=60, max=100206, avg=1331.72, stdev=6337.35
     lat (usec): min=61, max=100207, avg=1332.97, stdev=6337.58
    clat percentiles (usec):
     |  1.00th=[   72],  5.00th=[   78], 10.00th=[   83], 20.00th=[   92],
     | 30.00th=[  102], 40.00th=[  116], 50.00th=[  133], 60.00th=[  151],
     | 70.00th=[  175], 80.00th=[  211], 90.00th=[  302], 95.00th=[  466],
     | 99.00th=[36608], 99.50th=[39168], 99.90th=[47360], 99.95th=[54528],
     | 99.99th=[72192]
    bw (KB  /s): min=  809, max= 4445, per=12.50%, avg=2986.79, stdev=509.90
    lat (usec) : 100=27.73%, 250=58.01%, 500=9.62%, 750=0.92%, 1000=0.13%
    lat (msec) : 2=0.09%, 4=0.04%, 10=0.03%, 20=0.01%, 50=3.36%
    lat (msec) : 100=0.07%, 250=0.01%
  cpu          : usr=0.71%, sys=2.72%, ctx=729250, majf=0, minf=81
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=358505/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1400.5MB, aggrb=23899KB/s, minb=23899KB/s, maxb=23899KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sda: ios=138/358207, merge=0/2164, ticks=168/47172, in_queue=47212, util=78.36%

Speedtest (ГСрмания, Π€Π°Π»ΡŒΠΊΠ΅Π½ΡˆΡ‚Π΅ΠΉΠ½)

Retrieving speedtest.net configuration...
Testing from Hetzner Online GmbH (78.47.44.61)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by noris network AG (Nuremberg) [8.42 km]: 4.678 ms
Testing download speed................................................................................
Download: 1188.20 Mbit/s
Testing upload speed................................................................................................
Upload: 1315.40 Mbit/s

Speedtest (ГСрмания, ΠΡŽΡ€Π½Π±Π΅Ρ€Π³)

Retrieving speedtest.net configuration...
Testing from Hetzner Online GmbH (88.99.185.76)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Gemeindewerke Numbrecht GmbH (Numbrecht) [143.00 km]: 10.404 ms
Testing download speed................................................................................
Download: 781.57 Mbit/s
Testing upload speed................................................................................................
Upload: 487.99 Mbit/s

Speedtest (Ѐинляндия, Π₯Сльсинки)

Retrieving speedtest.net configuration...
Testing from Hetzner Online GmbH (95.217.11.133)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Cumulus Technologies (Helsinki) [0.18 km]: 1.283 ms
Testing download speed................................................................................
Download: 666.33 Mbit/s
Testing upload speed................................................................................................
Upload: 669.90 Mbit/s

Π Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ (ΠΊΠ°Ρ€Ρ‚ΠΈΠ½ΠΊΠΈ ΠΊΠ»ΠΈΠΊΠ°Π±Π΅Π»ΡŒΠ½Ρ‹):

Unixbench (ΠΏΠΎΠΏΡƒΠ³Π°ΠΈ)

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Π‘ΠΊΠΎΡ€ΠΎΡΡ‚ΡŒ чтСния/записи dd (MB/s)

1M

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

64K

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

ΠšΠΎΠ»ΠΈΡ‡Π΅ΡΡ‚Π²ΠΎ IOPS Π² Ρ€Π°Π·Π½Ρ‹Ρ… Ρ€Π΅ΠΆΠΈΠΌΠ°Ρ… Ρ€Π°Π±ΠΎΡ‚Ρ‹ fio

Random read

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Random write

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Random direct read

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Random direct write

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0

Π‘ΠΊΠΎΡ€ΠΎΡΡ‚ΡŒ сСти ΠΎΡ‚ speedtest (Mbit/s)

ВСстированиС Π²ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Ρ‹Ρ… сСрвСров ΠΎΡ‚ DigitalOcean, Vultr, Linode ΠΈ Hetzner. ЧСловСчСскиС ΠΆΠ΅Ρ€Ρ‚Π²Ρ‹: 0.0
Π‘ΠΈΠ½ΠΈΠΉ β€” download, красный β€” upload.

Как ΠΌΠ½Π΅ показалось, Π½Π°ΠΈΠ±ΠΎΠ»Π΅Π΅ ΡΡ‚Π°Π±ΠΈΠ»ΡŒΠ½Ρ‹Π΅ Ρ€Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ ΠΏΠΎΠΊΠ°Π·Π°Π»ΠΈ Linode ΠΈ Vultr. Hetzner Π½Π΅ΠΌΠ½ΠΎΠ³ΠΎ отстаСт, Π½ΠΎ ΠΎΠ½ ΠΈ относится ΠΊ нСсколько Π΄Ρ€ΡƒΠ³ΠΎΠΉ Ρ†Π΅Π½ΠΎΠ²ΠΎΠΉ ΠΊΠ°Ρ‚Π΅Π³ΠΎΡ€ΠΈΠΈ. DigitalOcean ΠΆΠ΅ Π² Π½Π΅ΠΊΠΎΡ‚ΠΎΡ€Ρ‹Ρ… тСстах (Π² частности Unixbench) уступил ΠΎΡΡ‚Π°Π»ΡŒΠ½Ρ‹ΠΌ, хотя Π² Ρ†Π΅Π»ΠΎΠΌ этот хостСр ΠΌΠ½Π΅ ΠΎΡ‡Π΅Π½ΡŒ нравится.

Волько зарСгистрированныС ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚Π΅Π»ΠΈ ΠΌΠΎΠ³ΡƒΡ‚ ΡƒΡ‡Π°ΡΡ‚Π²ΠΎΠ²Π°Ρ‚ΡŒ Π² опросС. Π’ΠΎΠΉΠ΄ΠΈΡ‚Π΅, поТалуйста.

ΠŸΠΎΠ»ΡŒΠ·ΡƒΠ΅Ρ‚Π΅ΡΡŒ Π»ΠΈ Π²Ρ‹ ΠΊΠ°ΠΊΠΈΠΌ-Π»ΠΈΠ±ΠΎ ΠΈΠ· ниТСпСрСчислСнных хостСров?

  • 50,9%DigitalOcean86

  • 11,2%Vultr19

  • 8,9%Linode15

  • 37,9%Hetzner64

  • 7,7%OVH13

  • 9,5%Scaleway16

  • 13,0%Π”Ρ€ΡƒΠ³ΠΎΠΉ22

  • 15,4%Π•ΡΡ‚ΡŒ свой сСрвСр Π½Π° Π±Π°Π»ΠΊΠΎΠ½Π΅26

ΠŸΡ€ΠΎΠ³ΠΎΠ»ΠΎΡΠΎΠ²Π°Π»ΠΈ 169 ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚Π΅Π»Π΅ΠΉ. Π’ΠΎΠ·Π΄Π΅Ρ€ΠΆΠ°Π»ΠΈΡΡŒ 30 ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚Π΅Π»Π΅ΠΉ.

Π˜ΡΡ‚ΠΎΡ‡Π½ΠΈΠΊ: habr.com