Ma RAM ambiri aulere, NVMe Intel P4500 ndipo chilichonse chimachedwa kwambiri - nkhani yakuwonjezera kosapambana kwa magawo osinthika.

M'nkhaniyi, ndilankhula za zomwe zinachitika posachedwapa ndi imodzi mwa ma seva mumtambo wathu wa VPS, yomwe inandisiya ine ndikupunthwa kwa maola angapo. Ndakhala ndikukonza ndi kuthetseratu ma seva a Linux kwa zaka pafupifupi 15, koma nkhaniyi sikugwirizana ndi zomwe ndimachita - ndinapanga malingaliro angapo abodza ndipo ndinakhala wosimidwa pang'ono ndisanathe kudziwa chomwe chayambitsa vutoli ndikulithetsa. .

Yambani

Timagwiritsa ntchito mtambo wapakatikati, womwe timamanga pa maseva wamba ndi masinthidwe awa - 32 cores, 256 GB RAM ndi 4500TB PCI-E Intel P4 NVMe drive. Timakonda kwambiri kasinthidwe awa chifukwa amachotsa kufunikira kodera nkhawa za IO popereka chiletso choyenera pamtundu wa mtundu wa VM. Chifukwa NVMe Intel P4500 ili ndi magwiridwe antchito ochititsa chidwi, titha kupereka nthawi imodzi zonse zathunthu za IOPS kumakina ndikusunga zosunga zobwezeretsera ku seva yosunga zobwezeretsera ndi zero IOWAIT.

Ndife m'modzi mwa okhulupirira akale omwe sagwiritsa ntchito hyperconverged SDN ndi zinthu zina zowoneka bwino, zamafashoni, zaunyamata kusunga ma VM, tikukhulupirira kuti dongosolo losavuta, ndikosavuta kuthana nalo mumikhalidwe ya "guru wamkulu wapita. kumapiri.” Zotsatira zake, timasunga ma VM mumtundu wa QCOW2 mu XFS kapena EXT4, yomwe imayikidwa pamwamba pa LVM2.

Timakakamizidwanso kugwiritsa ntchito QCOW2 ndi zomwe timagwiritsa ntchito poyimba - Apache CloudStack.

Kuti tichite zosunga zobwezeretsera, timatenga chithunzi chonse cha voliyumu ngati chithunzithunzi cha LVM2 (inde, tikudziwa kuti zithunzi za LVM2 zimachedwa, koma Intel P4500 imatithandizanso pano). Ife timatero lvmcreate -s .. ndi chithandizo dd timatumiza kopi yosunga zobwezeretsera ku seva yakutali yokhala ndi ZFS yosungirako. Pano tikupitabe patsogolo pang'ono - pambuyo pake, ZFS ikhoza kusunga deta mu mawonekedwe opanikizika, ndipo tikhoza kubwezeretsa mwamsanga pogwiritsa ntchito DD kapena pezani ma voliyumu a VM pawokha pogwiritsa ntchito mount -o loop ....

Mukhoza, ndithudi, kuchotsa osati chithunzi chonse cha voliyumu ya LVM2, koma kuyika fayilo mu fayilo RO ndikukopera zithunzi za QCOW2 okha, komabe, tidakumana ndi mfundo yakuti XFS idakhala yoyipa kuchokera ku izi, osati nthawi yomweyo, koma m'njira yosadziwika bwino. Sitimakonda kwambiri pamene hypervisor makamu "amamamatira" mwadzidzidzi kumapeto kwa sabata, usiku kapena patchuthi chifukwa cha zolakwika zomwe sizidziwika bwino kuti zidzachitika liti. Chifukwa chake, kwa XFS sitigwiritsa ntchito chithunzithunzi choyikamo RO kuti tichotse mabuku, timangotengera voliyumu yonse ya LVM2.

Kuthamanga kwa zosunga zobwezeretsera ku seva yosunga zobwezeretsera kumatsimikiziridwa kwa ife ndi magwiridwe antchito a seva yosunga zobwezeretsera, yomwe ili pafupifupi 600-800 MB / s pa data yosasunthika; malire ena ndi njira ya 10Gbit/s yomwe seva yosungira imalumikizidwa nayo. ku cluster.

Pankhaniyi, zosunga zobwezeretsera za ma seva 8 a hypervisor amatsitsidwa nthawi imodzi ku seva imodzi yosunga zobwezeretsera. Chifukwa chake, ma disk ndi ma network subsystems a seva yosunga zobwezeretsera, kukhala pang'onopang'ono, samalola kuti ma disk subsystems a makamu a hypervisor achuluke, chifukwa sangathe kukonza, kunena kuti, 8 GB / sec, yomwe ma hypervisor amatha mosavuta. panga.

Njira yokopera yomwe ili pamwambapa ndiyofunikira kwambiri pankhaniyi, kuphatikiza tsatanetsatane - kugwiritsa ntchito Intel P4500 drive yachangu, kugwiritsa ntchito NFS komanso, mwina, kugwiritsa ntchito ZFS.

Nkhani zosunga zobwezeretsera

Pamtundu uliwonse wa hypervisor tili ndi gawo laling'ono la SWAP la 8 GB kukula kwake, ndipo "timatulutsa" node ya hypervisor yokha pogwiritsa ntchito DD kuchokera pachithunzi cholozera. Pa voliyumu yamakina pa maseva, timagwiritsa ntchito 2xSATA SSD RAID1 kapena 2xSAS HDD RAID1 pa LSI kapena HP hardware controller. Nthawi zambiri, sitisamala za zomwe zili mkati, popeza voliyumu yathu imagwira ntchito "pafupifupi mongowerenga", kupatulapo SWAP. Ndipo popeza tili ndi RAM yambiri pa seva ndipo ndi 30-40% yaulere, sitiganizira za SWAP.

Njira zosunga zobwezeretsera. Ntchitoyi ikuwoneka motere:

#!/bin/bash

mkdir -p /mnt/backups/volumes

DIR=/mnt/images-snap
VOL=images/volume
DATE=$(date "+%d")
HOSTNAME=$(hostname)

lvcreate -s -n $VOL-snap -l100%FREE $VOL
ionice -c3 dd iflag=direct if=/dev/$VOL-snap bs=1M of=/mnt/backups/volumes/$HOSTNAME-$DATE.raw
lvremove -f $VOL-snap

Samalani ionice -c3, m'malo mwake, chinthu ichi ndichabechabe pazida za NVMe, popeza pulogalamu ya IO yawo yakhazikitsidwa motere:

cat /sys/block/nvme0n1/queue/scheduler
[none] 

Komabe, tili ndi malo angapo olowa ndi ma SSD RAID wamba, kwa iwo izi ndizofunikira, kotero akuyenda. NDIPO. Ponseponse, ichi ndi kachidutswa kosangalatsa komwe kamafotokoza zachabechabe ionice ngati kasinthidwe kotereku.

Samalani mbendera iflag=direct chifukwa DD. Timagwiritsa ntchito mwachindunji IO kudutsa posungira chitetezo kuti tipewe kusintha kosafunikira kwa ma buffer a IO powerenga. Komabe, oflag=direct sititero chifukwa takumana ndi zovuta za ZFS tikamagwiritsa ntchito.

Takhala tikugwiritsa ntchito njirayi bwino kwa zaka zingapo popanda mavuto.

Ndiyeno izo zinayamba... Tidazindikira kuti imodzi mwa mfundozo sinalinso kumbuyo, ndipo yapitayi inali kuyenda ndi IOWAIT yoopsa ya 50%. Poyesa kumvetsetsa chifukwa chake kukopera sikuchitika, tidakumana ndi izi:

Volume group "images" not found

Tinayamba kuganiza za "mapeto a Intel P4500 afika," komabe, tisanazimitse seva kuti ilowe m'malo mwa galimotoyo, kunali kofunikira kusunga zosunga zobwezeretsera. Tinakonza LVM2 pobwezeretsa metadata kuchokera ku zosunga zobwezeretsera za LVM2:

vgcfgrestore images

Tidayambitsa zosunga zobwezeretsera ndikuwona penti yamafuta iyi:

Apanso tinali achisoni kwambiri - zinali zoonekeratu kuti sitingathe kukhala motere, popeza ma VPS onse adzavutika, zomwe zikutanthauza kuti ifenso tidzavutika. Zomwe zidachitika sizikudziwika - iostat adawonetsa IOPS zachisoni komanso IOWAIT yapamwamba kwambiri. Panalibe malingaliro ena koma "tiyeni tisinthe NVMe," koma chidziwitso chinachitika munthawi yake.

Kuwunika momwe zinthu zilili pang'onopang'ono

Magazini ya Historical. Masiku angapo m'mbuyomo, pa seva iyi kunali koyenera kupanga VPS yaikulu ndi 128 GB RAM. Zinkawoneka kuti pali kukumbukira kokwanira, koma kuti tikhale otetezeka, tidagawa 32 GB ina yogawanitsa. VPS idapangidwa, idamaliza bwino ntchito yake ndipo chochitikacho chidayiwalika, koma magawo a SWAP adatsalira.

Kusintha Features. Kwa ma seva onse amtambo parameter vm.swappiness idakhazikitsidwa kukhala yosasintha 60. Ndipo SWAP idapangidwa pa SAS HDD RAID1.

Zomwe zidachitika (malinga ndi akonzi). Pothandizira DD idatulutsa zambiri zolembera, zomwe zidayikidwa muzosungira za RAM musanalembe ku NFS. Pansi pa dongosolo, motsogozedwa ndi ndondomeko swappiness, inali kusuntha masamba ambiri a kukumbukira kwa VPS kumalo osinthika, omwe anali pamtunda wochepa wa HDD RAID1. Izi zidapangitsa kuti IOWAIT ikule mwamphamvu kwambiri, koma osati chifukwa cha IO NVMe, koma chifukwa cha IO HDD RAID1.

Momwe vutolo linathetsedwa. Gawo la 32GB losinthana linayimitsidwa. Izi zidatenga maola 16; mutha kuwerenga padera momwe ndichifukwa chiyani SWAP imazimitsa pang'onopang'ono. Zokonda zasinthidwa swappiness ku mtengo wofanana ndi 5 pamtambo monse.

Kodi zimenezi sizikanatheka bwanji?. Choyamba, ngati SWAP inali pa SSD RAID kapena NVMe chipangizo, ndipo kachiwiri, ngati panalibe chipangizo cha NVMe, koma chipangizo chochepa chomwe sichikanatulutsa kuchuluka kwa deta yotere - modabwitsa, vuto linachitika chifukwa NVMe ndi yofulumira kwambiri.

Pambuyo pake, zonse zinayamba kugwira ntchito monga kale - ndi ziro IOWAIT.

Source: www.habr.com