Uninzi lwe-RAM yasimahla, iNVMe Intel P4500 kwaye yonke into icotha kakhulu-ibali lokongezwa okungaphumelelanga kwesahlulelo sokutshintsha

Kweli nqaku, ndiza kuthetha ngemeko esandul 'ukwenzeka ngenye yeeseva kwifu lethu leVPS, eyandishiya ndigxininise iiyure ezininzi. Bendicwangcisa kwaye ndilungisa iingxaki kwiiseva zeLinux malunga ne-15 leminyaka, kodwa eli tyala alingeni kwisenzo sam konke konke-ndenze iingcinga ezininzi ezingeyonyani kwaye ndaphelelwa lithemba ngaphambi kokuba ndikwazi ukuchonga unobangela wengxaki kwaye ndiyisombulule. .

Intshayelelo

Sisebenzisa ifu eliphakathi, esiyakha kwiiseva eziqhelekileyo kunye noqwalaselo olulandelayo - ii-cores ezingama-32, i-256 GB RAM kunye ne-4500TB PCI-E Intel P4 NVMe drive. Siluthanda kakhulu olu qwalaselo kuba lususa isidingo sokukhathazeka malunga ne-IO engaphezulu ngokubonelela ngothintelo oluchanekileyo kwinqanaba lohlobo lomzekelo we-VM. Ngenxa yokuba iNVMe Intel P4500 inomsebenzi oncomekayo, sinokubonelela ngaxeshanye zombini unikezelo olupheleleyo lwe-IOPS koomatshini kunye nokugcinwa kwe-backup kwi-server yogcino nge-zero IOWAIT.

Singabanye bamakholwa amadala angasebenzisi i-SDN eguquliweyo kunye nezinye izinto ezinesitayile, ezisefashonini, zolutsha ukugcina imiqulu ye-VM, sikholelwa ukuba inkqubo elula, kulula ukuyilungisa phantsi kweemeko "yeyona nto iphambili ihambile. ezintabeni.” Ngenxa yoko, sigcina imiqulu ye-VM kwifomathi ye-QCOW2 kwi-XFS okanye i-EXT4, ebekwe phezu kwe-LVM2.

Sikwanyanzelwa ukuba sisebenzise i-QCOW2 ngemveliso esiyisebenzisayo kwi-orchestration-Apache CloudStack.

Ukwenza i-backup, sithatha umfanekiso opheleleyo wevolumu njenge-snapshot ye-LVM2 (ewe, siyazi ukuba ii-snapshots ze-LVM2 ziyacotha, kodwa i-Intel P4500 iyasinceda apha nayo). Siya lvmcreate -s .. kunye noncedo dd sithumela ikopi yogcino kwiseva ekude enokugcinwa kweZFS. Apha sisaqhuba kancinci - emva kwayo yonke loo nto, i-ZFS inokugcina idatha kwifom ecinezelweyo, kwaye sinokuyibuyisela ngokukhawuleza sisebenzisa. DD okanye ufumane imiqulu yeVM nganye usebenzisa mount -o loop ....

Unako, ewe, ungawususi umfanekiso opheleleyo wevolumu yeLVM2, kodwa unyuse inkqubo yefayile kwi RO kwaye ukopishe imifanekiso ye-QCOW2 ngokwabo, nangona kunjalo, sasijongene nenyaniso yokuba i-XFS yaba yimbi kule nto, kwaye kungekhona ngokukhawuleza, kodwa ngendlela engalindelekanga. Asiyithandi ngokwenene xa i-hypervisor hosts "inamathela" ngokukhawuleza ngeempelaveki, ebusuku okanye ngeeholide ngenxa yeempazamo ezingacacanga ukuba ziya kwenzeka nini. Ke ngoko, kwi-XFS asisebenzisi i-snapshot mounting RO ukukhupha imiqulu, sikhuphela yonke ivolumu yeLVM2.

Isantya sokugcinwa kwe-backup kumncedisi we-backup sigqitywe kwimeko yethu ngokusebenza kweseva yokugcina, malunga ne-600-800 MB / s yedatha engabonakaliyo; enye i-limiter yi-10Gbit / s ishaneli apho iseva yokugcina idibaniswe nayo. kwiqela.

Kule meko, iikopi ezigciniweyo zeeseva ezi-8 ze-hypervisor zilayishwa ngaxeshanye kwiseva enye yokugcina. Ke, idisk kunye nenkqubo esezantsi yenethiwekhi ye-server yogcino, ngokucotha, ayivumeli idisk subsystems ze-hypervisor hosts ukuba zilayishe kakhulu, kuba zingakwazi ukuqhubekeka, zitsho, i-8 GB/sec, apho i-hypervisor host iyakwazi ngokulula. velisa.

Inkqubo yokukhuphela engentla ibaluleke kakhulu kwibali elongezelelweyo, kubandakanywa neenkcukacha - usebenzisa i-Intel P4500 drive ekhawulezayo, usebenzisa i-NFS kwaye, mhlawumbi, usebenzisa i-ZFS.

Backup ibali

Kwi-node nganye ye-hypervisor sinesahlulo esincinci se-SWAP se-8 GB ngobukhulu, kwaye "sikhupha" i-hypervisor node ngokwayo sisebenzisa. DD ukusuka kumfanekiso wereferensi. Ngomthamo wenkqubo kwiiseva, sisebenzisa i-2xSATA SSD RAID1 okanye i-2xSAS HDD RAID1 kwi-LSI okanye umlawuli we-hardware we-HP. Ngokuqhelekileyo, asikhathaleli konke okungaphakathi, ekubeni umthamo wenkqubo yethu usebenza "phantse ukufunda kuphela", ngaphandle kwe-SWAP. Kwaye ekubeni sine-RAM eninzi kumncedisi kwaye i-30-40% yamahhala, asicingi nge-SWAP.

Inkqubo yogcino. Lo msebenzi ujongeka ngolu hlobo:

#!/bin/bash

mkdir -p /mnt/backups/volumes

DIR=/mnt/images-snap
VOL=images/volume
DATE=$(date "+%d")
HOSTNAME=$(hostname)

lvcreate -s -n $VOL-snap -l100%FREE $VOL
ionice -c3 dd iflag=direct if=/dev/$VOL-snap bs=1M of=/mnt/backups/volumes/$HOSTNAME-$DATE.raw
lvremove -f $VOL-snap

naka u ionice -c3, eneneni, le nto ayinamsebenzi kwaphela kwizixhobo ze-NVMe, kuba umcwangcisi we-IO wabo umiselwe njenge:

cat /sys/block/nvme0n1/queue/scheduler
[none] 

Nangona kunjalo, sinenani leendawo zelifa ezinee-SSD RAID eziqhelekileyo, kubo oku kubalulekile, ngoko bayahamba. NJENGE. Lilonke, eli liqhekeza nje lekhowudi elinomdla elichaza into engekhoyo ionice kwimeko yoqwalaselo olunjalo.

Nika ingqalelo kwiflegi iflag=direct kuba DD. Sisebenzisa i-IO ethe ngqo ngokugqitha i-buffer cache ukunqanda ukutshintshwa ngokungeyomfuneko kwe-IO buffers xa ufunda. Nangona kunjalo, oflag=direct Akunjalo kuba siye sadibana nemiba yokusebenza kweZFS xa siyisebenzisa.

Sele sisisebenzisa ngempumelelo esi sikimu iminyaka emininzi ngaphandle kweengxaki.

Kwaye kwaqala... Sifumene ukuba enye yeenodi yayingasaxhaswa, kwaye enye yangaphambili yayiqhuba nge-IOWAIT enyantisayo ye-50%. Xa uzama ukuqonda ukuba kutheni ukukopa kungenzeki, siye sadibana nale meko ilandelayo:

Volume group "images" not found

Saqala ukucinga malunga "nesiphelo sifikile kwi-Intel P4500," nangona kunjalo, ngaphambi kokuvala iseva ukuze ithathe indawo ye-drive, kwakuseyimfuneko ukwenza i-backup. Silungise i-LVM2 ngokubuyisela i-metadata kwi-backup ye-LVM2:

vgcfgrestore images

Siphehlelele i-backup kwaye sabona lo mzobo weoyile:

Kwakhona sasibuhlungu kakhulu - kwacaca ukuba asikwazi ukuhlala ngolu hlobo, ekubeni zonke iiVPS ziza kubandezeleka, oku kuthetha ukuba nathi siya kubandezeleka. Okwenzekileyo akucaci ngokupheleleyo - iostat ibonise i-IOPS elusizi kunye neyona IOWAIT iphezulu. Kwakungekho mibono ngaphandle kokuthi "masitshintshe iNVMe," kodwa ukuqonda kwenzeka kanye ngexesha.

Uhlalutyo lwemeko yenyathelo ngenyathelo

Imagazini yezembali. Kwiintsuku ezimbalwa ngaphambili, kule seva kwakuyimfuneko ukudala iVPS enkulu kunye ne-128 GB RAM. Kwabonakala kukho inkumbulo eyaneleyo, kodwa ukuba kwicala elikhuselekileyo, sabele enye i-32 GB yokwahlulahlula. I-VPS yadalwa, yagqiba ngempumelelo umsebenzi wayo kwaye isiganeko salibaleka, kodwa isahlulelo se-SWAP sahlala.

Uqwalaselo Iimpawu. Kubo bonke abancedisi belifu ipharamitha vm.swappiness isetelwe kokungagqibekanga 60. Kwaye i-SWAP yenziwa kwi-SAS HDD RAID1.

Kwenzeke ntoni (ngokutsho kwabahleli). Xa uxhasa DD ivelise idatha eninzi yokubhala, eyafakwa kwi-RAM buffers ngaphambi kokubhala kwi-NFS. Undoqo wenkqubo, ekhokelwa ngumgaqo-nkqubo swappiness, yayihambisa amaphepha amaninzi ememori yeVPS kwindawo yokutshintshiselana, eyayibekwe kwi-HDD RAID1 yevolumu ecothayo. Oku kwakhokelela ekubeni i-IOWAIT ikhule ngamandla kakhulu, kodwa kungekhona ngenxa ye-IO NVMe, kodwa ngenxa ye-IO HDD RAID1.

Yasonjululwa njani ingxaki. Ukwahlulahlula kwe-32GB kuvaliwe. Oku kuthathe iiyure ezili-16; ungafunda ngokwahlukeneyo malunga nokuba kutheni kwaye kutheni i-SWAP icima ngokucothayo. Iisetingi zitshintshiwe swappiness kwixabiso elilingana ne 5 phezu kwelifu lonke.

Inokwenzeka njani loo nto?. Okokuqala, ukuba i-SWAP yayikwi-SSD RAID okanye i-NVMe isixhobo, kwaye okwesibini, ukuba kwakungekho sixhobo se-NVMe, kodwa isixhobo esicothayo esingayi kuvelisa umthamo onjalo wedatha - ngokumangalisayo, ingxaki yenzeka ngenxa yokuba i-NVMe ikhawuleza kakhulu.

Emva koko, yonke into yaqala ukusebenza njengangaphambili - nge-zero IOWAIT.

umthombo: www.habr.com