ProHoster > Blog > Utongi > Chii chinonzi LVM neMatryoshka chakafanana?

Chii chinonzi LVM neMatryoshka chakafanana?

Nenguva yezuva.
Ndinoda kugovera nenharaunda ruzivo rwangu runoshanda rwekuvaka dhata rekuchengetedza data reKVM uchishandisa md RAID + LVM.

Iyo purogiramu ichasanganisira:

  • Kuvaka md RAID 1 kubva kuNVMe SSD.
  • Kuunganidza md RAID 6 kubva kuSATA SSD uye madhiraivha akajairika.
  • Zvimiro zveTRIM/DISCARD kushanda paSSD RAID 1/6.
  • Kugadzira bootable md RAID 1/6 array pane yakajairwa seti yemadhisiki.
  • Kuisa sisitimu paNVMe RAID 1 kana pasina NVMe rutsigiro muBIOS.
  • Kushandisa LVM cache uye LVM yakaonda.
  • Uchishandisa BTRFS snapshots uye tumira / gamuchira kuti uchengetedze.
  • Uchishandisa LVM yakaonda snapshots uye yakaonda_delta yeBTRFS maitiro ekuchengetedza.

Kana uchifarira, ndapota ona katsi.

Statement

Munyori haana mutoro wemhedzisiro yekushandisa kana kusashandisa zvinhu / mienzaniso / kodhi / matipi / data kubva kuchinyorwa ichi. Nekuverenga kana kushandisa chinyorwa ichi munzira ipi neipi, unotora mutoro wemhedzisiro yezviito izvi. Migumisiro inobvira inosanganisira:

  • Crisp-yakakangwa NVMe SSDs.
  • Yakanyatso shandiswa kurekodha sosi uye kutadza kweSSD madhiraivha.
  • Kurasika kwakazara kwese data pane ese madhiraivha, kusanganisira makopi ekuchengetedza.
  • Yakashata komputa Hardware.
  • Yakatambisa nguva, tsinga nemari.
  • Chero mimwe mibairo isina kunyorwa pamusoro.

Iron

Aivepo aive:

Bhobho reamai kubva kutenderera 2013 ine Z87 chipset, yakazara neIntel Core i7 / Haswell.

  • processor 4 cores, 8 shinda
  • 32 GB DDR3 RAM
  • 1 x 16 kana 2 x 8 PCIe 3.0
  • 1 x 4 + 1 x 1 PCIe 2.0
  • 6 x 6 GBps SATA 3 zvibatanidza

SAS adapta LSI SAS9211-8I yakapenya kuIT / HBA modhi. RAID-enabled firmware yakatsiviwa nemaune neHBA firmware ku:

  1. Unogona kukanda adapta iyi chero nguva woitsiva neimwe yaunosangana nayo.
  2. TRIM/Discard yakashanda zvakajairwa pamadisiki, nekuti... muRAID firmware iyi mirairo haitsigirwe zvachose, uye HBA, kazhinji, haina basa kuti ndeipi mirairo inofambiswa pamusoro pebhazi.

Madhiraivha - 8 zvidimbu zveHGST Travelstar 7K1000 ine huwandu hwe1 TB mune 2.5 fomu chinhu, sekunge malaptop. Aya madhiraivha aimbove ari muRAID 6 array. Vachavawo nekushandiswa muhurongwa hutsva. Kuchengeta mabhapu emunharaunda.

Kuwedzera akawedzera:

6 zvidimbu SATA SSD modhi Samsung 860 QVO 2TB. Aya maSSD aida vhoriyamu hombe, kuvapo kweSLC cache, kuvimbika, uye mutengo wakaderera zvaidiwa. Tsigiro yekurasa / zero yaidiwa, iyo inotariswa nemutsara mu dmesg:

kernel: ata1.00: Enabling discard_zeroes_data

2 zvidimbu zveNVMe SSD modhi Samsung SSD 970 EVO 500GB.

Kune aya maSSD, kungoverengera kuverenga / kunyora kumhanya uye simba rekushandisa kune zvaunoda kwakakosha. Radiator kwavari. Zvakakodzera. Zvamazvirokwazvo. Zvikasadaro, fry ivo kusvika crispy panguva yekutanga RAID kuwiriranisa.

StarTech PEX8M2E2 adapta ye2 x NVMe SSD yakaiswa muPCIe 3.0 8x slot. Iyi, zvakare, ingori HBA, asi yeNVMe. Iyo inosiyana neyakachipa adapta pakuti haidi PCIe bifurcation rutsigiro kubva kubhodhi reamai nekuda kwekuvapo kweiyo yakavakirwa-mukati PCIe switch. Ichashanda kunyangwe mune yekare sisitimu ine PCIe, kunyangwe iri x1 PCIe 1.0 slot. Sezvingatarisirwa, pakumhanya kwakakodzera. Iko hakuna maRAID ipapo. Iko hakuna yakavakirwa-mukati BIOS pabhodhi. Saka, sisitimu yako haizodzidzi zvemashiripiti kubhutsu neNVMe, ndoda kuita NVMe RAID nekuda kwechinhu ichi.

Ichi chikamu chaikonzerwa nekuvapo kweiyo chete yemahara 8x PCIe 3.0 muhurongwa, uye, kana paine 2 emahara slots, inogona kutsiviwa zviri nyore nepenny mbiri PEX4M2E1 kana analogues, inogona kutengwa chero nemutengo we600. rubles.

Kurambwa kwemarudzi ese ezvishandiso kana akavakirwa-mukati chipset/BIOS RAIDs akaitwa nemaune, kuitira kuti akwanise kutsiva zvachose hurongwa hwese, kunze kweiyo SSD/HDD ivo pachavo, vachichengeta data rese. Zvakanaka, kuitira kuti iwe ugone kuchengetedza kunyange iyo yakaiswa sisitimu yekushandisa paunenge uchienda kune nyowani / yakasiyana Hardware. Chinhu chikuru ndechekuti kune SATA uye PCIe ports. Zvakafanana neCD mhenyu kana bootable flash drive, inokurumidza chaizvo uye ine bulky shoma.

HumorZvikasadaro, iwe unoziva zvinoitika - dzimwe nguva iwe unofanirwa kukurumidza kutora iyo yese hurongwa kuti uende nayo. Asi ini handidi kurasikirwa nedata. Kuti uite izvi, ese ataurwa midhiya anowanikwa zviri nyore pamasiraidhi mu5.25 bays yeyakajairwa kesi.

Zvakanaka, uye, hongu, yekuyedza nenzira dzakasiyana dzeSSD caching muLinux.

Hardware raids inofinha. Batidza. Zvinoshanda kana kuti hazvishande. Uye na mdadm pane nguva dzose sarudzo.

Nyoro

Pakutanga, Debian 8 Jessie yakaiswa pane Hardware, iri padyo neEOL. RAID 6 yakaunganidzwa kubva kune yataurwa pamusoro HDDs akapetwa neLVM. Yakamhanyisa michina chaiyo mu kvm/libvirt.

Nokuti Munyori ane ruzivo rwakakodzera mukugadzira inotakurika SATA/NVMe flash madhiraivha, uye zvakare, kuti asapwanye yakajairwa apt template, Ubuntu 18.04 yakasarudzwa seyakananga system, iyo yakatove yakagadzikana zvakakwana, asi ichine makore matatu rutsigiro mune ramangwana.

Iyo yakataurwa sisitimu ine ese madhiraivha emahara atinoda kunze kwebhokisi. Hatidi chero software yechitatu-bato kana madhiraivha.

Kugadzirira kuisirwa

Kuisa sisitimu tinoda Ubuntu Desktop Image. Iyo sevha system ine imwe mhando yekuisa yakasimba, iyo inoratidza kusununguka kwakanyanya kusingagone kuvharwa nekusundira UEFI system partition pane imwe yemadhisiki, ichiparadza runako rwese. Saizvozvowo, inongoiswa chete muEFI mode. Haipe chero sarudzo.

Hatisi kufara nazvo.

Sei?Nehurombo, UEFI boot haina kunyatso kuenderana neboot software RAID, nekuti ... Hapana anotipa kuchengetedza kweiyo UEFI ESP chikamu. Kune mabikirwo epamhepo anoratidza kuisa ESP chikamu pane flash drive mune USB port, asi iyi inzvimbo yekutadza. Kune mabikirwo anoshandisa software mdadm RAID 1 ine metadata vhezheni 0.9 iyo isingatadzise iyo UEFI BIOS kuona chikamu ichi, asi ichi chinorarama kusvika nguva yekufara apo BIOS kana imwe Hardware OS inonyora chimwe chinhu kuESP uye ichikanganwa kuiwiriranisa kune imwe. magirazi.

Mukuwedzera, UEFI boot inoenderana neNVRAM, iyo isingafambi pamwe chete nema disks kune itsva system, nokuti chikamu chebhodhi reamai.

Saka, hatisi kuzogadzirazve vhiri idzva. Isu tatova nebhasikoro rasekuru rakagadzirwa, rakaedzwa nguva, rave kunzi Legacy/BIOS boot, rine zita rekudada reCSM pane UEFI-inoenderana masisitimu. Isu tichangoibvisa pasherufu, tiise mafuta, pombi kumusoro matai uye toipukuta nemucheka wakanyorova.

Iyo desktop desktop yeUbuntu zvakare haigone kuisirwa nemazvo neLegacy bootloader, asi pano, sezvavanotaura, zvirinani pane zvingasarudzwa.

Uye saka, isu tinounganidza Hardware uye kurodha sisitimu kubva kuUbuntu Live bootable flash drive. Tichada kudhawunirodha mapakeji, saka isu tichagadzira network inokushandira iwe. Kana ikasashanda, unogona kurodha mapakeji anodiwa pane flash drive pamberi.

Isu tinoenda munzvimbo yeDesktop, tovhura iyo terminal emulator, uye toenda:

#sudo bash

Sei…?Mutsara uri pamusoro ndiyo inokonzeresa inokonzeresa holiwars nezve sudo. C bΠΎmikana mikuru kuuya uyeΠΎmutoro mukuru. Mubvunzo ndewokuti iwe unogona kuzvitora iwe pachako. Vanhu vazhinji vanofunga kuti kushandisa sudo nenzira iyi hakusi kungwarira. Zvisinei:

#apt-get install mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc

Sei ZFS...?Patinoisa software pakombuta yedu, isu tinokweretesa hardware yedu kune vanogadzira software iyi kuti vatyaire.
Kana isu tichivimba iyi software nekuchengetedzeka kwedata redu, tinotora chikwereti chakaenzana nemutengo wekudzoreredza iyi data, yatichafanira kubhadhara rimwe zuva.

Kubva pane iyi maonero, ZFS iFerrari, uye mdadm+lvm yakafanana nebhasikoro.

Sezvineiwo, munyori anofarira kukweretesa bhasikoro nechikwereti kune vanhu vasingazivikanwe panzvimbo peFerrari. Ikoko, mutengo wenyaya yacho hauna kukwirira. Hapana chikonzero chekodzero. Nyore kupfuura mitemo yemigwagwa. Kupaka mahara. Kugona kuyambuka nyika kuri nani. Iwe unogona kugara uchibatanidza makumbo kune bhasikoro, uye iwe unogona kugadzirisa bhasikoro nemaoko ako pachako.

Sei zvino BTRFS...?Kuti tivhure sisitimu yekushandisa, tinoda faira system inotsigirwa muLegacy/BIOS GRUB kunze kwebhokisi, uye panguva imwe chete inotsigira mhenyu snapshots. Isu tichaishandisa kune iyo / boot partition. Mukuwedzera, munyori anofarira kushandisa iyi FS ye / (mudzi), usingakanganwe kuziva kuti kune chero imwe software unogona kugadzira zvikamu zvakasiyana paLVM uye kuzviisa mumadhairekitori anodiwa.

Hatisi kuzochengeta chero mifananidzo yemakina chaiwo kana dhatabhesi pane iyi FS.
Iyi FS inongo shandiswa kugadzira snapshots ye system pasina kuidzima uye wozoendesa aya ma snapshots kune backup disk uchishandisa send/recieve.

Pamusoro pezvo, munyori anowanzofarira kuchengetedza hushoma hwesoftware zvakanangana nehardware uye kumhanya mamwe ese software mumashini chaiwo uchishandisa zvinhu zvakaita sekutumira maGPU uye PCI-USB Host controllers kuKVM kuburikidza neIOMMU.

Izvo chete zvinhu zvasara pane Hardware kuchengetedza data, virtualization uye backup.

Kana iwe uchivimba neZFS zvakanyanya, saka, musimboti, kune yakatsanangurwa application ivo vanochinjika.

Nekudaro, munyori anofuratira nemaune yakavakirwa-mukati girazi / RAID uye redundancy maficha ane ZFS, BRTFS uye LVM.

Seimwe gakava rekuwedzera, BTRFS inokwanisa kushandura zvinyorwa zvisina tsarukano kuita zvakatevedzana, izvo zvine mhedzisiro yakanaka pakumhanya kwekufananidza snapshots/backups paHDD.

Ngationgororei zvese zvishandiso:

#udevadm control --reload-rules && udevadm trigger

Ngatitarisei kumativi ose:

#lsscsi && nvme list
[0:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sda
[1:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdb
[2:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdc
[3:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdd
[4:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sde
[5:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdf
[6:0:0:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdg
[6:0:1:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdh
[6:0:2:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdi
[6:0:3:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdj
[6:0:4:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdk
[6:0:5:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdl
[6:0:6:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdm
[6:0:7:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdn
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 S466NXXXXXXX15L Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7
/dev/nvme1n1 S5H7NXXXXXXX48N Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7

Disk marongerwo

NVMe SSD

Asi isu hatizovamaka neimwe nzira. Zvakangofanana, BIOS yedu haione madhiraivha aya. Saka, ivo vanoenda zvachose kune software RAID. Hatizotombogadzira zvikamu ipapo. Kana iwe uchida kutevera "canon" kana "kunyanya", gadzira chikamu chimwe chikuru, seHDD.

SATA HDD

Hapana chikonzero chekugadzira chero chinhu chakakosha pano. Tichagadzira chikamu chimwe chezvose. Isu tichagadzira chikamu nekuti iyo BIOS inoona aya madhisiki uye inogona kutoedza kubhutsu kubva kwavari. Isu tichatoisa GRUB pane aya madhisiki gare gare kuitira kuti sisitimu igone kuita izvi kamwe kamwe.

#cat >hdd.part << EOF
label: dos
label-id: 0x00000000
device: /dev/sdg
unit: sectors

/dev/sdg1 : start= 2048, size= 1953523120, type=fd, bootable
EOF
#sfdisk /dev/sdg < hdd.part
#sfdisk /dev/sdh < hdd.part
#sfdisk /dev/sdi < hdd.part
#sfdisk /dev/sdj < hdd.part
#sfdisk /dev/sdk < hdd.part
#sfdisk /dev/sdl < hdd.part
#sfdisk /dev/sdm < hdd.part
#sfdisk /dev/sdn < hdd.part

SATA SSD

Apa ndipo panotinakidza zvinhu.

Chekutanga, madhiraivha edu ane 2 TB muhukuru. Izvi zviri mukati mechikamu chinogamuchirwa cheMBR, ndicho chatichashandisa. Kana zvichidikanwa, inogona kutsiviwa neGPT. Madhisiki eGPT ane dhisiki yekuenderana inobvumira MBR-inoenderana masisitimu kuona ekutanga 4 partitions kana ari mukati mekutanga 2 terabytes. Chinhu chikuru ndechekuti kupatsanurwa kwebhoti uye bios_grub kugovera pama diski aya kunofanirwa kunge kuri pakutanga. Izvi zvinotokutendera kuti utange kubva kuGPT Legacy/BIOS madhiraivha.

Asi iyi haisi nyaya yedu.

Pano tichagadzira zvikamu zviviri. Yekutanga ichave 1 GB muhukuru uye inoshandiswa RAID 1 / boot.

Yechipiri ichashandiswa RAID 6 uye inotora nzvimbo yese yasara yemahara kunze kwenzvimbo diki isina kugoverwa pakupera kwedhiraivha.

Ndeipi nzvimbo iyi isina chiratidzo?Zvinoenderana nezvinyorwa panetiweki, edu maSATA SSDs ane mubhodhi ane simba anowedzera SLC cache kubva muhukuru kubva pa6 kusvika 78 gigabytes. Isu tinowana 6 gigabytes "yemahara" nekuda kwekusiyana pakati pe "gigabytes" uye "gibibytes" mune data sheet yedhiraivha. Iwo asara 72 gigabytes akagoverwa kubva munzvimbo isina kushandiswa.

Pano zvinofanirwa kucherechedzwa kuti isu tine SLC cache, uye nzvimbo inogarwa mu4 bit MLC mode. Izvo kwatiri zvinobudirira zvinoreva kuti kune yega yega 4 gigabytes yemahara nzvimbo isu tichangowana 1 gigabyte yeSLC cache.

Wedzera 72 gigabytes ne 4 uye uwane 288 gigabytes. Iyi ndiyo nzvimbo yemahara yatisingazomaka kuitira kubvumidza madhiraivha kuti ashandise zvizere SLC cache.

Nekudaro, isu tichanyatso kusimuka kusvika ku312 gigabytes yeSLC cache kubva pamadhiraivha matanhatu akazara. Pamadhiraivha ese, 2 ichashandiswa muRAID yekudzosera.

Uhu huwandu hwecache huchatitendera kuti tisanyanya kashoma muhupenyu kusangana nemamiriro ezvinhu apo kunyora kusingaendi kune cache. Izvi zvakanyanya kutsiva iyo inosiririsa yekudzosa ndangariro yeQLC - iyo yakanyanya kuderera kunyora kumhanya kana data rakanyorwa nekupfuura cache. Kana mitoro yako isingaenderane neizvi, saka ini ndinokurudzira kuti ufunge zvakaoma kuti SSD yako ichagara kwenguva yakareba sei pasi pemutoro wakadaro, uchifunga nezveTBW kubva pane data sheet.

#cat >ssd.part << EOF
label: dos
label-id: 0x00000000
device: /dev/sda
unit: sectors

/dev/sda1 : start= 2048, size= 2097152, type=fd, bootable
/dev/sda2 : start= 2099200, size= 3300950016, type=fd
EOF
#sfdisk /dev/sda < ssd.part
#sfdisk /dev/sdb < ssd.part
#sfdisk /dev/sdc < ssd.part
#sfdisk /dev/sdd < ssd.part
#sfdisk /dev/sde < ssd.part
#sfdisk /dev/sdf < ssd.part

Kugadzira Arrays

Kutanga, tinoda kutumidza zita remuchina. Izvi zvinodikanwa nekuti zita remuenzi chikamu chezita rezita pane imwe nzvimbo mukati memdadm uye rinokanganisa chimwe chinhu kumwe. Ehe, arrays anogona kupihwa zita gare gare, asi iyi inhanho isingakoshi.

#mcedit /etc/hostname
#mcedit /etc/hosts
#hostname
vdesk0

NVMe SSD

#mdadm --create --verbose --assume-clean /dev/md0 --level=1 --raid-devices=2 /dev/nvme[0-1]n1

Sei -kufungidzira-kuchena ...?Kuti udzivise kutanga arrays. Kune ese ari maviri RAID mazinga 1 uye 6 izvi zvinoshanda. Zvese zvinogona kushanda pasina kutanga kana iri nyowani nyowani. Zvakare, kutanga iyo SSD array pane kusikwa kutambisa kweTBW sosi. Isu tinoshandisa TRIM/DISCARD pazvinogoneka pane yakaunganidzwa SSD arrays kuti "vatange" ivo.

Kune SSD arrays, RAID 1 DISCARD inotsigirwa kunze kwebhokisi.

Kune SSD RAID 6 DISCARD arrays, unofanira kuigonesa mu kernel module parameters.

Izvi zvinofanirwa kuitwa chete kana ese maSSD anoshandiswa padanho 4/5/6 arrays mune ino system aine tsigiro yekushanda yekudisard_zeroes_data. Dzimwe nguva unosangana nemadhiraivha asinganzwisisike anoudza kernel kuti basa iri rinotsigirwa, asi chokwadi harisipo, kana basa racho harishande nguva dzose. Parizvino, rutsigiro runowanikwa kwese kwese, zvisinei, kune madhiraivha ekare uye firmware ine zvikanganiso. Nechikonzero ichi, DISCARD tsigiro yakavharwa nekusarudzika kweRAID 6.

Chenjerera, murairo unotevera uchaparadza data rese paNVMe madhiraivha ne "kutanga" hurongwa ne "zeros".

#blkdiscard /dev/md0

Kana chimwe chinhu chisina kumira zvakanaka, edza kudoma danho.

#blkdiscard --step 65536 /dev/md0

SATA SSD

#mdadm --create --verbose --assume-clean /dev/md1 --level=1 --raid-devices=6 /dev/sd[a-f]1
#blkdiscard /dev/md1
#mdadm --create --verbose --assume-clean /dev/md2 --chunk-size=512 --level=6 --raid-devices=6 /dev/sd[a-f]2

Sei zvakakura kudaro...?Kuwedzera chunk-saizi kune mhedzisiro yakanaka pakumhanya kwekuverenga zvisina tsarukano mumabhuroko kusvika chunk-saizi inosanganisirwa. Izvi zvinoitika nekuti kumwe kushanda kwehukuru hwakakodzera kana diki kunogona kupedzwa zvachose pane imwe mudziyo. Naizvozvo, iyo IOPS kubva kumidziyo yese inopfupikiswa. Maererano nehuwandu, 99% yeIO haipfuure 512K.

RAID 6 IOPS pakunyora nguva dzose isingasviki kana kuenzana neIOPS yeimwe drive. Apo, sekuverenga zvisina tsarukano, IOPS inogona kuva yakakura kakawanda kupfuura iyo yedhiraivha imwe, uye pano saizi yebhuroka ndiyo yakakosha.
Munyori haaone poindi yekuyedza kukwirisa parameter iyo yakaipa muRAID 6 ne-dhizaini uye panzvimbo pacho anokwirisa izvo RAID 6 yakanaka pairi.
Isu ticharipa iyo isina kurongeka yakanyorwa yeRAID 6 ine NVMe cache uye dzakatetepa-yekupa matipi.

Isu hatisati tagonesa DISCARD yeRAID 6. Saka isu hati "tangisa" iyi array ikozvino. Tichaita izvi gare gare, mushure mekuisa iyo OS.

SATA HDD

#mdadm --create --verbose --assume-clean /dev/md3 --chunk-size=512 --level=6 --raid-devices=8 /dev/sd[g-n]1

LVM paNVMe RAID

Nekumhanyisa, isu tinoda kuisa iyo midzi faira system paNVMe RAID 1 iyo iri /dev/md0.
Nekudaro, isu ticharamba tichida iyi inokurumidza kuronga kune zvimwe zvinodiwa, sekuchinjanisa, metadata uye LVM-cache uye LVM-yakaonda metadata, saka isu tichagadzira LVM VG pane iyi array.

#pvcreate /dev/md0
#vgcreate root /dev/md0

Ngatigadzirei chikamu cheiyo midzi faira system.

#lvcreate -L 128G --name root root

Ngatigadzirei chikamu chekuchinjana zvinoenderana nehukuru hwe RAM.

#lvcreate -L 32G --name swap root

Kuiswa kweOS

Pakazara, isu tine zvese zvinodiwa kuti tiise system.

Tangisa iyo system yekuisa wizard kubva kuUbuntu Live nharaunda. Kuiswa kwakajairika. Chete padanho rekusarudza dhisiki yekuisa, iwe unofanirwa kutsanangura zvinotevera:

  • / dev/md1, - gomo poindi / boot, FS - BTRFS
  • / dev / mudzi / mudzi (aka / dev / mapper / mudzi-mudzi), - gomo poindi / (mudzi), FS - BTRFS
  • / dev / mudzi / swap (aka / dev / mapper / mudzi-chinja), - shandisa sekuchinjanisa chikamu
  • Isa iyo bootloader pa /dev/sda

Paunosarudza BTRFS semudziyo faira system, mugadziri anongogadzira maviri BTRFS mavhoriyamu anonzi "@" ye / (mudzi), uye "@home" ye / kumba.

Ngatitange kuisa ...

Iko kuisirwa kunopera nemodal dialog box inoratidza kukanganisa mukuisa iyo bootloader. Nehurombo, haugone kubuda iyi dialog uchishandisa yakajairwa nzira uye kuenderera mberi nekuisirwa. Isu tinobuda kunze kweiyo sisitimu uye tinopinda zvakare, tichiguma mune yakachena Ubuntu Live desktop. Vhura iyo terminal, uye zvakare:

#sudo bash

Gadzira chroot nharaunda kuti uenderere mberi nekuisa:

#mkdir /mnt/chroot
#mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@ /dev/mapper/root-root /mnt/chroot
#mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@home /dev/mapper/root-root /mnt/chroot/home
#mount -o defaults,space_cache,noatime,nodiratime,discard /dev/md1 /mnt/chroot/boot
#mount --bind /proc /mnt/chroot/proc
#mount --bind /sys /mnt/chroot/sys
#mount --bind /dev /mnt/chroot/dev

Ngatigadzirise network uye hostname muchroot:

#cat /etc/hostname >/mnt/chroot/etc/hostname
#cat /etc/hosts >/mnt/chroot/etc/hosts
#cat /etc/resolv.conf >/mnt/chroot/etc/resolv.conf

Ngatipindei munzvimbo yechroot:

#chroot /mnt/chroot

Chokutanga pane zvose, tichaendesa mapakeji:

apt-get install --reinstall mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc debsums hdparm

Ngatitarisei uye tigadzirise mapakeji ese akaiswa zvisina tsarukano nekuda kwekusakwana kwekuisa system:

#CORRUPTED_PACKAGES=$(debsums -s 2>&1 | awk '{print $6}' | uniq)
#apt-get install --reinstall $CORRUPTED_PACKAGES

Kana chimwe chinhu chikasashanda, ungangoda kugadzirisa /etc/apt/sources.list kutanga

Ngatigadzirise maparamita eRAID 6 module kuti tigonese TRIM/DISCARD:

#cat >/etc/modprobe.d/raid456.conf << EOF
options raid456 devices_handle_discard_safely=1
EOF

Ngatigadzirise mitsara yedu zvishoma:

#cat >/etc/udev/rules.d/60-md.rules << EOF
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/stripe_cache_size", ATTR{md/stripe_cache_size}="32768"
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_min", ATTR{md/sync_speed_min}="48000"
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_max", ATTR{md/sync_speed_max}="300000"
EOF
#cat >/etc/udev/rules.d/62-hdparm.rules << EOF
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/hdparm -B 254 /dev/%k"
EOF
#cat >/etc/udev/rules.d/63-blockdev.rules << EOF
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/blockdev --setra 1024 /dev/%k"
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="md*", RUN+="/sbin/blockdev --setra 0 /dev/%k"
EOF

Chaive chii..?Isu takagadzira seti yemitemo yeudev ichaita zvinotevera:

  • Gadzira saizi ye block cache ye RAID 2020 kuti ikwane ye 6. Iyo yakasarudzika kukosha, inoratidzika, haina kushanduka kubva pakusikwa kweLinux, uye haina kukwana kwenguva yakareba.
  • Chengetedza hushoma hweIO panguva yekutarisa / kuwiriranisa. Izvi ndezvekudzivirira arrays ako kubva kunamira muchimiro chekusingaperi kuwiriranisa pasi pemutoro.
  • Deredza iyo yakanyanya IO panguva yekutarisa / kuwiriranisa kwearrays. Izvi zvinodikanwa kuitira kuti kuwiriranisa / kutarisa SSD RAID kurege fry madhiraivha ako kune crisp. Izvi ndezvechokwadi kunyanya kune NVMe. (Unorangarira nezve radiator? Ndanga ndisiri kuita zvejee.)
  • Rambidza madhisiki kumisa spindle rotation (HDD) kuburikidza neAPM uye isa nguva yekurara yevanodzora dhisiki kusvika maawa manomwe. Unogona kudzima zvachose APM kana madhiraivha ako achigona kuzviita (-B 7). Neiyo default kukosha, madhiraivha anomira mushure memasekonzi mashanu. Ipapo iyo OS inoda kuseta zvakare dhisiki cache, madhisiki anotenderera zvakare, uye zvese zvinotanga zvakare. Madhisiki ane shoma shoma nhamba yekutenderera kwe spindle. Yakadaro yakapusa default kutenderera inogona kuuraya zviri nyore madhisiki ako mumakore mashoma. Haasi madhisiki ose anotambura neizvi, asi yedu ndeye "laptop" iyo, ine zvigadziro zvakakodzera, izvo zvinoita kuti RAID iite semini-MAID.
  • Isa kumberi pamadisiki (kutenderera) 1 megabyte - maviri akateedzana mabhuroko / chunk RAID 6
  • Dzima kuverenga kumberi pazvirongwa zvacho.

Ngatigadzirise /etc/fstab:

#cat >/etc/fstab << EOF
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# file-system mount-point type options dump pass
/dev/mapper/root-root / btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@ 0 1
UUID=$(blkid -o value -s UUID /dev/md1) /boot btrfs defaults,space_cache,noatime,nodiratime,discard 0 2
/dev/mapper/root-root /home btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@home 0 2
/dev/mapper/root-swap none swap sw 0 0
EOF
Sei zvakadaro..?Isu tichatsvaga iyo / boot partition neUUID. Array mazita anogona ne theoretically kuchinja.

Tichatsvaga zvikamu zvakasara nemazita eLVM mune /dev/mapper/vg-lv notation, nekuti. vanoratidza zvikamu zvakasiyana chaizvo.

Isu hatishandise UUID yeLVM nekuti Iyo UUID yeLVM mavhoriyamu uye mapikicha avo anogona kunge akafanana.Mount /dev/mapper/root-root.. kaviri?Ehe. Ndizvozvo. Chimiro cheBTRFS. Iyi faira system inogona kukwidzwa kakawanda nema subvols akasiyana.

Nekuda kwechinhu chimwe chete ichi, ini ndinokurudzira kusambogadzira LVM snapshots yeanoshanda BTRFS mavhoriyamu. Unogona kuwana kushamisika paunotangazve.

Ngatitangezve iyo mdadm config:

#/usr/share/mdadm/mkconf | sed 's/#DEVICE/DEVICE/g' >/etc/mdadm/mdadm.conf

Ngatigadzirise marongero eLVM:

#cat >>/etc/lvm/lvmlocal.conf << EOF

activation {
thin_pool_autoextend_threshold=90
thin_pool_autoextend_percent=5
}
allocation {
cache_pool_max_chunks=2097152
}
devices {
global_filter=["r|^/dev/.*_corig$|","r|^/dev/.*_cdata$|","r|^/dev/.*_cmeta$|","r|^/dev/.*gpv$|","r|^/dev/images/.*$|","r|^/dev/mapper/images.*$|","r|^/dev/backup/.*$|","r|^/dev/mapper/backup.*$|"] issue_discards=1
}
EOF

Chaive chii..?Isu takagonesa kuwedzera otomatiki kweLVM madziva matete pakusvika 90% yenzvimbo yakagarwa ne5% yevhoriyamu.

Isu takawedzera huwandu hwepamusoro hwevhavhavhavha yeLVM cache.

Isu takadzivirira LVM kutsvaga mavhoriyamu eLVM (PV) pa:

  • zvishandiso zvine LVM cache (cdata)
  • midziyo yakavharirwa uchishandisa LVM cache, ichipfuura cache ( _corig). Mune ino kesi, iyo cached mudziyo pachayo icharamba ichiongororwa kuburikidza necache (chete )
  • zvishandiso zvine LVM cache metadata (cmeta)
  • zvishandiso zvese muVG zvine mazita emifananidzo. Pano tichave nedhisiki mifananidzo yemakina chaiwo, uye isu hatidi LVM pane iyo host kuti ishandise mavhoriyamu emuenzi OS.
  • zvese zvishandiso muVG zvine zita rekuchengetedza. Pano tichava nemakopi ekuchengetedza emifananidzo yemuchina chaiwo.
  • zvishandiso zvese zvine zita rinopera ne "gpv" (vhoriyamu yevaenzi)

Isu takagonesa DISCARD rutsigiro kana uchisunungura nzvimbo yemahara paLVM VG. Chenjera. Izvi zvichaita kuti kudzima maLV paSSD kutore nguva. Izvi zvinonyanya kushanda kune SSD RAID 6. Zvisinei, maererano neurongwa, tichashandisa kuonda kupa, saka izvi hazvizotitadzisi zvachose.

Ngatigadzirise iyo initramfs mufananidzo:

#update-initramfs -u -k all

Isa uye gadzirisa grub:

#apt-get install grub-pc
#apt-get purge os-prober
#dpkg-reconfigure grub-pc

Ndeapi madhisiki aunofanira kusarudza?Vese vari sd*. Iyo sisitimu inofanirwa kukwanisa kubhutsu kubva kune chero inoshanda SATA drive kana SSD.

Sei vakawedzera os-prober..?Nokuda kwekusununguka kwakanyanya uye nemaoko ekutamba.

Izvo hazvishande nemazvo kana imwe yeRAID iri muhurombo. Inoedza kutsvaga OS pane zvikamu zvinoshandiswa mumashini chaiwo anoshanda pane iyi hardware.

Kana uchichida, unogona kuchisiya, asi ramba uchifunga zvose zviri pamusoro apa. Ndinokurudzira kutsvaga mabikirwo ekubvisa maoko ane misikanzwa online.

Neizvi tapedza kuiswa kwekutanga. Yasvika nguva yekutangazve muiyo ichangobva kuiswa OS. Usakanganwa kubvisa bootable Live CD/USB.

#exit
#reboot

Sarudza chero yeSATA SSD semudziyo webhoti.

LVM paSATA SSD

Panguva ino, isu takatopinda mu OS itsva, takagadzirisa network, apt, yakavhura iyo terminal emulator, uye yakatanga:

#sudo bash

Ngatienderere mberi.

"Tanga" rondedzero kubva kuSATA SSD:

#blkdiscard /dev/md2

Kana ikasashanda, edza:

#blkdiscard --step 65536 /dev/md2
Gadzira LVM VG paSATA SSD:

#pvcreate /dev/md2
#vgcreate data /dev/md2

Nei imwe VG..?Muchokwadi, isu tatova neVG inonzi mudzi. Wadii kuwedzera zvese muVG imwe chete?

Kana paine akati wandei maPV muVG, saka kuti VG ishandiswe nemazvo, ese maPV anofanira kunge aripo (online). Kunze kweLVM RAID, yatisingashandise nemaune.

Isu tinoda chaizvo kuti kana paine kukundikana (kuverenga kurasikirwa kwedata) pane chero yeRAID 6 arrays, iyo inoshanda sisitimu inozobhowa kazhinji uye inotipa mukana wekugadzirisa dambudziko.

Kuti tiite izvi, padanho rekutanga rekubvisa isu tichaparadzanisa yega yega yemuviri "media" mune yakaparadzana VG.

Nesainzi kutaura, akasiyana RAID arrays ndeeakasiyana "akavimbika madomasi". Iwe haufanirwe kugadzira imwe yakajairika pfungwa yekutadza kwavari nekuvarovera muVG imwe.

Kuvapo kweLVM padanho re "hardware" kunotitendera kucheka zvimedu zvemhando dzakasiyana dzeRAID nekuzvisanganisa nenzira dzakasiyana. Somuenzaniso - kumhanya panguva imwe chete bcache + LVM nhete, bcache + BTRFS, LVM cache + LVM yakaonda, yakaomesesa ZFS kumisikidzwa ine cache, kana chero imwe gehena musanganiswa kuyedza nekuzvienzanisa zvese.

Pa "hardware" level, isu hatizoshandisa chero chinhu kunze kweyakanaka yekare "yakakora" mavhoriyamu eLVM. Kusarudzika kumutemo uyu kunogona kunge kuri kwegadziriro yekugovera.

Ndinofunga panguva ino, vaverengi vazhinji vakange vatotanga kufungidzira chimwe chinhu pamusoro pechidhori chekugara.

LVM paSATA HDD

#pvcreate /dev/md3
#vgcreate backup /dev/md3

VG itsva zvakare..?Isu tinoda chaizvo kuti kana iyo disk array yatichashandisa kuchengetedza data ikatadza, sisitimu yedu yekushandisa icharamba ichishanda zvakajairika, tichichengetedza kuwana kune isiri-backup data semazuva ese. Naizvozvo, kudzivirira VG activation matambudziko, isu tinogadzira yakaparadzana VG.

Kumisikidza LVM cache

Ngatigadzirei LV paNVMe RAID 1 kuti tiishandise sechinhu checaching.

#lvcreate -L 70871154688B --name cache root

Sei pane zvishoma ...?Icho chokwadi ndechekuti edu maNVMe SSD anewo SLC cache. 4 gigabytes ye "yemahara" uye 18 gigabytes yesimba nekuda kwenzvimbo yemahara inogarwa mu3-bit MLC. Kana iyi cache yapera, NVMe SSDs haizove yakakurumidza kupfuura yedu SATA SSD ine cache. Chaizvoizvo, nechikonzero ichi, hazvina musoro kuti isu tiite iyo LVM cache partition yakakura kupfuura kaviri saizi yeSLC cache yeNVMe drive. Kune maNVMe madhiraivha anoshandiswa, munyori anoona zvine musoro kugadzira 32-64 gigabytes yecache.

Saizi yakapihwa yekuparadzanisa inodiwa kuronga 64 gigabytes yecache, cache metadata, uye metadata backup.

Pamusoro pezvo, ini ndinocherekedza kuti mushure mekuvharwa kwetsvina system, LVM inomaka iyo cache yese seyakasviba uye ichawiriranisa zvakare. Uyezve, izvi zvichadzokororwa pese panoshandiswa lvchange pachigadzirwa ichi kusvikira sisitimu yatangwa zvakare. Naizvozvo, ini ndinokurudzira nekukurumidza kudzoreredza cache uchishandisa iyo yakakodzera script.

Ngatigadzirei LV paSATA RAID 6 kuti tiishandise sechinhu chakavharirwa.

#lvcreate -L 3298543271936B --name cache data

Sei materabytes matatu chete..?Saka kuti, kana zvichidikanwa, unogona kushandisa SATA SSD RAID 6 kune zvimwe zvinodiwa. Saizi yenzvimbo yakavharirwa inogona kuwedzerwa zvine simba, panhunzi, pasina kumisa system. Kuti uite izvi, unofanirwa kumira kwenguva uye kugonesa cache, asi iyo yakasarudzika mukana weLVM-cache pamusoro, semuenzaniso, bcache ndeyekuti izvi zvinogona kuitwa panhunzi.

Ngatigadzire VG itsva ye caching.

#pvcreate /dev/root/cache
#pvcreate /dev/data/cache
#vgcreate cache /dev/root/cache /dev/data/cache

Ngatigadzirei LV pane cached mudziyo.

#lvcreate -L 3298539077632B --name cachedata cache /dev/data/cache

Pano takabva tangotora nzvimbo yese yemahara pa/dev/data/cache kuitira kuti mamwe ese anodiwa partitions akagadzirwa ipapo pa/dev/root/cache. Kana iwe wakagadzira chimwe chinhu munzvimbo isiri iyo, unogona kuchifambisa uchishandisa pvmove.

Ngatigadzire uye tikwanise cache:

#lvcreate -y -L 64G -n cache cache /dev/root/cache
#lvcreate -y -L 1G -n cachemeta cache /dev/root/cache
#lvconvert -y --type cache-pool --cachemode writeback --chunksize 64k --poolmetadata cache/cachemeta cache/cache
#lvconvert -y --type cache --cachepool cache/cache cache/cachedata

Why chunksize yakadai..?Kuburikidza nemiedzo inoshanda, munyori akakwanisa kuona kuti mhedzisiro yakanakisa inowanikwa kana saizi yeLVM cache block ichienderana nehukuru hweLVM yakatetepa block. Uyezve, iyo diki saizi, zviri nani iyo gadziriso inoita mukurekodha isina kujairika.

64k ndiyo yakaderera block saizi inotenderwa yeLVM yakaonda.

Ngwarira kunyora ..!Ehe. Iyi mhando ye cache inoregedza kunyora kuwiriranisa kune yakavharirwa mudziyo. Izvi zvinoreva kuti kana cache ikarasika, unogona kurasikirwa nedata pane cached mudziyo. Gare gare, munyori achakuudza kuti ndeapi matanho, kuwedzera kuNVMe RAID 1, anogona kutorwa kutsiva njodzi iyi.

Iyi cache type yakasarudzwa nemaune kuripira kusarongeka kwekunyora kweRAID 6.

Ngatitarisei zvatinazvo:

#lvs -a -o lv_name,lv_size,devices --units B cache
LV LSize Devices
[cache] 68719476736B cache_cdata(0)
[cache_cdata] 68719476736B /dev/root/cache(0)
[cache_cmeta] 1073741824B /dev/root/cache(16384)
cachedata 3298539077632B cachedata_corig(0)
[cachedata_corig] 3298539077632B /dev/data/cache(0)
[lvol0_pmspare] 1073741824B /dev/root/cache(16640)

Chete [cachedata_corig] inofanira kunge iri pa/dev/data/cache. Kana chimwe chinhu chisina kumira zvakanaka, saka shandisa pvmove.

Iwe unogona kudzima cache kana zvichidikanwa nemurairo mumwe chete:

#lvconvert -y --uncache cache/cachedata

Izvi zvinoitwa online. LVM inongowiriranisa cache ku diski, kuibvisa, uye kutumidza zita cachedata_corig kudzokera kucachedata.

Kumisikidza LVM yakaonda

Ngatimbofungidzira kuti ingani nzvimbo yatinoda yeLVM yakaonda metadata:

#thin_metadata_size --block-size=64k --pool-size=6terabytes --max-thins=100000 -u bytes
thin_metadata_size - 3385794560 bytes estimated metadata area size for "--block-size=64kibibytes --pool-size=6terabytes --max-thins=100000"

Kutenderera kusvika ku4 gigabytes: 4294967296B

Wedzerai mbiri uye wedzera 4194304B yeLVM PV metadata: 8594128896B
Ngatigadzirei chikamu chakaparadzana paNVMe RAID 1 kuti tiise LVM yakaonda metadata uye kopi yavo yekuchengetedza pairi:

#lvcreate -L 8594128896B --name images root

Yechii..?Pano mubvunzo unogona kumuka: sei uchiisa LVM yakaonda metadata zvakasiyana kana ichiri cached paNVMe uye ichashanda nekukurumidza.

Kunyangwe kumhanya kwakakosha pano, kuri kure nechikonzero chikuru. Chinhu chacho ndechekuti cache inzvimbo yekukundikana. Chimwe chinhu chinogona kuitika kwairi, uye kana iyo LVM yakaonda metadata ikavharirwa, ichaita kuti zvese zvirasikirwe zvachose. Pasina metadata yakakwana, zvingave zvisingaite kuunganidza mavhoriyamu matete.

Nekufambisa metadata kune yakaparadzana isina-cached, asi nekukurumidza, vhoriyamu, tinovimbisa kuchengetedzwa kwemetadata muchiitiko chekurasikirwa kwecache kana huwori. Muchiitiko ichi, kukuvadzwa kwese kunokonzerwa nekurasikirwa kwecache kuchaiswa mukati mehutete mavhoriyamu, izvo zvinorerutsa maitiro ekudzoreredza nemirairo yehukuru. Neine mukana wepamusoro, kukuvara uku kuchadzoserwa uchishandisa FS matanda.

Uyezve, kana pikicha yevhoriyamu yakatetepa yakambotorwa, uye mushure meizvozvo cache yakanyatso kuwiriraniswa kamwechete, saka, nekuda kwemukati dhizaini yeLVM yakaonda, kutendeseka kweiyo snapshot kunovimbiswa muchiitiko chekurasikirwa kwecache. .

Ngatigadzirei VG nyowani inozoita basa rekuonda-kupa:

#pvcreate /dev/root/images
#pvcreate /dev/cache/cachedata
#vgcreate images /dev/root/images /dev/cache/cachedata
Ngatigadzire dziva:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T images/thin-pool
Sei -Z yPamusoro peizvo iyo modhi iyi yakanyatso kurongedzerwa - kudzivirira data kubva kune imwe chaiyo muchina kubva pakudonhedza kune mumwe muchina chaiwo kana uchigoverazve nzvimbo - zeroing inoshandiswa kuwedzera kumhanya kwekunyora zvisina tsarukano mumabhururu madiki pane 64k. Chero nyora isingasviki 64k kune yaimbove isina kugoverwa nzvimbo yevhoriyamu yakatetepa ichava 64K mupendero-yakarongedzwa mu cache. Izvi zvinotendera kuti oparesheni iitwe zvachose kuburikidza necache, nekupfuura iyo cached mudziyo.

Ngatifambise maLV kune maPV anoenderana:

#pvmove -n images/thin-pool_tdata /dev/root/images /dev/cache/cachedata
#pvmove -n images/lvol0_pmspare /dev/cache/cachedata /dev/root/images
#pvmove -n images/thin-pool_tmeta /dev/cache/cachedata /dev/root/images

Ngatitarisei:

#lvs -a -o lv_name,lv_size,devices --units B images
LV LSize Devices
[lvol0_pmspare] 4294967296B /dev/root/images(0)
thin-pool 274877906944B thin-pool_tdata(0)
[thin-pool_tdata] 274877906944B /dev/cache/cachedata(0)
[thin-pool_tmeta] 4294967296B /dev/root/images(1024)

Ngatigadzirei vhoriyamu yakatetepa yebvunzo:

#lvcreate -V 64G --thin-pool thin-pool --name test images

Isu tichaisa mapakeji ebvunzo uye yekutarisa:

#apt-get install sysstat fio

Aya ndiwo maitiro aunokwanisa kuona maitiro ekugadzirisa kwedu kuchengetedza munguva chaiyo:

#watch 'lvs --rows --reportformat basic --quiet -ocache_dirty_blocks,cache_settings cache/cachedata && (lvdisplay cache/cachedata | grep Cache) && (sar -p -d 2 1 | grep -E "sd|nvme|DEV|md1|md2|md3|md0" | grep -v Average | sort)'

Aya ndiwo maitiro atinogona kuyedza gadziriso yedu:

#fio --loops=1 --size=64G --runtime=4 --filename=/dev/images/test --stonewall --ioengine=libaio --direct=1
--name=4kQD32read --bs=4k --iodepth=32 --rw=randread
--name=8kQD32read --bs=8k --iodepth=32 --rw=randread
--name=16kQD32read --bs=16k --iodepth=32 --rw=randread
--name=32KQD32read --bs=32k --iodepth=32 --rw=randread
--name=64KQD32read --bs=64k --iodepth=32 --rw=randread
--name=128KQD32read --bs=128k --iodepth=32 --rw=randread
--name=256KQD32read --bs=256k --iodepth=32 --rw=randread
--name=512KQD32read --bs=512k --iodepth=32 --rw=randread
--name=4Kread --bs=4k --rw=read
--name=8Kread --bs=8k --rw=read
--name=16Kread --bs=16k --rw=read
--name=32Kread --bs=32k --rw=read
--name=64Kread --bs=64k --rw=read
--name=128Kread --bs=128k --rw=read
--name=256Kread --bs=256k --rw=read
--name=512Kread --bs=512k --rw=read
--name=Seqread --bs=1m --rw=read
--name=Longread --bs=8m --rw=read
--name=Longwrite --bs=8m --rw=write
--name=Seqwrite --bs=1m --rw=write
--name=512Kwrite --bs=512k --rw=write
--name=256write --bs=256k --rw=write
--name=128write --bs=128k --rw=write
--name=64write --bs=64k --rw=write
--name=32write --bs=32k --rw=write
--name=16write --bs=16k --rw=write
--name=8write --bs=8k --rw=write
--name=4write --bs=4k --rw=write
--name=512KQD32write --bs=512k --iodepth=32 --rw=randwrite
--name=256KQD32write --bs=256k --iodepth=32 --rw=randwrite
--name=128KQD32write --bs=128k --iodepth=32 --rw=randwrite
--name=64KQD32write --bs=64k --iodepth=32 --rw=randwrite
--name=32KQD32write --bs=32k --iodepth=32 --rw=randwrite
--name=16KQD32write --bs=16k --iodepth=32 --rw=randwrite
--name=8KQD32write --bs=8k --iodepth=32 --rw=randwrite
--name=4kQD32write --bs=4k --iodepth=32 --rw=randwrite
| grep -E 'read|write|test' | grep -v ioengine

Kungwarira! Resource!Iyi kodhi inomhanyisa 36 bvunzo dzakasiyana, imwe neimwe ichimhanya kwemasekonzi mana. Hafu yebvunzo ndeyekurekodha. Unogona kurekodha zvakawanda paNVMe mumasekonzi mana. Kusvika 4 gigabytes pasekondi. Saka, yega yega yekunyora bvunzo inogona kudya kusvika ku4 gigabytes yeSSD sosi kubva kwauri.

Kuverenga nekunyora zvakasanganiswa?Ehe. Zvine musoro kumhanyisa kuverenga nekunyora bvunzo zvakasiyana. Zvakare, zvine musoro kuve nechokwadi chekuti macache ese anowiriraniswa kuitira kuti kunyora kwakamboita kusakanganisa kuverenga.

Mhedzisiro yacho ichasiyana zvakanyanya panguva yekutanga kuvhurwa uye inotevera iyo cache uye yakatetepa vhoriyamu ichizadza, uye zvakare zvichienderana nekuti iyo system yakakwanisa kuwiriranisa macache akazadzwa panguva yekupedzisira kwekutanga.

Pakati pezvimwe zvinhu, ini ndinokurudzira kuyera kumhanya pane yakatozara yakazara yakaonda vhoriyamu kubva iyo snapshot ichangotorwa. Munyori akawana mukana wekuona kuti zvinyorwa zvisina tsarukano zvinomhanyisa sei nekukurumidza mushure mekugadzira mufananidzo wekutanga, kunyanya kana cache isati yazara. Izvi zvinoitika nekuda kwekukopa-pa-kunyora kunyora semantics, kurongeka kwecache uye yakatetepa vhoriyamu mabhuroko, uye chokwadi chekuti chinyorwa chisina kurongeka kuRAID 6 chinoshanduka kuita chiverengero chekuverenga kubva kuRAID 6 ichiteverwa nekunyora kune cache. Mukugadzirisa kwedu, kuverenga kusina kubva kuRAID 6 kunosvika ku6 nguva (nhamba yeSATA SSDs muhurongwa) nekukurumidza kupfuura kunyora. Nokuti zvidhinha zveCoW zvakagovaniswa zvakatevedzana kubva padziva rakatetepa, ipapo kurekodha, kwechikamu chikuru, zvakare kunoshanduka kuita sequential.

Zvose zviri zviviri izvi zvinogona kushandiswa kune mukana wako.

Cache "coherent" snapshots

Kuti kuderedze njodzi yekurasikirwa nedata kana cache kukuvadzwa / kurasikirwa, munyori anoronga kuunza tsika yekutenderera snapshots kuvimbisa kutendeseka kwavo mune iyi kesi.

Chekutanga, nekuti yakaonda vhoriyamu metadata inogara pane isina kuvharwa mudziyo, iyo metadata ichaenderana uye inokwanisika kurasikirwa ichave yakaparadzaniswa mukati me data block.

Iyo inotevera snapshot kutenderera kutenderera inovimbisa kutendeseka kweiyo data mukati mezvipfupi kana cache kurasikirwa:

  1. Pavhoriyamu yega yega yakatetepa ine zita rekuti <zita>, gadzira mufananidzo une zita rekuti <zita> .cached
  2. Ngatiisei chikumbaridzo chekutama kune kukosha kwakanyanya. #lvchange --quiet --cachesettings "migration_threshold=16384" cache/cachedata
  3. Muchiuno tinotarisa nhamba yezvivharo zvakasviba mu cache: #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' kusvika tawana zero. Kana zero ichishaikwa kwenguva yakareba, inogona kugadzirwa nekuchinja cache kwechinguva kuti inyore. Nekudaro, uchifunga nezve kukurumidza hunhu hweSATA yedu uye NVMe SSD arrays, pamwe neiyo TBW sosi, iwe unogona kukurumidza kubata iyo nguva pasina kushandura cache mode, kana Hardware yako ichapedza zvachose sosi yayo mukati. mazuva mashoma. Nekuda kwekushomeka kwezviwanikwa, sisitimu yacho, mumusimboti, haikwanise kuve pasi pe100% kunyora mutoro nguva dzese. Yedu NVMe SSDs pasi pe100% kunyora mutoro inopedza zvachose sosi mukati 3-4 mazuva. SATA SSDs inongogara kaviri kwenguva yakareba. Naizvozvo, isu tichafungidzira kuti mizhinji yemutoro inoenda pakuverenga, uye isu tine pfupi-pfupi kuputika kwekuita kwakanyanya kwakanyanya kusanganiswa nemutoro wakaderera paavhareji yekunyora.
  4. Patakangobata (kana kuita) zero, tinopa zita rokuti <zita> .cached ku <zita> .committed. Chekare <zita> .committed chadzimwa.
  5. Sarudzo, kana cache izere 100%, inogona kudzokororwa ne script, nokudaro kuibvisa. Iine hafu-isina cache, iyo system inoshanda nekukurumidza zvakanyanya pakunyora.
  6. Isa chikumbaridzo chekutama kusvika zero: #lvchange --quiet --cachesettings "migration_threshold=0" cache/cachedata Izvi zvichadzivirira kwenguva pfupi cache kubva pakuyananisa kune midhiya huru.
  7. Isu tinomirira kusvika shanduko zhinji dzaungana mu cache #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' kana kuti timer ichadzima.
  8. Tinodzokorora zvakare.

Sei matambudziko ane migration chikumbaridzo...?Chinhu chacho ndechekuti mukuita chaiko, kurekodha "random" hakusi kurongeka zvachose. Kana isu takanyora chimwe chinhu kuchikamu che4 kilobytes muhukuru, pane mukana wakakura wekuti mumaminetsi mashoma anotevera rekodhi ichaitwa kune imwechete kana imwe yeakavakidzana (+- 32K) zvikamu.

Nekumisikidza chikumbaridzo chekutama kusvika zero, tinoverengera kunyora kuwiriranisa paSATA SSD uye tobatanidza shanduko dzinoverengeka kune imwe 64K block mune cache. Izvi zvakanyanya kuchengetedza sosi yeSATA SSD.

Ko code iripi..?Nehurombo, munyori anozviona seasina kukwana mukugadzirwa kwezvinyorwa zvebash nekuti ari 100% akazvidzidzisa uye anoita budiriro inotungamirwa ne "google", saka anotenda kuti iyo inotyisa kodhi inobuda mumaoko ake haifanirwe kushandiswa nemunhu. zvimwe.

Ini ndinofunga kuti nyanzvi mune ino ndima ichakwanisa kuzvimiririra kuratidzira zvese zvinotsanangurwa pamusoro, kana zvichidikanwa, uye, pamwe, kunyange zvinoyevedza kuigadzira sehurongwa hwesevhisi, sekuyedza kuita nemunyori.

Yakareruka snapshot rotation scheme ichatibvumira kwete kungogara tine imwe snapshot yakanyatsoenderana paSATA SSD, asi ichaitawo kutibvumira, tichishandisa thin_delta utility, kuti tione kuti zvivharo zvakachinjwa mushure mekusikwa kwayo, uye nokudaro kugadzirisa kukanganisa mavhoriyamu makuru, kurerutsa zvikuru kupora .

TRIM/DISCARD mu libvirt/KVM

Nokuti iyo data yekuchengetedza ichashandiswa kuKVM ichimhanya libvirt, zvino ingava pfungwa yakanaka kudzidzisa maVM edu kwete kungotora nzvimbo yemahara, asiwo kusunungura izvo zvisingachadiwi.

Izvi zvinoitwa nekutevedzera TRIM/DISCARD rutsigiro pamadhisiki chaiwo. Kuti uite izvi, unofanirwa kushandura iyo controller mhando kune virtio-scsi uye gadzirisa iyo xml.

#virsh edit vmname
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='writethrough' io='threads' discard='unmap'/>
<source dev='/dev/images/vmname'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

<controller type='scsi' index='0' model='virtio-scsi'>
<alias name='scsi0'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</controller>

MaDISCARD akadaro kubva kumaOS evaenzi anogadziriswa nemazvo neLVM, uye zvidhinha zvinosunungurwa nenzira kwayo zvese mucache uye mudziva rakatetepa. Muchiitiko chedu, izvi zvinonyanya kuitika nenzira yakanonoka, kana uchibvisa mufananidzo unotevera.

BTRFS Backup

Shandisa zvinyorwa zvakagadzirirwa ne zvakanyanyisa kungwarira uye panjodzi yomunhu. Munyori akanyora iyi kodhi pachake uye iye oga. Ndine chokwadi chekuti vazhinji vane ruzivo vashandisi veLinux vane maturusi akafanana, uye hapana chikonzero chekutevedzera cheumwe munhu.

Ngatigadzirei vhoriyamu pane backup mudziyo:

#lvcreate -L 256G --name backup backup

Ngatiifomate muBTRFS:

#mkfs.btrfs /dev/backup/backup

Ngatigadzire mapoinzi uye tiise midzi midzi yefaira system:

#mkdir /backup
#mkdir /backup/btrfs
#mkdir /backup/btrfs/root
#mkdir /backup/btrfs/back
#ln -s /boot /backup/btrfs
# cat >>/etc/fstab << EOF

/dev/mapper/root-root /backup/btrfs/root btrfs defaults,space_cache,noatime,nodiratime 0 2
/dev/mapper/backup-backup /backup/btrfs/back btrfs defaults,space_cache,noatime,nodiratime 0 2
EOF
#mount -a
#update-initramfs -u
#update-grub

Ngatigadzire madhairekitori ekuchengetedza ma backups:

#mkdir /backup/btrfs/back/remote
#mkdir /backup/btrfs/back/remote/root
#mkdir /backup/btrfs/back/remote/boot

Ngatigadzire dhairekitori rezvinyorwa zvekuchengetedza:

#mkdir /root/btrfs-backup

Ngatikopei script:

Yakawanda inotyisa bash kodhi. Shandisa panjodzi yako. Usanyore tsamba dzehasha kumunyori...#cat >/root/btrfs-backup/btrfs-backup.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"
DATE_PREFIX='%Y-%m-%d'
DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'
DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
BASE_SUFFIX=".@base"
PEND_SUFFIX=".@pend"
SNAP_SUFFIX=".@snap"
MOUNTS="/backup/btrfs/"
BACKUPS="/backup/btrfs/back/remote/"

function terminate ()
{
echo "$1" >&2
exit 1
}

function wait_lock()
{
flock 98
}

function wait_lock_or_terminate()
{
echo "Wating for lock..."
wait_lock || terminate "Failed to get lock. Exiting..."
echo "Got lock..."
}

function suffix()
{
FORMATTED_DATE=$(date +"$DATE_FORMAT")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function filter()
{
FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function backup()
{
SOURCE_PATH="$MOUNTS$1"
TARGET_PATH="$BACKUPS$1"
SOURCE_BASE_PATH="$MOUNTS$1$BASE_SUFFIX"
TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
TARGET_BASE_DIR="$(dirname $TARGET_BASE_PATH)"
SOURCE_PEND_PATH="$MOUNTS$1$PEND_SUFFIX"
TARGET_PEND_PATH="$BACKUPS$1$PEND_SUFFIX"
if [ -d "$SOURCE_BASE_PATH" ] then
echo "$SOURCE_BASE_PATH found"
else
echo "$SOURCE_BASE_PATH File not found creating snapshot of $SOURCE_PATH to $SOURCE_BASE_PATH"
btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_BASE_PATH
sync
if [ -d "$TARGET_BASE_PATH" ] then
echo "$TARGET_BASE_PATH found out of sync with source... removing..."
btrfs subvolume delete -c $TARGET_BASE_PATH
sync
fi
fi
if [ -d "$TARGET_BASE_PATH" ] then
echo "$TARGET_BASE_PATH found"
else
echo "$TARGET_BASE_PATH not found. Synching to $TARGET_BASE_DIR"
btrfs send $SOURCE_BASE_PATH | btrfs receive $TARGET_BASE_DIR
sync
fi
if [ -d "$SOURCE_PEND_PATH" ] then
echo "$SOURCE_PEND_PATH found removing..."
btrfs subvolume delete -c $SOURCE_PEND_PATH
sync
fi
btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_PEND_PATH
sync
if [ -d "$TARGET_PEND_PATH" ] then
echo "$TARGET_PEND_PATH found removing..."
btrfs subvolume delete -c $TARGET_PEND_PATH
sync
fi
echo "Sending $SOURCE_PEND_PATH to $TARGET_PEND_PATH"
btrfs send -p $SOURCE_BASE_PATH $SOURCE_PEND_PATH | btrfs receive $TARGET_BASE_DIR
sync
TARGET_DATE_SUFFIX=$(suffix)
btrfs subvolume snapshot -r $TARGET_PEND_PATH "$TARGET_PATH$TARGET_DATE_SUFFIX"
sync
btrfs subvolume delete -c $SOURCE_BASE_PATH
sync
btrfs subvolume delete -c $TARGET_BASE_PATH
sync
mv $SOURCE_PEND_PATH $SOURCE_BASE_PATH
mv $TARGET_PEND_PATH $TARGET_BASE_PATH
sync
}

function list()
{
LIST_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
LIST_TARGET_BASE_DIR="$(dirname $LIST_TARGET_BASE_PATH)"
LIST_TARGET_BASE_NAME="$(basename -s .$BASE_SUFFIX $LIST_TARGET_BASE_PATH)"
find "$LIST_TARGET_BASE_DIR" -maxdepth 1 -mindepth 1 -type d -printf "%fn" | grep "${LIST_TARGET_BASE_NAME/$BASE_SUFFIX/$SNAP_SUFFIX}.$DATE_REGEX"
}

function remove()
{
REMOVE_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
REMOVE_TARGET_BASE_DIR="$(dirname $REMOVE_TARGET_BASE_PATH)"
btrfs subvolume delete -c $REMOVE_TARGET_BASE_DIR/$2
sync
}

function removeall()
{
DATE_OFFSET="$2"
FILTER="$(filter "$DATE_OFFSET")"
while read -r SNAPSHOT ; do
remove "$1" "$SNAPSHOT"
done < <(list "$1" | grep "$FILTER")

}

(
COMMAND="$1"
shift

case "$COMMAND" in
"--help")
echo "Help"
;;
"suffix")
suffix
;;
"filter")
filter "$1"
;;
"backup")
wait_lock_or_terminate
backup "$1"
;;
"list")
list "$1"
;;
"remove")
wait_lock_or_terminate
remove "$1" "$2"
;;
"removeall")
wait_lock_or_terminate
removeall "$1" "$2"
;;
*)
echo "None.."
;;
esac
) 98>$LOCK_FILE

EOF

Zvinoita sei..?Iine seti yemirairo yakapusa yekugadzira BTRFS snapshots uye kuikopa kune imwe FS uchishandisa BTRFS kutumira/kugamuchira.

Kutanga kwekutanga kunogona kuve kwakareba, nekuti ... Pakutanga, data yese ichakopwa. Kumwe kutangwa kuchave nekukurumidza, nekuti... Chete shanduko dzichakopwa.

Imwe script yatinoisa mu cron:

Imwe bash kodhi#cat >/root/btrfs-backup/cron-daily.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

BACKUP_SCRIPT="$SCRIPT_DIR/btrfs-backup.sh"
RETENTION="-60 day"
$BACKUP_SCRIPT backup root/@
$BACKUP_SCRIPT removeall root/@ "$RETENTION"
$BACKUP_SCRIPT backup root/@home
$BACKUP_SCRIPT removeall root/@home "$RETENTION"
$BACKUP_SCRIPT backup boot/
$BACKUP_SCRIPT removeall boot/ "$RETENTION"
EOF

Zvinoita sei..?Inogadzira uye inowiriranisa akawedzera snapshots eakarongwa BTRFS mavhoriyamu pane backup FS. Mushure meizvi, inodzima mifananidzo yese yakagadzirwa mazuva makumi matanhatu apfuura. Mushure mekutangwa, snapshots emazuva emavhoriyamu akanyorwa achaonekwa mu /backup/btrfs/back/remote/ subdirectories.

Ngatipei kodzero dzekuita kodhi:

#chmod +x /root/btrfs-backup/cron-daily.sh
#chmod +x /root/btrfs-backup/btrfs-backup.sh

Ngatitarise uye tiise mu cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup
#cat /var/log/syslog | grep btrfs-backup
#crontab -e
0 2 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup

LVM yakaonda backup

Ngatigadzire dziva rakatetepa pane backup mudziyo:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T backup/thin-pool

Ngatiisei ddrescue, nekuti... zvinyorwa zvichashandisa chishandiso ichi:

#apt-get install gddrescue

Ngatigadzire dhairekitori rezvinyorwa:

#mkdir /root/lvm-thin-backup

Ngatikopei zvinyorwa:

Zvakawanda bash mukati ...#cat >/root/lvm-thin-backup/lvm-thin-backup.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"
DATE_PREFIX='%Y-%m-%d'
DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'
DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
BASE_SUFFIX=".base"
PEND_SUFFIX=".pend"
SNAP_SUFFIX=".snap"
BACKUPS="backup"
BACKUPS_POOL="thin-pool"

export LVM_SUPPRESS_FD_WARNINGS=1

function terminate ()
{
echo "$1" >&2
exit 1
}

function wait_lock()
{
flock 98
}

function wait_lock_or_terminate()
{
echo "Wating for lock..."
wait_lock || terminate "Failed to get lock. Exiting..."
echo "Got lock..."
}

function suffix()
{
FORMATTED_DATE=$(date +"$DATE_FORMAT")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function filter()
{
FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function read_thin_id {
lvs --rows --reportformat basic --quiet -othin_id "$1/$2" | awk '{print $2}'
}

function read_pool_lv {
lvs --rows --reportformat basic --quiet -opool_lv "$1/$2" | awk '{print $2}'
}

function read_lv_dm_path {
lvs --rows --reportformat basic --quiet -olv_dm_path "$1/$2" | awk '{print $2}'
}

function read_lv_active {
lvs --rows --reportformat basic --quiet -olv_active "$1/$2" | awk '{print $2}'
}

function read_lv_chunk_size {
lvs --rows --reportformat basic --quiet --units b --nosuffix -ochunk_size "$1/$2" | awk '{print $2}'
}

function read_lv_size {
lvs --rows --reportformat basic --quiet --units b --nosuffix -olv_size "$1/$2" | awk '{print $2}'
}

function activate_volume {
lvchange -ay -Ky "$1/$2"
}

function deactivate_volume {
lvchange -an "$1/$2"
}

function read_thin_metadata_snap {
dmsetup status "$1" | awk '{print $7}'
}

function thindiff()
{
DIFF_VG="$1"
DIFF_SOURCE="$2"
DIFF_TARGET="$3"
DIFF_SOURCE_POOL=$(read_pool_lv $DIFF_VG $DIFF_SOURCE)
DIFF_TARGET_POOL=$(read_pool_lv $DIFF_VG $DIFF_TARGET)

if [ "$DIFF_SOURCE_POOL" == "" ] then
(>&2 echo "Source LV is not thin.")
exit 1
fi

if [ "$DIFF_TARGET_POOL" == "" ] then
(>&2 echo "Target LV is not thin.")
exit 1
fi

if [ "$DIFF_SOURCE_POOL" != "$DIFF_TARGET_POOL" ] then
(>&2 echo "Source and target LVs belong to different thin pools.")
exit 1
fi

DIFF_POOL_PATH=$(read_lv_dm_path $DIFF_VG $DIFF_SOURCE_POOL)
DIFF_SOURCE_ID=$(read_thin_id $DIFF_VG $DIFF_SOURCE)
DIFF_TARGET_ID=$(read_thin_id $DIFF_VG $DIFF_TARGET)
DIFF_POOL_PATH_TPOOL="$DIFF_POOL_PATH-tpool"
DIFF_POOL_PATH_TMETA="$DIFF_POOL_PATH"_tmeta
DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)

if [ "$DIFF_POOL_METADATA_SNAP" != "-" ] then
(>&2 echo "Thin pool metadata snapshot already exist. Assuming stale one. Will release metadata snapshot in 5 seconds.")
sleep 5
dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap
fi

dmsetup message $DIFF_POOL_PATH_TPOOL 0 reserve_metadata_snap
DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)

if [ "$DIFF_POOL_METADATA_SNAP" == "-" ] then
(>&2 echo "Failed to create thin pool metadata snapshot.")
exit 1
fi

#We keep output in variable because metadata snapshot need to be released early.
DIFF_DATA=$(thin_delta -m$DIFF_POOL_METADATA_SNAP --snap1 $DIFF_SOURCE_ID --snap2 $DIFF_TARGET_ID $DIFF_POOL_PATH_TMETA)

dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap

echo $"$DIFF_DATA" | grep -E 'different|left_only|right_only' | sed 's/</"/g' | sed 's/ /"/g' | awk -F'"' '{print $6 "t" $8 "t" $11}' | sed 's/different/copy/g' | sed 's/left_only/copy/g' | sed 's/right_only/discard/g'

}

function thinsync()
{
SYNC_VG="$1"
SYNC_PEND="$2"
SYNC_BASE="$3"
SYNC_TARGET="$4"
SYNC_PEND_POOL=$(read_pool_lv $SYNC_VG $SYNC_PEND)
SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SYNC_VG $SYNC_PEND_POOL)
SYNC_PEND_PATH=$(read_lv_dm_path $SYNC_VG $SYNC_PEND)

activate_volume $SYNC_VG $SYNC_PEND

while read -r SYNC_ACTION SYNC_OFFSET SYNC_LENGTH ; do
SYNC_OFFSET_BYTES=$((SYNC_OFFSET * SYNC_BLOCK_SIZE))
SYNC_LENGTH_BYTES=$((SYNC_LENGTH * SYNC_BLOCK_SIZE))
if [ "$SYNC_ACTION" == "copy" ] then
ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SYNC_PEND_PATH" "$SYNC_TARGET"
fi

if [ "$SYNC_ACTION" == "discard" ] then
blkdiscard -o $SYNC_OFFSET_BYTES -l $SYNC_LENGTH_BYTES "$SYNC_TARGET"
fi
done < <(thindiff "$SYNC_VG" "$SYNC_PEND" "$SYNC_BASE")
}

function discard_volume()
{
DISCARD_VG="$1"
DISCARD_LV="$2"
DISCARD_LV_PATH=$(read_lv_dm_path "$DISCARD_VG" "$DISCARD_LV")
if [ "$DISCARD_LV_PATH" != "" ] then
echo "$DISCARD_LV_PATH found"
else
echo "$DISCARD_LV not found in $DISCARD_VG"
exit 1
fi
DISCARD_LV_POOL=$(read_pool_lv $DISCARD_VG $DISCARD_LV)
DISCARD_LV_SIZE=$(read_lv_size "$DISCARD_VG" "$DISCARD_LV")
lvremove -y --quiet "$DISCARD_LV_PATH" || exit 1
lvcreate --thin-pool "$DISCARD_LV_POOL" -V "$DISCARD_LV_SIZE"B --name "$DISCARD_LV" "$DISCARD_VG" || exit 1
}

function backup()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
SOURCE_PEND_LV="$SOURCE_LV$PEND_SUFFIX"
TARGET_PEND_LV="$TARGET_LV$PEND_SUFFIX"
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "Source base not found creating snapshot of $SOURCE_VG/$SOURCE_LV to $SOURCE_VG/$SOURCE_BASE_LV"
lvcreate --quiet --snapshot --name "$SOURCE_BASE_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
echo "Discarding $SOURCE_BASE_LV_PATH as we need to bootstrap."
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)
discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
sync
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found out of sync with source... removing..."
lvremove -y --quiet $TARGET_BASE_LV_PATH || exit 1
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
sync
fi
fi
SOURCE_BASE_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_BASE_LV")
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_VG/$TARGET_LV not found. Creating empty volume."
lvcreate --thin-pool "$BACKUPS_POOL" -V "$SOURCE_BASE_SIZE"B --name "$TARGET_BASE_LV" "$TARGET_VG" || exit 1
echo "Have to rebootstrap. Discarding source at $SOURCE_BASE_LV_PATH"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)
discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
TARGET_BASE_POOL=$(read_pool_lv $TARGET_VG $TARGET_BASE_LV)
TARGET_BASE_CHUNK_SIZE=$(read_lv_chunk_size $TARGET_VG $TARGET_BASE_POOL)
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
echo "Discarding target at $TARGET_BASE_LV_PATH"
discard_volume "$TARGET_VG" "$TARGET_BASE_LV"
sync
fi
if [ "$SOURCE_PEND_LV_PATH" != "" ] then
echo "$SOURCE_PEND_LV_PATH found removing..."
lvremove -y --quiet "$SOURCE_PEND_LV_PATH" || exit 1
sync
fi
lvcreate --quiet --snapshot --name "$SOURCE_PEND_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1
SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")
sync
if [ "$TARGET_PEND_LV_PATH" != "" ] then
echo "$TARGET_PEND_LV_PATH found removing..."
lvremove -y --quiet $TARGET_PEND_LV_PATH
sync
fi
lvcreate --quiet --snapshot --name "$TARGET_PEND_LV" "$TARGET_VG/$TARGET_BASE_LV" || exit 1
TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")
SOURCE_PEND_LV_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_PEND_LV")
lvresize -L "$SOURCE_PEND_LV_SIZE"B "$TARGET_PEND_LV_PATH"
activate_volume "$TARGET_VG" "$TARGET_PEND_LV"
echo "Synching $SOURCE_PEND_LV_PATH to $TARGET_PEND_LV_PATH"
thinsync "$SOURCE_VG" "$SOURCE_PEND_LV" "$SOURCE_BASE_LV" "$TARGET_PEND_LV_PATH" || exit 1
sync

TARGET_DATE_SUFFIX=$(suffix)
lvcreate --quiet --snapshot --name "$TARGET_LV$TARGET_DATE_SUFFIX" "$TARGET_VG/$TARGET_PEND_LV" || exit 1
sync
lvremove --quiet -y "$SOURCE_BASE_LV_PATH" || exit 1
sync
lvremove --quiet -y "$TARGET_BASE_LV_PATH" || exit 1
sync
lvrename -y "$SOURCE_VG/$SOURCE_PEND_LV" "$SOURCE_BASE_LV" || exit 1
lvrename -y "$TARGET_VG/$TARGET_PEND_LV" "$TARGET_BASE_LV" || exit 1
sync
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function verify()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "$SOURCE_BASE_LV_PATH not found"
exit 1
fi
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_BASE_LV_PATH not found"
exit 1
fi
activate_volume "$TARGET_VG" "$TARGET_BASE_LV"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
echo Comparing "$SOURCE_BASE_LV_PATH" with "$TARGET_BASE_LV_PATH"
cmp "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"
echo Done...
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function resync()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "$SOURCE_BASE_LV_PATH not found"
exit 1
fi
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_BASE_LV_PATH not found"
exit 1
fi
activate_volume "$TARGET_VG" "$TARGET_BASE_LV"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)

echo Syncronizing "$SOURCE_BASE_LV_PATH" to "$TARGET_BASE_LV_PATH"

CMP_OFFSET=0
while [[ "$CMP_OFFSET" != "" ]] ; do
CMP_MISMATCH=$(cmp -i "$CMP_OFFSET" "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH" | grep differ | awk '{print $5}' | sed 's/,//g' )
if [[ "$CMP_MISMATCH" != "" ]] ; then
CMP_OFFSET=$(( CMP_MISMATCH + CMP_OFFSET ))
SYNC_OFFSET_BYTES=$(( ( CMP_OFFSET / SYNC_BLOCK_SIZE ) * SYNC_BLOCK_SIZE ))
SYNC_LENGTH_BYTES=$(( SYNC_BLOCK_SIZE ))
echo "Synching $SYNC_LENGTH_BYTES bytes at $SYNC_OFFSET_BYTES from $SOURCE_BASE_LV_PATH to $TARGET_BASE_LV_PATH"
ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"
else
CMP_OFFSET=""
fi
done
echo Done...
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function list()
{
LIST_SOURCE_VG="$1"
LIST_SOURCE_LV="$2"
LIST_TARGET_VG="$BACKUPS"
LIST_TARGET_LV="$LIST_SOURCE_VG-$LIST_SOURCE_LV"
LIST_TARGET_BASE_LV="$LIST_TARGET_LV$SNAP_SUFFIX"
lvs -olv_name | grep "$LIST_TARGET_BASE_LV.$DATE_REGEX"
}

function remove()
{
REMOVE_TARGET_VG="$BACKUPS"
REMOVE_TARGET_LV="$1"
lvremove -y "$REMOVE_TARGET_VG/$REMOVE_TARGET_LV"
sync
}

function removeall()
{
DATE_OFFSET="$3"
FILTER="$(filter "$DATE_OFFSET")"
while read -r SNAPSHOT ; do
remove "$SNAPSHOT"
done < <(list "$1" "$2" | grep "$FILTER")

}

(
COMMAND="$1"
shift

case "$COMMAND" in
"--help")
echo "Help"
;;
"suffix")
suffix
;;
"filter")
filter "$1"
;;
"backup")
wait_lock_or_terminate
backup "$1" "$2"
;;
"list")
list "$1" "$2"
;;
"thindiff")
thindiff "$1" "$2" "$3"
;;
"thinsync")
thinsync "$1" "$2" "$3" "$4"
;;
"verify")
wait_lock_or_terminate
verify "$1" "$2"
;;
"resync")
wait_lock_or_terminate
resync "$1" "$2"
;;
"remove")
wait_lock_or_terminate
remove "$1"
;;
"removeall")
wait_lock_or_terminate
removeall "$1" "$2" "$3"
;;
*)
echo "None.."
;;
esac
) 98>$LOCK_FILE

EOF

Zvinoita sei...?Ine seti yemirairo yekunyengedza zvidhori zvitete uye kuwiriranisa mutsauko pakati pezvipikicha zvitete zvakatambirwa kuburikidza ne thin_delta kune imwe block mudziyo uchishandisa ddrescue uye blkdiscard.

Imwe script yatinozoisa mu cron:

Imwe bash zvishoma#cat >/root/lvm-thin-backup/cron-daily.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

BACKUP_SCRIPT="$SCRIPT_DIR/lvm-thin-backup.sh"
RETENTION="-60 days"

$BACKUP_SCRIPT backup images linux-dev
$BACKUP_SCRIPT backup images win8
$BACKUP_SCRIPT backup images win8-data
#etc

$BACKUP_SCRIPT removeall images linux-dev "$RETENTION"
$BACKUP_SCRIPT removeall images win8 "$RETENTION"
$BACKUP_SCRIPT removeall images win8-data "$RETENTION"
#etc

EOF

Zvinoita sei...?Inoshandisa script yakapfuura kugadzira uye kuwiriranisa mabhapu emavhoriyamu akatetepa akanyorwa. Iyo script inosiya isingashande snapshots emavhoriyamu akanyorwa, ayo anodiwa kuronda shanduko kubva pakupedzisira kuwiriraniswa.

Ichi chinyorwa chinofanira kupepetwa, chichitsanangura rondedzero yemavhoriyamu matete ayo makopi ekuchengetedza anofanirwa kuitwa. Mazita akapihwa ndeekuenzanisira chete. Kana uchida, unogona kunyora chinyorwa chinozobatanidza mavhoriyamu ese.

Ngatipei kodzero:

#chmod +x /root/lvm-thin-backup/cron-daily.sh
#chmod +x /root/lvm-thin-backup/lvm-thin-backup.sh

Ngatitarise uye tiise mu cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup
#cat /var/log/syslog | grep lvm-thin-backup
#crontab -e
0 3 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup

Kutanga kwekutanga kuchave kwenguva refu, nekuti ... mavhoriyamu matete anozonyatso wiriraniswa nekukopa ese anoshandiswa nzvimbo. Nekuda kweLVM yakatetepa metadata, isu tinoziva kuti ndeapi mabhururu ari kunyatso shandiswa, saka chete anoshandiswa matete mavhoriyamu mabhuroki anokopwa.

Inotevera inomhanya ichakopa iyo data zvakanyanya nekutenda nekuchinja tracking kuburikidza neLVM nhete metadata.

Ngationei zvakaitika:

#time /root/btrfs-backup/cron-daily.sh
real 0m2,967s
user 0m0,225s
sys 0m0,353s

#time /root/lvm-thin-backup/cron-daily.sh
real 1m2,710s
user 0m12,721s
sys 0m6,671s

#ls -al /backup/btrfs/back/remote/*
/backup/btrfs/back/remote/boot:
total 0
drwxr-xr-x 1 root root 1260 ΠΌΠ°Ρ€ 26 09:11 .
drwxr-xr-x 1 root root 16 ΠΌΠ°Ρ€ 6 09:30 ..
drwxr-xr-x 1 root root 322 ΠΌΠ°Ρ€ 26 02:00 .@base
drwxr-xr-x 1 root root 516 ΠΌΠ°Ρ€ 6 09:39 [email protected]
drwxr-xr-x 1 root root 516 ΠΌΠ°Ρ€ 6 09:39 [email protected]
...
/backup/btrfs/back/remote/root:
total 0
drwxr-xr-x 1 root root 2820 ΠΌΠ°Ρ€ 26 09:11 .
drwxr-xr-x 1 root root 16 ΠΌΠ°Ρ€ 6 09:30 ..
drwxr-xr-x 1 root root 240 ΠΌΠ°Ρ€ 26 09:11 @.@base
drwxr-xr-x 1 root root 22 ΠΌΠ°Ρ€ 26 09:11 @home.@base
drwxr-xr-x 1 root root 22 ΠΌΠ°Ρ€ 6 09:39 @[email protected]
drwxr-xr-x 1 root root 22 ΠΌΠ°Ρ€ 6 09:39 @[email protected]
...
drwxr-xr-x 1 root root 240 ΠΌΠ°Ρ€ 6 09:39 @[email protected]
drwxr-xr-x 1 root root 240 ΠΌΠ°Ρ€ 6 09:39 @[email protected]
...

#lvs -olv_name,lv_size images && lvs -olv_name,lv_size backup
LV LSize
linux-dev 128,00g
linux-dev.base 128,00g
thin-pool 1,38t
win8 128,00g
win8-data 2,00t
win8-data.base 2,00t
win8.base 128,00g
LV LSize
backup 256,00g
images-linux-dev.base 128,00g
images-linux-dev.snap.2020-03-08-10-09-11 128,00g
images-linux-dev.snap.2020-03-08-10-09-25 128,00g
...
images-win8-data.base 2,00t
images-win8-data.snap.2020-03-16-14-11-55 2,00t
images-win8-data.snap.2020-03-16-14-19-50 2,00t
...
images-win8.base 128,00g
images-win8.snap.2020-03-17-04-51-46 128,00g
images-win8.snap.2020-03-18-03-02-49 128,00g
...
thin-pool <2,09t

Izvi zvinei nechekuita nezvidhori zvekugara?

Zvingangodaro, zvichipihwa kuti LVM LV mavhoriyamu ane musoro anogona kuve LVM PV mavhoriyamu emuviri kune mamwe maVG. LVM inogona kudzokororwa, senge madhiri. Izvi zvinopa LVM yakanyanyisa kushanduka.

PS

Muchinyorwa chinotevera, tichaedza kushandisa akati wandei akafanana enharembozha ekuchengetedza masisitimu/KVM sehwaro hwekugadzira geo-yakagoverwa chengetedzo/vm cluster ine redundancy pamakondinendi akati wandei tichishandisa matafura epamba, iyo Internet yekumba uye P2P network.

Source: www.habr.com

Voeg