He aha te hononga o LVM me Matryoshka?

Tuhinga o mua.
E hiahia ana ahau ki te whakapuaki ki te hapori i taku wheako mahi mo te hanga i tetahi punaha rokiroki raraunga mo KVM ma te whakamahi md RAID + LVM.

Kei roto i te kaupapa:

• Te hanga md RAID 1 mai i te NVMe SSD.
• Ko te kohikohi md RAID 6 mai i te SATA SSD me nga puku auau.
• Nga ahuatanga o te mahi TRIM/DISCARD i runga i te SSD RAID 1/6.
• Te hanga i te huinga bootable md RAID 1/6 i runga i te huinga kōpae noa.
• Te whakauru i te punaha i runga i te NVMe RAID 1 karekau he tautoko NVMe i te BIOS.
• Te whakamahi i te keteroki LVM me te LVM kikokore.
• Ma te whakamahi i nga whakaahua BTRFS me te tuku / whiwhi mo te taapiri.
• Ma te whakamahi i nga whakaahua kikokore LVM me te thin_delta mo nga taapiri ahua BTRFS.

Mena kei te pirangi koe, tirohia koa te ngeru.

Puka tono

Karekau he kawenga a te kaituhi mo nga hua o te whakamahi, te kore ranei e whakamahi rauemi/tauira/waehere/tohutohu/raraunga mai i tenei tuhinga. Ma te panui me te whakamahi i tenei rauemi ahakoa he aha, kei a koe te kawenga mo nga hua katoa o enei mahi. Ko nga hua ka puta ko:

• NVMe SSDs parai parai.
• Kua pau katoa te rauemi tuhi me te kore o nga puku SSD.
• Kua ngaro nga raraunga katoa i runga i nga puku katoa, tae atu ki nga kape taapiri.
• Ko nga taputapu rorohiko hapa.
• Moumau te taima, nga uaua me te moni.
• Ko etahi atu paanga kaore i te raarangi i runga ake nei.

Iron

Ko:

Papapapa matua mai i te tau 2013 me te maramara Z87, kua oti me Intel Core i7 / Haswell.

• Tukatuka 4 matua, 8 miro
• 32 GB DDR3 RAM
• 1 x 16 ranei 2 x 8 PCIe 3.0
• 1 x 4 + 1 x 1 PCIe 2.0
• 6 x 6 GBps SATA 3 tūhonohono

SAS whāurutau LSI SAS9211-8I i rarapa ki te aratau IT / HBA. Kua whakakapia te maru whakaahei RAID ki te firmware HBA ki:

1. Ka taea e koe te maka atu i tenei urutau i nga wa katoa ka whakakapi ki tetahi atu ka kitea e koe.
2. I mahi te TRIM/Discard i runga i nga kōpae, na te mea... i roto i te RAID firmware kaore enei whakahau e tautokohia, a, ko te HBA, i te nuinga o te waa, kaore e aro ki nga whakahau e tukuna ana i runga i te pahi.

puku mārō - 8 mongamonga o HGST Travelstar 7K1000 ki te kaha o te 1 TB i roto i te 2.5 ahua tauwehe, rite mo rorohiko. Ko enei puku i mua i te rarangi RAID 6. Ka whai waahi ano ratou ki te punaha hou. Hei rokiroki i nga taapiri o te rohe.

Hei taapiri atu:

6 nga wahi SATA SSD tauira Samsung 860 QVO 2TB. Ko enei SSD e hiahiatia ana he nui te rahi, ko te waahi o te keteroki SLC, te pono, me te utu iti i hiahiatia. I hiahiatia te tautoko mo te whakakore/kore, ka tirohia e te raina i dmesg:

kernel: ata1.00: Enabling discard_zeroes_data

2 nga waahanga o te tauira NVMe SSD Samsung SSD 970 EVO 500GB.

Mo enei SSD, he mea nui te tere panui/tuhi matapōkere me te kaha rauemi mo o hiahia. Radiator mo ratou. Ka tika. Tino. Ki te kore, parai kia maoa i te wa tuatahi o te tukutahi RAID.

StarTech PEX8M2E2 urutau mo te 2 x NVMe SSD i whakauruhia ki te mokamoka PCIe 3.0 8x. Ko tenei, ano, he HBA noa, engari mo NVMe. He rereke mai i nga taapiri utu iti na te mea kaore e hiahiatia te tautoko a te PCIe bifurcation mai i te papahoa papa na te mea kei te uru mai te whakahuri PCIe i roto. Ka mahi ahakoa i roto i te punaha tawhito me PCIe, ahakoa he mokamoka x1 PCIe 1.0. Ko te tikanga, i te tere e tika ana. Karekau he RAID kei reira. Karekau he BIOS hanga-i runga i te papa. Na, kaore to punaha e ako makutu ki te whawhai me te NVMe, he iti ake te mihi ki a NVMe RAID ki tenei taputapu.

Ko tenei waahanga na te mea kotahi noa te 8x PCIe 3.0 kore utu i roto i te punaha, a, mena he 2 nga mokowhiti kore utu, ka taea te whakakapi ki te rua pene PEX4M2E1 me nga taapiri ranei, ka taea te hoko ki hea i te utu 600 rubles.

Ko te whakakore i nga momo taputapu katoa, i hangaia ranei nga chipset / BIOS RAID i hangaia ma te whakaaro, kia taea ai te whakakapi katoa i te punaha, haunga te SSD / HDD ake, me te pupuri i nga raraunga katoa. Ko te mea pai, kia taea ai e koe te tiaki ahakoa te punaha whakahaere kua whakauruhia ina neke ki nga taputapu hou/rereke. Ko te mea nui kei reira nga tauranga SATA me PCIe. He rite ki te CD ora, puku kohiko bootable ranei, he tere noa, he iti te rahi.

AtaahuaKi te kore, ka mohio koe ka aha - i etahi wa ka hiahia koe ki te kawe i te huinga katoa hei tango. Engari kaore au e hiahia kia ngaro nga raraunga. Ki te mahi i tenei, ko nga panui kua whakahuahia katoa kei runga i nga kiriata i nga kokoru 5.25 o te keehi paerewa.

Ae, me te mea, mo te whakamatautau i nga tikanga rereke o te SSD caching i Linux.

He hoha nga whakaeke taputapu. Hurihia. Ka mahi, kaore ranei. A me te mdadm kei nga wa katoa nga whiringa.

Pūmanawa

I mua, i whakauruhia a Debian 8 Jessie ki te taputapu, e tata ana ki te EOL. Ko te RAID 6 i kohia mai i nga HDD kua whakahuahia i runga ake nei i honoa ki te LVM. I whakahaerehia e ia nga miihini mariko i roto i te kvm/libvirt.

No te mea He pai te wheako o te kaituhi ki te hanga i nga puku kohiko SATA / NVMe ka taea te kawe, me te kore e pakaru i te tauira tika o mua, ko te Ubuntu 18.04 i whiriwhiria hei punaha whaainga, kua oti kee te whakapumau, engari he 3 tau tonu. tautoko a meake nei.

Kei roto i te punaha kua whakahuahia nga taraiwa taputapu katoa e hiahiatia ana i waho o te pouaka. Kare matou e hiahia ana i tetahi rorohiko tuatoru, taraiwa ranei.

Te whakarite mo te whakauru

Hei whakauru i te punaha ka hiahia matou ki te Whakaahua Papamahi Ubuntu. Ko te punaha tūmau he momo whakaurunga kaha, e whakaatu ana i te mana motuhake e kore e taea te whakakore ma te pana i te wehewehenga punaha UEFI ki tetahi o nga kopae, ka pahuatia te ataahua katoa. No reira, ka whakauruhia ki te aratau UEFI anake. Karekau he whiringa.

Kaore matou i te harikoa ki tenei.

He aha?Ko te mea kino, he kino te hototahi a UEFI boot ki te RAID software boot, na te mea... Karekau he tangata e tuku rahui mo te wehenga UEFI ESP. He tohutao kei runga i te Ipurangi e kii ana ki te whakatakoto i te wehenga ESP ki runga i te puku kohiko i roto i te tauranga USB, engari he waahi kore tenei. He tohutoro kei te whakamahi i te raupaparorohiko mdadm RAID 1 me te putanga metadata 0.9 e kore e aukati i te UEFI BIOS kia kite i tenei wehewehenga, engari ka ora tenei tae noa ki te wa koa ka tuhia e te BIOS tetahi atu taputapu OS ki te ESP ka wareware ki te tukutahi ki etahi atu. whakaata.

Hei taapiri, ko te UEFI boot ka whakawhirinaki ki te NVRAM, kaore e neke me nga kopae ki te punaha hou, na te mea he wahanga o te papahoahoa.

Na, e kore matou e hanga hou i te wira hou. Kei a matou he paihikara a koroua kua rite, kua oti te whakamatau i te waa, e kiia nei ko Legacy/BIOS boot, e mau ana te ingoa whakakake o CSM i runga i nga punaha UEFI-hototahi. Ka tangohia noa e matou i te whata, ka whakahinuhinu, ka pupuhi i nga potae ka muru ki te kakahu makuku.

Ko te putanga papamahi o Ubuntu kaore e taea te whakauru tika me te Kaihoko Tukutuku, engari i konei, e kii ana ratou, he iti rawa nga whiringa.

Na, ka kohia e matou nga taputapu ka utaina te punaha mai i te puku kohiko bootable Ubuntu Live. Ka hiahia matou ki te tango i nga kohinga, na ka whakarite matou i te whatunga e pai ana mo koe. Ki te kore e mahi, ka taea e koe te uta i nga kete e tika ana ki runga i te puku kohiko i mua.

Ka haere matou ki roto i te taiao Papamahi, whakarewahia te emulator kāpeka, ka haere:

#sudo bash

Pehea…?Ko te raina i runga ake nei ko te keu kanona mo nga holiwars mo sudo. C bоnui atu nga whai wāhitanga ka tae maiоkawenga nui ake. Ko te patai mehemea ka taea e koe te kawe i a koe ano. He maha nga tangata e whakaaro ana ko te whakamahi i te sudo i tenei ara kaore i te tupato. Heoi ano:

#apt-get install mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc

He aha te kore ZFS...?Ina whakauruhia e matou he rorohiko ki to rorohiko, ka tukuna e matou a matou taputapu ki nga kaihanga o tenei rorohiko hei taraiwa.
Ina whakawhirinaki ana matou ki tenei rorohiko me te haumaru o a matou raraunga, ka tangohia e matou he nama e rite ana ki te utu mo te whakahoki mai i enei raraunga, ka utua e matou i tetahi ra.

Mai i tenei tirohanga, he Ferrari a ZFS, he rite tonu te mdadm+lvm ki te paihikara.

Ko te tikanga, he pai ake te kaituhi ki te tuku i te paihikara i runga i te nama ki nga tangata e kore e mohiotia, kaua ki te Ferrari. I reira, ko te utu o te take kaore i te teitei. Kaore he take mo nga tika. He ngawari ake i nga ture waka. He kore utu te waka waka. He pai ake te kaha whakawhiti whenua. Ka taea e koe te hono waewae ki te pahikara, ka taea e koe te whakatika i te paihikara me o ringaringa.

No te aha BTRFS...?Hei whakaara i te punaha whakahaere, me whai punaha konae e tautokohia ana i te Legacy/BIOS GRUB i waho o te pouaka, me te tautoko i nga whakaahua ora. Ka whakamahia e matou mo te wehewehenga /boot. I tua atu, he pai ake te kaituhi ki te whakamahi i tenei FS mo / (pakiaka), kaua e wareware ki te tuhi mo etahi atu rorohiko ka taea e koe te hanga wehewehenga motuhake i runga i te LVM ka whakairihia ki roto i nga raarangi e tika ana.

E kore matou e rokiroki i nga whakaahua o nga miihini mariko, papaa raraunga ranei i runga i tenei FS.
Ka whakamahia noa tenei FS ki te hanga pikitia o te punaha me te kore e whakawetohia, katahi ka whakawhiti i enei whakaahua ki te kōpae taapiri ma te tuku / whiwhi.

I tua atu, he pai ake te kaituhi ki te pupuri i te iti rawa o nga raupaparorohiko i runga i te taputapu me te whakahaere i era atu rorohiko katoa i roto i nga miihini mariko ma te whakamahi i nga mea penei i te tuku atu i nga GPU me nga kaiwhakahaere Kaihautu PCI-USB ki KVM ma IOMMU.

Ko nga mea e toe ana i runga i te taputapu ko te rokiroki raraunga, te mariko me te taapiri.

Mena ka whakawhirinaki atu koe ki a ZFS, na, i roto i te kaupapa, mo te tono kua tohua ka taea te whakawhiti.

Heoi ano, kaore te kaituhi e aro ki nga ahuatanga whakaata / RAID me nga waahanga taapiri kei a ZFS, BRTFS me LVM.

Hei tohenga taapiri, kei a BTRFS te kaha ki te huri i nga tuhi matapōkere ki roto i nga raupapa raupapa, he tino pai te paanga ki te tere o te tukutahi i nga whakaahua / taapiri i runga i te HDD.

Ka matawai ano i nga taputapu katoa:

#udevadm control --reload-rules && udevadm trigger

Kia titiro a tawhio noa:

#lsscsi && nvme list
[0:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sda
[1:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdb
[2:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdc
[3:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdd
[4:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sde
[5:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdf
[6:0:0:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdg
[6:0:1:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdh
[6:0:2:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdi
[6:0:3:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdj
[6:0:4:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdk
[6:0:5:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdl
[6:0:6:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdm
[6:0:7:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdn
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 S466NXXXXXXX15L Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7
/dev/nvme1n1 S5H7NXXXXXXX48N Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7


tahora kōpae

NVMe SSD

Engari kaore matou e tohu i a raatau i tetahi huarahi. He rite tonu, kaore to tatou BIOS e kite i enei puku. Na, ka haere katoa ratou ki te RAID pūmanawa. Kaore matou e hanga i nga waahanga ki reira. Mena kei te hiahia koe ki te whai i te "canon" me te "matua", hangahia tetahi waahanga nui, penei i te HDD.

SATA HDD

Kaore he take ki te hanga i tetahi mea motuhake i konei. Ka hanga e matou tetahi waahanga mo nga mea katoa. Ka hangaia e matou he wehewehenga na te mea ka kite te BIOS i enei kopae ka ngana pea ki te tarai mai i a raatau. Ka whakauruhia e matou te GRUB ki runga i enei kopae a muri ake nei ka taea e te punaha te mahi ohorere.

#cat >hdd.part << EOF
label: dos
label-id: 0x00000000
device: /dev/sdg
unit: sectors

/dev/sdg1 : start= 2048, size= 1953523120, type=fd, bootable
EOF
#sfdisk /dev/sdg < hdd.part
#sfdisk /dev/sdh < hdd.part
#sfdisk /dev/sdi < hdd.part
#sfdisk /dev/sdj < hdd.part
#sfdisk /dev/sdk < hdd.part
#sfdisk /dev/sdl < hdd.part
#sfdisk /dev/sdm < hdd.part
#sfdisk /dev/sdn < hdd.part


SATA SSD

Koinei te waahi ka pai nga mea mo tatou.

Tuatahi, he 2 TB te rahi o a maatau puku. Kei roto tenei i te awhe e whakaaetia ana mo te MBR, koinei te mea ka whakamahia e matou. Mena e tika ana, ka taea te whakakapi ki te GPT. Ko nga kopae GPT he paparanga hototahi ka taea e nga punaha hototahi MBR te kite i nga waahanga 4 tuatahi mena kei roto i te 2 terabytes tuatahi. Ko te mea nui ko te wehenga boot me te wehenga bios_grub i runga i enei kōpae me noho i te timatanga. Ma tenei ka taea e koe te peke mai i nga puku a GPT Legacy/BIOS.

Engari ehara tenei i a matou.

I konei ka hanga e matou nga waahanga e rua. Ko te mea tuatahi ka 1 GB te rahi ka whakamahia mo te RAID 1 /boot.

Ko te tuarua ka whakamahia mo te RAID 6 ka tango i nga waahi waatea katoa engari mo tetahi waahi iti kaore i tohatohahia i te mutunga o te puku.

He aha tenei waahi kore tohu?E ai ki nga puna korero i runga i te whatunga, kei runga i a maatau SATA SSDs tetahi keteroki SLC ka taea te whakawhanui mai i te 6 ki te 78 gigabytes. Ka whiwhi matou i te 6 gigabytes "mo te kore utu" na te rereketanga o "gigabytes" me "gibibytes" i roto i te pepa raraunga o te puku. Ko te toenga 72 gigabytes ka tohatohahia mai i te waahi kaore i whakamahia.

I konei me tohu kei a matou he keteroki SLC, a kei te noho te waahi ki te aratau MLC 4 bit. Ko te tikanga mo ia 4 gigabytes o te waahi waatea ka whiwhi 1 anake te gigabyte o te keteroki SLC.

Whakareatia te 72 gigabytes ki te 4 ka whiwhi 288 gigabytes. Koinei te waahi kore e tohuhia e matou kia taea ai e nga puku te whakamahi katoa i te keteroki SLC.

No reira, ka eke maatau ki te 312 gigabytes o te keteroki SLC mai i te katoa o nga puku e ono. O nga puku katoa, ka whakamahia te 2 ki te RAID mo te taapiri.

Ko tenei nui o te keteroki ka taea e tatou te tino uaua i roto i te ao tuuturu ka pa ki tetahi ahuatanga kaore e uru te tuhi ki te keteroki. He tino pai tenei ki te utu mo te whakahokinga pouri rawa atu o te mahara QLC - he iti rawa te tere o te tuhi i te wa e tuhia ana nga raraunga ma te keteroki. Mena kaore e rite ana o kawenga ki tenei, katahi ahau ka tūtohu kia whakaaro nui koe mo te roa o to SSD i raro i taua kawenga, me te whakaaro ki te TBW mai i te pepa raraunga.

#cat >ssd.part << EOF
label: dos
label-id: 0x00000000
device: /dev/sda
unit: sectors

/dev/sda1 : start= 2048, size= 2097152, type=fd, bootable
/dev/sda2 : start= 2099200, size= 3300950016, type=fd
EOF
#sfdisk /dev/sda < ssd.part
#sfdisk /dev/sdb < ssd.part
#sfdisk /dev/sdc < ssd.part
#sfdisk /dev/sdd < ssd.part
#sfdisk /dev/sde < ssd.part
#sfdisk /dev/sdf < ssd.part


Te Waihanga Huanga

Tuatahi, me whakaingoa ano te miihini. He mea tika tenei na te mea ko te ingoa kaihautu tetahi waahanga o te ingoa hurangi i tetahi waahi kei roto mdadm ka pa ki tetahi waahi. Ko te tikanga, ka taea te whakaingoa ano i nga rarangi i muri mai, engari he mahi koretake tenei.

#mcedit /etc/hostname
#mcedit /etc/hosts
#hostname
vdesk0


NVMe SSD

#mdadm --create --verbose --assume-clean /dev/md0 --level=1 --raid-devices=2 /dev/nvme[0-1]n1

He aha -whakaaro-ma...?Hei karo i te arawhiti huānga. Mo nga taumata RAID e rua 1 me te 6 he tika tenei. Ka taea e nga mea katoa te mahi me te kore arawhiti mena he huinga hou. I tua atu, ko te tiimata i te raupapa SSD i te wa i hangaia he ururua o te rauemi TBW. Ka whakamahia e matou te TRIM/DISCARD i nga waahi ka taea i runga i nga huinga SSD kua whakaemihia hei "whakaara" i a raatau.

Mo nga raupapa SSD, ka tautokohia te RAID 1 DISCARD i waho o te pouaka.

Mo nga raupapa SSD RAID 6 DISCARD, me whakaahei koe i roto i nga tawhā kōwae kernel.

Me mahi noa mena kei te tautoko nga SSD katoa e whakamahia ana i te taumata 4/5/6 i roto i tenei punaha mo te discard_zeroes_data. I etahi wa ka kite koe i nga puku rereke e kii ana ki te kernel kei te tautokohia tenei mahi, engari kaore i reira, kaore ranei te mahi e mahi i nga wa katoa. I tenei wa, kei te waatea te tautoko tata ki nga waahi katoa, engari, he puku tawhito me te firmware me nga hapa. Mo konei, kua monoa te tautoko DISCARD ma te taunoa mo RAID 6.

Kia tupato, ko te whakahau e whai ake nei ka whakangaro i nga raraunga katoa i runga i nga puku NVMe ma te "whakaara" i te huinga me te "kore".

#blkdiscard /dev/md0

Ki te he tetahi, ngana ki te tohu i tetahi taahiraa.

#blkdiscard --step 65536 /dev/md0

SATA SSD

#mdadm --create --verbose --assume-clean /dev/md1 --level=1 --raid-devices=6 /dev/sd[a-f]1
#blkdiscard /dev/md1
#mdadm --create --verbose --assume-clean /dev/md2 --chunk-size=512 --level=6 --raid-devices=6 /dev/sd[a-f]2

He aha te nui...?Ko te whakanui ake i te rahi o te tiihi ka pa ki te tere o te panui matapōkere i roto i nga poraka tae atu ki te rahi o te tiihi whakauru. Ka puta tenei na te mea ka taea te whakaoti i tetahi mahinga o te rahi e tika ana, iti ake ranei i runga i te taputapu kotahi. Na reira, ko nga IOPS mai i nga taputapu katoa ka whakarapopotohia. E ai ki nga tatauranga, 99% o IO kaore e neke ake i te 512K.

RAID 6 IOPS mo ia tuhi i nga wa katoa iti iho i te rite ranei ki te IOPS o te puku kotahi. Ina, hei panui matapōkeretia, he maha nga wa ka nui ake te IOPS i tera o te puku kotahi, na konei ko te rahi o te poraka he mea nui.
Kare te kaituhi e kite i te take ki te ngana ki te arotau i tetahi tawhā he kino i roto i te RAID 6-hoahoa, engari ka arotau i te pai o te RAID 6.
Ka utua e matou mo te tuhi ohorere o RAID 6 me te keteroki NVMe me nga tinihanga kikokore.

Kaore ano matou kia whakahohea te DISCARD mo RAID 6. No reira e kore matou e "whakaara" i tenei huinga mo tenei wa. Ka mahia e matou i muri mai, i muri i te whakauru i te OS.

SATA HDD

#mdadm --create --verbose --assume-clean /dev/md3 --chunk-size=512 --level=6 --raid-devices=8 /dev/sd[g-n]1

LVM i runga i te NVMe RAID

Mo te tere, e hiahia ana matou ki te tuu i te punaha konae pakiaka ki te NVMe RAID 1 ko /dev/md0.
Heoi, ka hiahia tonu matou ki tenei rarangi tere mo etahi atu hiahia, penei i te whakawhiti, metadata me te LVM-cache me te LVM-kikokore metadata, na reira ka hangaia e matou he LVM VG i runga i tenei huinga.

#pvcreate /dev/md0
#vgcreate root /dev/md0

Me hanga he wehewehenga mo te punaha konae pakiaka.

#lvcreate -L 128G --name root root

Me hanga he wehewehenga mo te whakawhiti i runga i te rahi o te RAM.

#lvcreate -L 32G --name swap root

tāutanga OS

I te katoa, kei a matou nga mea katoa e tika ana hei whakauru i te punaha.

Whakarewahia te ruānuku whakaurunga pūnaha mai i te taiao Ubuntu Live. Te whakaurunga noa. I te wa o te kowhiri i nga kōpae mo te whakaurunga, me tohu e koe nga mea e whai ake nei:

• /dev/md1, - ira maunga /boot, FS - BTRFS
• /dev/root/root (aka /dev/mapper/root-root), - mount point / (pakiaka), FS - BTRFS
• /dev/root/swap (aka /dev/mapper/root-swap), - whakamahia hei wehewehe wehewehe
• Tāutahia te bootloader ki runga /dev/sda

Ina kowhiria e koe te BTRFS hei punaha konae pakiaka, ka hanga aunoa e te kaiwhakaputa nga pukapuka BTRFS e rua kua tapaina ko "@" mo / (pakiaka), me "@home" mo /home.

Me timata te whakaurunga...

Ka mutu te whakaurunga me te pouaka korero aratau e tohu ana he hapa i te whakaurunga o te bootloader. Kia aroha mai, kare e taea e koe te puta i tenei korero ma te whakamahi tikanga paerewa ka haere tonu te whakaurunga. Ka puta atu matou i te punaha ka takiuru ano, ka mutu ki te papamahi Ubuntu Live ma. Whakatuwheratia te tauranga, ka hoki ano:

#sudo bash

Waihangatia he taiao chroot hei haere tonu i te whakaurunga:

#mkdir /mnt/chroot
#mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@ /dev/mapper/root-root /mnt/chroot
#mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@home /dev/mapper/root-root /mnt/chroot/home
#mount -o defaults,space_cache,noatime,nodiratime,discard /dev/md1 /mnt/chroot/boot
#mount --bind /proc /mnt/chroot/proc
#mount --bind /sys /mnt/chroot/sys
#mount --bind /dev /mnt/chroot/dev

Me whirihora i te whatunga me te ingoa kaihautu i roto i te chroot:

#cat /etc/hostname >/mnt/chroot/etc/hostname
#cat /etc/hosts >/mnt/chroot/etc/hosts
#cat /etc/resolv.conf >/mnt/chroot/etc/resolv.conf

Me haere tatou ki te taiao chroot:

#chroot /mnt/chroot

Tuatahi, ka tukuna e matou nga kete:

apt-get install --reinstall mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc debsums hdparm

Me tirotiro me te whakatika i nga kohinga katoa i whakauruhia pohehe na te mea kaore i oti te whakaurunga o te punaha:

#CORRUPTED_PACKAGES=$(debsums -s 2>&1 | awk '{print $6}' | uniq)
#apt-get install --reinstall $CORRUPTED_PACKAGES

Mena karekau he mahi, me whakatika /etc/apt/sources.list i te tuatahi

Me whakatika nga tawhā mo te kōwae RAID 6 kia taea ai te TRIM/DISCARD:

#cat >/etc/modprobe.d/raid456.conf << EOF
options raid456 devices_handle_discard_safely=1
EOF

Kia iti ake ta maatau huinga:

#cat >/etc/udev/rules.d/60-md.rules << EOF
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/stripe_cache_size", ATTR{md/stripe_cache_size}="32768"
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_min", ATTR{md/sync_speed_min}="48000"
SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_max", ATTR{md/sync_speed_max}="300000"
EOF
#cat >/etc/udev/rules.d/62-hdparm.rules << EOF
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/hdparm -B 254 /dev/%k"
EOF
#cat >/etc/udev/rules.d/63-blockdev.rules << EOF
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/blockdev --setra 1024 /dev/%k"
SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="md*", RUN+="/sbin/blockdev --setra 0 /dev/%k"
EOF

He aha tera..?Kua hangahia e matou he huinga ture udev e whai ake nei:

• Whakaritehia te rahi o te keteroki poraka mo te RAID 2020 kia tika mo te 6. Ko te uara taunoa, te ahua nei, kaore i rereke mai i te hanganga o Linux, a kaore i te tika mo te wa roa.
• Rahuihia te iti rawa o te IO mo te roanga o nga arowhai / tukutahi. He mea tenei kia kore ai o rarangi e mau ki roto i te ahua o te tukutahitanga mau tonu i raro i te kawenga.
• Whakawhāitihia te IO mōrahi i te wā o te arowhai/tukutahitanga o ngā huānga. He mea tika tenei kia kore ai te tukutahi/te tirotiro i nga RAID SSD e parai i o puku ki te parai. He tino pono tenei mo NVMe. (Kia maumahara mo te radiator? Kaore au i te katakata.)
• Aukati i nga kopae kia whakamutua te hurihanga porowhita (HDD) ma te APM me te whakarite i te wa moe mo nga kaiwhakahaere kōpae ki te 7 haora. Ka taea e koe te whakakore katoa i te APM mena ka taea e o puku (-B 255). Ma te uara taunoa, ka mutu nga puku i muri i te rima hēkona. Na ka hiahia te OS ki te tautuhi i te keteroki kōpae, ka huri ano nga kopae, ka timata ano nga mea katoa. He iti noa te maha o nga hurihanga porowhita o nga kōpae. Ka taea e tenei huringa taunoa ngawari te patu i o kōpae i roto i nga tau e rua. Ehara i te mea ka pangia e nga kopae katoa, engari ko o maatau "pona rorohiko", me nga tautuhinga taunoa e tika ana, ka rite te ahua o RAID ki te mini-MAID.
• Tāutahia te panui i mua i runga i nga kopae (hurihuri) 1 megabyte - e rua nga poraka karapīpiti/tuhinga RAID 6
• Monokia te panui i mua i runga i nga rarangi.

Me whakatika /etc/fstab:

#cat >/etc/fstab << EOF
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# file-system mount-point type options dump pass
/dev/mapper/root-root / btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@ 0 1
UUID=$(blkid -o value -s UUID /dev/md1) /boot btrfs defaults,space_cache,noatime,nodiratime,discard 0 2
/dev/mapper/root-root /home btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@home 0 2
/dev/mapper/root-swap none swap sw 0 0
EOF
He aha tena..?Ka rapua e matou te wehewehenga /boot na UUID. Ka taea e te whakaingoa huānga te huri.

Ka rapua e matou nga waahanga e toe ana ma nga ingoa LVM i te tohu /dev/mapper/vg-lv, na te mea he tino ahurei te tautuhi i nga wehewehenga.

Kaore matou e whakamahi i te UUID mo te LVM na te mea Ko te UUID o nga pukapuka LVM me o raatau whakaahua ka taea te rite.Maunga /dev/mapper/root-root.. rua?Ae. Tika tonu. Ko nga waahanga mo te BTRFS. Ka taea te whakauru i tenei punaha konae i nga wa maha me nga waahanga iti rereke.

Na tenei ahuatanga ano, ka tūtohu ahau kia kaua e hanga he whakaahua LVM o nga pukapuka BTRFS kaha. Ka ohorere pea koe ina whakaara ano koe.

Me whakahou ano i te whirihora mdadm:

#/usr/share/mdadm/mkconf | sed 's/#DEVICE/DEVICE/g' >/etc/mdadm/mdadm.conf

Me whakatika nga tautuhinga LVM:

#cat >>/etc/lvm/lvmlocal.conf << EOF

activation {
thin_pool_autoextend_threshold=90
thin_pool_autoextend_percent=5
}
allocation {
cache_pool_max_chunks=2097152
}
devices {
global_filter=["r|^/dev/.*_corig$|","r|^/dev/.*_cdata$|","r|^/dev/.*_cmeta$|","r|^/dev/.*gpv$|","r|^/dev/images/.*$|","r|^/dev/mapper/images.*$|","r|^/dev/backup/.*$|","r|^/dev/mapper/backup.*$|"] issue_discards=1
}
EOF

He aha tera..?Kua taea e matou te roha aunoa o nga puna angiangi LVM ina tae ki te 90% o te waahi kua nohoia e te 5% o te rōrahi.

Kua piki ake te maha o nga poraka keteroki mo te keteroki LVM.

Kua aukatihia e matou a LVM ki te rapu i nga pukapuka LVM (PV) i runga:

• nga taputapu kei roto te keteroki LVM (cdata)
• nga taputapu keteroki ma te whakamahi i te keteroki LVM, ka maataki i te keteroki ( _korig). I tenei keehi, ko te taputapu keteroki tonu ka tirotirohia ma te keteroki (tika ).
• nga taputapu kei roto LVM keteroki metadata (cmeta)
• nga taputapu katoa i roto i te VG me nga whakaahua ingoa. I konei ka whai whakaahua kopae o nga miihini mariko, kaore matou e hiahia kia LVM i runga i te kaihautu kia whakahohe nga pukapuka no te OS manuhiri.
• nga taputapu katoa i roto i te VG me te ingoa taapiri. I konei ka whai kape taapiri o nga whakaahua miihini mariko.
• nga taputapu katoa ka mutu te ingoa ki te "gpv" (te rōrahi tinana manuhiri)

Kua whakahohea e matou te tautoko DISCARD i te wa e whakawātea ana te mokowā wātea i runga i te LVM VG. Kia tupato. Ma tenei ka whakakore i nga LV i runga i te SSD ka pau te wa. E pa ana tenei ki te SSD RAID 6. Heoi, i runga i te mahere, ka whakamahia e matou nga waahanga kikokore, na tenei ka kore e aukati i a maatau.

Me whakahou ake te whakaahua initramfs:

#update-initramfs -u -k all

Tāuta me te whirihora i te grub:

#apt-get install grub-pc
#apt-get purge os-prober
#dpkg-reconfigure grub-pc

Ko ēhea kōpae hei kōwhiri?Te hunga katoa sd*. Me taea e te punaha te peke mai i tetahi puku SATA mahi, SSD ranei.

He aha ratou i tapiri ai os-prober..?Mo te mana motuhake me nga ringaringa takaro.

Kare e mahi tika mena kei te ahua kino tetahi o nga RAID. Ka ngana ki te rapu i te OS i runga i nga wehewehenga e whakamahia ana i roto i nga miihini mariko e rere ana i runga i tenei taputapu.

Mena ka hiahia koe, ka taea e koe te waiho, engari kia maumahara ki nga mea katoa o runga ake nei. Ka tūtohu ahau ki te rapu tohutoro mo te whakakore i nga ringaringa kino i runga ipurangi.

Na tenei kua oti te whakaurunga tuatahi. Kua tae ki te wa ki te whakaara ano ki te OS hou kua whakauruhia. Kaua e wareware ki te tango i te CD/USB Live bootable.

#exit
#reboot

Tīpakohia tetahi o nga SSD SATA hei taputapu whawhai.

LVM i runga i te SATA SSD

I tenei wa, kua eke matou ki te OS hou, whirihorahia te whatunga, apt, whakatuwheratia te emulator tauranga, ka oma:

#sudo bash

Kia haere tonu.

"Whakaarohia" te huinga mai i te SATA SSD:

#blkdiscard /dev/md2

Ki te kore e mahi, ka ngana:

#blkdiscard --step 65536 /dev/md2
Waihangahia te LVM VG i runga i te SATA SSD:

#pvcreate /dev/md2
#vgcreate data /dev/md2

He aha tetahi atu VG ..?Inaa, he VG ano to tatou ingoa pakiaka. He aha e kore ai e taapiri nga mea katoa ki te VG kotahi?

Mena he maha nga PV kei roto i tetahi VG, na kia tika te whakahohe i te VG, me noho nga PV katoa (ipurangi). Ko te rereke ko LVM RAID, kaore e whakamahia e matou.

Kei te tino pirangi matou mena he rahua (panuihia te ngaronga raraunga) i runga i tetahi o nga rarangi RAID 6, ka pupuhi te punaha whakahaere me te whai waahi ki te whakaoti rapanga.

Hei mahi i tenei, i te taumata tuatahi o te tangohanga ka wehea e matou ia momo "paapaho" tinana ki tetahi VG motuhake.

E ai ki nga korero a te tangata, he rereke nga momo RAID o nga momo "rohe pono". Kaua koe e hanga i tetahi atu waahi noa o te korenga mo ratou ma te whakakii i a raatau ki te VG kotahi.

Ko te noho mai o te LVM i te taumata "maamaa" ka taea e taatau ki te tapahi i nga waahanga rereke o nga momo RAID ma te whakakotahi i a raatau i nga huarahi rereke. Hei tauira - oma i te wa ano bcache + LVM kikokore, bcache + BTRFS, LVM cache + LVM kikokore, he whirihoranga ZFS matatini me nga keteroki, tetahi atu ranunga reinga ranei hei whakamatautau me te whakataurite i te katoa.

I te taumata "pūmārō", e kore matou e whakamahi i tetahi mea ke atu i nga pukapuka LVM "matotoru" tawhito pai. Ko te rereke ki tenei ture ko te arai taapiri.

Ki taku whakaaro i tenei wa, he maha nga kaipanui kua timata ki te whakapae i tetahi mea mo te tare kohanga.

LVM i runga i te SATA HDD

#pvcreate /dev/md3
#vgcreate backup /dev/md3

VG hou ano..?E tino hiahia ana matou ki te rahua te raupapa kōpae ka whakamahia e matou mo te taapiri raraunga, ka mahi tonu to maatau punaha whakahaere, me te mau tonu te uru ki nga raraunga kore-purua pera i o mua. No reira, ki te karo i nga raru whakahohenga VG, ka hangaia he VG motuhake.

Te whakarite keteroki LVM

Me hanga he LV i runga i te NVMe RAID 1 hei whakamahi hei taputapu keteroki.

#lvcreate -L 70871154688B --name cache root

He aha te iti...?Ko te meka ko o maatau NVMe SSDs he keteroki SLC ano. 4 gigabytes o te "kore utu" me te 18 gigabytes o te hihiri na te waahi noa e nohoia ana i roto i te MLC 3-bit. Kia pau tenei keteroki, e kore e tere ake nga NVMe SSD i to tatou SATA SSD me te keteroki. Inaa, mo tenei take, kaore he tikanga mo taatau ki te hanga i te wehenga keteroki LVM kia nui ake i te rua o te rahi o te keteroki SLC o te puku NVMe. Mo nga puku NVMe i whakamahia, ka whakaarohia e te kaituhi he mea tika ki te hanga 32-64 gigabytes o te keteroki.

Ko te rahinga wehewehenga e hiahiatia ana hei whakarite i te 64 gigabytes o te keteroki, te keteroki metadata, me te taapiri metadata.

I tua atu, ka kite ahau i muri i te katinga o te punaha paru, ka tohuhia e LVM te keteroki katoa he paru, ka tukutahi ano. I tua atu, ka tukuna ano tenei i nga wa katoa ka whakamahia te lvchange ki runga i tenei taputapu tae noa ki te whakaara ano i te punaha. Na reira, ka tūtohu ahau kia hanga ano i te keteroki ma te whakamahi i te tuhinga e tika ana.

Me hanga he LV mo SATA RAID 6 hei whakamahi hei taputapu keteroki.

#lvcreate -L 3298543271936B --name cache data

He aha noa e toru terabytes..?Na, ki te tika, ka taea e koe te whakamahi SATA SSD RAID 6 mo etahi atu hiahia. Ka taea te whakanui ake i te rahi o te waahi keteroki, i runga i te rere, me te kore e aukati i te punaha. Ki te mahi i tenei, me whakamutu mo te wa poto me te whakahohe ano i te keteroki, engari ko te painga motuhake o te LVM-cache ki runga, hei tauira, bcache ka taea tenei ma te rere.

Hangaia he VG hou mo te keteroki.

#pvcreate /dev/root/cache
#pvcreate /dev/data/cache
#vgcreate cache /dev/root/cache /dev/data/cache

Me hanga he LV i runga i te taputapu keteroki.

#lvcreate -L 3298539077632B --name cachedata cache /dev/data/cache

I konei ka tangohia e matou nga waahi kore utu katoa i runga i /dev/data/cache kia hanga tonu nga waahanga katoa e tika ana ki runga /dev/root/cache. Mena i hanga e koe tetahi mea i te waahi he, ka taea e koe te neke ma te whakamahi pvmove.

Me hanga ka taea te keteroki:

#lvcreate -y -L 64G -n cache cache /dev/root/cache
#lvcreate -y -L 1G -n cachemeta cache /dev/root/cache
#lvconvert -y --type cache-pool --cachemode writeback --chunksize 64k --poolmetadata cache/cachemeta cache/cache
#lvconvert -y --type cache --cachepool cache/cache cache/cachedata

He aha te chunksize penei ..?Na roto i nga whakamatautau mahi, i kitea e te kaituhi ko te hua pai ka tutuki mena ka rite te rahi o te poraka keteroki LVM ki te rahi o te poraka kikokore LVM. I tua atu, ko te iti o te rahi, ko te pai ake o te whirihoranga i roto i te rekoata matapōkere.

Ko te 64k te rahi poraka iti e whakaaetia ana mo te LVM kikokore.

Kia tupato ki te tuhi hoki..!Ae. Ko tenei momo keteroki ka whakaroa te tuhi tukutahitanga ki te taputapu keteroki. Ko te tikanga mena ka ngaro te keteroki, ka ngaro pea nga raraunga i runga i te taputapu keteroki. I muri mai, ka korero te kaituhi ki a koe he aha nga mehua, hei taapiri atu ki te NVMe RAID 1, ka taea te tango hei utu mo tenei raru.

I whiriwhiria tenei momo keteroki hei utu mo te ngoikore o te mahi tuhi matapōkere o RAID 6.

Kia tirohia e tatou he aha ta tatou i whiwhi:

#lvs -a -o lv_name,lv_size,devices --units B cache
LV LSize Devices
[cache] 68719476736B cache_cdata(0)
[cache_cdata] 68719476736B /dev/root/cache(0)
[cache_cmeta] 1073741824B /dev/root/cache(16384)
cachedata 3298539077632B cachedata_corig(0)
[cachedata_corig] 3298539077632B /dev/data/cache(0)
[lvol0_pmspare] 1073741824B /dev/root/cache(16640)

Ko [cachedata_corig] anake me noho ki runga /dev/data/cache. Mena kei te he tetahi mea, whakamahia te pvmove.

Ka taea e koe te whakakore i te keteroki mena e tika ana ma te whakahau kotahi:

#lvconvert -y --uncache cache/cachedata

Ka mahia tenei i runga ipurangi. Ka tukutahi noa e LVM te keteroki ki te kōpae, ka tango, ka whakaingoa ano i te cachedata_corig ki te cachedata.

Te whakatu LVM kikokore

Me whakatau tata tatou e hia te waahi e hiahiatia ana mo te LVM metadata kikokore:

#thin_metadata_size --block-size=64k --pool-size=6terabytes --max-thins=100000 -u bytes
thin_metadata_size - 3385794560 bytes estimated metadata area size for "--block-size=64kibibytes --pool-size=6terabytes --max-thins=100000"

Whakaawhiwhia ki te 4 gigabytes: 4294967296B

Whakareatia ki te rua ka taapirihia te 4194304B mo te LVM PV metadata: 8594128896B
Me hanga he wehewehenga motuhake i runga i te NVMe RAID 1 hei whakanoho i nga metadata kikokore LVM me o raatau kape taapiri ki runga:

#lvcreate -L 8594128896B --name images root

Mo te aha..?I konei ka ara ake pea te patai: he aha te mea ka wehea te LVM metadata kikokore mena ka keteroki tonu ki runga i te NVMe ka tere te mahi.

Ahakoa he mea nui te tere i konei, kei tawhiti atu i te take matua. Ko te mea ko te keteroki he waahi o te kore. Ka pa mai tetahi mea ki a ia, a ki te keterokihia nga metadata kikokore LVM, ka ngaro katoa nga mea katoa. Ki te kore he metadata kua oti, ka kore e taea te whakahiato pukapuka kikokore.

Ma te neke i nga metadata ki tetahi keteroki kore-keteroki, engari tere, rōrahi, ka whakamanahia e matou te haumaru o te metadata mena ka ngaro te keteroki, te pirau ranei. I tenei keehi, ko nga kino katoa i puta mai i te ngaro o te keteroki ka noho ki roto i nga pukapuka angiangi, ka whakamaarama te tikanga whakaora ma te nui o nga whakahau. Ma te nui o te tupono, ka whakahokia enei pakaru ma te whakamahi i nga raarangi FS.

I tua atu, ki te tangohia he whakaahua o te riipene angiangi i mua, a i muri i tera ka tino tukutahia te keteroki i te iti rawa i te kotahi, na, na te hoahoa o roto o te LVM kikokore, ka whakamanahia te tapatahi o te whakaahua mena ka ngaro te keteroki. .

Me hanga he VG hou hei kawenga mo te tuku kikokore:

#pvcreate /dev/root/images
#pvcreate /dev/cache/cachedata
#vgcreate images /dev/root/images /dev/cache/cachedata
Me hanga he puna:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T images/thin-pool
He aha -Z yI tua atu i te tikanga o tenei aratau - kia kore e pakaru nga raraunga mai i tetahi miihini mariko ki tetahi atu miihini mariko i te wa e tohatoha ana i te waahi - ka whakamahia ano te korekore hei whakanui ake i te tere o te tuhi matapōkere i roto i nga poraka iti ake i te 64k. Ko nga tuhi iti iho i te 64k ki te waahi kaore i tohatohahia i mua o te rōrahi angiangi ka 64K te taha-tiaaro i roto i te keteroki. Ma tenei ka taea te mahi katoa ma te keteroki, ma te maataki i te taputapu keteroki.

Me nukuhia nga LV ki nga PV e rite ana:

#pvmove -n images/thin-pool_tdata /dev/root/images /dev/cache/cachedata
#pvmove -n images/lvol0_pmspare /dev/cache/cachedata /dev/root/images
#pvmove -n images/thin-pool_tmeta /dev/cache/cachedata /dev/root/images

Me taki tatou:

#lvs -a -o lv_name,lv_size,devices --units B images
LV LSize Devices
[lvol0_pmspare] 4294967296B /dev/root/images(0)
thin-pool 274877906944B thin-pool_tdata(0)
[thin-pool_tdata] 274877906944B /dev/cache/cachedata(0)
[thin-pool_tmeta] 4294967296B /dev/root/images(1024)

Me hanga he rōrahi angiangi mō ngā whakamātautau:

#lvcreate -V 64G --thin-pool thin-pool --name test images

Ka whakauruhia e matou nga kohinga mo nga whakamatautau me te aroturuki:

#apt-get install sysstat fio

Ma tenei ka taea e koe te kite i te whanonga o ta maatau whirihoranga rokiroki i te waa tuuturu:

#watch 'lvs --rows --reportformat basic --quiet -ocache_dirty_blocks,cache_settings cache/cachedata && (lvdisplay cache/cachedata | grep Cache) && (sar -p -d 2 1 | grep -E "sd|nvme|DEV|md1|md2|md3|md0" | grep -v Average | sort)'

Koinei te huarahi e taea ai e matou te whakamatautau i ta matou whirihoranga:

#fio --loops=1 --size=64G --runtime=4 --filename=/dev/images/test --stonewall --ioengine=libaio --direct=1
--name=4kQD32read --bs=4k --iodepth=32 --rw=randread
--name=8kQD32read --bs=8k --iodepth=32 --rw=randread
--name=16kQD32read --bs=16k --iodepth=32 --rw=randread
--name=32KQD32read --bs=32k --iodepth=32 --rw=randread
--name=64KQD32read --bs=64k --iodepth=32 --rw=randread
--name=128KQD32read --bs=128k --iodepth=32 --rw=randread
--name=256KQD32read --bs=256k --iodepth=32 --rw=randread
--name=512KQD32read --bs=512k --iodepth=32 --rw=randread
--name=4Kread --bs=4k --rw=read
--name=8Kread --bs=8k --rw=read
--name=16Kread --bs=16k --rw=read
--name=32Kread --bs=32k --rw=read
--name=64Kread --bs=64k --rw=read
--name=128Kread --bs=128k --rw=read
--name=256Kread --bs=256k --rw=read
--name=512Kread --bs=512k --rw=read
--name=Seqread --bs=1m --rw=read
--name=Longread --bs=8m --rw=read
--name=Longwrite --bs=8m --rw=write
--name=Seqwrite --bs=1m --rw=write
--name=512Kwrite --bs=512k --rw=write
--name=256write --bs=256k --rw=write
--name=128write --bs=128k --rw=write
--name=64write --bs=64k --rw=write
--name=32write --bs=32k --rw=write
--name=16write --bs=16k --rw=write
--name=8write --bs=8k --rw=write
--name=4write --bs=4k --rw=write
--name=512KQD32write --bs=512k --iodepth=32 --rw=randwrite
--name=256KQD32write --bs=256k --iodepth=32 --rw=randwrite
--name=128KQD32write --bs=128k --iodepth=32 --rw=randwrite
--name=64KQD32write --bs=64k --iodepth=32 --rw=randwrite
--name=32KQD32write --bs=32k --iodepth=32 --rw=randwrite
--name=16KQD32write --bs=16k --iodepth=32 --rw=randwrite
--name=8KQD32write --bs=8k --iodepth=32 --rw=randwrite
--name=4kQD32write --bs=4k --iodepth=32 --rw=randwrite
| grep -E 'read|write|test' | grep -v ioengine

Kia tupato! Rauemi!Ka whakahaerehia e tenei waehere nga whakamatautau rereke 36, ka rere ia mo te 4 hēkona. Ko te haurua o nga whakamatautau mo te tuhi. Ka taea e koe te tuhi nui ki runga i te NVMe i roto i te 4 hēkona. Tae atu ki te 3 gigabytes ia hekona. Na, ka taea e ia o nga whakamatautau tuhi te kai ake ki te 216 gigabytes o te rauemi SSD mai i a koe.

Ka hanumi te panui me te tuhi?Ae. He mea tika ki te whakahaere motuhake i nga whakamatautau panui me te tuhi. I tua atu, he mea tika ki te whakarite kia tukutahia nga keteroki katoa kia kore ai te tuhi i mahia i mua e pa ki te panui.

Ka rereke nga hua i te wa o te whakarewatanga tuatahi me nga mea o muri mai ka whakakiia te keteroki me te rōrahi angiangi, me te whakawhirinaki ano ki te whakahaere a te punaha ki te tukutahi i nga keteroki kua whakakiia i te whakarewatanga whakamutunga.

I roto i etahi atu mea, ka tūtohu ahau ki te ine i te tere i runga i te rōrahi angiangi kua ki tonu i tangohia mai ai he whakaahua. I whai waahi te kaituhi ki te kite i te tere tere o nga tuhi matapōkere i muri tonu mai i te waihanga i te whakaahua tuatahi, ina koa kare ano te keteroki kia ki tonu. Ka puta tenei na te kape-i-tuhi-tuhi-tuhi tuhi, te tirohanga o te keteroki me nga poraka rōrahi angiangi, me te mea ko te tuhi matapōkere ki te RAID 6 ka huri ki te panui matapōkere mai i te RAID 6 ka whai i te tuhi ki te keteroki. I roto i ta maatau whirihoranga, ko te panui matapōkere mai i te RAID 6 ka eke ki te 6 nga wa (te maha o nga SATA SSD i roto i te rarangi) tere atu i te tuhi. No te mea Ka tohatohahia nga poraka mo te CoW mai i te puna angiangi, katahi ka huri ano te rekoata, mo te nuinga o te waa, ki te raupapa.

Ko enei ahuatanga e rua ka taea te whakamahi mo to painga.

Keteroki "honohono" whakaahua

Hei whakaiti i te tupono o te ngaronga o nga raraunga mena he kino te keteroki/mate, ka whakaaro te kaituhi ki te whakauru i te tikanga o te huri i nga whakaahua hei whakapumau i to ratou pono i tenei keehi.

Tuatahi, na te mea kei te noho te metadata rōrahi angiangi i runga i te taputapu keteroki kore, ka rite tonu te metadata ka noho mokemoke te ngaronga i roto i nga poraka raraunga.

Ko te huringa hurihanga whakaahua e whai ake nei ka whakapumau i te tika o nga raraunga kei roto i nga mahanga ka ngaro te keteroki:

1. Mo ia rōrahi angiangi me te ingoa <ingoa>, hanga he whakaahua me te ingoa <name>.keteroki
2. Me whakarite te paepae heke ki te uara teitei tika: #lvchange --quiet --cachesettings "migration_threshold=16384" cache/cachedata
3. I roto i te koropiko ka tirohia e matou te maha o nga poraka paru kei roto i te keteroki: #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' tae noa ki te whiwhi kore. Mena kei te ngaro te kore mo te wa roa, ka taea te hanga ma te huri i te keteroki mo te wa poto ki te aratau tuhi-roto. Heoi, ma te whai whakaaro ki nga ahuatanga tere o a maatau raupapa SATA me te NVMe SSD, me o raatau rauemi TBW, ka taea e koe te hopu tere i te waa me te kore e whakarereke i te aratau keteroki, ka pau ranei i to taputapu te katoa o ana rauemi i roto. he ra torutoru. Na te iti o nga rauemi, ko te tikanga, kaore e taea e te punaha te noho i raro i te 100% te utaina tuhi i nga wa katoa. Ko a maatau NVMe SSD kei raro i te 100% te uta tuhituhi ka pau katoa te rauemi kei roto 3-4 ra. Ka rua noa te roa o nga SSD SATA. No reira, ka whakaaro matou ko te nuinga o nga kawenga ka haere ki te panui, a he wa poto nei nga pakaru o nga mahi tino tiketike me te iti o te uta mo te tuhi.
4. I te wa i mau ai (ka hanga ranei) he kore, ka whakaingoatia ano <name>.keteroki ki <name>.committed. Kua mukua te <name>.committed tawhito.
5. Ko te tikanga, ki te 100% te ki tonu o te keteroki, ka taea te hanga ano e te tuhinga, na reira ka whakakorea. Ma te keteroki haurua-kore, ka tere ake te mahi a te punaha i te wa e tuhi ana.
6. Tautuhia te paepae heke ki te kore: #lvchange --quiet --cachesettings "migration_threshold=0" cache/cachedata Ma tenei ka aukati i te keteroki ki te tukutahi ki te papapāho matua.
7. Ka tatari tatou kia maha nga huringa ka whakaemi i roto i te keteroki #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' ka pahemo ranei te taima.
8. Ka tuaruatia ano e matou.

He aha nga uauatanga ki te paepae heke...?Ko te mea i roto i nga mahi tuuturu, he rekoata "tupurangi" ehara i te mea tino tupurangi. Mena ka tuhia e matou tetahi mea ki tetahi waahanga e 4 kiropaita te rahi, he nui te tupono kei roto i nga meneti e rua e whai ake nei ka tuhia he rekoata ki tetahi o nga waahanga tata (+- 32K) ranei.

Ma te tautuhi i te paepae heke ki te kore, ka hikitia e matou te tukutahitanga tuhi ki te SATA SSD me te whakahiato i etahi huringa ki tetahi poraka 64K kei roto i te keteroki. Ka tino tiakina e tenei te rauemi o SATA SSD.

Kei hea te waehere..?Heoi, ki te whakaaro o te kaituhi kaore ia i te tino mohio ki te whakawhanake i nga tuhinga bash na te mea he 100% tana ako-a-whaiaro me te mahi i te whanaketanga "google", na reira e whakapono ana ia ko te waehere whakamataku ka puta mai i ona ringa kaua e whakamahia e tetahi. atu.

Ki taku whakaaro ka taea e nga tohunga ngaio o tenei mara te whakaatu takitahi i nga arorau katoa i whakaahuahia i runga ake nei, mena e tika ana, a, tera pea, he ataahua te hoahoa hei ratonga punaha, pera i ta te kaituhi i ngana ki te mahi.

Ko te kaupapa hurihuri maataki ngawari ka taea e taatau kia kore noa e tukutahi i nga wa katoa ki te SATA SSD, engari ka taea ano e tatou, ma te whakamahi i te thin_delta whaipainga, ki te mohio ko wai nga poraka i whakarerekehia i muri i tana hanganga, na reira ka taea e tatou te whakaatu i te kino ki runga. Ko nga pukapuka matua, he tino ngawari te whakaora.

TRIM/DISCARD i roto i te libvirt/KVM

No te mea ka whakamahia te rokiroki raraunga mo te KVM e whakahaere ana i te libvirt, katahi ka pai ki te whakaako i a tatou VM kia kaua e tango i nga waahi kore noa, engari ki te waatea i nga mea kaore e hiahiatia ana.

Ka mahia tenei ma te whai i te tautoko TRIM/DISCARD ki nga kōpae mariko. Ki te mahi i tenei, me huri koe i te momo kaiwhakahaere ki virtio-scsi me te whakatika i te xml.

#virsh edit vmname
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='writethrough' io='threads' discard='unmap'/>
<source dev='/dev/images/vmname'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

<controller type='scsi' index='0' model='virtio-scsi'>
<alias name='scsi0'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</controller>

Ko enei momo DISCARD mai i nga OS manuhiri ka tika te tukatuka e LVM, ka tukuna tika nga poraka i roto i te keteroki me te puna angiangi. I roto i to maatau, ka puta tenei i roto i te wa roa, i te wa e mukua ana te pikitia e whai ake nei.

BTRFS Pūrua

Whakamahia nga tuhinga kua oti te hanga me te tino tūpato me i runga i to tupono. Na te kaituhi tenei waehere nana i tuhi, mo ia anake. E mohio ana ahau he maha nga kaiwhakamahi Linux mohio he rite nga taputapu, a kaore he take ki te kape i ta tetahi atu.

Me hanga he rōrahi i runga i te taputapu pūrua:

#lvcreate -L 256G --name backup backup

Me whakahōputu ki te BTRFS:

#mkfs.btrfs /dev/backup/backup

Me hanga e tatou nga tohu maunga me te whakairi i nga waahanga pakiaka o te punaha konae:

#mkdir /backup
#mkdir /backup/btrfs
#mkdir /backup/btrfs/root
#mkdir /backup/btrfs/back
#ln -s /boot /backup/btrfs
# cat >>/etc/fstab << EOF

/dev/mapper/root-root /backup/btrfs/root btrfs defaults,space_cache,noatime,nodiratime 0 2
/dev/mapper/backup-backup /backup/btrfs/back btrfs defaults,space_cache,noatime,nodiratime 0 2
EOF
#mount -a
#update-initramfs -u
#update-grub

Me hanga whaiaronga mo nga taapiri:

#mkdir /backup/btrfs/back/remote
#mkdir /backup/btrfs/back/remote/root
#mkdir /backup/btrfs/back/remote/boot

Me hanga he whaiaronga mo nga tuhinga taapiri:

#mkdir /root/btrfs-backup

Me kape te tuhinga:

He maha nga waehere bash whakamataku. Whakamahia i runga i to tupono. Kaua e tuhi reta riri ki te kaituhi...#cat >/root/btrfs-backup/btrfs-backup.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"
DATE_PREFIX='%Y-%m-%d'
DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'
DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
BASE_SUFFIX=".@base"
PEND_SUFFIX=".@pend"
SNAP_SUFFIX=".@snap"
MOUNTS="/backup/btrfs/"
BACKUPS="/backup/btrfs/back/remote/"

function terminate ()
{
echo "$1" >&2
exit 1
}

function wait_lock()
{
flock 98
}

function wait_lock_or_terminate()
{
echo "Wating for lock..."
wait_lock || terminate "Failed to get lock. Exiting..."
echo "Got lock..."
}

function suffix()
{
FORMATTED_DATE=$(date +"$DATE_FORMAT")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function filter()
{
FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function backup()
{
SOURCE_PATH="$MOUNTS$1"
TARGET_PATH="$BACKUPS$1"
SOURCE_BASE_PATH="$MOUNTS$1$BASE_SUFFIX"
TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
TARGET_BASE_DIR="$(dirname $TARGET_BASE_PATH)"
SOURCE_PEND_PATH="$MOUNTS$1$PEND_SUFFIX"
TARGET_PEND_PATH="$BACKUPS$1$PEND_SUFFIX"
if [ -d "$SOURCE_BASE_PATH" ] then
echo "$SOURCE_BASE_PATH found"
else
echo "$SOURCE_BASE_PATH File not found creating snapshot of $SOURCE_PATH to $SOURCE_BASE_PATH"
btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_BASE_PATH
sync
if [ -d "$TARGET_BASE_PATH" ] then
echo "$TARGET_BASE_PATH found out of sync with source... removing..."
btrfs subvolume delete -c $TARGET_BASE_PATH
sync
fi
fi
if [ -d "$TARGET_BASE_PATH" ] then
echo "$TARGET_BASE_PATH found"
else
echo "$TARGET_BASE_PATH not found. Synching to $TARGET_BASE_DIR"
btrfs send $SOURCE_BASE_PATH | btrfs receive $TARGET_BASE_DIR
sync
fi
if [ -d "$SOURCE_PEND_PATH" ] then
echo "$SOURCE_PEND_PATH found removing..."
btrfs subvolume delete -c $SOURCE_PEND_PATH
sync
fi
btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_PEND_PATH
sync
if [ -d "$TARGET_PEND_PATH" ] then
echo "$TARGET_PEND_PATH found removing..."
btrfs subvolume delete -c $TARGET_PEND_PATH
sync
fi
echo "Sending $SOURCE_PEND_PATH to $TARGET_PEND_PATH"
btrfs send -p $SOURCE_BASE_PATH $SOURCE_PEND_PATH | btrfs receive $TARGET_BASE_DIR
sync
TARGET_DATE_SUFFIX=$(suffix)
btrfs subvolume snapshot -r $TARGET_PEND_PATH "$TARGET_PATH$TARGET_DATE_SUFFIX"
sync
btrfs subvolume delete -c $SOURCE_BASE_PATH
sync
btrfs subvolume delete -c $TARGET_BASE_PATH
sync
mv $SOURCE_PEND_PATH $SOURCE_BASE_PATH
mv $TARGET_PEND_PATH $TARGET_BASE_PATH
sync
}

function list()
{
LIST_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
LIST_TARGET_BASE_DIR="$(dirname $LIST_TARGET_BASE_PATH)"
LIST_TARGET_BASE_NAME="$(basename -s .$BASE_SUFFIX $LIST_TARGET_BASE_PATH)"
find "$LIST_TARGET_BASE_DIR" -maxdepth 1 -mindepth 1 -type d -printf "%fn" | grep "${LIST_TARGET_BASE_NAME/$BASE_SUFFIX/$SNAP_SUFFIX}.$DATE_REGEX"
}

function remove()
{
REMOVE_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"
REMOVE_TARGET_BASE_DIR="$(dirname $REMOVE_TARGET_BASE_PATH)"
btrfs subvolume delete -c $REMOVE_TARGET_BASE_DIR/$2
sync
}

function removeall()
{
DATE_OFFSET="$2"
FILTER="$(filter "$DATE_OFFSET")"
while read -r SNAPSHOT ; do
remove "$1" "$SNAPSHOT"
done < <(list "$1" | grep "$FILTER")

}

(
COMMAND="$1"
shift

case "$COMMAND" in
"--help")
echo "Help"
;;
"suffix")
suffix
;;
"filter")
filter "$1"
;;
"backup")
wait_lock_or_terminate
backup "$1"
;;
"list")
list "$1"
;;
"remove")
wait_lock_or_terminate
remove "$1" "$2"
;;
"removeall")
wait_lock_or_terminate
removeall "$1" "$2"
;;
*)
echo "None.."
;;
esac
) 98>$LOCK_FILE

EOF


He aha te mahi..?Kei roto he huinga whakahau ngawari mo te hanga whakaahua BTRFS me te kape ki tetahi atu FS ma te BTRFS tuku/whiwhi.

Ko te whakarewanga tuatahi ka roa pea, na te mea... I te timatanga, ka kapehia nga raraunga katoa. Ko etahi atu whakarewatanga ka tino tere, na te mea... Ko nga huringa anake ka kape.

Ko tetahi atu tuhinga ka whakauruhia e matou ki te cron:

Ko etahi atu waehere bash#cat >/root/btrfs-backup/cron-daily.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

BACKUP_SCRIPT="$SCRIPT_DIR/btrfs-backup.sh"
RETENTION="-60 day"
$BACKUP_SCRIPT backup root/@
$BACKUP_SCRIPT removeall root/@ "$RETENTION"
$BACKUP_SCRIPT backup root/@home
$BACKUP_SCRIPT removeall root/@home "$RETENTION"
$BACKUP_SCRIPT backup boot/
$BACKUP_SCRIPT removeall boot/ "$RETENTION"
EOF


He aha tana mahi..?Ka waihanga me te tukutahi i nga whakaahua taapiri o nga pukapuka BTRFS kua whakarārangihia ki te FS taapiri. Whai muri i tenei, ka mukua e ia nga pikitia katoa i hanga 60 ra ki muri. Whai muri i te whakarewatanga, ka puta nga whakaahua o nga pukapuka kua whakarārangihia ki te /backup/btrfs/back/remote/ subdirectories.

Me hoatu e tatou nga mana whakahaere waehere:

#chmod +x /root/btrfs-backup/cron-daily.sh
#chmod +x /root/btrfs-backup/btrfs-backup.sh

Me taki ka maka ki te cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup
#cat /var/log/syslog | grep btrfs-backup
#crontab -e
0 2 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup


LVM pūrua kikokore

Me hanga he puna angiangi ki runga i te taputapu taapiri:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T backup/thin-pool

Me whakauru te ddrescue, na te mea... ka whakamahi nga tuhinga i tenei taputapu:

#apt-get install gddrescue

Me hanga he whaiaronga mo nga tuhinga:

#mkdir /root/lvm-thin-backup

Me kape nga tuhinga:

He nui te pakaru o roto ...#cat >/root/lvm-thin-backup/lvm-thin-backup.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"
DATE_PREFIX='%Y-%m-%d'
DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'
DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
BASE_SUFFIX=".base"
PEND_SUFFIX=".pend"
SNAP_SUFFIX=".snap"
BACKUPS="backup"
BACKUPS_POOL="thin-pool"

export LVM_SUPPRESS_FD_WARNINGS=1

function terminate ()
{
echo "$1" >&2
exit 1
}

function wait_lock()
{
flock 98
}

function wait_lock_or_terminate()
{
echo "Wating for lock..."
wait_lock || terminate "Failed to get lock. Exiting..."
echo "Got lock..."
}

function suffix()
{
FORMATTED_DATE=$(date +"$DATE_FORMAT")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function filter()
{
FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")
echo "$SNAP_SUFFIX.$FORMATTED_DATE"
}

function read_thin_id {
lvs --rows --reportformat basic --quiet -othin_id "$1/$2" | awk '{print $2}'
}

function read_pool_lv {
lvs --rows --reportformat basic --quiet -opool_lv "$1/$2" | awk '{print $2}'
}

function read_lv_dm_path {
lvs --rows --reportformat basic --quiet -olv_dm_path "$1/$2" | awk '{print $2}'
}

function read_lv_active {
lvs --rows --reportformat basic --quiet -olv_active "$1/$2" | awk '{print $2}'
}

function read_lv_chunk_size {
lvs --rows --reportformat basic --quiet --units b --nosuffix -ochunk_size "$1/$2" | awk '{print $2}'
}

function read_lv_size {
lvs --rows --reportformat basic --quiet --units b --nosuffix -olv_size "$1/$2" | awk '{print $2}'
}

function activate_volume {
lvchange -ay -Ky "$1/$2"
}

function deactivate_volume {
lvchange -an "$1/$2"
}

function read_thin_metadata_snap {
dmsetup status "$1" | awk '{print $7}'
}

function thindiff()
{
DIFF_VG="$1"
DIFF_SOURCE="$2"
DIFF_TARGET="$3"
DIFF_SOURCE_POOL=$(read_pool_lv $DIFF_VG $DIFF_SOURCE)
DIFF_TARGET_POOL=$(read_pool_lv $DIFF_VG $DIFF_TARGET)

if [ "$DIFF_SOURCE_POOL" == "" ] then
(>&2 echo "Source LV is not thin.")
exit 1
fi

if [ "$DIFF_TARGET_POOL" == "" ] then
(>&2 echo "Target LV is not thin.")
exit 1
fi

if [ "$DIFF_SOURCE_POOL" != "$DIFF_TARGET_POOL" ] then
(>&2 echo "Source and target LVs belong to different thin pools.")
exit 1
fi

DIFF_POOL_PATH=$(read_lv_dm_path $DIFF_VG $DIFF_SOURCE_POOL)
DIFF_SOURCE_ID=$(read_thin_id $DIFF_VG $DIFF_SOURCE)
DIFF_TARGET_ID=$(read_thin_id $DIFF_VG $DIFF_TARGET)
DIFF_POOL_PATH_TPOOL="$DIFF_POOL_PATH-tpool"
DIFF_POOL_PATH_TMETA="$DIFF_POOL_PATH"_tmeta
DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)

if [ "$DIFF_POOL_METADATA_SNAP" != "-" ] then
(>&2 echo "Thin pool metadata snapshot already exist. Assuming stale one. Will release metadata snapshot in 5 seconds.")
sleep 5
dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap
fi

dmsetup message $DIFF_POOL_PATH_TPOOL 0 reserve_metadata_snap
DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)

if [ "$DIFF_POOL_METADATA_SNAP" == "-" ] then
(>&2 echo "Failed to create thin pool metadata snapshot.")
exit 1
fi

#We keep output in variable because metadata snapshot need to be released early.
DIFF_DATA=$(thin_delta -m$DIFF_POOL_METADATA_SNAP --snap1 $DIFF_SOURCE_ID --snap2 $DIFF_TARGET_ID $DIFF_POOL_PATH_TMETA)

dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap

echo $"$DIFF_DATA" | grep -E 'different|left_only|right_only' | sed 's/</"/g' | sed 's/ /"/g' | awk -F'"' '{print $6 "t" $8 "t" $11}' | sed 's/different/copy/g' | sed 's/left_only/copy/g' | sed 's/right_only/discard/g'

}

function thinsync()
{
SYNC_VG="$1"
SYNC_PEND="$2"
SYNC_BASE="$3"
SYNC_TARGET="$4"
SYNC_PEND_POOL=$(read_pool_lv $SYNC_VG $SYNC_PEND)
SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SYNC_VG $SYNC_PEND_POOL)
SYNC_PEND_PATH=$(read_lv_dm_path $SYNC_VG $SYNC_PEND)

activate_volume $SYNC_VG $SYNC_PEND

while read -r SYNC_ACTION SYNC_OFFSET SYNC_LENGTH ; do
SYNC_OFFSET_BYTES=$((SYNC_OFFSET * SYNC_BLOCK_SIZE))
SYNC_LENGTH_BYTES=$((SYNC_LENGTH * SYNC_BLOCK_SIZE))
if [ "$SYNC_ACTION" == "copy" ] then
ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SYNC_PEND_PATH" "$SYNC_TARGET"
fi

if [ "$SYNC_ACTION" == "discard" ] then
blkdiscard -o $SYNC_OFFSET_BYTES -l $SYNC_LENGTH_BYTES "$SYNC_TARGET"
fi
done < <(thindiff "$SYNC_VG" "$SYNC_PEND" "$SYNC_BASE")
}

function discard_volume()
{
DISCARD_VG="$1"
DISCARD_LV="$2"
DISCARD_LV_PATH=$(read_lv_dm_path "$DISCARD_VG" "$DISCARD_LV")
if [ "$DISCARD_LV_PATH" != "" ] then
echo "$DISCARD_LV_PATH found"
else
echo "$DISCARD_LV not found in $DISCARD_VG"
exit 1
fi
DISCARD_LV_POOL=$(read_pool_lv $DISCARD_VG $DISCARD_LV)
DISCARD_LV_SIZE=$(read_lv_size "$DISCARD_VG" "$DISCARD_LV")
lvremove -y --quiet "$DISCARD_LV_PATH" || exit 1
lvcreate --thin-pool "$DISCARD_LV_POOL" -V "$DISCARD_LV_SIZE"B --name "$DISCARD_LV" "$DISCARD_VG" || exit 1
}

function backup()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
SOURCE_PEND_LV="$SOURCE_LV$PEND_SUFFIX"
TARGET_PEND_LV="$TARGET_LV$PEND_SUFFIX"
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "Source base not found creating snapshot of $SOURCE_VG/$SOURCE_LV to $SOURCE_VG/$SOURCE_BASE_LV"
lvcreate --quiet --snapshot --name "$SOURCE_BASE_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
echo "Discarding $SOURCE_BASE_LV_PATH as we need to bootstrap."
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)
discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
sync
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found out of sync with source... removing..."
lvremove -y --quiet $TARGET_BASE_LV_PATH || exit 1
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
sync
fi
fi
SOURCE_BASE_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_BASE_LV")
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_VG/$TARGET_LV not found. Creating empty volume."
lvcreate --thin-pool "$BACKUPS_POOL" -V "$SOURCE_BASE_SIZE"B --name "$TARGET_BASE_LV" "$TARGET_VG" || exit 1
echo "Have to rebootstrap. Discarding source at $SOURCE_BASE_LV_PATH"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)
discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
TARGET_BASE_POOL=$(read_pool_lv $TARGET_VG $TARGET_BASE_LV)
TARGET_BASE_CHUNK_SIZE=$(read_lv_chunk_size $TARGET_VG $TARGET_BASE_POOL)
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
echo "Discarding target at $TARGET_BASE_LV_PATH"
discard_volume "$TARGET_VG" "$TARGET_BASE_LV"
sync
fi
if [ "$SOURCE_PEND_LV_PATH" != "" ] then
echo "$SOURCE_PEND_LV_PATH found removing..."
lvremove -y --quiet "$SOURCE_PEND_LV_PATH" || exit 1
sync
fi
lvcreate --quiet --snapshot --name "$SOURCE_PEND_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1
SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")
sync
if [ "$TARGET_PEND_LV_PATH" != "" ] then
echo "$TARGET_PEND_LV_PATH found removing..."
lvremove -y --quiet $TARGET_PEND_LV_PATH
sync
fi
lvcreate --quiet --snapshot --name "$TARGET_PEND_LV" "$TARGET_VG/$TARGET_BASE_LV" || exit 1
TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")
SOURCE_PEND_LV_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_PEND_LV")
lvresize -L "$SOURCE_PEND_LV_SIZE"B "$TARGET_PEND_LV_PATH"
activate_volume "$TARGET_VG" "$TARGET_PEND_LV"
echo "Synching $SOURCE_PEND_LV_PATH to $TARGET_PEND_LV_PATH"
thinsync "$SOURCE_VG" "$SOURCE_PEND_LV" "$SOURCE_BASE_LV" "$TARGET_PEND_LV_PATH" || exit 1
sync

TARGET_DATE_SUFFIX=$(suffix)
lvcreate --quiet --snapshot --name "$TARGET_LV$TARGET_DATE_SUFFIX" "$TARGET_VG/$TARGET_PEND_LV" || exit 1
sync
lvremove --quiet -y "$SOURCE_BASE_LV_PATH" || exit 1
sync
lvremove --quiet -y "$TARGET_BASE_LV_PATH" || exit 1
sync
lvrename -y "$SOURCE_VG/$SOURCE_PEND_LV" "$SOURCE_BASE_LV" || exit 1
lvrename -y "$TARGET_VG/$TARGET_PEND_LV" "$TARGET_BASE_LV" || exit 1
sync
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function verify()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "$SOURCE_BASE_LV_PATH not found"
exit 1
fi
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_BASE_LV_PATH not found"
exit 1
fi
activate_volume "$TARGET_VG" "$TARGET_BASE_LV"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
echo Comparing "$SOURCE_BASE_LV_PATH" with "$TARGET_BASE_LV_PATH"
cmp "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"
echo Done...
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function resync()
{
SOURCE_VG="$1"
SOURCE_LV="$2"
TARGET_VG="$BACKUPS"
TARGET_LV="$SOURCE_VG-$SOURCE_LV"
SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"
TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"
TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")
SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

if [ "$SOURCE_BASE_LV_PATH" != "" ] then
echo "$SOURCE_BASE_LV_PATH found"
else
echo "$SOURCE_BASE_LV_PATH not found"
exit 1
fi
if [ "$TARGET_BASE_LV_PATH" != "" ] then
echo "$TARGET_BASE_LV_PATH found"
else
echo "$TARGET_BASE_LV_PATH not found"
exit 1
fi
activate_volume "$TARGET_VG" "$TARGET_BASE_LV"
activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)
SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)

echo Syncronizing "$SOURCE_BASE_LV_PATH" to "$TARGET_BASE_LV_PATH"

CMP_OFFSET=0
while [[ "$CMP_OFFSET" != "" ]] ; do
CMP_MISMATCH=$(cmp -i "$CMP_OFFSET" "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH" | grep differ | awk '{print $5}' | sed 's/,//g' )
if [[ "$CMP_MISMATCH" != "" ]] ; then
CMP_OFFSET=$(( CMP_MISMATCH + CMP_OFFSET ))
SYNC_OFFSET_BYTES=$(( ( CMP_OFFSET / SYNC_BLOCK_SIZE ) * SYNC_BLOCK_SIZE ))
SYNC_LENGTH_BYTES=$(( SYNC_BLOCK_SIZE ))
echo "Synching $SYNC_LENGTH_BYTES bytes at $SYNC_OFFSET_BYTES from $SOURCE_BASE_LV_PATH to $TARGET_BASE_LV_PATH"
ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"
else
CMP_OFFSET=""
fi
done
echo Done...
deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"
deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"
}

function list()
{
LIST_SOURCE_VG="$1"
LIST_SOURCE_LV="$2"
LIST_TARGET_VG="$BACKUPS"
LIST_TARGET_LV="$LIST_SOURCE_VG-$LIST_SOURCE_LV"
LIST_TARGET_BASE_LV="$LIST_TARGET_LV$SNAP_SUFFIX"
lvs -olv_name | grep "$LIST_TARGET_BASE_LV.$DATE_REGEX"
}

function remove()
{
REMOVE_TARGET_VG="$BACKUPS"
REMOVE_TARGET_LV="$1"
lvremove -y "$REMOVE_TARGET_VG/$REMOVE_TARGET_LV"
sync
}

function removeall()
{
DATE_OFFSET="$3"
FILTER="$(filter "$DATE_OFFSET")"
while read -r SNAPSHOT ; do
remove "$SNAPSHOT"
done < <(list "$1" "$2" | grep "$FILTER")

}

(
COMMAND="$1"
shift

case "$COMMAND" in
"--help")
echo "Help"
;;
"suffix")
suffix
;;
"filter")
filter "$1"
;;
"backup")
wait_lock_or_terminate
backup "$1" "$2"
;;
"list")
list "$1" "$2"
;;
"thindiff")
thindiff "$1" "$2" "$3"
;;
"thinsync")
thinsync "$1" "$2" "$3" "$4"
;;
"verify")
wait_lock_or_terminate
verify "$1" "$2"
;;
"resync")
wait_lock_or_terminate
resync "$1" "$2"
;;
"remove")
wait_lock_or_terminate
remove "$1"
;;
"removeall")
wait_lock_or_terminate
removeall "$1" "$2" "$3"
;;
*)
echo "None.."
;;
esac
) 98>$LOCK_FILE

EOF


He aha tana mahi...?Kei roto he huinga whakahau mo te raweke i nga whakaahua kikokore me te tukutahi i te rereketanga i waenga i nga whakaahua angiangi e rua i riro mai ma thin_delta ki tetahi atu taputapu paraka ma te whakamahi i te ddrescue me te blkdiscard.

Ko tetahi atu tuhinga ka whakauruhia e matou ki te cron:

He iti ake te bash#cat >/root/lvm-thin-backup/cron-daily.sh << EOF
#!/bin/bash
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

SCRIPT_FILE="$(realpath $0)"
SCRIPT_DIR="$(dirname $SCRIPT_FILE)"
SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

BACKUP_SCRIPT="$SCRIPT_DIR/lvm-thin-backup.sh"
RETENTION="-60 days"

$BACKUP_SCRIPT backup images linux-dev
$BACKUP_SCRIPT backup images win8
$BACKUP_SCRIPT backup images win8-data
#etc

$BACKUP_SCRIPT removeall images linux-dev "$RETENTION"
$BACKUP_SCRIPT removeall images win8 "$RETENTION"
$BACKUP_SCRIPT removeall images win8-data "$RETENTION"
#etc

EOF


He aha tana mahi...?Ka whakamahi i te tuhinga o mua ki te hanga me te tukutahi i nga taapiri o nga pukapuka kikokore kua whakarārangihia. Ka waiho e te tuhinga nga whakaahua hohekore o nga pukapuka kua whakarārangihia, e hiahiatia ana hei whai i nga huringa mai i te tukutahitanga whakamutunga.

Me whakatika tenei tuhinga, me whakaatu te rarangi o nga pukapuka angiangi hei mahi kape taapiri. Ko nga ingoa kua hoatu mo nga kaupapa whakaahua anake. Ki te hiahia koe, ka taea e koe te tuhi i tetahi tuhinga hei tukutahi i nga pukapuka katoa.

Me hoatu e tatou nga mana:

#chmod +x /root/lvm-thin-backup/cron-daily.sh
#chmod +x /root/lvm-thin-backup/lvm-thin-backup.sh

Me taki ka maka ki te cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup
#cat /var/log/syslog | grep lvm-thin-backup
#crontab -e
0 3 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup

Ka roa te whakarewanga tuatahi, no te mea... ka tino tukutahia nga pukapuka angiangi ma te kape i nga waahi kua whakamahia katoa. He mihi ki te LVM metadata angiangi, e mohio ana matou ko ehea nga poraka e whakamahia ana, na reira ko nga poraka rōrahi angiangi anake ka kapehia.

Ko nga oma ka whai ake ka kape i nga raraunga ka mihi ki te whakarereke i te aroturuki ma te LVM metadata kikokore.

Kia kite tatou i te mea i tupu:

#time /root/btrfs-backup/cron-daily.sh
real 0m2,967s
user 0m0,225s
sys 0m0,353s

#time /root/lvm-thin-backup/cron-daily.sh
real 1m2,710s
user 0m12,721s
sys 0m6,671s

#ls -al /backup/btrfs/back/remote/*
/backup/btrfs/back/remote/boot:
total 0
drwxr-xr-x 1 root root 1260 мар 26 09:11 .
drwxr-xr-x 1 root root 16 мар 6 09:30 ..
drwxr-xr-x 1 root root 322 мар 26 02:00 .@base
drwxr-xr-x 1 root root 516 мар 6 09:39 [email protected]
drwxr-xr-x 1 root root 516 мар 6 09:39 [email protected]
...
/backup/btrfs/back/remote/root:
total 0
drwxr-xr-x 1 root root 2820 мар 26 09:11 .
drwxr-xr-x 1 root root 16 мар 6 09:30 ..
drwxr-xr-x 1 root root 240 мар 26 09:11 @.@base
drwxr-xr-x 1 root root 22 мар 26 09:11 @home.@base
drwxr-xr-x 1 root root 22 мар 6 09:39 @[email protected]
drwxr-xr-x 1 root root 22 мар 6 09:39 @[email protected]
...
drwxr-xr-x 1 root root 240 мар 6 09:39 @[email protected]
drwxr-xr-x 1 root root 240 мар 6 09:39 @[email protected]
...

#lvs -olv_name,lv_size images && lvs -olv_name,lv_size backup
LV LSize
linux-dev 128,00g
linux-dev.base 128,00g
thin-pool 1,38t
win8 128,00g
win8-data 2,00t
win8-data.base 2,00t
win8.base 128,00g
LV LSize
backup 256,00g
images-linux-dev.base 128,00g
images-linux-dev.snap.2020-03-08-10-09-11 128,00g
images-linux-dev.snap.2020-03-08-10-09-25 128,00g
...
images-win8-data.base 2,00t
images-win8-data.snap.2020-03-16-14-11-55 2,00t
images-win8-data.snap.2020-03-16-14-19-50 2,00t
...
images-win8.base 128,00g
images-win8.snap.2020-03-17-04-51-46 128,00g
images-win8.snap.2020-03-18-03-02-49 128,00g
...
thin-pool <2,09t


He aha te hononga o tenei ki nga tare kohanga?

Ko te nuinga pea, i te mea ko nga pukapuka arorau LVM LV ka taea te LVM PV pukapuka tinana mo etahi atu VG. Ka taea e LVM te recursive, penei i nga tare kohanga. Ma tenei ka tino ngawari te LVM.

PS

I roto i te tuhinga e whai ake nei, ka ngana matou ki te whakamahi i etahi punaha rokiroki pūkoro rite tonu / KVM hei putake mo te hanga i tetahi roopu rokiroki / vm kua tohatoha geo me te koretake i runga i nga whenua maha ma te whakamahi i nga papamahi kaainga, te Ipurangi kaainga me nga whatunga P2P.

Source: will.com