🥇 Menene LVM da matryoshka suka haɗu?

Kyakkyawan lokaci na rana.
Ina so in raba tare da al'umma gwaninta mai amfani na gina tsarin adana bayanai don KVM ta amfani da md RAID + LVM.

Shirin zai hada da:

Gina md RAID 1 daga NVMe SSD.
Haɗa md RAID 6 daga SATA SSD da fayafai na yau da kullun.
Siffofin aikin TRIM/DISCARD akan SSD RAID 1/6.
Ƙirƙirar tsararrun md RAID 1/6 mai bootable akan saitin fayafai na gama gari.
Shigar da tsarin akan NVMe RAID 1 lokacin da babu tallafin NVMe a cikin BIOS.
Yin amfani da cache na LVM da LVM bakin ciki.
Amfani da hotuna na BTRFS kuma aika/ karɓa don madadin.
Amfani da Scanpshots na bakin ciki na LVM da thin_delta don madadin salon BTRFS.

Idan kuna sha'awar, da fatan za a duba cat.

Fayil aikace-aikacen

Marubucin ba shi da alhakin sakamakon amfani ko rashin amfani da kayan / misalai / lambar / nasihu / bayanai daga wannan labarin. Ta hanyar karantawa ko amfani da wannan abu ta kowace hanya, kuna ɗaukar alhakin duk sakamakon waɗannan ayyukan. Sakamakon da zai iya yiwuwa sun haɗa da:

Soyayyen NVMe SSDs.
An yi amfani da shi gabaɗaya don yin rikodi da gazawar SSD.
Cikakkun asarar duk bayanai akan duk fayafai, gami da kwafin madadin.
Kayan aikin kwamfuta mara kyau.
Bata lokaci, jijiyoyi da kudi.
Duk wani sakamakon da ba a lissafa a sama ba.

Iron

Akwai su:

Motherboard daga kusan 2013 tare da Z87 chipset, cikakke tare da Intel Core i7 / Haswell.

Processor 4 cores, 8 zaren
32GB DDR3 RAM
1 x 16 ko 2 x 8 PCIe 3.0
1 x 4 + 1 x 1 PCIe 2.0
6 x 6 GBps SATA 3 masu haɗawa

Adaftar SAS LSI SAS9211-8I ya haskaka zuwa yanayin IT / HBA. An maye gurbin firmware mai kunna RAID da gangan tare da firmware HBA zuwa:

Kuna iya jefa wannan adaftar a kowane lokaci kuma ku maye gurbinsa da duk wani wanda kuka ci karo da shi.
TRIM/Yiwa aiki akai-akai akan faifai, saboda... a cikin firmware RAID waɗannan umarnin ba su da tallafi kwata-kwata, kuma HBA, gabaɗaya, ba ta damu da waɗanne umarni ake watsawa akan bas ɗin ba.

Hard Drive - guda 8 na HGST Travelstar 7K1000 tare da karfin TB 1 a cikin nau'i nau'i 2.5, kamar na kwamfyutocin. Waɗannan tutocin sun kasance a baya a cikin tsararrun RAID 6. Hakanan za su sami amfani a cikin sabon tsarin. Don adana bayanan gida.

Ƙarin ƙarawa:

6 guda SATA SSD samfurin Samsung 860 QVO 2TB. Waɗannan SSDs suna buƙatar babban girma, kasancewar cache na SLC, amintacce, da ƙarancin farashi ana so. An buƙaci tallafi don jefar/sifili, wanda aka duba ta layin a dmesg:

kernel: ata1.00: Enabling discard_zeroes_data

2 guda na samfurin NVMe SSD Samsung SSD 970 EVO 500GB.

Ga waɗannan SSDs, saurin karantawa/rubutu bazuwar da ƙarfin albarkatu don buƙatunku suna da mahimmanci. Radiator gare su. Lallai. Lallai. In ba haka ba, toya su har sai sun yi laushi yayin aiki tare na RAID na farko.

StarTech PEX8M2E2 adaftar don 2 x NVMe SSD wanda aka shigar a cikin PCIe 3.0 8x slot. Wannan, kuma, HBA ne kawai, amma don NVMe. Ya bambanta da masu adaftar masu arha domin baya buƙatar tallafin bifurcation na PCIe daga uwayen uwa saboda kasancewar haɗin ginin PCIe. Zai yi aiki har ma a cikin mafi tsoho tsarin tare da PCIe, koda kuwa x1 PCIe 1.0 Ramin. A zahiri, a saurin da ya dace. Babu RAIDs a wurin. Babu ginanniyar BIOS akan jirgin. Don haka, tsarin ku ba zai koyi yin taya tare da NVMe da sihiri ba, ƙasa da yin NVMe RAID godiya ga wannan na'urar.

Wannan bangaren ya kasance kawai saboda kasancewar 8x PCIe 3.0 kyauta ɗaya kawai a cikin tsarin, kuma, idan akwai ramummuka na kyauta 2, ana iya maye gurbinsa cikin sauƙi tare da dinari biyu PEX4M2E1 ko analogues, waɗanda za'a iya siye a ko'ina akan farashin 600. rubles.

An yi watsi da duk nau'ikan kayan aiki ko ginanniyar chipset/BIOS RAIDs da gangan, don samun damar maye gurbin gaba ɗaya tsarin gaba ɗaya, ban da SSD/HDD da kansu, yayin adana duk bayanan. Da kyau, ta yadda za ku iya ajiyewa ko da tsarin aiki da aka shigar yayin matsawa zuwa sabbin kayan aiki daban-daban. Babban abu shi ne cewa akwai SATA da PCIe tashar jiragen ruwa. Yana kama da CD kai tsaye ko filasha mai bootable, kawai mai sauri da ɗan girma.

HumorIn ba haka ba, kun san abin da zai faru - wani lokaci kuna buƙatar ɗaukar dukkan tsararrun tare da ku don ɗauka. Amma ba na son rasa bayanai. Don yin wannan, duk kafofin watsa labaru da aka ambata suna dacewa a kan nunin faifai a cikin 5.25 bays na daidaitaccen shari'ar.

To, kuma, ba shakka, don gwaji tare da hanyoyi daban-daban na caching SSD a cikin Linux.

Hardware hari yana da ban sha'awa. Kunna shi. Ko dai yana aiki ko baya yi. Kuma tare da mdadm koyaushe akwai zaɓuɓɓuka.

Software

A baya can, an shigar da Debian 8 Jessie akan kayan masarufi, wanda ke kusa da EOL. RAID 6 an haɗa shi daga HDD ɗin da aka ambata a sama wanda aka haɗa tare da LVM. Yana gudanar da injunan kama-da-wane a cikin kvm/libvirt.

Domin Marubucin yana da ƙwarewar da ta dace wajen ƙirƙirar faifan faifan SATA/NVMe masu ɗaukar hoto, haka kuma, don kar a karya samfurin da ya dace, an zaɓi Ubuntu 18.04 a matsayin tsarin da aka yi niyya, wanda an riga an daidaita shi sosai, amma har yanzu yana da shekaru 3. goyon baya a nan gaba.

Tsarin da aka ambata ya ƙunshi duk direbobin kayan aikin da muke buƙata daga cikin akwatin. Ba ma buƙatar kowane software na ɓangare na uku ko direbobi.

Ana shirin shigar

Don shigar da tsarin muna buƙatar Hoton Desktop na Ubuntu. Tsarin uwar garken yana da wani nau'in mai sakawa mai ƙarfi, wanda ke nuna yancin kai mai wuce gona da iri wanda ba za a iya kashe shi ba ta hanyar tura sashin tsarin UEFI akan ɗayan faifai, yana lalata duk kyawawan abubuwa. Saboda haka, an shigar dashi kawai a yanayin UEFI. Ba ya bayar da kowane zaɓi.

Ba mu ji dadin wannan ba.

Me ya sa?Abin takaici, taya UEFI bai dace sosai ba tare da RAID software na boot, saboda ... Babu wanda yayi mana tanadi don ɓangaren UEFI ESP. Akwai girke-girke akan Intanet waɗanda ke ba da shawarar sanya sashin ESP akan faifan filasha a cikin tashar USB, amma wannan mahimmin gazawa ne. Akwai girke-girke ta amfani da software mdadm RAID 1 tare da nau'in metadata 0.9 waɗanda ba su hana UEFI BIOS ganin wannan ɓangaren ba, amma wannan yana rayuwa har zuwa lokacin farin ciki lokacin da BIOS ko wani OS na OS ya rubuta wani abu zuwa ESP kuma ya manta don daidaita shi zuwa wasu. madubai.

Bugu da ƙari, taya na UEFI ya dogara da NVRAM, wanda ba zai motsa tare da faifai zuwa sabon tsarin ba, saboda wani bangare ne na motherboard.

Don haka, ba za mu sake ƙirƙira sabon dabaran ba. Mun riga muna da keken kakan da aka yi gwajin lokaci, wanda yanzu ake kira Legacy/BIOS boot, mai girman sunan CSM akan tsarin da ya dace da UEFI. Za mu cire shi daga kan shiryayye, sa mai da shi, famfo tayoyin kuma mu shafe shi da rigar datti.

Hakanan ba za a iya shigar da sigar tebur ta Ubuntu da kyau tare da Legacy bootloader ba, amma a nan, kamar yadda suke faɗa, aƙalla akwai zaɓuɓɓuka.

Don haka, muna tattara kayan aikin kuma muna loda tsarin daga Ubuntu Live bootable flash drive. Za mu buƙaci zazzage fakiti, don haka za mu saita hanyar sadarwar da ke aiki a gare ku. Idan bai yi aiki ba, za ku iya loda buƙatun da suka dace a kan filasha a gaba.

Mun shiga cikin yanayin Desktop, ƙaddamar da kwaikwaiyon tashar tashar, sannan mu tafi:

#sudo bash

Yaya…?Layin da ke sama shine abin faɗakarwa ga holiwars game da sudo. C bоmafi girma damar zo daоnauyi mafi girma. Tambayar ita ce ko za ku iya ɗauka a kan kanku. Mutane da yawa suna tunanin cewa yin amfani da sudo ta wannan hanyar ba shi da hankali. Duk da haka:

#apt-get install mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc

Me yasa ba ZFS...?Lokacin da muka shigar da software a kan kwamfutarmu, muna ba da rancen kayan aikin mu ga masu haɓaka wannan software don tuƙi.
Lokacin da muka amince da wannan software tare da amincin bayananmu, muna karɓar lamuni daidai da farashin maido da wannan bayanan, wanda za mu biya wata rana.

Daga wannan ra'ayi, ZFS Ferrari ne, kuma mdadm+lvm ya fi kama da keke.

A zahiri, marubucin ya gwammace ya ba da rancen babur ga mutanen da ba a san su ba maimakon Ferrari. A can, farashin batun ba shi da yawa. Babu bukatar hakki. Mafi sauki fiye da dokokin zirga-zirga. Yin kiliya kyauta ne. Ƙarfin ƙetare ya fi kyau. Kullum kuna iya haɗa ƙafafu da keke, kuma kuna iya gyara keke da hannuwanku.

Me yasa BTRFS...?Domin kunna tsarin aiki, muna buƙatar tsarin fayil wanda aka goyan bayan Legacy/BIOS GRUB daga cikin akwatin, kuma a lokaci guda yana goyan bayan hotuna masu rai. Za mu yi amfani da shi don / boot partition. Bugu da kari, marubucin ya fi son yin amfani da wannan FS don / (tushen), ba tare da mantawa da lura cewa ga kowane software ba zaku iya ƙirƙirar ɓangarori daban-daban akan LVM kuma saka su a cikin kundayen adireshi.

Ba za mu adana kowane hoto na inji mai kama-da-wane ko bayanan bayanai akan wannan FS ba.
Za a yi amfani da wannan FS ne kawai don ƙirƙirar hotuna na tsarin ba tare da kashe shi ba sannan kuma canja wurin waɗannan hotunan zuwa faifan ajiyar ajiya ta amfani da aikawa / karɓa.

Bugu da kari, marubucin gabaɗaya ya fi son kiyaye ƙarancin software kai tsaye akan kayan masarufi da gudanar da duk sauran software a cikin injina ta amfani da abubuwa kamar tura GPUs da masu kula da Mai watsa shiri na PCI-USB zuwa KVM ta IOMMU.

Abubuwan da suka rage kawai akan kayan aikin sune ma'ajin bayanai, haɓakawa da maajiyar bayanai.

Idan kun ƙara amincewa da ZFS, to, bisa ƙa'ida, don ƙayyadaddun aikace-aikacen suna canzawa.

Koyaya, da gangan marubucin yayi watsi da ginanniyar madubi/RAID da fasalin sake fasalin da ZFS, BRTFS da LVM suke da shi.

A matsayin ƙarin gardama, BTRFS yana da ikon juyar da rubuce-rubucen bazuwar zuwa jeri-nauyi, wanda ke da tasiri mai matuƙar tasiri akan saurin aiki tare na hotuna / madadin akan HDD.

Bari mu sake duba duk na'urori:

#udevadm control --reload-rules && udevadm trigger

Mu leka wajen:

#lsscsi && nvme list [0:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sda [1:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdb [2:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdc [3:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdd [4:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sde [5:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdf [6:0:0:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdg [6:0:1:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdh [6:0:2:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdi [6:0:3:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdj [6:0:4:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdk [6:0:5:0] disk ATA HGST HTS721010A9 A3B0 /dev/sdl [6:0:6:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdm [6:0:7:0] disk ATA HGST HTS721010A9 A3J0 /dev/sdn Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 S466NXXXXXXX15L Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7 /dev/nvme1n1 S5H7NXXXXXXX48N Samsung SSD 970 EVO 500GB 1 0,00 GB / 500,11 GB 512 B + 0 B 2B2QEXE7

Tsarin diski

NVMe SSD

Amma ba za mu yi musu alama ta kowace hanya ba. Duk iri ɗaya ne, BIOS ɗinmu baya ganin waɗannan abubuwan tafiyarwa. Don haka, za su tafi gaba ɗaya zuwa software RAID. Ba ma za mu ƙirƙiri sassan can ba. Idan kana son bin "canon" ko "mafi mahimmanci", ƙirƙirar babban bangare guda ɗaya, kamar HDD.

SATA HDD

Babu buƙatar ƙirƙira wani abu na musamman a nan. Za mu ƙirƙiri sashe ɗaya don komai. Za mu ƙirƙiri bangare saboda BIOS yana ganin waɗannan faifai kuma yana iya ƙoƙarin yin taya daga gare su. Za mu ma shigar da GRUB a kan waɗannan faifai daga baya don tsarin zai iya yin hakan kwatsam.

#cat >hdd.part << EOF label: dos label-id: 0x00000000 device: /dev/sdg unit: sectors

/dev/sdg1 : start= 2048, size= 1953523120, type=fd, bootable EOF #sfdisk /dev/sdg < hdd.part #sfdisk /dev/sdh < hdd.part #sfdisk /dev/sdi < hdd.part #sfdisk /dev/sdj < hdd.part #sfdisk /dev/sdk < hdd.part #sfdisk /dev/sdl < hdd.part #sfdisk /dev/sdm < hdd.part #sfdisk /dev/sdn < hdd.part

SATA SSD

Wannan shi ne inda abubuwa ke burge mu.

Da fari dai, na'urorin mu suna da girman TB 2. Wannan yana cikin kewayon karɓuwa don MBR, wanda shine abin da za mu yi amfani da shi. Idan ya cancanta, ana iya maye gurbinsu da GPT. Fayilolin GPT suna da matakan daidaitawa wanda ke ba da damar tsarin MBR masu jituwa don ganin ɓangarori 4 na farko idan suna cikin terabytes 2 na farko. Babban abu shi ne cewa boot partition da bios_grub partition a kan wadannan faifai ya kamata a farkon. Wannan har ma yana ba ku damar yin taya daga abubuwan tafiyar GPT Legacy/BIOS.

Amma wannan ba al'amarinmu bane.

A nan za mu ƙirƙiri sassa biyu. Na farko zai zama 1 GB a girman kuma ana amfani dashi don RAID 1 / boot.

Za'a yi amfani da na biyun don RAID 6 kuma zai ɗauki duk sauran sarari kyauta sai dai ƙaramin yanki da ba a ware a ƙarshen tuƙi.

Menene wannan yanki mara alama?Dangane da majiyoyi akan hanyar sadarwar, SATA SSDs ɗinmu suna kan jirgin cache na SLC mai faɗaɗawa mai ƙarfi daga girman 6 zuwa 78 gigabytes. Muna samun gigabytes 6 "a kyauta" saboda bambanci tsakanin "gigabyte" da "gibibytes" a cikin takardar bayanan drive. Sauran gigabytes 72 an ware su ne daga sararin da ba a yi amfani da su ba.

Anan ya kamata a lura cewa muna da cache na SLC, kuma sararin yana shagaltar da shi a cikin yanayin 4 bit MLC. Wanda a gare mu da kyau yana nufin cewa kowane gigabytes 4 na sarari kyauta za mu sami gigabyte 1 na cache SLC kawai.

A ninka gigabytes 72 da 4 kuma sami 288 gigabytes. Wannan shi ne sarari kyauta wanda ba za mu yi alama ba don ba da damar faifai don yin cikakken amfani da cache na SLC.

Don haka, za mu sami inganci har zuwa 312 gigabytes na cache SLC daga jimlar faifai shida. Daga cikin dukkan abubuwan tafiyarwa, 2 za a yi amfani da su a cikin RAID don sakewa.

Wannan adadin cache zai ba mu damar da wuya a rayuwa ta ainihi gamu da yanayin da rubutu ba ya zuwa wurin cache. Wannan yana da matuƙar ramawa ga mafi girman koma baya na ƙwaƙwalwar QLC - ƙarancin saurin rubutu lokacin da aka rubuta bayanai ta tsallake cache. Idan nauyinka bai dace da wannan ba, to, ina ba da shawarar cewa kayi tunani sosai game da tsawon lokacin da SSD ɗinka zai kasance a ƙarƙashin irin wannan nauyin, la'akari da TBW daga takardar bayanan.

#cat >ssd.part << EOF label: dos label-id: 0x00000000 device: /dev/sda unit: sectors

/dev/sda1 : start= 2048, size= 2097152, type=fd, bootable /dev/sda2 : start= 2099200, size= 3300950016, type=fd EOF #sfdisk /dev/sda < ssd.part #sfdisk /dev/sdb < ssd.part #sfdisk /dev/sdc < ssd.part #sfdisk /dev/sdd < ssd.part #sfdisk /dev/sde < ssd.part #sfdisk /dev/sdf < ssd.part

Ƙirƙirar Arrays

Da farko, muna buƙatar sake suna na'urar. Wannan ya zama dole saboda sunan mai watsa shiri wani bangare ne na sunan tsararru a wani wuri a cikin mdadm kuma yana shafar wani abu a wani wuri. Tabbas, za'a iya canza sunan tsararru daga baya, amma wannan mataki ne da ba dole ba.

#mcedit /etc/hostname #mcedit /etc/hosts #hostname vdesk0

NVMe SSD

#mdadm --create --verbose --assume-clean /dev/md0 --level=1 --raid-devices=2 /dev/nvme[0-1]n1

Me yasa - zato-tsabta...?Don gujewa fara tsarawa. Domin duka matakan RAID 1 da 6 wannan yana aiki. Komai na iya aiki ba tare da farawa ba idan sabon tsararru ne. Bugu da ƙari, ƙaddamar da tsararrun SSD akan ƙirƙira ɓarna ce ta albarkatun TBW. Muna amfani da TRIM/DISCARD inda zai yiwu akan haɗe-haɗen tsarin SSD don “fara” su.

Don tsararrakin SSD, RAID 1 DISCARD ana tallafawa daga cikin akwatin.

Don SSD RAID 6 DISCARD arrays, dole ne ku kunna shi a cikin sigogin kernel module.

Ya kamata a yi wannan kawai idan duk SSDs da aka yi amfani da su a cikin matakan 4/5/6 a cikin wannan tsarin suna da tallafin aiki don discard_zeroes_data. Wani lokaci za ku ci karo da wasu abubuwa masu ban mamaki waɗanda ke gaya wa kernel cewa ana tallafawa wannan aikin, amma a zahiri ba ya nan, ko kuma aikin ba koyaushe yake aiki ba. A halin yanzu, tallafi yana samuwa kusan ko'ina, duk da haka, akwai tsofaffin faifai da firmware tare da kurakurai. Saboda wannan dalili, an kashe tallafin DISCARD ta tsohuwa don RAID 6.

Hankali, umarni mai zuwa zai lalata duk bayanai akan abubuwan tafiyar NVMe ta hanyar “fara” tsararru tare da “zeros”.

#blkdiscard /dev/md0

Idan wani abu ba daidai ba, gwada ƙayyade mataki.

#blkdiscard --step 65536 /dev/md0

SATA SSD

#mdadm --create --verbose --assume-clean /dev/md1 --level=1 --raid-devices=6 /dev/sd[a-f]1
#blkdiscard /dev/md1
#mdadm --create --verbose --assume-clean /dev/md2 --chunk-size=512 --level=6 --raid-devices=6 /dev/sd[a-f]2

Me yasa girman haka...?Ƙara girman chunk yana da tasiri mai kyau akan saurin karatun bazuwar a cikin tubalan har zuwa girman da ya haɗa. Wannan yana faruwa ne saboda ana iya kammala aiki ɗaya na girman da ya dace ko ƙarami gaba ɗaya akan na'ura ɗaya. Saboda haka, IOPS daga duk na'urori an taƙaita su. A cewar kididdiga, 99% na IO bai wuce 512K ba.

RAID yana da IOPS 6 a kowane rubutu koyaushe ƙasa da ko daidai da IOPS na tuƙi ɗaya. Lokacin, azaman karatun bazuwar, IOPS na iya girma sau da yawa fiye da na tuƙi ɗaya, kuma a nan girman toshe yana da mahimmancin mahimmanci.
Marubucin bai ga ma'anar ƙoƙarin inganta ma'aunin da ba shi da kyau a cikin ƙirar RAID 6 kuma a maimakon haka yana inganta abin da RAID 6 ke da kyau a ciki.
Za mu ramawa matalauta rubutaccen bazuwar RAID 6 tare da ma'ajin NVMe da dabaru na samar da bakin ciki.

Har yanzu ba mu kunna DISCARD don RAID 6 ba. Don haka ba za mu “fara” wannan tsararru ba a yanzu. Za mu yi wannan daga baya, bayan shigar da OS.

SATA HDD

#mdadm --create --verbose --assume-clean /dev/md3 --chunk-size=512 --level=6 --raid-devices=8 /dev/sd[g-n]1

LVM akan NVMe RAID

Don saurin, muna so mu sanya tushen tsarin fayil akan NVMe RAID 1 wanda shine / dev/md0.
Koyaya, har yanzu za mu buƙaci wannan tsari mai sauri don wasu buƙatu, kamar musanyawa, metadata da LVM-cache da metadata na LVM-baƙi, don haka za mu ƙirƙiri LVM VG akan wannan tsarar.

#pvcreate /dev/md0 #vgcreate root /dev/md0

Bari mu ƙirƙiri bangare don tsarin fayil ɗin tushen.

#lvcreate -L 128G --name root root

Bari mu ƙirƙiri bangare don musanya gwargwadon girman RAM.

#lvcreate -L 32G --name swap root

Shigar da OS

Gabaɗaya, muna da duk abin da ake buƙata don shigar da tsarin.

Kaddamar da mayen shigar da tsarin daga mahallin Ubuntu Live. Shigarwa na al'ada. Sai kawai a matakin zabar faifai don shigarwa, kuna buƙatar ƙayyade masu zuwa:

/dev/md1, - Dutsen batu /boot, FS - BTRFS
/ dev / tushen / tushen (aka / dev / mapper / tushen-tushen), - Dutsen batu / (tushen), FS - BTRFS
/dev/tushen/swap (aka / dev/mapper/tushen-swap), - yi amfani da matsayin musanya bangare
Shigar da bootloader akan /dev/sda

Lokacin da ka zaɓi BTRFS a matsayin tushen tsarin fayil, mai sakawa zai ƙirƙiri kundin BTRFS guda biyu ta atomatik mai suna "@" don / (tushen), da "@home" don / gida.

Bari mu fara shigarwa ...

Shigarwa zai ƙare tare da akwatin maganganu na modal yana nuna kuskure wajen shigar da bootloader. Abin takaici, ba za ku iya fita daga wannan tattaunawa ta amfani da daidaitattun hanyoyi ba kuma ku ci gaba da shigarwa. Mun fita daga tsarin kuma mu sake shiga, muna ƙarewa a cikin tebur na Ubuntu Live mai tsabta. Bude tasha, kuma:

#sudo bash

Ƙirƙiri yanayi na chroot don ci gaba da shigarwa:

#mkdir /mnt/chroot #mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@ /dev/mapper/root-root /mnt/chroot #mount -o defaults,space_cache,noatime,nodiratime,discard,subvol=@home /dev/mapper/root-root /mnt/chroot/home #mount -o defaults,space_cache,noatime,nodiratime,discard /dev/md1 /mnt/chroot/boot #mount --bind /proc /mnt/chroot/proc #mount --bind /sys /mnt/chroot/sys #mount --bind /dev /mnt/chroot/dev
Bari mu saita cibiyar sadarwar da sunan mai masauki a cikin chroot:

#cat /etc/hostname >/mnt/chroot/etc/hostname #cat /etc/hosts >/mnt/chroot/etc/hosts #cat /etc/resolv.conf >/mnt/chroot/etc/resolv.conf
Bari mu shiga cikin yanayin chroot:

#chroot /mnt/chroot
Da farko, za mu isar da fakitin:

apt-get install --reinstall mdadm lvm2 thin-provisioning-tools btrfs-tools util-linux lsscsi nvme-cli mc debsums hdparm

Bari mu bincika kuma mu gyara duk fakitin da aka girka ta karkata saboda rashin cika tsarin shigar da su:

#CORRUPTED_PACKAGES=$(debsums -s 2>&1 | awk '{print $6}' | uniq) #apt-get install --reinstall $CORRUPTED_PACKAGES
Idan wani abu bai yi aiki ba, kuna iya buƙatar gyara /etc/apt/sources.list da farko

Bari mu daidaita sigogi don tsarin RAID 6 don kunna TRIM/DISCARD:

#cat >/etc/modprobe.d/raid456.conf << EOF options raid456 devices_handle_discard_safely=1 EOF
Bari mu ɗan ɗan ɗan daidaita tsararrun mu:

#cat >/etc/udev/rules.d/60-md.rules << EOF SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/stripe_cache_size", ATTR{md/stripe_cache_size}="32768" SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_min", ATTR{md/sync_speed_min}="48000" SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/sync_speed_max", ATTR{md/sync_speed_max}="300000" EOF #cat >/etc/udev/rules.d/62-hdparm.rules << EOF SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/hdparm -B 254 /dev/%k" EOF #cat >/etc/udev/rules.d/63-blockdev.rules << EOF SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", RUN+="/sbin/blockdev --setra 1024 /dev/%k" SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="md*", RUN+="/sbin/blockdev --setra 0 /dev/%k" EOF
Menene ya kasance..?Mun ƙirƙiri saitin ƙa'idodin udev waɗanda zasu yi masu zuwa:

Saita girman cache na toshe don RAID 2020 don isa ga 6. Ƙimar da ta dace, da alama, ba ta canza ba tun ƙirƙirar Linux, kuma ba ta isa ba na dogon lokaci.
Ajiye mafi ƙanƙanta na IO na tsawon lokacin dubawa/aiki tare. Wannan shi ne don hana tsararrun ku su makale cikin yanayin aiki tare na har abada ƙarƙashin kaya.
Iyakance madaidaicin IO yayin dubawa/aiki tare na tsararraki. Wannan ya zama dole don aiki tare/duba SSD RAIDs baya soya abubuwan tuki zuwa kintsattse. Wannan gaskiya ne musamman ga NVMe. (Ka tuna game da radiator? Ba wasa nake ba.)
Hana fayafai daga dakatar da jujjuyawar spindle (HDD) ta hanyar APM kuma saita lokacin bacci don masu sarrafa diski zuwa awanni 7. Kuna iya kashe APM gaba ɗaya idan abubuwan tafiyarku zasu iya yin ta (-B 255). Tare da ƙimar tsoho, faifai za su tsaya bayan daƙiƙa biyar. Sannan OS yana so ya sake saita cache ɗin diski, diski ɗin zai sake juyawa, kuma komai zai sake farawa. Fayafai suna da iyakataccen adadin jujjuyawar igiya. Irin wannan tsarin tsoho mai sauƙi zai iya kashe diski ɗinku cikin sauƙi cikin shekaru biyu. Ba duk faifai ke fama da wannan ba, amma namu “laptop” ne, tare da saitunan da suka dace, wanda ke sa RAID yayi kama da mini-MAID.
Sanya readhead a kan faifai (juyawa) 1 megabyte - tubalan jere guda biyu/chunk RAID 6
Kashe karatun gaba a kan tsararrun kansu.

Bari mu gyara /etc/fstab:

#cat >/etc/fstab << EOF # /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # file-system mount-point type options dump pass /dev/mapper/root-root / btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@ 0 1 UUID=$(blkid -o value -s UUID /dev/md1) /boot btrfs defaults,space_cache,noatime,nodiratime,discard 0 2 /dev/mapper/root-root /home btrfs defaults,space_cache,noatime,nodiratime,discard,subvol=@home 0 2 /dev/mapper/root-swap none swap sw 0 0 EOF
Me yasa haka..?Za mu nemo ɓangaren /boot ta UUID. Tsari suna na iya canzawa bisa ka'ida.

Za mu nemo sauran sassan da sunayen LVM a cikin /dev/mapper/vg-lv notation, saboda sun gano partitions quite musamman.

Ba ma amfani da UUID don LVM saboda UUID na kundin LVM da hotunan su na iya zama iri ɗaya.Dutsen /dev/mapper/tushen-tushen.. sau biyu?Ee. Daidai. Farashin BTRFS. Za'a iya shigar da wannan tsarin fayil sau da yawa tare da maɓalli daban-daban.

Saboda wannan fasalin, Ina ba da shawarar kada ku ƙirƙiri hotunan LVM na kundin BTRFS masu aiki. Kuna iya samun mamaki lokacin da kuka sake yi.

Bari mu sake haɓaka tsarin mdadm:

#/usr/share/mdadm/mkconf | sed 's/#DEVICE/DEVICE/g' >/etc/mdadm/mdadm.conf
Bari mu daidaita saitunan LVM:

#cat >>/etc/lvm/lvmlocal.conf << EOF

activation { thin_pool_autoextend_threshold=90 thin_pool_autoextend_percent=5 } allocation { cache_pool_max_chunks=2097152 } devices { global_filter=["r|^/dev/.*_corig$|","r|^/dev/.*_cdata$|","r|^/dev/.*_cmeta$|","r|^/dev/.*gpv$|","r|^/dev/images/.*$|","r|^/dev/mapper/images.*$|","r|^/dev/backup/.*$|","r|^/dev/mapper/backup.*$|"] issue_discards=1 } EOF
Menene ya kasance..?Mun ba da damar fadada ta atomatik na LVM bakin ciki wuraren waha a kan kai 90% na sararin samaniya da 5% na girma.

Mun ƙara matsakaicin adadin tubalan cache don cache na LVM.

Mun hana LVM neman kundin LVM (PV) akan:

na'urori masu ɗauke da cache LVM (cdata)
na'urorin da aka adana ta amfani da cache na LVM, suna ƙetare cache ( _kori). A wannan yanayin, na'urar da aka adana kanta har yanzu za a bincika ta cikin cache (kawai ).
na'urorin da ke ɗauke da metadata na cache LVM (cmeta)
duk na'urori a cikin VG tare da hotunan suna. Anan za mu sami hotunan faifai na injunan kama-da-wane, kuma ba ma son LVM akan mai watsa shiri don kunna kundin na OS ɗin baƙo.
duk na'urori a cikin VG tare da madadin sunan. Anan za mu sami kwafi na hotunan injin kama-da-wane.
duk na'urorin da sunansu ya ƙare da "gpv" (ƙarar jiki na baƙo)

Mun ba da damar DISCARD goyon baya lokacin da aka ba da sarari kyauta akan LVM VG. Yi hankali. Wannan zai sa share LVs akan SSD ya ɗauki lokaci sosai. Wannan musamman ya shafi SSD RAID 6. Duk da haka, bisa ga shirin, za mu yi amfani da tanadi na bakin ciki, don haka wannan ba zai hana mu ba.

Bari mu sabunta hoton initramfs:

#update-initramfs -u -k all

Shigar kuma saita grub:

#apt-get install grub-pc #apt-get purge os-prober #dpkg-reconfigure grub-pc

Wadanne faifai za a zaɓa?Duk wanda yake sd*. Dole ne tsarin ya iya yin taya daga kowane SATA drive ko SSD.

Me yasa suka kara os-prober..?Don wuce gona da iri da yancin kai da hannuwa na wasa.

Ba ya aiki daidai idan ɗaya daga cikin RAIDs yana cikin ƙasƙantaccen yanayi. Yana ƙoƙarin bincika OS akan ɓangarori waɗanda ake amfani da su a cikin injunan kama-da-wane da ke gudana akan wannan kayan aikin.

Idan kuna buƙatar shi, zaku iya barin shi, amma ku tuna duk abubuwan da ke sama. Ina ba da shawarar neman girke-girke don kawar da hannaye mara kyau akan layi.

Da wannan mun kammala shigarwa na farko. Lokaci ya yi da za a sake yi cikin sabuwar OS da aka shigar. Kar a manta da cire CD/USB Live mai bootable.

#exit #reboot

Zaɓi kowane SATA SSDs azaman na'urar taya.

LVM akan SATA SSD

A wannan gaba, mun riga mun shiga cikin sabon OS, mun saita hanyar sadarwa, dacewa, buɗe kwaikwaiyon tashar, kuma mun ƙaddamar da:

#sudo bash

Mu ci gaba.

"Fara" tsararru daga SATA SSD:

#blkdiscard /dev/md2

Idan bai yi aiki ba, to gwada:

#blkdiscard --step 65536 /dev/md2
Ƙirƙiri LVM VG akan SATA SSD:

#pvcreate /dev/md2 #vgcreate data /dev/md2

Me yasa wani VG..?A gaskiya ma, muna da VG mai suna root. Me zai hana a saka komai cikin VG guda daya?

Idan akwai PV da yawa a cikin VG, to, don kunna VG daidai, duk PVs dole ne su kasance (kan layi). Banda shi ne LVM RAID, wanda ba ma amfani da shi da gangan.

Muna son gaske cewa idan akwai gazawa (karanta asarar bayanai) akan kowane nau'ikan RAID 6, tsarin aiki zai yi kullun kuma ya ba mu damar magance matsalar.

Don yin wannan, a matakin farko na abstraction za mu ware kowane nau'in "kafofin watsa labaru" na jiki zuwa cikin VG daban.

A ilimin kimiyya, nau'ikan RAID daban-daban suna cikin "yankin dogaro" daban-daban. Kada ku ƙirƙiri ƙarin maƙasudin gama gari na gazawa gare su ta hanyar cusa su cikin VG ɗaya.

Kasancewar LVM a matakin "hardware" zai ba mu damar yanke sassa daban-daban na RAID daban-daban ta hanyar haɗa su ta hanyoyi daban-daban. Misali - gudu a lokaci guda bcache + LVM bakin ciki, bcache + BTRFS, LVM cache + LVM bakin ciki, hadadden tsari na ZFS tare da caches, ko duk wani cakuda jahannama don gwadawa da kwatanta shi duka.

A matakin “hardware”, ba za mu yi amfani da wani abu ba in ban da tsoffin kundin LVM mai “kauri” mai kyau. Banda wannan ka'ida na iya zama ɓangaren wariyar ajiya.

Ina tsammanin a wannan lokacin, masu karatu da yawa sun riga sun fara zargin wani abu game da 'yar tsana.

LVM akan SATA HDD

#pvcreate /dev/md3 #vgcreate backup /dev/md3

Sabon VG kuma..?Muna son da gaske idan tsarin faifan diski da za mu yi amfani da shi don adana bayanan ya gaza, tsarin mu zai ci gaba da aiki akai-akai, tare da ci gaba da samun damar yin amfani da bayanan da ba a ajiyewa kamar yadda aka saba. Don haka, don guje wa matsalolin kunnawa VG, muna ƙirƙirar VG daban.

Saita cache na LVM

Bari mu ƙirƙiri LV akan NVMe RAID 1 don amfani da shi azaman na'urar ɓoyewa.

#lvcreate -L 70871154688B --name cache root

Me yasa kadan ne...?Gaskiyar ita ce NVMe SSDs ɗinmu kuma suna da cache na SLC. 4 gigabytes na "kyauta" da 18 gigabytes na tsauri saboda sararin samaniya da ke cikin 3-bit MLC. Da zarar wannan cache ya ƙare, NVMe SSDs ba za su yi sauri fiye da SATA SSD ɗin mu tare da cache ba. A zahiri, saboda wannan dalili, ba shi da ma'ana a gare mu mu sanya sashin cache na LVM ya fi girma sau biyu girman cache na SLC na drive NVMe. Don faifan NVMe da aka yi amfani da su, marubucin yana ganin ya dace don yin gigabytes 32-64 na cache.

Ana buƙatar girman ɓangaren da aka bayar don tsara 64 gigabytes na cache, cache metadata, da madadin metadata.

Bugu da ƙari, na lura cewa bayan rufewar tsarin datti, LVM zai yiwa cache duka alama kuma zai sake aiki tare. Haka kuma, za a maimaita wannan duk lokacin da aka yi amfani da lvchange akan wannan na'urar har sai an sake kunna tsarin. Saboda haka, ina ba da shawarar nan da nan sake ƙirƙirar cache ta amfani da rubutun da ya dace.

Bari mu ƙirƙiri LV akan SATA RAID 6 don amfani da shi azaman na'urar da aka adana.

#lvcreate -L 3298543271936B --name cache data

Me yasa terabyte uku kawai..?Don haka, idan ya cancanta, zaku iya amfani da SATA SSD RAID 6 don wasu buƙatu. Girman sararin da aka adana za a iya ƙara haɓakawa, a kan tashi, ba tare da dakatar da tsarin ba. Don yin wannan, kuna buƙatar tsayawa na ɗan lokaci kuma ku sake kunna cache, amma fa'idar ta musamman na LVM-cache akan, misali, bcache shine ana iya yin hakan akan tashi.

Bari mu ƙirƙiri sabon VG don caching.

#pvcreate /dev/root/cache #pvcreate /dev/data/cache #vgcreate cache /dev/root/cache /dev/data/cache

Bari mu ƙirƙiri LV akan na'urar da aka adana.

#lvcreate -L 3298539077632B --name cachedata cache /dev/data/cache

Nan da nan muka ɗauki duk sarari kyauta akan /dev/data/cache ta yadda duk sauran sassan da ake buƙata an ƙirƙira su nan da nan akan /dev/root/cache. Idan kun ƙirƙiri wani abu a wuri mara kyau, zaku iya motsa shi ta amfani da pvmove.

Bari mu ƙirƙira kuma mu kunna cache:

#lvcreate -y -L 64G -n cache cache /dev/root/cache #lvcreate -y -L 1G -n cachemeta cache /dev/root/cache #lvconvert -y --type cache-pool --cachemode writeback --chunksize 64k --poolmetadata cache/cachemeta cache/cache #lvconvert -y --type cache --cachepool cache/cache cache/cachedata
Me yasa irin wannan chunksize..?Ta hanyar gwaje-gwaje masu amfani, marubucin ya sami damar gano cewa an sami sakamako mafi kyau idan girman shingen cache na LVM ya dace da girman shingen bakin ciki na LVM. Bugu da ƙari, ƙarami girman, mafi kyawun tsarin aiki a cikin rikodin bazuwar.

64k shine mafi ƙarancin girman toshe da aka yarda don bakin ciki na LVM.

Yi hankali da sake rubutawa..!Ee. Wannan nau'in cache yana jinkirta rubuta aiki tare zuwa na'urar da aka adana. Wannan yana nufin cewa idan cache ya ɓace, kuna iya rasa bayanai akan na'urar da aka adana. Daga baya, marubucin zai gaya muku matakan, ban da NVMe RAID 1, za a iya ɗauka don rama wannan haɗarin.

An zaɓi wannan nau'in cache da gangan don rama ƙarancin aikin rubuta bazuwar RAID 6.

Mu duba abin da muka samu:

#lvs -a -o lv_name,lv_size,devices --units B cache LV LSize Devices [cache] 68719476736B cache_cdata(0) [cache_cdata] 68719476736B /dev/root/cache(0) [cache_cmeta] 1073741824B /dev/root/cache(16384) cachedata 3298539077632B cachedata_corig(0) [cachedata_corig] 3298539077632B /dev/data/cache(0) [lvol0_pmspare] 1073741824B /dev/root/cache(16640)
Kawai [cachedata_corig] yakamata a kasance akan /dev/data/cache. Idan wani abu ba daidai ba, to yi amfani da pvmove.

Kuna iya kashe cache idan ya cancanta tare da umarni ɗaya:

#lvconvert -y --uncache cache/cachedata

Ana yin hakan akan layi. LVM kawai zai daidaita cache ɗin zuwa faifai, cire shi, sannan ya sake suna cachedata_corig zuwa cachedata.

Saita LVM bakin ciki

Bari mu kimanta sararin samaniya da muke buƙata don ƙananan metadata na LVM:

#thin_metadata_size --block-size=64k --pool-size=6terabytes --max-thins=100000 -u bytes thin_metadata_size - 3385794560 bytes estimated metadata area size for "--block-size=64kibibytes --pool-size=6terabytes --max-thins=100000"

Zagaye har zuwa 4 gigabytes: 4294967296B

ninka ta biyu kuma ƙara 4194304B don metadata na LVM PV: 8594128896B
Bari mu ƙirƙiri wani bangare daban akan NVMe RAID 1 don sanya LVM bakin ciki metadata da kwafin ajiyar su akansa:

#lvcreate -L 8594128896B --name images root

Don me..?Anan tambayar zata iya tasowa: me yasa LVM na bakin ciki metadata daban idan har yanzu za'a adana shi akan NVMe kuma zaiyi aiki da sauri.

Kodayake gudun yana da mahimmanci a nan, yana da nisa daga babban dalili. Abun shine cewa cache shine wurin gazawar. Wani abu zai iya faruwa da shi, kuma idan LVM na bakin ciki metadata aka adana, zai sa komai ya ɓace gaba ɗaya. Idan ba tare da cikakken metadata ba, zai zama kusan ba zai yuwu a haɗa ƙarar sirara ba.

Ta hanyar matsar da metadata zuwa keɓantaccen ma'ajin da ba a ɓoye ba, amma sauri, ƙara, muna ba da garantin amincin metadata a yayin asarar cache ko ɓarna. A wannan yanayin, duk lalacewar da asarar cache ta haifar za a gurɓata su a cikin ƙananan kundila, wanda zai sauƙaƙa hanyar dawo da umarni mai girma. Tare da babban yuwuwar, waɗannan lalacewa za a dawo dasu ta amfani da rajistan ayyukan FS.

Haka kuma, idan an ɗauki hoton ƙaramin ƙarar bakin ciki a baya, kuma bayan haka cache ɗin an daidaita shi gabaɗaya aƙalla sau ɗaya, to, saboda ƙirar ciki na LVM bakin ciki, za a tabbatar da amincin hoton hoton a yayin asarar cache. .

Bari mu ƙirƙiri sabon VG wanda zai ɗauki alhakin samar da bakin ciki:

#pvcreate /dev/root/images #pvcreate /dev/cache/cachedata #vgcreate images /dev/root/images /dev/cache/cachedata
Bari mu ƙirƙiri tafkin:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T images/thin-pool
Domin -Z yBaya ga abin da ainihin wannan yanayin ake nufi da shi - don hana bayanai daga na'ura mai kama-da-wane daga zubewa zuwa wata na'ura mai kama-da-wane yayin da ake sake rarraba sararin samaniya - ana kuma amfani da sifiri don ƙara saurin rubutun bazuwar cikin tubalan ƙasa da 64k. Duk wani rubutu kasa da 64k zuwa yankin da ba a kasaftawa a baya na ƙarar bakin ciki ba zai zama 64K gefen-daidaitacce a cikin cache. Wannan zai ba da damar yin aikin gabaɗaya ta wurin cache, ta ƙetare na'urar da aka adana.

Bari mu matsar da LVs zuwa PVs masu dacewa:

#pvmove -n images/thin-pool_tdata /dev/root/images /dev/cache/cachedata #pvmove -n images/lvol0_pmspare /dev/cache/cachedata /dev/root/images #pvmove -n images/thin-pool_tmeta /dev/cache/cachedata /dev/root/images
Mu duba:

#lvs -a -o lv_name,lv_size,devices --units B images LV LSize Devices [lvol0_pmspare] 4294967296B /dev/root/images(0) thin-pool 274877906944B thin-pool_tdata(0) [thin-pool_tdata] 274877906944B /dev/cache/cachedata(0) [thin-pool_tmeta] 4294967296B /dev/root/images(1024)
Bari mu ƙirƙiri ƙarar bakin ciki don gwaje-gwaje:

#lvcreate -V 64G --thin-pool thin-pool --name test images

Za mu shigar da fakiti don gwaje-gwaje da saka idanu:

#apt-get install sysstat fio

Wannan shine yadda zaku iya lura da halayen tsarin ajiyar mu a ainihin lokacin:

#watch 'lvs --rows --reportformat basic --quiet -ocache_dirty_blocks,cache_settings cache/cachedata && (lvdisplay cache/cachedata | grep Cache) && (sar -p -d 2 1 | grep -E "sd|nvme|DEV|md1|md2|md3|md0" | grep -v Average | sort)'

Wannan shine yadda zamu iya gwada tsarin mu:

#fio --loops=1 --size=64G --runtime=4 --filename=/dev/images/test --stonewall --ioengine=libaio --direct=1 --name=4kQD32read --bs=4k --iodepth=32 --rw=randread --name=8kQD32read --bs=8k --iodepth=32 --rw=randread --name=16kQD32read --bs=16k --iodepth=32 --rw=randread --name=32KQD32read --bs=32k --iodepth=32 --rw=randread --name=64KQD32read --bs=64k --iodepth=32 --rw=randread --name=128KQD32read --bs=128k --iodepth=32 --rw=randread --name=256KQD32read --bs=256k --iodepth=32 --rw=randread --name=512KQD32read --bs=512k --iodepth=32 --rw=randread --name=4Kread --bs=4k --rw=read --name=8Kread --bs=8k --rw=read --name=16Kread --bs=16k --rw=read --name=32Kread --bs=32k --rw=read --name=64Kread --bs=64k --rw=read --name=128Kread --bs=128k --rw=read --name=256Kread --bs=256k --rw=read --name=512Kread --bs=512k --rw=read --name=Seqread --bs=1m --rw=read --name=Longread --bs=8m --rw=read --name=Longwrite --bs=8m --rw=write --name=Seqwrite --bs=1m --rw=write --name=512Kwrite --bs=512k --rw=write --name=256write --bs=256k --rw=write --name=128write --bs=128k --rw=write --name=64write --bs=64k --rw=write --name=32write --bs=32k --rw=write --name=16write --bs=16k --rw=write --name=8write --bs=8k --rw=write --name=4write --bs=4k --rw=write --name=512KQD32write --bs=512k --iodepth=32 --rw=randwrite --name=256KQD32write --bs=256k --iodepth=32 --rw=randwrite --name=128KQD32write --bs=128k --iodepth=32 --rw=randwrite --name=64KQD32write --bs=64k --iodepth=32 --rw=randwrite --name=32KQD32write --bs=32k --iodepth=32 --rw=randwrite --name=16KQD32write --bs=16k --iodepth=32 --rw=randwrite --name=8KQD32write --bs=8k --iodepth=32 --rw=randwrite --name=4kQD32write --bs=4k --iodepth=32 --rw=randwrite | grep -E 'read|write|test' | grep -v ioengine
A hankali! Albarkatu!Wannan lambar za ta gudanar da gwaje-gwaje daban-daban 36, kowanne yana gudana na daƙiƙa 4. Rabin gwaje-gwajen na yin rikodi ne. Kuna iya yin rikodin da yawa akan NVMe a cikin daƙiƙa 4. Har zuwa 3 gigabytes a sakan daya. Don haka, kowane gwajin rubuce-rubuce na iya ci har zuwa 216 gigabytes na albarkatun SSD daga gare ku.

Karatu da rubutu gauraye?Ee. Yana da ma'ana don gudanar da gwajin karantawa da rubutawa daban. Bugu da ƙari, yana da ma'ana don tabbatar da cewa duk caches suna aiki tare ta yadda rubutun da aka yi a baya bai shafi karatun ba.

Sakamakon zai bambanta sosai yayin ƙaddamarwar farko da na gaba yayin da cache da ƙarar ƙarar sirara suka cika, haka kuma ya danganta ko tsarin ya yi aiki tare da cache ɗin da aka cika yayin ƙaddamarwar ƙarshe.

Daga cikin wasu abubuwa, Ina ba da shawarar auna saurin a kan ƙaramin ƙarar sirara wanda tuni aka ɗauki hoto. Marubucin ya sami damar lura da yadda rubuce-rubucen bazuwar ke ƙaruwa da sauri nan da nan bayan ƙirƙirar hoton farko, musamman lokacin da cache ɗin bai cika gaba ɗaya ba. Wannan yana faruwa ne saboda kwafi-kan-rubutu na mahangar rubutu, daidaita cache da ɓangarorin ƙarar sirara, da gaskiyar cewa bazuwar rubutu zuwa RAID 6 ya juya zuwa karatun bazuwar daga RAID 6 sannan rubuta zuwa cache. A cikin tsarin mu, karatun bazuwar daga RAID 6 ya kai har sau 6 (yawan SATA SSDs a cikin tsararru) da sauri fiye da rubutu. Domin tubalan na CoW ana keɓance su a jere daga wani tafki mai bakin ciki, sa'an nan rikodin, a mafi yawancin, kuma yana juya zuwa jeri.

Ana iya amfani da waɗannan abubuwan biyu don amfanin ku.

Cache "daidaitacce" hotunan hoto

Don rage haɗarin asarar bayanai idan akwai lalacewar cache / asara, marubucin ya ba da shawarar gabatar da al'adar jujjuya hotuna don tabbatar da amincin su a wannan yanayin.

Na farko, saboda ƙananan ƙananan metadata yana zaune akan na'urar da ba a ɓoye ba, metadata za ta kasance daidai kuma za a keɓance asarar da aka yi a cikin toshe bayanai.

Zagayowar zagayowar hoton hoto mai zuwa yana ba da garantin amincin bayanan da ke cikin hotunan hotunan idan an sami asarar cache:

Ga kowane ƙarar ƙarar ƙarar mai suna <name>, ƙirƙirar hoto mai suna <name>.cached
Bari mu saita iyakar ƙaura zuwa madaidaicin ƙimar ƙima: #lvchange --quiet --cachesettings "migration_threshold=16384" cache/cachedata
A cikin madauki muna duba adadin ƙazantattun tubalan a cikin cache: #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' sai mun samu sifili. Idan sifilin ya ɓace na dogon lokaci, ana iya ƙirƙira ta ta hanyar canza cache na ɗan lokaci zuwa yanayin rubutu. Koyaya, la'akari da halayen saurin tsarin SATA da NVMe SSD, da kuma albarkatun su na TBW, ko dai zaku iya kama lokacin da sauri ba tare da canza yanayin cache ba, ko kuma kayan aikin ku gaba ɗaya zasu cinye albarkatun sa gaba ɗaya. 'yan kwanaki. Saboda gazawar albarkatu, tsarin shine, bisa ka'ida, ba zai iya kasancewa ƙarƙashin nauyin rubutu 100% koyaushe ba. NVMe SSDs ɗin mu a ƙarƙashin nauyin rubutu 100% zai ƙare da albarkatu gaba ɗaya 3-4 na ranar. SATA SSDs za su šauki sau biyu kawai. Saboda haka, za mu ɗauka cewa yawancin nauyin yana zuwa karatu, kuma muna da ɗan gajeren lokaci fashewa na ayyuka masu girma da yawa tare da ƙananan nauyi akan matsakaici don rubutu.
Da zaran mun kama (ko sanya) sifili, sai mu sake suna <name>.cache zuwa <name>.committed. An share tsohon <name>.committed.
Optionally, idan cache ya cika 100%, ana iya sake ƙirƙira shi ta hanyar rubutun, don haka share shi. Tare da cache rabin-rabi, tsarin yana aiki da sauri lokacin rubutu.
Saita iyakar ƙaura zuwa sifili: #lvchange --quiet --cachesettings "migration_threshold=0" cache/cachedata Wannan zai hana cache na ɗan lokaci aiki tare da manyan kafofin watsa labarai.
Muna jira har sai canje-canje da yawa sun taru a cikin cache #lvs --rows --reportformat basic --quiet -ocache_dirty_blocks cache/cachedata | awk '{print $2}' ko kuma lokacin zai tafi.
Mu sake maimaitawa.

Me yasa matsaloli tare da iyakar ƙaura...?Abun shine cewa a cikin aikin gaske, rikodin "bazuwar" a zahiri ba gaba ɗaya ba ne. Idan muka rubuta wani abu zuwa wani yanki na kilobytes 4 a girman, akwai yuwuwar cewa a cikin mintuna biyu masu zuwa za a yi rikodin zuwa ɗaya ko ɗaya daga cikin sassan makwabta (+- 32K).

Ta hanyar saita iyakar ƙaura zuwa sifili, muna jinkirta rubuta aiki tare akan SATA SSD kuma muna tara canje-canje da yawa zuwa toshe 64K ɗaya a cikin cache. Wannan yana adana mahimmancin albarkatun SATA SSD.

Ina code din yake..?Abin baƙin ciki shine, marubucin yana ɗaukan kansa bai isa ba wajen haɓaka rubutun bash saboda ya kasance 100% wanda ya koyar da kansa kuma yana aiwatar da ci gaban "google", don haka ya yi imanin cewa mummunan lambar da ke fitowa daga hannunsa bai kamata ya yi amfani da shi ba ga kowa. wani.

Ina tsammanin masu sana'a a cikin wannan filin za su iya nuna kansu da kansu, idan ya cancanta, kuma, watakila, har ma da kyau su tsara shi azaman sabis na tsarin, kamar yadda marubucin ya yi ƙoƙari ya yi.

Irin wannan tsarin jujjuyawar hoto mai sauƙi zai ba mu damar ba kawai samun cikakken hoto ɗaya aiki tare akan SATA SSD ba, amma kuma zai ba mu damar, ta amfani da mai amfani thin_delta, don gano waɗanne tubalan da aka canza bayan halittarsa, kuma ta haka ne za a iya gano lalacewa a kan. babban kundin, mai sauƙaƙawa sosai farfadowa .

TRIM/DISKARD a cikin libvirt/KVM

Domin za a yi amfani da ajiyar bayanan don KVM mai gudana libvirt, to, zai zama kyakkyawan ra'ayi don koyar da VMs ba kawai don ɗaukar sarari kyauta ba, har ma don yantar da abin da ba a buƙata.

Ana yin wannan ta hanyar kwaikwayon goyan bayan TRIM/DISCARD akan faifai masu kama-da-wane. Don yin wannan, kuna buƙatar canza nau'in sarrafawa zuwa virtio-scsi kuma gyara xml.

#virsh edit vmname <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='threads' discard='unmap'/> <source dev='/dev/images/vmname'/> <backingStore/> <target dev='sda' bus='scsi'/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk>

<controller type='scsi' index='0' model='virtio-scsi'> <alias name='scsi0'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </controller>
Irin wannan DISCARDs daga OSes ɗin baƙo ana sarrafa su daidai ta hanyar LVM, kuma an 'yantar da tubalan daidai duka a cikin cache da a cikin bakin ruwa. A cikin yanayinmu, wannan yana faruwa musamman a cikin jinkiri, lokacin share hoto na gaba.

BTRFS Ajiyayyen

Yi amfani da shirye-shiryen rubutun da matsananci hankali da a kan kansa. Marubucin ya rubuta wannan lambar da kansa kuma kawai don kansa. Na tabbata da yawa gogaggun masu amfani da Linux suna da irin wannan kayan aikin, kuma babu buƙatar kwafi na wani.

Bari mu ƙirƙiri ƙara akan na'urar ajiyar waje:

#lvcreate -L 256G --name backup backup

Bari mu tsara shi a cikin BTRFS:

#mkfs.btrfs /dev/backup/backup

Bari mu ƙirƙiri wuraren tudu kuma mu hau tushen tushen tsarin fayil:

#mkdir /backup #mkdir /backup/btrfs #mkdir /backup/btrfs/root #mkdir /backup/btrfs/back #ln -s /boot /backup/btrfs # cat >>/etc/fstab << EOF

/dev/mapper/root-root /backup/btrfs/root btrfs defaults,space_cache,noatime,nodiratime 0 2 /dev/mapper/backup-backup /backup/btrfs/back btrfs defaults,space_cache,noatime,nodiratime 0 2 EOF #mount -a #update-initramfs -u #update-grub
Bari mu ƙirƙiri kundayen adireshi don madadin:

#mkdir /backup/btrfs/back/remote #mkdir /backup/btrfs/back/remote/root #mkdir /backup/btrfs/back/remote/boot
Bari mu ƙirƙiri kundin adireshi don rubutun madadin:

#mkdir /root/btrfs-backup

Mu kwafi rubutun:

Yawancin lambar bash mai ban tsoro. Yi amfani da haɗarin ku. Kada ku rubuta wasiƙun fushi ga marubucin...#cat >/root/btrfs-backup/btrfs-backup.sh << EOF #!/bin/bash PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"


SCRIPT_FILE="$(realpath $0)"

SCRIPT_DIR="$(dirname $SCRIPT_FILE)"

SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"
LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"

DATE_PREFIX='%Y-%m-%d'

DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'

DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'

BASE_SUFFIX=".@base"

PEND_SUFFIX=".@pend"

SNAP_SUFFIX=".@snap"

MOUNTS="/backup/btrfs/"

BACKUPS="/backup/btrfs/back/remote/"
function terminate ()

{

echo "$1" >&2

exit 1

}
function wait_lock()

{

flock 98

}
function wait_lock_or_terminate()

{

echo "Wating for lock..."

wait_lock || terminate "Failed to get lock. Exiting..."

echo "Got lock..."

}
function suffix()

{

FORMATTED_DATE=$(date +"$DATE_FORMAT")

echo "$SNAP_SUFFIX.$FORMATTED_DATE"

}
function filter()

{

FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")

echo "$SNAP_SUFFIX.$FORMATTED_DATE"

}
function backup()

{

SOURCE_PATH="$MOUNTS$1"

TARGET_PATH="$BACKUPS$1"

SOURCE_BASE_PATH="$MOUNTS$1$BASE_SUFFIX"

TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"

TARGET_BASE_DIR="$(dirname $TARGET_BASE_PATH)"

SOURCE_PEND_PATH="$MOUNTS$1$PEND_SUFFIX"

TARGET_PEND_PATH="$BACKUPS$1$PEND_SUFFIX"

if [ -d "$SOURCE_BASE_PATH" ]
then

echo "$SOURCE_BASE_PATH found"

else

echo "$SOURCE_BASE_PATH File not found creating snapshot of $SOURCE_PATH to $SOURCE_BASE_PATH"

btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_BASE_PATH

sync

if [ -d "$TARGET_BASE_PATH" ]
then

echo "$TARGET_BASE_PATH found out of sync with source... removing..."

btrfs subvolume delete -c $TARGET_BASE_PATH

sync

fi

fi

if [ -d "$TARGET_BASE_PATH" ]
then

echo "$TARGET_BASE_PATH found"

else

echo "$TARGET_BASE_PATH not found. Synching to $TARGET_BASE_DIR"

btrfs send $SOURCE_BASE_PATH | btrfs receive $TARGET_BASE_DIR

sync

fi

if [ -d "$SOURCE_PEND_PATH" ]
then

echo "$SOURCE_PEND_PATH found removing..."

btrfs subvolume delete -c $SOURCE_PEND_PATH

sync

fi

btrfs subvolume snapshot -r $SOURCE_PATH $SOURCE_PEND_PATH

sync

if [ -d "$TARGET_PEND_PATH" ]
then

echo "$TARGET_PEND_PATH found removing..."

btrfs subvolume delete -c $TARGET_PEND_PATH

sync

fi

echo "Sending $SOURCE_PEND_PATH to $TARGET_PEND_PATH"

btrfs send -p $SOURCE_BASE_PATH $SOURCE_PEND_PATH | btrfs receive $TARGET_BASE_DIR

sync

TARGET_DATE_SUFFIX=$(suffix)

btrfs subvolume snapshot -r $TARGET_PEND_PATH "$TARGET_PATH$TARGET_DATE_SUFFIX"

sync

btrfs subvolume delete -c $SOURCE_BASE_PATH

sync

btrfs subvolume delete -c $TARGET_BASE_PATH

sync

mv $SOURCE_PEND_PATH $SOURCE_BASE_PATH

mv $TARGET_PEND_PATH $TARGET_BASE_PATH

sync

}
function list()

{

LIST_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"

LIST_TARGET_BASE_DIR="$(dirname $LIST_TARGET_BASE_PATH)"

LIST_TARGET_BASE_NAME="$(basename -s .$BASE_SUFFIX $LIST_TARGET_BASE_PATH)"

find "$LIST_TARGET_BASE_DIR" -maxdepth 1 -mindepth 1 -type d -printf "%fn" | grep "${LIST_TARGET_BASE_NAME/$BASE_SUFFIX/$SNAP_SUFFIX}.$DATE_REGEX"

}
function remove()

{

REMOVE_TARGET_BASE_PATH="$BACKUPS$1$BASE_SUFFIX"

REMOVE_TARGET_BASE_DIR="$(dirname $REMOVE_TARGET_BASE_PATH)"

btrfs subvolume delete -c $REMOVE_TARGET_BASE_DIR/$2

sync

}
function removeall()

{

DATE_OFFSET="$2"

FILTER="$(filter "$DATE_OFFSET")"

while read -r SNAPSHOT ; do

remove "$1" "$SNAPSHOT"

done < <(list "$1" | grep "$FILTER")
}
(

COMMAND="$1"

shift
 case "$COMMAND" in

"--help") 

echo "Help"

;;

"suffix") 

suffix

;;

"filter") 

filter "$1"

;;

"backup") 

wait_lock_or_terminate

backup "$1"

;;

"list") 

list "$1"

;;

"remove") 

wait_lock_or_terminate

remove "$1" "$2"

;;

"removeall") 

wait_lock_or_terminate

removeall "$1" "$2"

;;

*)

echo "None.."

;;

esac

) 98>$LOCK_FILE

EOF

Menene ma yake yi..?Ya ƙunshi saitin umarni masu sauƙi don ƙirƙirar hotunan BTRFS da kwafa su zuwa wani FS ta amfani da BTRFS aika/ karɓa.

Ƙaddamarwar farko na iya ɗaukar dogon lokaci, saboda ... A farkon, duk bayanai za a kwafi. Ƙarin ƙaddamarwa zai yi sauri sosai, saboda ... Canje-canje kawai za a kwafi.

Wani rubutun da za mu sanya cikin cron:

Wasu ƙarin bash code#cat >/root/btrfs-backup/cron-daily.sh << EOF #!/bin/bash PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"


SCRIPT_FILE="$(realpath $0)"

SCRIPT_DIR="$(dirname $SCRIPT_FILE)"

SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"

BACKUP_SCRIPT="$SCRIPT_DIR/btrfs-backup.sh" RETENTION="-60 day" $BACKUP_SCRIPT backup root/@ $BACKUP_SCRIPT removeall root/@ "$RETENTION" $BACKUP_SCRIPT backup root/@home $BACKUP_SCRIPT removeall root/@home "$RETENTION" $BACKUP_SCRIPT backup boot/ $BACKUP_SCRIPT removeall boot/ "$RETENTION" EOF

Me yake yi..?Yana ƙirƙira da aiki tare da ƙarin hotuna na ɗimbin BTRFS da aka jera akan madadin FS. Bayan wannan, yana goge duk hotunan da aka kirkira kwanaki 60 da suka gabata. Bayan ƙaddamarwa, hotuna masu kwanan wata na lissafin da aka jera za su bayyana a cikin /backup/btrfs/back/remote/ subdirectories.

Bari mu ba da haƙƙin aiwatar da lambar:

#chmod +x /root/btrfs-backup/cron-daily.sh #chmod +x /root/btrfs-backup/btrfs-backup.sh
Bari mu duba shi kuma mu sanya shi a cikin cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup #cat /var/log/syslog | grep btrfs-backup #crontab -e 0 2 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/btrfs-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t btrfs-backup

LVM bakin ciki madadin

Bari mu ƙirƙiri tafki mai bakin ciki akan na'urar ajiyar waje:

#lvcreate -L 274877906944B --poolmetadataspare y --poolmetadatasize 4294967296B --chunksize 64k -Z y -T backup/thin-pool

Mu shigar da ddrescue, saboda... scripts za su yi amfani da wannan kayan aiki:

#apt-get install gddrescue

Bari mu ƙirƙiri kundin adireshi don rubutun:

#mkdir /root/lvm-thin-backup

Bari mu kwafi rubutun:

Yawan bash a ciki...#cat >/root/lvm-thin-backup/lvm-thin-backup.sh << EOF #!/bin/bash PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"


SCRIPT_FILE="$(realpath $0)"

SCRIPT_DIR="$(dirname $SCRIPT_FILE)"

SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"
LOCK_FILE="/dev/shm/$SCRIPT_NAME.lock"

DATE_PREFIX='%Y-%m-%d'

DATE_FORMAT=$DATE_PREFIX'-%H-%M-%S'

DATE_REGEX='[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9]'

BASE_SUFFIX=".base"

PEND_SUFFIX=".pend"

SNAP_SUFFIX=".snap"

BACKUPS="backup"

BACKUPS_POOL="thin-pool"
export LVM_SUPPRESS_FD_WARNINGS=1
function terminate ()

{

echo "$1" >&2

exit 1

}
function wait_lock()

{

flock 98

}
function wait_lock_or_terminate()

{

echo "Wating for lock..."

wait_lock || terminate "Failed to get lock. Exiting..."

echo "Got lock..."

}
function suffix()

{

FORMATTED_DATE=$(date +"$DATE_FORMAT")

echo "$SNAP_SUFFIX.$FORMATTED_DATE"

}
function filter()

{

FORMATTED_DATE=$(date --date="$1" +"$DATE_PREFIX")

echo "$SNAP_SUFFIX.$FORMATTED_DATE"

}
function read_thin_id {

lvs --rows --reportformat basic --quiet -othin_id "$1/$2" | awk '{print $2}'

}
function read_pool_lv {

lvs --rows --reportformat basic --quiet -opool_lv "$1/$2" | awk '{print $2}'

}
function read_lv_dm_path {

lvs --rows --reportformat basic --quiet -olv_dm_path "$1/$2" | awk '{print $2}'

}
function read_lv_active {

lvs --rows --reportformat basic --quiet -olv_active "$1/$2" | awk '{print $2}'

}
function read_lv_chunk_size {

lvs --rows --reportformat basic --quiet --units b --nosuffix -ochunk_size "$1/$2" | awk '{print $2}'

}
function read_lv_size {

lvs --rows --reportformat basic --quiet --units b --nosuffix -olv_size "$1/$2" | awk '{print $2}'

}
function activate_volume {

lvchange -ay -Ky "$1/$2"

}
function deactivate_volume {

lvchange -an "$1/$2"

}
function read_thin_metadata_snap {

dmsetup status "$1" | awk '{print $7}'

}
function thindiff()

{

DIFF_VG="$1"

DIFF_SOURCE="$2"

DIFF_TARGET="$3"

DIFF_SOURCE_POOL=$(read_pool_lv $DIFF_VG $DIFF_SOURCE)

DIFF_TARGET_POOL=$(read_pool_lv $DIFF_VG $DIFF_TARGET)
if [ "$DIFF_SOURCE_POOL" == "" ]
then

(>&2 echo "Source LV is not thin.")

exit 1

fi
if [ "$DIFF_TARGET_POOL" == "" ]
then

(>&2 echo "Target LV is not thin.")

exit 1

fi
if [ "$DIFF_SOURCE_POOL" != "$DIFF_TARGET_POOL" ]
then

(>&2 echo "Source and target LVs belong to different thin pools.")

exit 1

fi
DIFF_POOL_PATH=$(read_lv_dm_path $DIFF_VG $DIFF_SOURCE_POOL)

DIFF_SOURCE_ID=$(read_thin_id $DIFF_VG $DIFF_SOURCE)

DIFF_TARGET_ID=$(read_thin_id $DIFF_VG $DIFF_TARGET)

DIFF_POOL_PATH_TPOOL="$DIFF_POOL_PATH-tpool"

DIFF_POOL_PATH_TMETA="$DIFF_POOL_PATH"_tmeta

DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)
if [ "$DIFF_POOL_METADATA_SNAP" != "-" ]
then

(>&2 echo "Thin pool metadata snapshot already exist. Assuming stale one. Will release metadata snapshot in 5 seconds.")

sleep 5

dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap

fi
dmsetup message $DIFF_POOL_PATH_TPOOL 0 reserve_metadata_snap

DIFF_POOL_METADATA_SNAP=$(read_thin_metadata_snap $DIFF_POOL_PATH_TPOOL)
if [ "$DIFF_POOL_METADATA_SNAP" == "-" ]
then

(>&2 echo "Failed to create thin pool metadata snapshot.")

exit 1

fi
#We keep output in variable because metadata snapshot need to be released early.

DIFF_DATA=$(thin_delta -m$DIFF_POOL_METADATA_SNAP --snap1 $DIFF_SOURCE_ID --snap2 $DIFF_TARGET_ID $DIFF_POOL_PATH_TMETA)
dmsetup message $DIFF_POOL_PATH_TPOOL 0 release_metadata_snap
echo $"$DIFF_DATA" | grep -E 'different|left_only|right_only' | sed 's/</"/g' | sed 's/ /"/g' | awk -F'"' '{print $6 "t" $8 "t" $11}' | sed 's/different/copy/g' | sed 's/left_only/copy/g' | sed 's/right_only/discard/g'
}
function thinsync()

{

SYNC_VG="$1"

SYNC_PEND="$2"

SYNC_BASE="$3"

SYNC_TARGET="$4"

SYNC_PEND_POOL=$(read_pool_lv $SYNC_VG $SYNC_PEND)

SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SYNC_VG $SYNC_PEND_POOL)

SYNC_PEND_PATH=$(read_lv_dm_path $SYNC_VG $SYNC_PEND)
activate_volume $SYNC_VG $SYNC_PEND
while read -r SYNC_ACTION SYNC_OFFSET SYNC_LENGTH ; do

SYNC_OFFSET_BYTES=$((SYNC_OFFSET * SYNC_BLOCK_SIZE))

SYNC_LENGTH_BYTES=$((SYNC_LENGTH * SYNC_BLOCK_SIZE))

if [ "$SYNC_ACTION" == "copy" ]
then

ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SYNC_PEND_PATH" "$SYNC_TARGET"

fi
if [ "$SYNC_ACTION" == "discard" ]
then

blkdiscard -o $SYNC_OFFSET_BYTES -l $SYNC_LENGTH_BYTES "$SYNC_TARGET"

fi

done < <(thindiff "$SYNC_VG" "$SYNC_PEND" "$SYNC_BASE")

}
function discard_volume()

{

DISCARD_VG="$1"

DISCARD_LV="$2"

DISCARD_LV_PATH=$(read_lv_dm_path "$DISCARD_VG" "$DISCARD_LV")

if [ "$DISCARD_LV_PATH" != "" ]
then

echo "$DISCARD_LV_PATH found"

else

echo "$DISCARD_LV not found in $DISCARD_VG"

exit 1

fi

DISCARD_LV_POOL=$(read_pool_lv $DISCARD_VG $DISCARD_LV)

DISCARD_LV_SIZE=$(read_lv_size "$DISCARD_VG" "$DISCARD_LV")

lvremove -y --quiet "$DISCARD_LV_PATH" || exit 1

lvcreate --thin-pool "$DISCARD_LV_POOL" -V "$DISCARD_LV_SIZE"B --name "$DISCARD_LV" "$DISCARD_VG" || exit 1

}
function backup()

{

SOURCE_VG="$1"

SOURCE_LV="$2"

TARGET_VG="$BACKUPS"

TARGET_LV="$SOURCE_VG-$SOURCE_LV"

SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"

TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"

SOURCE_PEND_LV="$SOURCE_LV$PEND_SUFFIX"

TARGET_PEND_LV="$TARGET_LV$PEND_SUFFIX"

SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")

TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")

TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")
if [ "$SOURCE_BASE_LV_PATH" != "" ]
then

echo "$SOURCE_BASE_LV_PATH found"

else

echo "Source base not found creating snapshot of $SOURCE_VG/$SOURCE_LV to $SOURCE_VG/$SOURCE_BASE_LV"

lvcreate --quiet --snapshot --name "$SOURCE_BASE_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1

SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")

activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

echo "Discarding $SOURCE_BASE_LV_PATH as we need to bootstrap."

SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)

SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)

discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

sync

if [ "$TARGET_BASE_LV_PATH" != "" ]
then

echo "$TARGET_BASE_LV_PATH found out of sync with source... removing..."

lvremove -y --quiet $TARGET_BASE_LV_PATH || exit 1

TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")

sync

fi

fi

SOURCE_BASE_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_BASE_LV")

if [ "$TARGET_BASE_LV_PATH" != "" ]
then

echo "$TARGET_BASE_LV_PATH found"

else

echo "$TARGET_VG/$TARGET_LV not found. Creating empty volume."

lvcreate --thin-pool "$BACKUPS_POOL" -V "$SOURCE_BASE_SIZE"B --name "$TARGET_BASE_LV" "$TARGET_VG" || exit 1

echo "Have to rebootstrap. Discarding source at $SOURCE_BASE_LV_PATH"

activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)

SOURCE_BASE_CHUNK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)

discard_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

TARGET_BASE_POOL=$(read_pool_lv $TARGET_VG $TARGET_BASE_LV)

TARGET_BASE_CHUNK_SIZE=$(read_lv_chunk_size $TARGET_VG $TARGET_BASE_POOL)

TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")

echo "Discarding target at $TARGET_BASE_LV_PATH"

discard_volume "$TARGET_VG" "$TARGET_BASE_LV"

sync

fi

if [ "$SOURCE_PEND_LV_PATH" != "" ]
then

echo "$SOURCE_PEND_LV_PATH found removing..."

lvremove -y --quiet "$SOURCE_PEND_LV_PATH" || exit 1

sync

fi

lvcreate --quiet --snapshot --name "$SOURCE_PEND_LV" "$SOURCE_VG/$SOURCE_LV" || exit 1

SOURCE_PEND_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_PEND_LV")

sync

if [ "$TARGET_PEND_LV_PATH" != "" ]
then

echo "$TARGET_PEND_LV_PATH found removing..."

lvremove -y --quiet $TARGET_PEND_LV_PATH

sync

fi

lvcreate --quiet --snapshot --name "$TARGET_PEND_LV" "$TARGET_VG/$TARGET_BASE_LV" || exit 1

TARGET_PEND_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_PEND_LV")

SOURCE_PEND_LV_SIZE=$(read_lv_size "$SOURCE_VG" "$SOURCE_PEND_LV")

lvresize -L "$SOURCE_PEND_LV_SIZE"B "$TARGET_PEND_LV_PATH"

activate_volume "$TARGET_VG" "$TARGET_PEND_LV"

echo "Synching $SOURCE_PEND_LV_PATH to $TARGET_PEND_LV_PATH"

thinsync "$SOURCE_VG" "$SOURCE_PEND_LV" "$SOURCE_BASE_LV" "$TARGET_PEND_LV_PATH" || exit 1

sync
TARGET_DATE_SUFFIX=$(suffix)

lvcreate --quiet --snapshot --name "$TARGET_LV$TARGET_DATE_SUFFIX" "$TARGET_VG/$TARGET_PEND_LV" || exit 1

sync

lvremove --quiet -y "$SOURCE_BASE_LV_PATH" || exit 1

sync

lvremove --quiet -y "$TARGET_BASE_LV_PATH" || exit 1

sync

lvrename -y "$SOURCE_VG/$SOURCE_PEND_LV" "$SOURCE_BASE_LV" || exit 1

lvrename -y "$TARGET_VG/$TARGET_PEND_LV" "$TARGET_BASE_LV" || exit 1

sync

deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"

deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

}
function verify()

{

SOURCE_VG="$1"

SOURCE_LV="$2"

TARGET_VG="$BACKUPS"

TARGET_LV="$SOURCE_VG-$SOURCE_LV"

SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"

TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"

TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")

SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
if [ "$SOURCE_BASE_LV_PATH" != "" ]
then

echo "$SOURCE_BASE_LV_PATH found"

else

echo "$SOURCE_BASE_LV_PATH not found"

exit 1

fi

if [ "$TARGET_BASE_LV_PATH" != "" ]
then

echo "$TARGET_BASE_LV_PATH found"

else

echo "$TARGET_BASE_LV_PATH not found"

exit 1

fi

activate_volume "$TARGET_VG" "$TARGET_BASE_LV"

activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

echo Comparing "$SOURCE_BASE_LV_PATH" with "$TARGET_BASE_LV_PATH"

cmp "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"

echo Done...

deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"

deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

}
function resync()

{

SOURCE_VG="$1"

SOURCE_LV="$2"

TARGET_VG="$BACKUPS"

TARGET_LV="$SOURCE_VG-$SOURCE_LV"

SOURCE_BASE_LV="$SOURCE_LV$BASE_SUFFIX"

TARGET_BASE_LV="$TARGET_LV$BASE_SUFFIX"

TARGET_BASE_LV_PATH=$(read_lv_dm_path "$TARGET_VG" "$TARGET_BASE_LV")

SOURCE_BASE_LV_PATH=$(read_lv_dm_path "$SOURCE_VG" "$SOURCE_BASE_LV")
if [ "$SOURCE_BASE_LV_PATH" != "" ]
then

echo "$SOURCE_BASE_LV_PATH found"

else

echo "$SOURCE_BASE_LV_PATH not found"

exit 1

fi

if [ "$TARGET_BASE_LV_PATH" != "" ]
then

echo "$TARGET_BASE_LV_PATH found"

else

echo "$TARGET_BASE_LV_PATH not found"

exit 1

fi

activate_volume "$TARGET_VG" "$TARGET_BASE_LV"

activate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

SOURCE_BASE_POOL=$(read_pool_lv $SOURCE_VG $SOURCE_BASE_LV)

SYNC_BLOCK_SIZE=$(read_lv_chunk_size $SOURCE_VG $SOURCE_BASE_POOL)
echo Syncronizing "$SOURCE_BASE_LV_PATH" to "$TARGET_BASE_LV_PATH"
CMP_OFFSET=0

while [[ "$CMP_OFFSET" != "" ]] ; do

CMP_MISMATCH=$(cmp -i "$CMP_OFFSET" "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH" | grep differ | awk '{print $5}' | sed 's/,//g' )

if [[ "$CMP_MISMATCH" != "" ]] ; then

CMP_OFFSET=$(( CMP_MISMATCH + CMP_OFFSET ))

SYNC_OFFSET_BYTES=$(( ( CMP_OFFSET / SYNC_BLOCK_SIZE ) * SYNC_BLOCK_SIZE ))

SYNC_LENGTH_BYTES=$(( SYNC_BLOCK_SIZE ))

echo "Synching $SYNC_LENGTH_BYTES bytes at $SYNC_OFFSET_BYTES from $SOURCE_BASE_LV_PATH to $TARGET_BASE_LV_PATH"

ddrescue --quiet --force --input-position=$SYNC_OFFSET_BYTES --output-position=$SYNC_OFFSET_BYTES --size=$SYNC_LENGTH_BYTES "$SOURCE_BASE_LV_PATH" "$TARGET_BASE_LV_PATH"

else

CMP_OFFSET=""

fi

done

echo Done...

deactivate_volume "$TARGET_VG" "$TARGET_BASE_LV"

deactivate_volume "$SOURCE_VG" "$SOURCE_BASE_LV"

}
function list()

{

LIST_SOURCE_VG="$1"

LIST_SOURCE_LV="$2"

LIST_TARGET_VG="$BACKUPS"

LIST_TARGET_LV="$LIST_SOURCE_VG-$LIST_SOURCE_LV"

LIST_TARGET_BASE_LV="$LIST_TARGET_LV$SNAP_SUFFIX"

lvs -olv_name | grep "$LIST_TARGET_BASE_LV.$DATE_REGEX"

}
function remove()

{

REMOVE_TARGET_VG="$BACKUPS"

REMOVE_TARGET_LV="$1"

lvremove -y "$REMOVE_TARGET_VG/$REMOVE_TARGET_LV"

sync

}
function removeall()

{

DATE_OFFSET="$3"

FILTER="$(filter "$DATE_OFFSET")"

while read -r SNAPSHOT ; do

remove "$SNAPSHOT"

done < <(list "$1" "$2" | grep "$FILTER")
}
(

COMMAND="$1"

shift
 case "$COMMAND" in

"--help") 

echo "Help"

;;

"suffix") 

suffix

;;

"filter") 

filter "$1"

;;

"backup") 

wait_lock_or_terminate

backup "$1" "$2"

;;

"list") 

list "$1" "$2"

;;

"thindiff") 

thindiff "$1" "$2" "$3"

;;

"thinsync") 

thinsync "$1" "$2" "$3" "$4"

;;

"verify") 

wait_lock_or_terminate

verify "$1" "$2"

;;

"resync") 

wait_lock_or_terminate

resync "$1" "$2"

;;

"remove") 

wait_lock_or_terminate

remove "$1"

;;

"removeall") 

wait_lock_or_terminate

removeall "$1" "$2" "$3"

;;

*)

echo "None.."

;;

esac

) 98>$LOCK_FILE

EOF

Me yake yi...?Ya ƙunshi saitin umarni don sarrafa ɓangarorin ɗaukar hoto na bakin ciki da daidaita bambanci tsakanin siraran hotuna biyu da aka karɓa ta hanyar thin_delta zuwa wani na'urar toshe ta amfani da ddrescue da blkdiscard.

Wani rubutun da za mu sanya a cikin cron:

Dan karin bash#cat >/root/lvm-thin-backup/cron-daily.sh << EOF #!/bin/bash PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"


SCRIPT_FILE="$(realpath $0)"

SCRIPT_DIR="$(dirname $SCRIPT_FILE)"

SCRIPT_NAME="$(basename -s .sh $SCRIPT_FILE)"
BACKUP_SCRIPT="$SCRIPT_DIR/lvm-thin-backup.sh"

RETENTION="-60 days"
$BACKUP_SCRIPT backup images linux-dev

$BACKUP_SCRIPT backup images win8

$BACKUP_SCRIPT backup images win8-data

#etc
$BACKUP_SCRIPT removeall images linux-dev "$RETENTION"

$BACKUP_SCRIPT removeall images win8 "$RETENTION"

$BACKUP_SCRIPT removeall images win8-data "$RETENTION"

#etc

EOF

Me yake yi...?Yana amfani da rubutun da ya gabata don ƙirƙira da aiki tare da ma'ajin ajiyar kundila na sirara da aka jera. Rubutun zai bar hotuna marasa aiki na kundin da aka jera, waɗanda ake buƙata don bin diddigin canje-canje tun aiki tare na ƙarshe.

Dole ne a gyara wannan rubutun, yana ƙayyadaddun jeri na ƙananan kundila waɗanda ya kamata a yi kwafin ajiya don su. Sunayen da aka bayar don dalilai ne kawai. Idan kuna so, zaku iya rubuta rubutun da zai daidaita duk kundin.

Bari mu ba da hakkoki:

#chmod +x /root/lvm-thin-backup/cron-daily.sh #chmod +x /root/lvm-thin-backup/lvm-thin-backup.sh

Bari mu duba shi kuma mu sanya shi a cikin cron:

#/usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup #cat /var/log/syslog | grep lvm-thin-backup #crontab -e 0 3 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c 3 /root/lvm-thin-backup/cron-daily.sh 2>&1 | /usr/bin/logger -t lvm-thin-backup
Ƙaddamarwar farko za ta daɗe, saboda... Za a daidaita juzu'i na bakin ciki ta hanyar kwafi duk sararin da aka yi amfani da su. Godiya ga metadata na bakin ciki na LVM, mun san waɗanne tubalan da ake amfani da su a zahiri, don haka kawai za a kwafi katantan ƙarar ƙarar da aka yi amfani da su.

Gudun da ke gaba za su kwafi bayanan da ƙari godiya ga canza bin diddigin ta hanyar ƙananan metadata na LVM.

Bari mu ga abin da ya faru:

#time /root/btrfs-backup/cron-daily.sh real 0m2,967s user 0m0,225s sys 0m0,353s


#time /root/lvm-thin-backup/cron-daily.sh

real 1m2,710s

user 0m12,721s

sys 0m6,671s
#ls -al /backup/btrfs/back/remote/*

/backup/btrfs/back/remote/boot:

total 0

drwxr-xr-x 1 root root 1260 мар 26 09:11 .

drwxr-xr-x 1 root root 16 мар 6 09:30 ..

drwxr-xr-x 1 root root 322 мар 26 02:00 .@base

drwxr-xr-x 1 root root 516 мар 6 09:39 [email protected]

drwxr-xr-x 1 root root 516 мар 6 09:39 [email protected]

...

/backup/btrfs/back/remote/root:

total 0

drwxr-xr-x 1 root root 2820 мар 26 09:11 .

drwxr-xr-x 1 root root 16 мар 6 09:30 ..

drwxr-xr-x 1 root root 240 мар 26 09:11 @.@base

drwxr-xr-x 1 root root 22 мар 26 09:11 @home.@base

drwxr-xr-x 1 root root 22 мар 6 09:39 @[email protected]

drwxr-xr-x 1 root root 22 мар 6 09:39 @[email protected]

...

drwxr-xr-x 1 root root 240 мар 6 09:39 @[email protected]

drwxr-xr-x 1 root root 240 мар 6 09:39 @[email protected]

...

#lvs -olv_name,lv_size images && lvs -olv_name,lv_size backup LV LSize linux-dev 128,00g linux-dev.base 128,00g thin-pool 1,38t win8 128,00g win8-data 2,00t win8-data.base 2,00t win8.base 128,00g LV LSize backup 256,00g images-linux-dev.base 128,00g images-linux-dev.snap.2020-03-08-10-09-11 128,00g images-linux-dev.snap.2020-03-08-10-09-25 128,00g ... images-win8-data.base 2,00t images-win8-data.snap.2020-03-16-14-11-55 2,00t images-win8-data.snap.2020-03-16-14-19-50 2,00t ... images-win8.base 128,00g images-win8.snap.2020-03-17-04-51-46 128,00g images-win8.snap.2020-03-18-03-02-49 128,00g ... thin-pool <2,09t

Menene alakar wannan da ƴan tsana?

Mafi mahimmanci, idan aka ba da kundin ma'ana na LVM LV na iya zama kundin jiki na LVM PV don sauran VGs. LVM na iya zama mai jujjuyawa, kamar tsana na gida. Wannan yana ba LVM matsananciyar sassauci.

PS

A cikin labarin na gaba, za mu yi ƙoƙari mu yi amfani da tsarin ajiya na wayar hannu da yawa / KVM a matsayin tushen don ƙirƙirar tari mai rarrabawa / vm tare da sakewa a kan nahiyoyi da yawa ta amfani da kwamfyutocin gida, Intanet na gida da kuma hanyoyin sadarwar P2P.

source: www.habr.com

Menene LVM da Matryoshka suka haɗu?

Fayil aikace-aikacen

Iron

Akwai su:

Ƙarin ƙarawa:

Software

Ana shirin shigar

Tsarin diski

NVMe SSD

SATA HDD

SATA SSD

Ƙirƙirar Arrays

NVMe SSD

SATA SSD

SATA HDD

LVM akan NVMe RAID

Shigar da OS

LVM akan SATA SSD

LVM akan SATA HDD

Saita cache na LVM

Saita LVM bakin ciki

Cache "daidaitacce" hotunan hoto

TRIM/DISKARD a cikin libvirt/KVM

BTRFS Ajiyayyen

LVM bakin ciki madadin

Bari mu ga abin da ya faru:

Menene alakar wannan da ƴan tsana?

PS

Add a comment Отменить ответ