Sannu, kwanan nan na ci karo da matsala mai ban sha'awa: saita ajiya don tallafawa babban adadin toshe na'urori.
Kowane mako muna adana duk injunan kama-da-wane a cikin gajimarenmu, don haka muna buƙatar samun damar adana dubunnan abubuwan ajiya kuma mu yi shi cikin sauri da inganci yadda ya kamata.
Abin takaici, daidaitattun saituna RAID5, RAID6 a wannan yanayin, ba za a ƙyale mu mu yi haka ba, tun da tsarin dawo da kan manyan faifai kamar namu zai yi tsayi mai zafi kuma wataƙila ba zai ƙare ba.
Bari mu kalli waɗanne hanyoyi ne:
- Mai kama da RAID5, RAID6, amma tare da matakan daidaitawa. A wannan yanayin, ana yin ajiyar ba tare da toshewa ba, amma ga kowane abu daban. Hanya mafi sauƙi don gwada goge coding ita ce faɗaɗa .
fasalin ZFS ne wanda ba a sake shi ba a halin yanzu. Ba kamar RAIDZ ba, DRAID yana da shinge mai rarrabawa kuma, yayin dawo da, yana amfani da duk faifai na tsararrun lokaci guda, wanda ke sa ya fi dacewa don tsira daga faɗuwar faifai da murmurewa da sauri bayan gazawar.
akwai uwar garken Fujitsu Primergy RX300 S7 da processor Intel Xeon CPU E5-2650L 0 @ 1.80GHz, sanduna tara na RAM Samsung DDR3-1333 8Gb PC3L-10600R ECC Rajista (M393B1K70DH0-YH9), faifan faifai Saukewa: Supermicro SuperChassis 847E26-RJBOD1, an haɗa ta Dual LSI SAS2X36 Expander da 45 fayafai Saukewa: ST6000NM0115-1YZ110 a kan 6TB kowa da kowa.
Kafin mu yanke shawarar wani abu, da farko muna buƙatar gwada komai yadda yakamata.
Don yin wannan, na shirya da kuma gwada daban-daban jeri. Don yin wannan, na yi amfani da minio, wanda ya yi aiki a matsayin baya na S3 kuma ya ƙaddamar da shi a cikin hanyoyi daban-daban tare da lambobi daban-daban na hari.
Ainihin, an gwada ƙarar minio a cikin erasure codeing vs software hari tare da adadin diski iri ɗaya da daidaiton diski, kuma waɗannan sune: RAID6, RAIDZ2 da DRAID2.
Don tunani: lokacin da kuka ƙaddamar da minio tare da manufa ɗaya kawai, minio yana aiki a yanayin ƙofa na S3, yana isar da tsarin fayil ɗin ku a cikin hanyar S3 ajiya. Idan ka ƙaddamar da minio mai ƙayyadaddun manufa da yawa, yanayin Erasure Codeing zai kunna kai tsaye, wanda zai yada bayanan tsakanin maƙasudin ku yayin ba da haƙuri ga kuskure.
Ta hanyar tsoho, minio yana raba maƙasudi zuwa ƙungiyoyi na fayafai 16, tare da nau'ikan nau'ikan nau'ikan nau'ikan nau'ikan nau'ikan nau'ikan 2. Wadancan. Disk guda biyu na iya kasawa a lokaci guda ba tare da rasa bayanai ba.
Don gwada aikin, na yi amfani da faifai 16 na 6TB kowanne na rubuta ƙananan abubuwa masu girman 1MB a kansu, wannan ya fi dacewa da siffanta nauyin da za mu ɗauka a nan gaba, tunda duk kayan aikin ajiya na zamani sun raba bayanai zuwa blocks na megabytes da yawa kuma na rubuta su ta wannan hanyar.
Don gudanar da ma'auni, mun yi amfani da s3bench utility, ƙaddamar a kan uwar garken nesa kuma mun aika dubunnan irin waɗannan abubuwa zuwa minio a cikin ɗaruruwan zaren. Bayan haka na yi ƙoƙari na dawo da su a cikin hanyar.
Ana nuna sakamakon maƙasudin a cikin tebur mai zuwa:
Kamar yadda muke iya gani, minio a cikin yanayin sa na gogewa yana yin muni sosai a rubuce fiye da minio da ke gudana a saman software RAID6, RAIDZ2 da DRAID2 a cikin tsari iri ɗaya.
Na dabam ni gwada minio akan ext4 vs XFS. Abin mamaki, don nau'in nauyin aiki na, XFS ya juya ya zama mai hankali fiye da ext4.
A cikin rukunin farko na gwaje-gwaje, Mdadm ya nuna fifiko akan ZFS, amma daga baya cewa zaku iya inganta aikin ZFS ta hanyar saita zaɓuɓɓuka masu zuwa:
xattr=sa atime=off recordsize=1Mkuma bayan haka gwaje-gwaje tare da ZFS sun zama mafi kyau.
Hakanan zaka iya lura cewa DRAID baya samar da riba mai yawa akan RAIDZ, amma a ka'idar yakamata ya zama mafi aminci.
A cikin gwaje-gwaje biyu na ƙarshe, Na kuma yi ƙoƙarin canja wurin metadata (na musamman) da ZIL (log) zuwa madubi daga SSD. Amma cire metadata bai ba da riba mai yawa a cikin saurin rikodi ba, kuma lokacin cire ZIL, na buga rufi tare da amfani 100%, don haka na yi la'akari da wannan gwajin gazawar. Ban ware cewa idan ina da faifan SSD masu sauri, to watakila wannan zai iya inganta sakamako na sosai, amma, da rashin alheri, ba ni da su.
A ƙarshe, na yanke shawarar yin amfani da DRAID kuma duk da matsayin beta, shine mafita mafi sauri kuma mafi inganci a cikin yanayin mu.
Na ƙirƙiri DRAID2 mai sauƙi a cikin tsari tare da ƙungiyoyi uku da madaidaitan rarraba guda biyu:
# zpool status data
pool: data
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
draid2:3g:2s-0 ONLINE 0 0 0
sdy ONLINE 0 0 0
sdam ONLINE 0 0 0
sdf ONLINE 0 0 0
sdau ONLINE 0 0 0
sdab ONLINE 0 0 0
sdo ONLINE 0 0 0
sdw ONLINE 0 0 0
sdak ONLINE 0 0 0
sdd ONLINE 0 0 0
sdas ONLINE 0 0 0
sdm ONLINE 0 0 0
sdu ONLINE 0 0 0
sdai ONLINE 0 0 0
sdaq ONLINE 0 0 0
sdk ONLINE 0 0 0
sds ONLINE 0 0 0
sdag ONLINE 0 0 0
sdi ONLINE 0 0 0
sdq ONLINE 0 0 0
sdae ONLINE 0 0 0
sdz ONLINE 0 0 0
sdan ONLINE 0 0 0
sdg ONLINE 0 0 0
sdac ONLINE 0 0 0
sdx ONLINE 0 0 0
sdal ONLINE 0 0 0
sde ONLINE 0 0 0
sdat ONLINE 0 0 0
sdaa ONLINE 0 0 0
sdn ONLINE 0 0 0
sdv ONLINE 0 0 0
sdaj ONLINE 0 0 0
sdc ONLINE 0 0 0
sdar ONLINE 0 0 0
sdl ONLINE 0 0 0
sdt ONLINE 0 0 0
sdah ONLINE 0 0 0
sdap ONLINE 0 0 0
sdj ONLINE 0 0 0
sdr ONLINE 0 0 0
sdaf ONLINE 0 0 0
sdao ONLINE 0 0 0
sdh ONLINE 0 0 0
sdp ONLINE 0 0 0
sdad ONLINE 0 0 0
spares
s0-draid2:3g:2s-0 AVAIL
s1-draid2:3g:2s-0 AVAIL
errors: No known data errorsTo, mun tsara ma'ajiyar, yanzu bari mu yi magana kan abin da za mu ajiye. Anan zan so nan da nan in yi magana game da mafita guda uku waɗanda na yi nasarar gwadawa, kuma waɗannan su ne:
- cokali mai yatsa , wani bayani na musamman don toshe na'urar madadin, yana da haɗin kai tare da Ceph. Zai iya ɗaukar bambance-bambance tsakanin hotunan hoto da samar da ƙarin wariyar ajiya daga gare su. Yana goyan bayan babban adadin bayanan ajiya, gami da na gida da S3. Yana buƙatar keɓantaccen bayanan bayanai don adana tebur hash ɗin cirewa. Hasara: an rubuta shi a cikin Python, yana da ɗan ƙaramin cli mara amsa.
- cokali mai yatsa , sanannen sanannen kuma ingantaccen kayan aiki na madadin, yana iya adana bayanai kuma ya cire shi da kyau. Mai ikon ajiyewa a gida da kuma zuwa uwar garken nesa ta hanyar scp. Za a iya madadin toshe na'urori idan an ƙaddamar da tutar --special, daya daga cikin minuses: lokacin ƙirƙirar madadin, an toshe ma'ajiyar gaba ɗaya, don haka ana ba da shawarar ƙirƙirar wurin ajiya daban don kowane injin kama-da-wane, a ka'ida wannan ba matsala bane, sa'a an halicce su cikin sauƙi.
wani aiki ne mai haɓakawa mai haɓakawa, rubuce-rubucen tafi, da sauri sosai kuma yana goyan bayan babban adadin bayanan ajiya, gami da ajiyar gida, scp, S3 da ƙari mai yawa. Na dabam, Ina so in lura cewa akwai na musamman halitta don restic, wanda ke ba ku damar fitarwa da sauri ma'ajiyar don amfani daga nesa. Daga cikin abubuwan da ke sama, na fi son shi. Za a iya ajiyewa daga stdin. Ba shi da kusan rashin lahani, amma akwai fasali da yawa:
-
Da fari dai, na yi ƙoƙarin yin amfani da shi a cikin yanayin ma'auni na gabaɗaya don duk injunan kama-da-wane (kamar Benji) har ma ya yi aiki sosai, amma ayyukan dawo da sun ɗauki lokaci mai tsawo, saboda ... Kowane lokaci kafin maidowa, restic yana ƙoƙarin karanta metadata na duk madadin. An warware wannan matsala cikin sauƙi, kamar yadda ta kasance tare da borg, ta hanyar ƙirƙirar keɓaɓɓen wurin ajiya na kowane injin kama-da-wane. Wannan hanya ta tabbatar da kasancewa mai tasiri sosai don sarrafa ma'ajin ma. Wuraren ajiya daban na iya samun keɓantaccen kalmar sirri don samun damar bayanai, kuma ba ma dole mu ji tsoron cewa ma'aunin ajiyar duniya na iya karya ko ta yaya. Kuna iya haifar da sabbin ma'ajiya kamar sauƙi kamar a madadin borg.
A kowane hali, ƙaddamarwa ana yin shi kawai dangane da sigar da ta gabata ta madadin; madadin baya an ƙaddara ta hanyar hanyar da aka ƙayyade, don haka idan kun adana abubuwa daban-daban daga stdin zuwa maajiyar gama gari, kar ku manta da saka faifan. zaɓi
--stdin-filename, ko a fayyace zabin kowane lokaci--parent.
-
Na biyu, dawowa zuwa stdout yana ɗaukar lokaci mai tsawo fiye da dawo da tsarin fayil saboda yanayin layi ɗaya. A nan gaba, muna shirin ƙara goyon baya na kusa don adanawa don toshe na'urori.
-
Na uku, a halin yanzu ana ba da shawarar yin amfani da shi , saboda version 0.9.6 yana da bug tare da dogon dawo da manyan fayiloli.
Don gwada tasirin wariyar ajiya da saurin rubutawa / maidowa daga maajiyar, Na ƙirƙiri wurin ajiya daban kuma na yi ƙoƙarin adana ƙaramin hoto na injin kama-da-wane (21 GB). An yi madogara biyu ba tare da canza ainihin asali ba, ta amfani da kowane ɗayan hanyoyin da aka jera don duba yadda sauri/hankali aka kwafi bayanan da aka kwafi.
Kamar yadda muke iya gani, Borg Ajiyayyen yana da mafi kyawun ƙimar ingantaccen aiki na farko, amma yana da ƙasa a cikin sharuddan rubutu da dawo da sauri.
Restic ya juya ya zama sauri fiye da Ajiyayyen Benji, amma yana ɗaukar lokaci mai tsawo don dawo da stdout, kuma, da rashin alheri, har yanzu bai san yadda ake rubuta kai tsaye zuwa na'urar toshe ba.
Bayan na auna duk ribobi da fursunoni, na yanke shawarar daidaitawa hutawa с uwar garken hutu a matsayin mafi dace da kuma alamar rahama madadin bayani.
A cikin wannan sigar allo zaku iya ganin yadda ake amfani da tashoshi mai girman gigabit 10 gabaɗaya yayin ayyukan madadin da yawa suna gudana lokaci guda. Ya kamata a lura cewa sake amfani da faifai ba ya tashi sama da 30%.
Na yi farin ciki da mafita da na samu!
source: www.habr.com