DBA bot Joe. UAnatoly Stansler (Postgres.ai)

Ngaba umphuhlisi we-backend uyaqonda njani ukuba umbuzo we-SQL uya kusebenza kakuhle kwi "prod"? Kwiinkampani ezinkulu okanye ezikhula ngokukhawuleza, ayinguye wonke umntu onokufikelela "kwimveliso". Kwaye ngokufikelela, ayizizo zonke izicelo ezinokujongwa ngaphandle kwentlungu, kwaye ukudala ikopi yedatha ihlala kuthatha iiyure. Ukusombulula ezi ngxaki, senze i-DBA eyenziweyo - uJoe. Sele iphunyezwe ngempumelelo kwiinkampani ezininzi kwaye inceda abaphuhlisi abangaphezu kweshumi elinesibini.

Ividiyo:

Molweni nonke! Igama lam ndinguAnatoly Stansler. Ndisebenzela inkampani postgres.ai. Sizimisele ukukhawuleza inkqubo yophuhliso ngokususa ukulibaziseka okuhambelana nomsebenzi wePostgres kubaphuhlisi, ii-DBA kunye nee-QAs.

Sinabathengi abakhulu kwaye namhlanje inxalenye yengxelo iya kunikezelwa kumatyala esiye sadibana nawo ngelixa sisebenza nabo. Ndiza kuthetha ngendlela esibancede ngayo bacombulule iingxaki ezinzulu.

Xa siphuhlisa kwaye senza ukufuduka kwemithwalo ephezulu enzima, sizibuza umbuzo: "Ngaba oku kufuduka kuya kuthatha?". Sisebenzisa uphononongo, sisebenzisa ulwazi lwabalingane abanamava ngakumbi, iingcali ze-DBA. Kwaye bayakwazi ukuxelela ukuba iya kubhabha okanye hayi.

Kodwa mhlawumbi kungangcono ukuba sinokuzivavanya ngokwethu kwiikopi ezipheleleyo. Kwaye namhlanje siza kuthetha nje malunga nokuba zeziphi iindlela zokuvavanya ngoku kwaye zingenziwa njani ngcono kwaye ziziphi izixhobo. Siza kuthetha nangezinto ezilungileyo nezingalunganga zeendlela ezinjalo, kwaye yintoni esinokuyilungisa apha.

Ngubani owakhe wenza izalathisi ngokuthe ngqo kwi-prod okanye wenza naluphi na utshintsho? Kancinci kancinci. Kwaye oku kwakhokelela kubani kwinto yokuba idatha yalahleka okanye kukho ixesha lokuphumla? Uyayazi ke le ntlungu. Ndiyambulela uThixo kukho ii-backups.

Indlela yokuqala kukuvavanya kwiprod. Okanye, xa umphuhlisi ehleli kumatshini wendawo, unedatha yokuvavanya, kukho uhlobo oluthile lokukhetha okulinganiselwe. Kwaye sikhuphela ukuvelisa, kwaye siyayifumana le meko.

Kubuhlungu, kuyabiza. Kungcono ukuba ungayenzi.

Kwaye yeyiphi eyona ndlela ilungileyo yokuyenza?

Masithathe iqonga kwaye sikhethe inxalenye yeprod apho. Okanye kungcono, masithathe iprod yokwenyani, yonke idatha. Kwaye emva kokuba siyiphuhlise ekuhlaleni, siya kuphinda sijonge iqonga.

Oku kuya kusivumela ukuba sisuse ezinye zeempazamo, o.k.t. zithintele ukuba zingabikho kwimveliso.

Ziziphi iingxaki?

• Ingxaki kukuba lo mboniso sabelane noogxa bethu. Kwaye kaninzi kwenzeka ukuba wenze uhlobo oluthile lotshintsho, bam - kwaye akukho datha, umsebenzi uphantsi kombhobho. Iqonga laliyi-multi-terabyte. Kwaye kufuneka ulinde ixesha elide ukuze ivuke kwakhona. Kwaye sigqiba ekubeni siyigqibe ngomso. Yiloo nto, sinophuhliso.
• Kwaye, kunjalo, sinabalingane abaninzi abasebenza apho, amaqela amaninzi. Kwaye kufuneka kwenziwe ngesandla. Kwaye oku akuncedi.

Kwaye kufanelekile ukutsho ukuba sinomzamo omnye kuphela, ukudubula okunye, ukuba sifuna ukwenza utshintsho oluthile kwisiseko sedatha, sichukumise idatha, sitshintshe isakhiwo. Kwaye ukuba kukho into engalunganga, ukuba bekukho impazamo ekufudukeni, ngoko asiyi kubuya ngokukhawuleza.

Oku kungcono kunendlela yangaphambili, kodwa kusekho ithuba eliphezulu lokuba uhlobo oluthile lwempazamo luya kuhamba kwimveliso.

Yintoni esithintelayo ukunika umphuhlisi ngamnye ibhentshi yovavanyo, ikopi epheleleyo? Ndicinga ukuba icacile into ehamba endleleni.

Ngubani onedatabase enkulu kune terabyte? Ngaphezu kwesiqingatha segumbi.

Kwaye kucacile ukuba ukugcina oomatshini kumthuthukisi ngamnye, xa kukho imveliso enkulu kangaka, kubiza kakhulu, kwaye ngaphandle koko, kuthatha ixesha elide.

Sinabathengi abaye baqaphela ukuba kubaluleke kakhulu ukuvavanya zonke iinguqu kwiikopi ezipheleleyo, kodwa i-database yabo ingaphantsi kwe-terabyte, kwaye akukho zixhobo zokugcina ibhentshi yokuvavanya kumthuthukisi ngamnye. Ke ngoko, kufuneka bakhuphele iindawo zokulahla kwindawo yabo kumatshini wabo kwaye bavavanye ngale ndlela. Kuthatha ixesha elininzi.

Nokuba uyenza ngaphakathi kweziseko zophuhliso, emva koko ukukhuphela i-terabyte enye yedatha ngeyure sele kulungile kakhulu. Kodwa basebenzisa iindawo zokulahla ezinengqiqo, bakhuphela kwalapha efini. Kubo, isantya simalunga nama-200 egigabytes ngeyure. Kwaye kusathatha ixesha ukujika ukusuka kwindawo yokulahla okunengqiqo, ukusonga izalathisi, njl.

Kodwa basebenzisa le ndlela kuba ivumela ukuba bagcine iprod ithembekile.

Singenza ntoni apha? Masenze iibhentshi zovavanyo zingabizi kakhulu kwaye sinike umphuhlisi ngamnye ibhentshi yakhe yovavanyo.

Kwaye oku kunokwenzeka.

Kwaye kule ndlela, xa sisenza ii-clones ezicekethekileyo zomphuhlisi ngamnye, singabelana ngayo kumatshini omnye. Umzekelo, ukuba unesiseko sedatha se-10TB kwaye ufuna ukunika abaphuhlisi abali-10, awudingi ukuba ne-XNUMX x XNUMXTB yogcino-lwazi. Ufuna kuphela umatshini omnye ukwenza iikopi ezicekethekileyo ezizimeleyo zomphuhlisi ngamnye usebenzisa umatshini omnye. Ndiza kukuxelela ukuba isebenza njani kamva.

Umzekelo wokwenyani:

• DB - 4,5 terabytes.

• Sinokufumana iikopi ezizimeleyo kwimizuzwana engama-30.

Akunyanzelekanga ukuba ulinde i-test stand kwaye ixhomekeke ekubeni inkulu kangakanani na. Ungayifumana ngemizuzwana. Iya kuba ziindawo ezizimeleyo ngokupheleleyo, kodwa ezabelana ngedatha phakathi kwazo.

Oku kakuhle. Apha sithetha ngomlingo kunye nendalo ehambelanayo.

Kwimeko yethu, oku kusebenza kusetyenziswa inkqubo ye-OpenZFS.

I-OpenZFS yinkqubo yefayile yokukhuphela-ngokubhala exhasa izifinyezo kunye neeclones ngaphandle kwebhokisi. Inokwethenjelwa kwaye iyakaleka. Kulula kakhulu ukuphatha. Inokusasazwa ngokwenyani ngokwamaqela amabini.

Kukho ezinye iinketho:

• I-LVM,

• Ukugcinwa (umzekelo, Ukugcinwa okucocekileyo).

IDatabase Lab ndithetha ngayo yimodyuli. Inokuphunyezwa ngokusebenzisa olu khetho. Kodwa okwangoku, sigxile kwi-OpenZFS, kuba bekukho iingxaki nge-LVM ngokuthe ngqo.

Ingaba isebenza kanjani? Endaweni yokubhala ngaphezulu idatha ngalo lonke ixesha siyitshintsha, siyigcina ngokumakisha nje ukuba le datha intsha isuka kwindawo entsha ngexesha, i-snapshot entsha.

Kwaye kwixesha elizayo, xa sifuna ukubuyisela umva okanye sifuna ukwenza i-clone entsha kuguqulelo oludala, sivele sithi: "Kulungile, sinike ezibhloko zedatha eziphawulwe ngolu hlobo."

Kwaye lo msebenzisi uya kusebenza ngeseti yedatha enjalo. Uya kuzitshintsha ngokuthe ngcembe, enze ezakhe izifinyezo.

Kwaye siya kumasebe. Umphuhlisi ngamnye kwimeko yethu uya kuba nethuba lokuba ne-clone yakhe ayihlelayo, kwaye idatha ekwabelwana ngayo iya kwabelwana phakathi komntu wonke.

Ukuhambisa inkqubo enjalo ekhaya, kufuneka usombulule iingxaki ezimbini:

• Eyokuqala ngumthombo wedatha, apho uya kuyithatha khona. Ungaseta uphindaphindo kunye nemveliso. Uyakwazi ukusebenzisa i-backups osele uyilungisile, ndiyathemba. WAL-E, WAL-G okanye Barman. Kwaye nokuba usebenzisa uhlobo oluthile lwesisombululo se-Cloud njenge-RDS okanye i-Cloud SQL, ngoko ungasebenzisa iindawo zokulahla ezinengqiqo. Kodwa sisakucebisa ukuba usebenzise ii-backups, kuba ngale ndlela uya kugcina ubume bomzimba weefayile, okuya kukuvumela ukuba usondele ngakumbi kwiimetriki oza kuzibona kwimveliso ukuze ubambe ezo ngxaki zikhoyo.

• Okwesibini kulapho ufuna ukubamba iDatabase Lab. Inokuba lilifu, inokuba li-On-premise. Kubalulekile ukutsho apha ukuba iZFS ixhasa ukucinezelwa kwedatha. Kwaye yenza kakuhle kakhulu.

Khawucinge ukuba kwi-clone nganye enjalo, kuxhomekeke kwimisebenzi esiyenzayo ngesiseko, uhlobo oluthile lwe-dev luya kukhula. Kule nto, i-dev iya kufuna indawo. Kodwa ngenxa yokuba sithathe isiseko se-4,5 terabytes, i-ZFS iya kuyicinezela kwi-3,5 terabytes. Oku kunokwahluka ngokuxhomekeke kwizicwangciso. Kwaye sisenayo indawo ye-dev.

Inkqubo enjalo ingasetyenziselwa iimeko ezahlukeneyo.

• Aba ngabaphuhlisi, ii-DBA zokuqinisekiswa kwemibuzo, ukulungiselela ukufaneleka.

• Oku kunokusetyenziswa kuvavanyo lwe-QA ukuvavanya imfuduko ethile ngaphambi kokuba siyikhuphele kwimveliso. Kwaye sinokuphakamisa iindawo ezikhethekileyo ze-QA kunye nedatha yangempela, apho banokuvavanya ukusebenza okutsha. Kwaye kuya kuthatha imizuzwana endaweni yeeyure zokulinda, kwaye mhlawumbi iintsuku kwezinye iimeko apho iikopi ezincinci zingasetyenziswanga.

• Kwaye elinye ityala. Ukuba inkampani ayinayo inkqubo yokuhlalutya esetyenzisiweyo, ngoko sinokuhlukanisa i-clone encinci yesiseko semveliso kwaye siyinike imibuzo emide okanye izalathisi ezikhethekileyo ezinokusetyenziswa kuhlalutyo.

Ngale ndlela:

1. Amathuba asezantsi eempazamo kwi-"prod", kuba sivavanye lonke utshintsho kwidatha yesayizi epheleleyo.

2. Sinenkcubeko yokuvavanya, kuba ngoku akufuneki ukuba ulinde iiyure kwi-stand yakho.

3. Kwaye akukho mqobo, akukho kulinda phakathi kweemvavanyo. Ungaya ngokwenene kwaye ujonge. Kwaye kuya kuba ngcono ngale ndlela njengoko sikhawulezisa uphuhliso.

• Kuya kubakho ukuhlengahlengiswa okuncinci. Iibugs ezimbalwa ziyakuphelela kwimveliso. Siza kuzihlaziya kancinci kamva.

• Singakwazi ukubuyisela umva utshintsho olungenakuguqulwa. Oku akusiyo indlela eqhelekileyo.

1. Oku kunenzuzo kuba sabelana ngezibonelelo zeebhentshi zokuvavanya.

Sele kulungile, kodwa yintoni enye enokukhawuleziswa?

Siyabulela kwinkqubo enjalo, sinokunciphisa kakhulu umda wokungena kuvavanyo olunjalo.

Ngoku kukho isangqa esikhohlakeleyo, xa umphuhlisi, ukuze afumane ukufikelela kwidatha epheleleyo, kufuneka abe yingcali. Umele athenjwe ngofikelelo olunjalo.

Kodwa indlela yokukhulisa ukuba ayikho. Kodwa kuthekani ukuba uneseti encinci kakhulu yedatha yovavanyo efumanekayo kuwe? Emva koko awuyi kufumana nawaphi na amava okwenene.

Uphuma njani kwesi sangqa? Njengojongano lokuqala, olulungele abaphuhlisi balo naliphi na inqanaba, sikhethe iSlack bot. Kodwa inokuba naluphi na olunye ujongano.

Ikuvumela ukuba wenze ntoni? Ungathatha umbuzo othile kwaye uwuthumele kwisitishi esikhethekileyo sesiseko sedatha. Siza kuthumela i-clone ebhityileyo ngokuzenzekelayo ngemizuzwana. Masiqhube esi sicelo. Siqokelela iimethrikhi kunye neengcebiso. Masibonise umfanekiso. Kwaye ke le clone iya kuhlala ukuze lo mbuzo uphuculwe ngandlela ithile, yongeza izalathisi, njl.

Kwaye kwakhona uSlack usinika amathuba entsebenziswano ngaphandle kwebhokisi. Kuba esi sisitishi nje, ungaqala ukuxoxa ngesi sicelo kanye apho kumsonto wesicelo esinjalo, ping oogxa bakho, ii-DBA ezingaphakathi kwinkampani.

Kodwa ke, zikho iingxaki. Kuba eli lilizwe lokwenyani, kwaye sisebenzisa iseva ebamba iiclones ezininzi ngaxeshanye, kufuneka sicinezele inani lememori kunye namandla e-CPU akhoyo kwiiclones.

Kodwa ukuze olu vavanyo lube nengqiqo, kufuneka uyisombulule le ngxaki ngandlela thile.

Kucacile ukuba ingongoma ebalulekileyo yidatha efanayo. Kodwa sele sinayo. Kwaye sifuna ukufezekisa uqwalaselo olufanayo. Kwaye sinokunika uqwalaselo oluphantse lufane.

Kuya kuba kuhle ukuba ne-hardware efanayo nanjengemveliso, kodwa inokwahluka.

Masikhumbule indlela iPostgres esebenza ngayo ngememori. Sine-cache ezimbini. Enye ukusuka kwisixokelelwano sefayile kunye nePostgres enye yemveli, o.k.

Kubalulekile ukuqaphela ukuba i-Shared Buffer Cache yabelwe xa i-Postgres iqala, kuxhomekeke kubungakanani obukhankanyiweyo kuqwalaselo.

Kwaye i-cache yesibini isebenzisa yonke indawo ekhoyo.

Kwaye xa sisenza ii-clones ezininzi kumatshini omnye, kuvela ukuba ngokuthe ngcembe sigcwalisa imemori. Kwaye ngendlela elungileyo, i-Shared Buffer Cache yi-25% yexabiso lilonke lememori ekhoyo kumatshini.

Kwaye kuyavela ukuba ukuba asiyitshintshi le parameter, ngoko siya kukwazi ukuqhuba iimeko ezi-4 kuphela kumatshini omnye, oko kukuthi, ezi-4 zala ma-cloni amancinci ewonke. Kwaye oku, ewe, kubi, kuba sifuna ukuba nokunye okuninzi.

Kodwa kwelinye icala, i-Buffer Cache isetyenziselwa ukwenza imibuzo yezalathisi, oko kukuthi, isicwangciso sixhomekeke ekubeni zinkulu kangakanani iicache zethu. Kwaye ukuba sithatha nje le parameter kwaye sinciphise, ngoko ke izicwangciso zethu zinokutshintsha kakhulu.

Umzekelo, ukuba sinogcino olukhulu kwiprod, emva koko iPostgres iya kukhetha ukusebenzisa isalathiso. Kwaye ukuba akunjalo, kuya kubakho iSeqScan. Yaye ibiya kuba yintoni ingongoma ukuba amacebo ethu ebengangqinelani?

Kodwa apha sifika kwisigqibo sokuba ngokwenene isicwangciso kwi-Postgres ayixhomekekanga kubukhulu obuthile obuchazwe kwi-Shared Buffer kwisicwangciso, kuxhomekeke kwi-effect_cache_size.

Effective_cache_size yimali eqikelelweyo yecache ekhoyo kuthi, o.t. isixa seBuffer Cache kunye nefayile yenkqubo yefayile. Oku kusetwa luqwalaselo. Kwaye le nkumbulo ayinikezelwanga.

Kwaye ngenxa yale parameter, sinokwenza iqhinga lePostgres, sisithi eneneni sinedatha eninzi ekhoyo, nokuba asinayo le datha. Kwaye ke, izicwangciso ziya kuhambelana ngokupheleleyo nemveliso.

Kodwa oku kunokuchaphazela ixesha. Kwaye sikhulisa imibuzo ngokubeka ixesha, kodwa kubalulekile ukuba ixesha lixhomekeke kwizinto ezininzi:

• Kuxhomekeke kumthwalo okhoyo ngoku kwi-prod.

• Kuxhomekeke kwiimpawu zomatshini ngokwawo.

Kwaye le yiparamitha engathanga ngqo, kodwa eneneni sinokwandisa kanye ngenani ledatha eliya kufundwa ngulo mbuzo ukuze sifumane isiphumo.

Kwaye ukuba ufuna ukuba ixesha lisondele kwinto esiza kuyibona kwiprod, ke kufuneka sithathe eyona hardware ifanayo kwaye, mhlawumbi, nangakumbi ukuze zonke iiclones zilingane. Kodwa oku kukuchasana, o.k.t. uya kufumana izicwangciso ezifanayo, uya kubona ukuba ingakanani idatha umbuzo othile oza kufunda kwaye uya kuba nakho ukugqiba ukuba ngaba lo mbuzo ulungile (okanye ukufuduka) okanye kubi, kusafuna ukulungiswa. .

Makhe sijonge ukuba uJoe ulungiswa njani ngokuthe ngqo.

Masithathe isicelo kwinkqubo yokwenyani. Kule meko, i-database yi-1 terabyte. Kwaye sifuna ukubala inani lezithuba ezitsha ezithande ngaphezu kwe-10.

Sibhala umyalezo kwitshaneli, i-clone isetyenziselwe thina. Kwaye siya kubona ukuba isicelo esinjalo siya kugqiba kwimizuzu ye-2,5. Le yinto yokuqala esiyiqaphelayo.

U-B Joe uya kukubonisa iingcebiso ezizenzekelayo ngokusekelwe kwisicwangciso kunye neemetriki.

Siza kubona ukuba umbuzo uqhuba idatha eninzi kakhulu ukufumana inani elincinci lemiqolo. Kwaye olunye uhlobo lwesalathiso esikhethekileyo luyafuneka, kuba siqaphele ukuba mininzi kakhulu imiqolo ehluziweyo kumbuzo.

Makhe sihlolisise oko kwenzekayo. Enyanisweni, siyabona ukuba sifunde phantse i-gigabytes enye enesiqingatha yedatha kwi-cache yefayile okanye nakwidiski. Kwaye oku akulunganga, kuba sinemigca eyi-142 kuphela.

Kwaye, kubonakala ngathi, sinesalathisi sokuskena apha kwaye bekufanele ukuba sisebenze ngokukhawuleza, kodwa njengoko sihluze imigca emininzi kakhulu (kufuneka siyibale), isicelo sisebenze kancinane.

Kwaye oku kwenzeka kwisicwangciso ngenxa yokuba iimeko zombuzo kunye neemeko zesalathiso azihambelani.

Masizame ukwenza isalathiso sichaneke ngakumbi kwaye sibone indlela ukuphunyezwa kombuzo otshintsha ngayo emva koko.

Ukwenziwa kwesalathiso kuthathe ixesha elide, kodwa ngoku sijonga umbuzo kwaye sibone ukuba ixesha endaweni ye-2,5 imizuzu yi-156 milliseconds kuphela, elungile ngokwaneleyo. Kwaye sifunda kuphela i-6 megabytes yedatha.

Kwaye ngoku sisebenzisa isalathisi kuphela scan.

Elinye ibali elibalulekileyo kukuba sifuna ukubonisa isicwangciso ngendlela eqondakala ngakumbi. Siphumeze ukubonwa kusetyenziswa iiGrafu zeFlame.

Esi sisicelo esahlukileyo, esiqatha. Kwaye sakha iiGrafu zeFlame ngokweeparamitha ezimbini: le yimali yedatha ukuba i-node ethile ibalwe kwisicwangciso kunye nexesha, oko kukuthi ixesha lokwenziwa kwe-node.

Apha sinokuthelekisa iindawo ezithile kunye nomnye. Kwaye kuya kucaca ukuba yeyiphi kuzo ethatha ngaphezulu okanye ngaphantsi, edla ngokuba nzima ukuyenza kwezinye iindlela zokunikezela.

Ewe, wonke umntu uyazi explain.depesz.com. Inqaku elilungileyo lokubonwayo kukuba sigcina isicwangciso sokubhaliweyo kwaye sikwabeka iiparameters ezisisiseko kwitafile ukuze sikwazi ukuhlela.

Kwaye abaphuhlisi abangekangeneli kwesi sihloko nabo basebenzisa i- explain.depesz.com, kuba kulula kubo ukufumanisa ukuba zeziphi ii-metrics ezibalulekileyo kwaye zeziphi ezingabalulekanga.

Kukho indlela entsha yokujonga - oku kuchaza.dalibo.com. Benza umfanekiso womthi, kodwa kunzima kakhulu ukuthelekisa iindawo zokuhlala kunye nomnye. Apha unokusiqonda kakuhle isakhiwo, nangona kunjalo, ukuba kukho isicelo esikhulu, ngoko kuya kufuneka upheqe ngasemva naphambili, kodwa kwakhona ukhetho.

intsebenziswano

Kwaye, njengoko benditshilo, uSlack usinika ithuba lokusebenzisana. Umzekelo, ukuba sidibana nombuzo ontsokothileyo ongacacanga ukuba ungawenza njani na, singawucacisa lo mbandela kunye noogxa bethu kumsonto weSlack.

Kubonakala ngathi kubalulekile ukuvavanya kwidatha epheleleyo. Ukwenza oku, senze isixhobo seLabhu yoHlaziyo lweDatha, efumaneka kumthombo ovulekileyo. Ungasebenzisa i-Joe bot nayo. Ungayithatha ngoku kwaye uyiphumeze kwindawo yakho. Zonke izikhokelo ziyafumaneka apho.

Kwakhona kubalulekile ukuqaphela ukuba isisombululo ngokwaso asiyi-revolutionary, kuba kukho i-Delphix, kodwa sisisombululo seshishini. Ivalwe ngokupheleleyo, ibiza kakhulu. Sisebenza ngokukodwa kwiPostgres. Zonke ezi ziimveliso zomthombo ovulekileyo. Dibana nathi!

Kulapho ndiphelela khona. Enkosi!

Imibuzo yakho

Mholo! Enkosi ngengxelo! Inika umdla kakhulu, ngakumbi kum, kuba ndisombulule ingxaki efanayo kwixesha elidlulileyo. Kwaye ke ndinenani lemibuzo. Ngethemba ndizakufumana nokuba yinxenye yayo.

Ndiyazibuza ukuba uyibala njani indawo yale ndawo? Itekhnoloji ithetha ukuba phantsi kweemeko ezithile, ii-clones zakho zinokukhula ziye kobona bungakanani obukhulu. Ukuthetha nje, ukuba une-database ye-terabyte elishumi kunye nee-clones ezili-10, ngoko kulula ukulinganisa imeko apho i-clone nganye inobunzima be-10 yedatha eyodwa. Uyibala njani le ndawo, oko kukuthi, laa delta uthethe ngayo, apho ezi clones ziya kuhlala khona?

Umbuzo omhle. Kubalulekile ukugcina umkhondo wee-clones ezithile apha. Kwaye ukuba i-clone inotshintsho olukhulu kakhulu, iqala ukukhula, ngoko sinokukhupha isilumkiso kumsebenzisi malunga noku, okanye simise ngokukhawuleza le clone ukuze singabi nayo imeko yokusilela.

Ewe, ndinombuzo endinawo. Oko kukuthi, uqinisekisa njani umjikelo wobomi bezi modyuli? Sinale ngxaki kunye nebali elahlukileyo. Kwenzeka njani oku?

Kukho i-ttl kwi-clone nganye. Ngokusisiseko, sine-ttl esisigxina.

Kuthekani, ukuba akuyomfihlo?

Iyure eyi-1, oko kukuthi ingasebenzi - 1 iyure. Ukuba ayisetyenziswanga, siyayibetha. Kodwa akukho nto imangalisayo apha, kuba sinokuyinyusa i-clone ngemizuzwana. Kwaye ukuba uyayifuna kwakhona, nceda.

Ndikwanomdla ekukhetheni iteknoloji, kuba, umzekelo, sisebenzisa iindlela ezininzi ngokuhambelana ngenxa yesizathu esinye okanye esinye. Kutheni iZFS? Kutheni ungasebenzisi i-LVM? Ukhankanye ukuba kukho iingxaki nge-LVM. Zaziyintoni iingxaki? Ngokombono wam, olona khetho lufanelekileyo kukugcinwa, ngokwemigaqo yokusebenza.

Yeyiphi eyona ngxaki iphambili nge-ZFS? Inyani yokuba kufuneka ubaleke kumamkeli omnye, okt zonke iimeko ziya kuhlala ngaphakathi kwe-OS efanayo. Kwaye kwimeko yokugcina, unokudibanisa izixhobo ezahlukeneyo. Kwaye i-bottleneck yizo kuphela iibhloko ezikwinkqubo yokugcina. Kwaye umbuzo wokukhethwa kobuchwepheshe unomdla. Kutheni kungenjalo i-LVM?

Ngokukodwa, sinokuxoxa nge-LVM ekudibaneni. Malunga nokugcinwa - kuyabiza nje. Singasebenzisa inkqubo ye-ZFS naphi na. Ungayisebenzisa kumatshini wakho. Unokukhuphela ngokulula indawo yokugcina kwaye uyibeke. I-ZFS ifakwe phantse kuyo yonke indawo ukuba sithetha ngeLinux. Oko kukuthi, sifumana isisombululo esiguquguqukayo kakhulu. Kwaye ngaphandle kwebhokisi, iZFS inika okuninzi. Ungalayisha idatha eninzi njengoko uthanda, qhagamshela inani elikhulu leediski, kukho i-snapshots. Kwaye, njengoko benditshilo, kulula ukuyilawula. Oko kukuthi, kubonakala kumnandi kakhulu ukusebenzisa. Uvavanyiwe, uneminyaka emininzi ubudala. Unoluntu olukhulu kakhulu olukhulayo. I-ZFS sisisombululo esithembeke kakhulu.

U-Nikolai Samokhvalov: Ndingaphinda ndiphawule? Igama lam nguNikolay, sisebenza kunye noAnatoly. Ndiyavuma ukuba ugcino lukhulu. Kwaye abanye abathengi bethu banoGcino olucocekileyo njl.

UAnatoly uphawule ngokuchanekileyo ukuba sigxile kwimodyuli. Kwaye kwixesha elizayo, unokuphumeza enye i-interface - thatha i-snapshot, wenze i-clone, utshabalalise i-clone. Konke kulula. Kwaye ukugcinwa kupholile, ukuba kunjalo.

Kodwa i-ZFS iyafumaneka kuye wonke umntu. I-DelPhix sele yanele, banabathengi abangama-300. Kule, i-fortune 100 inabaxhasi be-50, oko kukuthi bajolise kwi-NASA, njl. Ixesha lokuba wonke umntu afumane le teknoloji. Yiyo loo nto sinomthombo ovulekileyo weCore. Sinendawo yojongano engekho mthombo ovulekileyo. Eli liqonga esiza kulibonisa. Kodwa sifuna ukuba ifikeleleke kumntu wonke. Sifuna ukwenza uhlaziyo ukuze bonke abavavanyi bayeke ukuthelekelela kwiilaptops. Kufuneka sibhale KHETHA kwaye ngokukhawuleza sibone ukuba iyacotha. Yeka ukulinda ukuba i-DBA ikuxelele ngayo. Nantsi eyona njongo iphambili. Kwaye ndicinga ukuba sonke siza kuza kule nto. Kwaye senza le nto ukuba wonke umntu abe nayo. Ke ngoko iZFS, kuba iya kufumaneka kuyo yonke indawo. Enkosi kuluntu ngokusombulula iingxaki kunye nokuba nelayisensi yomthombo ovulekileyo, njl.

Molweni! Enkosi ngengxelo! Igama lam ndinguMaxim. Siye sajongana nemiba efanayo. Bagqiba ngokwabo. Wabelana njani ngezibonelelo phakathi kwezi zixhobo? I-clone nganye inokwenza into yayo nangaliphi na ixesha: omnye uvavanya enye into, enye enye, umntu wakha isalathisi, umntu unomsebenzi onzima. Kwaye ukuba usenokwahlula nge-CPU, ngoko nge-IO, uyahlula njani? Lo ngumbuzo wokuqala.

Kwaye umbuzo wesibini umalunga nokungafani kwezitendi. Masithi ndine-ZFS apha kwaye yonke into ipholile, kodwa umxhasi kwi-prod akanayo i-ZFS, kodwa i-ext4, umzekelo. Njani kule meko?

Imibuzo mihle kakhulu. Ndiyikhankanye le ngxaki kancinci ngento yokuba sabelana ngezibonelelo. Kwaye isisombululo yile. Khawucinge ukuba uvavanya eqongeni. Unokuba nemeko enjalo ngexesha elifanayo ukuba umntu unike umthwalo omnye, omnye umntu. Kwaye ngenxa yoko, ubona i-metrics engaqondakaliyo. Nkqu ingxaki efanayo inokuba kunye prod. Xa ufuna ukujonga isicelo esithile kwaye ubone ukuba kukho ingxaki nayo - isebenza ngokukhawuleza, ngoko ngokwenene ingxaki yayingekho kwisicelo, kodwa kwinto yokuba kukho uhlobo oluthile lomthwalo ohambelanayo.

Kwaye ngoko ke, kubalulekile apha ukuba sigxininise ukuba isicwangciso siya kuba yintoni, ngawaphi amanyathelo esiya kuwathatha kwisicwangciso kunye nokuba ingakanani idatha esiya kuyiphakamisa kule nto. Inyaniso yokuba iidiski zethu, umzekelo, ziya kulayishwa ngento ethile, iya kuchaphazela ngokuthe ngqo ixesha. Kodwa sinokuqikelela ukuba esi sicelo silayishwe kangakanani na ngokwenani ledatha. Akubalulekanga kakhulu ukuba ngexesha elifanayo kuya kubakho uhlobo oluthile lokubulawa.

Ndinemibuzo emibini. Yinto entle kakhulu le. Ngaba kukho iimeko apho idatha yemveliso ibaluleke kakhulu, njengamanani ekhadi letyala? Ngaba sele kukho into esele ilungile okanye ngumsebenzi owahlukileyo? Kwaye umbuzo wesibini-ingaba kukho into efana nale yeMySQL?

Malunga nedatha. Siza kwenza i-obfuscation de senze. Kodwa ukuba uhambisa ngqo uJoe, ukuba awuniki ufikelelo kubaphuhlisi, akukho ukufikelela kwidatha. Ngoba? Kuba uJoe akabonisi datha. Ibonisa kuphela iimethrikhi, izicwangciso kwaye yiloo nto. Oku kwenziwa ngenjongo, kuba le yenye yeemfuno zomxhasi wethu. Babefuna ukukwazi ukukhulisa ngaphandle kokunika wonke umntu ukufikelela.

Malunga neMySQL. Le nkqubo inokusetyenziselwa nantoni na egcina imeko kwidiski. Kwaye kuba sisenza iiPostgres, ngoku senza zonke ii-automation zePostgres kuqala. Sifuna ukwenza ngokuzenzekelayo ukufumana idatha kwi-backup. Siqwalasela iPostgres ngokuchanekileyo. Siyayazi indlela yokwenza izicwangciso zihambelane, njl.

Kodwa ekubeni inkqubo iyandiswa, ingasetyenziselwa iMySQL. Kwaye kukho imizekelo enjalo. IYandex inento efanayo, kodwa abayipapashi naphi na. Bayisebenzisa ngaphakathi kweYandex.Metrica. Kwaye kukho nje ibali malunga ne-MySQL. Kodwa itekhnoloji iyafana, ZFS.

Enkosi ngengxelo! Ndiphinde ndibe nemibuzo emibini. Ukhankanye ukuba i-cloning ingasetyenziselwa uhlalutyo, umzekelo ukwakha izalathisi ezongezelelweyo apho. Ngaba ungaxela ngakumbi kancinci malunga nendlela esebenza ngayo?

Kwaye ngokukhawuleza ndiya kubuza umbuzo wesibini malunga nokufana kwemigangatho, ukufana kwezicwangciso. Isicwangciso sikwaxhomekeke kwiinkcukacha-manani eziqokelelwe yi-Postgres. Uyisombulula njani le ngxaki?

Ngokwe-analytics, akukho matyala athile, kuba asizange siyisebenzise okwangoku, kodwa kukho ithuba elinjalo. Ukuba sithetha ngezalathisi, ngoko khawufane ucinge ukuba umbuzo uleqa itafile enamakhulu ezigidi zeerekhodi kunye nekholamu edla ngokungafakwa kwi-indexed in prod. Kwaye sifuna ukubala idatha ethile apho. Ukuba esi sicelo sithunyelwe kwi-prod, ngoko kukho ithuba lokuba kuya kuba lula kwi-prod, kuba isicelo siya kuqwalaselwa apho umzuzu.

Kulungile, masenze i-clone ebhityileyo engoyikiyo ukuyeka imizuzu embalwa. Kwaye ukwenzela ukuba kube lula ukufunda i-analytics, siya kongeza i-indices kule miqolo apho sinomdla kwidatha.

Isalathiso siyakwenziwa ngexesha ngalinye?

Ungayenza ukuze sichukumise idatha, senze i-snapshots, emva koko siya kubuyisela kule snapshot kwaye siqhube izicelo ezitsha. Oko kukuthi, ungayenza ukuze ukwazi ukuphakamisa ii-clones ezintsha ezine-indices esele zigxunyekiwe.

Ngokuphathelele umbuzo malunga nezibalo, ukuba sibuyisela kwi-backup, ukuba siyaphinda-phinda, izibalo zethu ziya kufana ngqo. Kuba sinalo lonke ulwakhiwo lwedatha yenyama, oko kukuthi, siya kuzisa idatha njengoko injalo kuzo zonke iimethrikhi zezibalo.

Nantsi enye ingxaki. Ukuba usebenzisa isisombululo sefu, ke ukulahla okunengqiqo kuphela kufumaneka apho, kuba i-Google, i-Amazon ayikuvumeli ukuba uthathe ikopi yomzimba. Kuya kubakho ingxaki.

Enkosi ngengxelo. Kwakukho imibuzo emibini elungileyo apha malunga ne-MySQL kunye nokwabelana ngobutyebi. Kodwa, eneneni, konke kuhla kwinto yokuba oku akusiyo isihloko seDBMS ethile, kodwa yenkqubo yefayile iyonke. Kwaye, ngokufanelekileyo, imiba yokwabelana ngezixhobo kufuneka kwakhona isonjululwe ukusuka apho, kungekhona ekugqibeleni ukuba yiPostgres, kodwa kwinkqubo yefayile, kumncedisi, ngokomzekelo.

Umbuzo wam wahlukile kancinci. Isondele kwisiseko sedatha enamaleko amaninzi, apho kukho iileya ezininzi. Umzekelo, siseta uhlaziyo lwe-ten-terabyte yomfanekiso, siphindaphinda. Kwaye sisebenzisa esi sisombululo ngokuthe ngqo kwiziko ledatha. Ukuphindaphinda kuyaqhubeka, idatha iyahlaziywa. Kukho abasebenzi abali-100 abasebenza ngokunxuseneyo apha, abathe gqolo ukuphehlelela ezi foto zahlukeneyo. Kwenziwe ntoni? Indlela yokuqinisekisa ukuba akukho ngquzulwano, ukuba basungule enye, kwaye ke inkqubo yefayile yatshintsha, kwaye le mifanekiso yonke yahamba?

Abazukuhamba kuba yindlela esebenza ngayo iZFS. Singakwazi ukugcina ngokwahlukeneyo kumsonto omnye utshintsho lwenkqubo yefayile oluza ngenxa yokuphindaphinda. Kwaye gcina iiclones ezisetyenziswa ngabaphuhlisi kwiinguqulelo ezindala zedatha. Kwaye kuyasisebenzela, yonke into ilungelelene nale nto.

Kuyavela ukuba uhlaziyo luya kwenzeka njengoluhlu olongezelelweyo, kwaye yonke imifanekiso emitsha iya kuhamba, ngokusekwe kolu luhlu, akunjalo?

Ukusuka kwiileya zangaphambili ebezisuka kwiimpinda zangaphambili.

Iileyile zangaphambili ziya kuwa, kodwa ziya kubhekisa kuluhlu oludala, kwaye ngaba baya kuthatha imifanekiso emitsha kuluhlu lokugqibela olufunyenwe kuhlaziyo?

Ngokubanzi, ewe.

Ngoko ke njengesiphumo siya kuba nekhiwane lomaleko. Kwaye ekuhambeni kwexesha baya kufuna ukucinezelwa?

Ewe yonke into ichanekile. Kukho ifestile. Sigcina i-snapshots ngeveki. Kuxhomekeke kubutyebi obo banabo. Ukuba unakho ukugcina idatha eninzi, unokugcina izifinyezo zexesha elide. Abayi kumka ngokwabo. Akuyi kubakho urhwaphilizo lwedatha. Ukuba i-snapshots iphelelwe yisikhathi, njengoko kubonakala ngathi, oko kukuthi kuxhomekeke kumgaqo-nkqubo kwinkampani, ngoko sinokuzicima ngokulula kwaye sikhulule indawo.

Molo, enkosi ngengxelo! Umbuzo ngoJoe. Uthe umthengi akafuni ukunika wonke umntu ukufikelela kwidatha. Ukuthetha ngokuthe ngqo, ukuba umntu unesiphumo sokuchaza Hlalutya, ngoko unokukhangela idatha.

Kunjalo. Umzekelo, sinokubhala: "KHETHA KUPHI i-imeyile = ukuya kuloo nto". Oko kukuthi, asiyi kubona idatha ngokwayo, kodwa sinokubona ezinye iimpawu ezingathanga ngqo. Oku kufuneka kuqondwe. Kodwa kwelinye icala, konke kukho. Sinophicotho lwelog, sinolawulo lwabanye oogxa nabo babona ukuba benza ntoni abaphuhlisi. Kwaye ukuba umntu uzama ukwenza oku, ke inkonzo yokhuseleko iya kuza kubo kwaye isebenze kulo mbandela.

Mholo Enkosi ngengxelo! Ndinombuzo omfutshane. Ukuba inkampani ayisebenzisi i-Slack, ngaba kukho nayiphi na into ebophelelayo kuyo ngoku, okanye ngaba kunokwenzeka ukuba abaphuhlisi basebenzise iimeko ukuze baqhagamshele isicelo sovavanyo kuvimba weenkcukacha?

Ngoku kukho ikhonkco kuSlack, okt akukho mnye umthunywa, kodwa ndifuna ukwenza inkxaso nakwabanye abathunywa. Ungenzani? Unokusebenzisa i-DB Lab ngaphandle kweJoe, hamba ngoncedo lwe-REST API okanye ngoncedo lweqonga lethu kwaye wenze ii-clones kwaye udibanise ne-PSQL. Kodwa oku kunokwenziwa ukuba ulungele ukunika abaphuhlisi bakho ukufikelela kwidatha, kuba akusayi kuphinda kubekho nasiphi na isikrini.

Andifuni lo maleko, kodwa ndifuna ithuba elinjalo.

Emva koko ewe, inokwenziwa.

umthombo: www.habr.com