Ngabe umthuthukisi we-backend uqonda kanjani ukuthi umbuzo we-SQL uzosebenza kahle "ku-prod"? Ezinkampanini ezinkulu noma ezikhula ngokushesha, akuwona wonke umuntu okwazi ukufinyelela "umkhiqizo". Futhi ngokufinyelela, akuzona zonke izicelo ezingabhekwa ngokungenabuhlungu, futhi ukudala ikhophi yesizindalwazi ngokuvamile kuthatha amahora. Ukuxazulula lezi zinkinga, sidale i-DBA yokwenziwa - Joe. Isivele isetshenziswe ngempumelelo ezinkampanini ezimbalwa futhi isiza onjiniyela abangaphezu kweshumi nambili.
Ividiyo:
Sanibonani nonke! Igama lami ngingu-Anatoly Stansler. Ngisebenzela inkampani . Sizibophezele ekusheshiseni inqubo yokuthuthukisa ngokususa ukubambezeleka okuhlobene nomsebenzi we-Postgres kubathuthukisi, ama-DBA nama-QA.
Sinamakhasimende amahle futhi namuhla ingxenye yombiko izosetshenziswa emacaleni esihlangane nawo ngesikhathi sisebenza nawo. Ngizokhuluma ngendlela esibasize ngayo ukuxazulula izinkinga ezingathi sΓna.
Lapho sithuthukisa futhi senza ukufuduka okuyinkimbinkimbi kwemithwalo ephezulu, sizibuza umbuzo: "Ingabe lokhu kufuduka kuzosuka?". Sisebenzisa ukubuyekeza, sisebenzisa ulwazi lozakwethu abanolwazi olunzulu, ochwepheshe be-DBA. Futhi bayakwazi ukubona ukuthi izondiza noma cha.
Kodwa mhlawumbe kungaba ngcono uma singazivivinya ngokwethu ngamakhophi aphelele. Futhi namuhla sizokhuluma nje ngokuthi yiziphi izindlela zokuhlola manje nokuthi kungenziwa kanjani kangcono futhi ngamaphi amathuluzi. Sizophinde sikhulume ngobuhle nobubi bezindlela ezinjalo, nokuthi yini esingayilungisa lapha.
Ubani owake wenza izinkomba ngokuqondile ku-prod noma wenza izinguquko? Kancane kancane. Futhi lokhu kuholele kubani ekutheni idatha ilahlekile noma kube nesikhathi sokuphumula? Uyabazi-ke lobu buhlungu. Siyabonga uNkulunkulu kukhona ama-backups.
Indlela yokuqala ukuhlola ku-prod. Noma, uma umthuthukisi ehlala emshinini wendawo, unedatha yokuhlola, kukhona uhlobo oluthile lokukhetha okulinganiselwe. Futhi siphuma ukuze sikhiqize, futhi siyasithola lesi simo.
Kubuhlungu, kuyabiza. Kungcono kakhulu ukuthi ungakwenzi.
Futhi iyiphi indlela engcono kakhulu yokukwenza?
Ake sithathe isiteji bese sikhetha ingxenye ethile ye-prod lapho. Noma okungcono kakhulu, ake sithathe i-prod yangempela, yonke idatha. Futhi ngemva kokuba sesiyithuthukisile endaweni, sizophinde sihlole ukuthi idlalwa kanjani.
Lokhu kuzosivumela ukuthi sisuse amanye amaphutha, i.e. ukuwavimbela ekubeni ku-prod.
Ziyini izinkinga?
- Inkinga wukuthi lesi siteji sihlanganyela nozakwethu. Futhi kaningi kwenzeka ukuthi wenze uhlobo oluthile loshintsho, bam - futhi akukho datha, umsebenzi uphansi. Isiteji bekuyi-multi-terabyte. Futhi kufanele ulinde isikhathi eside ukuze ivuke futhi. Futhi sinquma ukukuphothula kusasa. Yilokho-ke, sinentuthuko.
- Futhi, kunjalo, sinozakwethu abaningi abasebenza lapho, amaqembu amaningi. Futhi kufanele kwenziwe ngesandla. Futhi lokhu akulungile.
Futhi kufanelekile ukusho ukuthi sinomzamo owodwa kuphela, isibhamu esisodwa, uma sifuna ukwenza izinguquko ezithile ku-database, thinta idatha, ushintshe isakhiwo. Futhi uma kukhona okungahambanga kahle, uma kube nephutha ekufudukeni, lapho-ke ngeke sihlehle ngokushesha.
Lokhu kungcono kunendlela yangaphambilini, kodwa kusenethuba elikhulu lokuthi uhlobo oluthile lwephutha luzoya ekukhiqizeni.
Yini esivimbela ekunikezeni unjiniyela ngamunye ibhentshi lokuhlola, ikhophi yosayizi ogcwele? Ngicabanga ukuthi kuyacaca ukuthi yini ephazamisayo.
Ubani onesizindalwazi esikhulu kune-terabyte? Igumbi elingaphezu kwesigamu.
Futhi kusobala ukuthi ukugcina imishini kumthuthukisi ngamunye, uma kukhona umkhiqizo omkhulu kangaka, kubiza kakhulu, futhi ngaphandle kwalokho, kuthatha isikhathi eside.
Sinamakhasimende abonile ukuthi kubaluleke kakhulu ukuhlola zonke izinguquko kumakhophi anosayizi ogcwele, kodwa isizindalwazi sawo singaphansi kwe-terabyte, futhi azikho izinsiza zokugcina ibhentshi lokuhlola lomthuthukisi ngamunye. Ngakho-ke, kufanele balande izindawo zokulahla endaweni yangakini emshinini wabo futhi bahlole ngale ndlela. Kuthatha isikhathi esiningi.
Ngisho noma ukwenza ngaphakathi kwengqalasizinda, ukulanda i-terabyte eyodwa yedatha ngehora sekuvele kuhle kakhulu. Kodwa basebenzisa ukulahla okunengqondo, balanda endaweni kusuka efwini. Kubo, isivinini singamagigabhayithi angama-200 ngehora. Futhi kusathatha isikhathi ukujika endaweni yokulahla okunengqondo, ukugoqa izinkomba, njll.
Kodwa basebenzisa le ndlela ngoba ibavumela ukuthi bagcine i-prod ithembekile.
Singenzani lapha? Masenze amabhentshi okuhlola ashibhe futhi sinikeze wonke unjiniyela ibhentshi lakhe lokuhlola.
Futhi lokhu kungenzeka.
Futhi ngale ndlela, lapho senza ama-clone azacile kunjiniyela ngamunye, singabelana ngawo emshinini owodwa. Isibonelo, uma unesizindalwazi se-10TB futhi ufuna ukusinikeza onjiniyela abayi-10, awudingi ukuba nesizindalwazi esingu-XNUMX x XNUMXTB. Udinga umshini owodwa kuphela ukuze wenze amakhophi amancane angawodwa kunjiniyela ngamunye esebenzisa umshini owodwa. Ngizokutshela ukuthi kusebenza kanjani kamuva.
Isibonelo sangempela:
-
DB - 4,5 terabytes.
-
Singathola amakhophi azimele ngemizuzwana engama-30.
Akumele ulinde isitendi sokuhlola futhi uncike ekutheni sikhulu kangakanani. Ungayithola ngemizuzwana. Kuzoba yizindawo ezingazodwa ngokuphelele, kodwa ezabelana ngedatha phakathi kwazo.
Kuhle lokhu. Lapha sikhuluma ngomlingo kanye nendawo yonke ehambisanayo.
Esimweni sethu, lokhu kusebenza kusetshenziswa uhlelo lwe-OpenZFS.
I-OpenZFS iwuhlelo lwefayela lokukopisha-phezu kokubhala olusekela izifinyezo nama-clones ngaphandle kwebhokisi. Inokwethenjelwa futhi iyakaleka. Kulula kakhulu ukuphatha. Ingatshalwa ngokoqobo ngamaqembu amabili.
Kukhona ezinye izinketho:
-
I-LVM,
-
Isitoreji (isibonelo, Isitoreji Esihlanzekile).
I-Database Lab engikhuluma ngayo i-modular. Kungenziwa ngokusebenzisa lezi zinketho. Kodwa okwamanje, sigxile ku-OpenZFS, ngoba kube nezinkinga nge-LVM ngokuqondile.
Isebenza kanjani? Esikhundleni sokusula idatha njalo uma siyishintsha, siyilondoloza ngokumane siphawule ukuthi le datha entsha isuka endaweni entsha ngesikhathi, isifinyezo esisha.
Futhi ngokuzayo, uma sifuna ukubuyisela emuva noma sifuna ukwenza i-clone entsha kusuka kwenye inguqulo endala, sivele sithi: "KULUNGILE, sinikeze la mabhulokhi edatha amakwe kanje."
Futhi lo msebenzisi uzosebenza nesethi yedatha enjalo. Uzowashintsha kancane kancane, enze ezakhe izifinyezo.
Futhi sizogxila. Unjiniyela ngamunye kithi uzoba nethuba lokuba neyakhe imodeli ayihlelayo, futhi idatha eyabiwayo izokwabelwa wonke umuntu.
Ukuze usebenzise uhlelo olunjalo ekhaya, udinga ukuxazulula izinkinga ezimbili:
-
Okokuqala umthombo wedatha, lapho uzoyithatha khona. Ungasetha ukuphindaphinda ngokukhiqiza. Usuvele usebenzise izipele ozimisile, ngiyethemba. I-WAL-E, i-WAL-G noma i-Barman. Futhi noma ngabe usebenzisa uhlobo oluthile lwesixazululo se-Cloud njenge-RDS noma i-Cloud SQL, ungasebenzisa ukulahla okunengqondo. Kodwa sisakweluleka ukuthi usebenzise ama-backups, ngoba ngale ndlela uzophinde ugcine ukwakheka komzimba kwamafayela, okuzokuvumela ukuthi usondele nakakhulu kumamethrikhi ongawabona ekukhiqizeni ukuze ubambe lezo zinkinga ezikhona.
-
Okwesibili yilapho ufuna ukusingatha i-Database Lab. Kungaba Cloud, kungaba On-premise. Kubalulekile ukusho lapha ukuthi i-ZFS isekela ukucindezelwa kwedatha. Futhi ikwenza kahle impela.
Cabanga ukuthi ku-clone ngayinye enjalo, kuye ngemisebenzi esiyenzayo ngesisekelo, uhlobo oluthile lwe-dev luzokhula. Kulokhu, i-dev izodinga nesikhala. Kodwa ngenxa yokuthi sithathe isisekelo esingu-4,5 terabytes, i-ZFS izoyicindezela ibe ngu-3,5 terabytes. Lokhu kungahluka kuye ngezilungiselelo. Futhi sisenayo indawo ye-dev.
Uhlelo olunjalo lungasetshenziselwa amacala ahlukene.
-
Laba ngonjiniyela, ama-DBA okuqinisekisa umbuzo, ukuze kusetshenziswe ngokugcwele.
-
Lokhu kungasetshenziswa ekuhlolweni kwe-QA ukuhlola ukufuduka okuthile ngaphambi kokuthi sikukhiphele ekukhiqizeni. Futhi singaphakamisa izindawo ezikhethekile ze-QA ngedatha yangempela, lapho zingahlola khona ukusebenza okusha. Futhi kuzothatha imizuzwana esikhundleni samahora okulinda, futhi mhlawumbe nezinsuku kwezinye izimo lapho amakhophi amancane angasetshenziswa khona.
-
Futhi elinye icala. Uma inkampani ingenalo uhlelo lokuhlaziya olumisiwe, singase sihlukanise i-clone encane yesisekelo somkhiqizo futhi siyinikeze imibuzo emide noma izinkomba ezikhethekile ezingasetshenziswa kuzibalo.
Ngale ndlela:
-
Amathuba aphansi amaphutha ku-"prod", ngoba sihlole zonke izinguquko kudatha yosayizi ogcwele.
-
Sinesiko lokuhlola, ngoba manje awudingi ukulinda amahora amaningi ukuze uthole okwakho ukuma.
-
Futhi asikho isithiyo, akukho ukulinda phakathi kokuhlolwa. Empeleni ungahamba uyohlola. Futhi kuzoba ngcono ngale ndlela njengoba sisheshisa intuthuko.
-
Kuzoba nokuhlehlisa okuncane. Iziphazamisi ezimbalwa zizogcina ku-prod. Sizowenza kabusha kancane kamuva.
-
Singahlehlisa izinguquko ezingenakuhlehliswa. Lena akuyona indlela ejwayelekile.
- Lokhu kunenzuzo ngoba sabelana ngezisetshenziswa zamabhentshi okuhlola.
Kuhle kakade, kodwa yini enye engasheshiswa?
Ngenxa yesistimu enjalo, singanciphisa kakhulu umkhawulo wokungena ekuhlolweni okunjalo.
Manje sekunomjikelezo ononya, lapho umthuthukisi, ukuze afinyelele kudatha yosayizi ogcwele wangempela, kufanele abe uchwepheshe. Kumelwe athembeke ngokufinyelela okunjalo.
Kodwa indlela yokukhula uma ingekho. Kodwa kuthiwani uma unesethi encane kakhulu yedatha yokuhlola etholakalayo kuwe? Khona-ke ngeke uthole ulwazi lwangempela.
Ungaphuma kanjani kulo mbuthano? Njengesixhumi esibonakalayo sokuqala, esilungele abathuthukisi banoma yiliphi izinga, sikhethe i-Slack bot. Kodwa kungaba noma iyiphi enye i-interface.
Ikuvumela ukuthi wenzeni? Ungathatha umbuzo othile futhi uwuthumele esiteshini esikhethekile sesizindalwazi. Sizosebenzisa ngokuzenzakalelayo i-clone emincane ngemizuzwana. Asiqhube lesi sicelo. Siqoqa amamethrikhi nezincomo. Masibonise umbono. Bese-ke le clone izohlala ukuze lo mbuzo uthuthukiswe ngandlela thile, engeza izinkomba, njll.
Futhi i-Slack isinika amathuba okubambisana ngaphandle kwebhokisi. Njengoba lesi kuyisiteshi nje, ungaqala ukuxoxa ngalesi sicelo khona lapho ochungechungeni lwesicelo esinjalo, ngokucindezela ozakwenu, ama-DBA angaphakathi enkampanini.
Kodwa zikhona izinkinga. Ngoba lona umhlaba wangempela, futhi sisebenzisa iseva ebamba ama-clone amaningi ngesikhathi esisodwa, kufanele sicindezele inani lememori namandla e-CPU atholakala kuma-clones.
Kodwa ukuze lezi zivivinyo zibe sobala, udinga ukuxazulula le nkinga ngandlela thile.
Kuyacaca ukuthi iphuzu elibalulekile idatha efanayo. Kodwa sesinayo kakade. Futhi sifuna ukufeza ukucushwa okufanayo. Futhi singanikeza ukucushwa okucishe kufane.
Kungaba kuhle ukuba ne-hardware efanayo naleyo ekukhiqizeni, kodwa ingase ihluke.
Masikhumbule ukuthi i-Postgres isebenza kanjani ngenkumbulo. Sinezinqolobane ezimbili. Eyodwa evela kusistimu yefayela kanye ne-Postgres eyodwa yomdabu, okungukuthi Inqolobane Yesigcinalwazi Sabiwe.
Kubalulekile ukuqaphela ukuthi i-Shared Buffer Cache inikezwa uma i-Postgres iqala, kuye ngokuthi ucacisa usayizi onjani ekucushweni.
Futhi inqolobane yesibili isebenzisa sonke isikhala esitholakalayo.
Futhi lapho senza ama-clone amaningana emshinini owodwa, kuvela ukuthi kancane kancane sigcwalisa inkumbulo. Futhi ngendlela enhle, i-Shared Buffer Cache ingu-25% yenani eliphelele lememori elitholakala emshinini.
Futhi kuvela ukuthi uma singashintshi le parameter, khona-ke sizokwazi ukusebenzisa izimo ezi-4 kuphela emshinini owodwa, okungukuthi, ama-4 alawa ma-clones azacile esewonke. Futhi lokhu, kunjalo, kubi, ngoba sifuna ukuba nokunye okuningi.
Kodwa ngakolunye uhlangothi, i-Buffer Cache isetshenziselwa ukwenza imibuzo yezinkomba, okungukuthi, uhlelo luncike ekutheni makhulu kangakanani ama-caches ethu. Futhi uma nje sithatha le parameter futhi siyinciphisa, khona-ke izinhlelo zethu zingashintsha kakhulu.
Isibonelo, uma sinenqolobane enkulu ku-prod, i-Postgres izokhetha ukusebenzisa inkomba. Futhi uma kungenjalo, khona-ke kuzoba ne-SeqScan. Futhi kungaba yini iphuzu uma izinhlelo zethu zingahlangani?
Kodwa lapha sifika esiphethweni sokuthi empeleni uhlelo oluku-Postgres aluncikile kusayizi othize oshiwo ku-Shared Buffer ohlelweni, kuncike ku-effect_cache_size.
I-Effective_cache_size yinani elilinganiselwe lenqolobane elitholakala kithi, okungukuthi isamba se-Buffer Cache kanye nenqolobane yesistimu yefayela. Lokhu kusethwa yi-config. Futhi le nkumbulo ayinikezwanga.
Futhi ngenxa yale pharamitha, singakhohlisa ama-Postgres, sithi empeleni sinedatha eningi etholakalayo, noma singenayo le datha. Futhi ngaleyo ndlela, izinhlelo zizohambisana ngokuphelele nokukhiqiza.
Kodwa lokhu kungathinta isikhathi. Futhi sithuthukisa imibuzo ngokubeka isikhathi, kodwa kubalulekile ukuthi isikhathi sincike ezintweni eziningi:
-
Kuncike ekulayisheni okwamanje okuku-prod.
-
Kuncike ezicini zomshini ngokwawo.
Futhi lena ipharamitha engaqondile, kodwa empeleni singakwazi ukukhulisa ngokuthe ngqo ngenani ledatha elizofundwa yilo mbuzo ukuze sithole umphumela.
Futhi uma ufuna ukuthi isikhathi sisondele kulokho esizokubona ku-prod, ngakho-ke sidinga ukuthatha ihadiwe efanayo kakhulu futhi, ngokunokwenzeka, nangaphezulu ukuze wonke ama-clone alingane. Kodwa lokhu kuvumelana, okungukuthi uzothola izinhlelo ezifanayo, uzobona ukuthi ingakanani idatha umbuzo othile ozofundwa futhi uzokwazi ukuphetha ngokuthi lo mbuzo muhle (noma ukufuduka) noma mubi, usadinga ukuthuthukiswa. .
Ake sibheke ukuthi uJoe uthuthukiswa kanjani ngokukhethekile.
Ake sithathe isicelo ohlelweni lwangempela. Kulokhu, i-database ingu-1 terabyte. Futhi sifuna ukubala inani lamaposi amasha abe nokuthandwa okungaphezu kuka-10.
Sibhalela isiteshi umlayezo, umlingisi ujutshelwe thina. Futhi sizobona ukuthi isicelo esinjalo sizoqeda emaminithini angu-2,5. Lokhu kuyinto yokuqala esiyiqaphelayo.
U-B Joe uzokukhombisa izincomo ezizenzakalelayo ezisuselwe kuhlelo namamethrikhi.
Sizobona ukuthi umbuzo ucubungula idatha eningi kakhulu ukuze uthole inombolo encane yemigqa. Futhi uhlobo oluthile lwenkomba olukhethekile luyadingeka, njengoba siqaphele ukuthi kunemigqa ehlungiwe eminingi kakhulu embuzweni.
Ake sihlolisise okwenzeka. Ngempela, siyabona ukuthi sifunde cishe i-gigabytes eyodwa nengxenye yedatha kusuka kunqolobane yefayela noma ngisho nakudiski. Futhi lokhu akukuhle, ngoba sinemigqa eyi-142 kuphela.
Futhi, kubonakala sengathi sinokuskena kwenkomba lapha futhi bekufanele sisebenze ngokushesha, kodwa njengoba sihlunge imigqa eminingi kakhulu (bekumele siyibale), umbuzo usebenze kancane.
Futhi lokhu kwenzeke ohlelweni ngenxa yokuthi izimo embuzweni kanye nezimo zenkomba azifani.
Ake sizame ukwenza inkomba inembe kakhudlwana futhi sibone ukuthi ukusebenza kombuzo kushintsha kanjani ngemva kwalokho.
Ukwakhiwa kwenkomba kuthathe isikhathi eside impela, kodwa manje sibheka umbuzo futhi sibone ukuthi isikhathi esikhundleni semizuzu engu-2,5 singama-millisecond angu-156 kuphela, okwanele. Futhi sifunda kuphela amamegabhayithi angu-6 edatha.
Futhi manje sisebenzisa inkomba kuphela scan.
Enye indaba ebalulekile ukuthi sifuna ukwethula uhlelo ngendlela eqondakala kakhudlwana. Sisebenzise ukubonwa ngeso sisebenzisa i-Flame Graph.
Lesi isicelo esihlukile, esiqine kakhulu. Futhi sakha amaGrafu e-Flame ngokuya ngemingcele emibili: leli inani ledatha i-node ethile ebalwe ohlelweni nesikhathi, okungukuthi isikhathi sokwenza se-node.
Lapha singaqhathanisa ama-node athile nomunye nomunye. Futhi kuzocaca ukuthi yimuphi kubo othatha okungaphezulu noma okuncane, okuvame ukuba nzima ukukwenza kwezinye izindlela zokunikeza.
Yebo, wonke umuntu uyazi explain.depesz.com. Isici esihle salokhu kubukwa ngeso ukuthi sigcina uhlelo lombhalo futhi siphinde sibeke imingcele ethile etafuleni ukuze sikwazi ukuhlela.
Futhi onjiniyela abangakagxili kulesi sihloko baphinde basebenzise i- explain.depesz.com, ngoba kulula kubo ukuthola ukuthi yimaphi ama-metrics abalulekile nokuthi yimaphi angabalulekile.
Kukhona indlela entsha yokubuka - lena yi-explain.dalibo.com. Benza ukubonakala kwesihlahla, kodwa kunzima kakhulu ukuqhathanisa ama-node nomunye nomunye. Lapha ungakwazi ukuqonda kahle isakhiwo, noma kunjalo, uma kunesicelo esikhulu, uzodinga ukupheqa emuva naphambili, kodwa futhi inketho.
ukusebenzisana
Futhi, njengoba ngishilo, uSlack usinika ithuba lokuhlanganyela. Isibonelo, uma sihlangabezana nombuzo oyinkimbinkimbi ongacacile ukuthi ungasebenza kanjani ngokugcwele, singalucacisa lolu daba nozakwethu kuchungechunge lwe-Slack.
Kithina kubonakala kubalulekile ukuhlola kudatha yosayizi ogcwele. Ukuze senze lokhu, senze ithuluzi le-Update Database Lab, elitholakala kumthombo ovulekile. Ungasebenzisa futhi i-Joe bot. Ungakuthatha njengamanje futhi ukusebenzise endaweni yakho. Yonke imihlahlandlela iyatholakala lapho.
Kubalulekile futhi ukuqaphela ukuthi isixazululo ngokwaso akusona izinguquko, ngoba kukhona i-Delphix, kodwa yisisombululo sebhizinisi. Ivalwe ngokuphelele, ibiza kakhulu. Sisebenza ngokukhethekile ku-Postgres. Yonke lena imikhiqizo yomthombo ovulekile. Sijoyine!
Yilapho engigcina khona. Ngiyabonga!
Imibuzo yakho
Sawubona! Siyabonga ngombiko! Kuyathakazelisa kakhulu, ikakhulukazi kimi, ngoba ngaxazulula inkinga efanayo esikhathini esithile esidlule. Futhi ngakho nginemibuzo eminingana. Ngethemba ukuthi ngizothola okungenani ingxenye yakho.
Kazi uyibala kanjani indawo yale ndawo? Ubuchwepheshe busho ukuthi ngaphansi kwezimo ezithile, ama-clones akho angakhula afike kusayizi omkhulu. Uma sikhuluma nje, uma unesizindalwazi se-terabyte eyishumi kanye nama-clone angu-10, kulula-ke ukulingisa isimo lapho i-clone ngayinye inesisindo sedatha eyi-10 eyingqayizivele. Ubala kanjani le ndawo, okungukuthi, leyo delta okhulume ngayo, lapho la ma-clone azohlala kuyo?
Umbuzo omuhle. Kubalulekile ukulandelela ama-clones athile lapha. Futhi uma i-clone inoshintsho olukhulu kakhulu, iqala ukukhula, lapho-ke singaqala sikhiphe isexwayiso kumsebenzisi mayelana nalokhu, noma simise ngokushesha le clone ukuze singabi nesimo sokuhluleka.
Yebo, nginombuzo owodwa. Okusho ukuthi, uwuqinisekisa kanjani umjikelezo wempilo walawa mamojula? Sinale nkinga kanye nendaba ehlukene. Kwenzeka kanjani lokhu?
Kukhona i-ttl ye-clone ngayinye. Ngokuyisisekelo, sine-ttl elungisiwe.
Kuthiwani, uma kungeyona imfihlo?
Ihora elingu-1, i.e. engenzi lutho - ihora elingu-1. Uma ingasetshenziswa, siyayishaya. Kodwa akumangazi lapha, njengoba singakwazi ukuphakamisa i-clone ngemizuzwana. Futhi uma uyidinga futhi, sicela.
Ngiphinde nginesithakazelo ekukhethweni kobuchwepheshe, ngoba, isibonelo, sisebenzisa izindlela eziningana ngokuhambisana ngesizathu esisodwa noma esinye. Kungani i-ZFS? Kungani ungazange usebenzise i-LVM? Ushilo ukuthi kube nezinkinga nge-LVM. Zaziyini izinkinga? Ngokubona kwami, inketho efanelekile kakhulu ngesitoreji, ngokuya ngokusebenza.
Iyiphi inkinga enkulu nge-ZFS? Iqiniso lokuthi kufanele ugijime kumsingathi ofanayo, okungukuthi zonke izimo zizophila ngaphakathi kwe-OS efanayo. Futhi esimweni sokugcina, ungaxhuma imishini ehlukene. Futhi i-bottleneck yilawo mabhulokhi kuphela akusistimu yokugcina. Futhi umbuzo wokukhethwa kobuchwepheshe uyathakazelisa. Kungani kungenjalo i-LVM?
Ngokukhethekile, singaxoxa nge-LVM emhlanganweni. Mayelana nesitoreji - kuyabiza nje. Singasebenzisa uhlelo lweZFS noma yikuphi. Ungawusebenzisa emshinini wakho. Ungamane ulande inqolobane bese uyiphakela. I-ZFS ifakwe cishe yonke indawo uma sikhuluma ngeLinux. Okusho ukuthi, sithola isisombululo esivumelana nezimo kakhulu. Futhi i-ZFS ngokwayo inikeza okuningi ngaphandle kwebhokisi. Ungalayisha idatha eningi ngendlela othanda ngayo, uxhume inombolo enkulu yamadiski, kukhona izifinyezo. Futhi, njengoba ngishilo, kulula ukuphatha. Okusho ukuthi, kubonakala kumnandi kakhulu ukusebenzisa. Uhlolwe, uneminyaka eminingi. Unomphakathi omkhulu kakhulu okhulayo. I-ZFS iyisixazululo esithembeke kakhulu.
U-Nikolai Samokhvalov: Ngicela ukuphawula okwengeziwe? Igama lami nginguNikolay, sisebenza ndawonye no-Anatoly. Ngiyavuma ukuthi isitoreji sikhulu. Futhi amanye amakhasimende ethu aneSitoreji Esihlanzekile njll.
U-Anatoly uphawule kahle ukuthi sigxile ekumodemeni. Futhi ngokuzayo, ungasebenzisa isikhombimsebenzisi esisodwa - thatha isifinyezo, wenze i-clone, ucekele phansi i-clone. Konke kulula. Futhi isitoreji sipholile, uma kunjalo.
Kodwa i-ZFS itholakala kuwo wonke umuntu. I-DelPhix isivele yanele, inamakhasimende angama-300. Kulezi, i-fortune 100 inamakhasimende angu-50, okungukuthi ihloselwe i-NASA, njll. Isikhathi sokuthi wonke umuntu athole lobu buchwepheshe. Futhi yingakho sinomthombo ovulekile Core. Sinengxenye yesixhumi esibonakalayo esingewona umthombo ovulekile. Lena inkundla esizoyikhombisa. Kodwa sifuna ukuthi ifinyeleleke kuwo wonke umuntu. Sifuna ukwenza uguquko ukuze bonke abahloli bayeke ukuqagela kumakhompyutha aphathekayo. Kufanele sibhale KHETHA futhi ngokushesha sibone ukuthi ihamba kancane. Yeka ukulinda i-DBA ukuthi ikutshele ngakho. Nansi umgomo oyinhloko. Futhi ngicabanga ukuthi sonke sizofika kulokhu. Futhi senza le nto ukuze wonke umuntu abe nayo. Ngakho-ke i-ZFS, ngoba izotholakala yonke indawo. Sibonga umphakathi ngokuxazulula izinkinga nokuba nelayisensi yomthombo ovulekile, njll.*
Sanibonani! Siyabonga ngombiko! Igama lami nginguMaxim. Sike sabhekana nezindaba ezifanayo. Bazinqumela bodwa. Wabelana kanjani ngezinsiza phakathi kwalawa ma-clones? I-clone ngayinye ingenza into yayo nganoma yisiphi isikhathi: omunye uvivinya into eyodwa, enye enye, omunye wakha inkomba, omunye unomsebenzi onzima. Futhi uma usengakwazi ukuhlukanisa nge-CPU, bese nge-IO, uhlukanisa kanjani? Lona umbuzo wokuqala.
Futhi umbuzo wesibili umayelana nokungafani kwezitendi. Ake sithi ngine-ZFS lapha futhi yonke into ipholile, kodwa iklayenti ku-prod ayinayo i-ZFS, kodwa i-ext4, isibonelo. Kanjani kulokhu?
Imibuzo mihle kakhulu. Ngikhulume le nkinga kancane ngokuthi sabelana ngezinsiza. Futhi isixazululo yilesi. Zicabange uhlola esiteji. Ungase futhi ube nesimo esinjalo ngesikhathi esifanayo ukuthi othile unikeza omunye umthwalo, omunye umuntu. Futhi ngenxa yalokho, ubona amamethrikhi angaqondakali. Ngisho nenkinga efanayo ingaba nge-prod. Uma ufuna ukuhlola isicelo esithile futhi ubona ukuthi kunenkinga ethile ngakho - isebenza kancane, khona-ke empeleni inkinga yayingekho esicelweni, kodwa eqinisweni lokuthi kukhona uhlobo oluthile lomthwalo ofanayo.
Ngakho-ke, kubalulekile lapha ukugxila kulokho uhlelo oluzoba khona, yiziphi izinyathelo esizozithatha ohlelweni nokuthi ingakanani idatha esizoyiphakamisa ngalokhu. Iqiniso lokuthi amadiski ethu, ngokwesibonelo, azolayishwa ngokuthile, azothinta ngokuqondile isikhathi. Kodwa singalinganisela ukuthi lesi sicelo silayishwe kangakanani ngenani ledatha. Akubalulekile kakhulu ukuthi ngesikhathi esifanayo kuzoba nohlobo oluthile lokubulawa.
Nginemibuzo emibili. Izinto ezinhle kakhulu lezi. Ingabe kube khona izimo lapho idatha yokukhiqiza ibalulekile, njengezinombolo zekhadi lesikweletu? Ingabe kukhona osekulungile kakade noma kuwumsebenzi ohlukile? Futhi umbuzo wesibili - ingabe kukhona okufana nalokhu ku-MySQL?
Mayelana nedatha. Sizokwenza obfuscation kuze sikwenze. Kodwa uma uphakela ngqo uJoe, uma ungabaniki ukufinyelela konjiniyela, akukho ukufinyelela kudatha. Kungani? Ngoba uJoe akabonisi idatha. Ibonisa kuphela amamethrikhi, izinhlelo futhi yilokho kuphela. Lokhu kwenziwa ngenhloso, ngoba lokhu kungenye yezidingo zeklayenti lethu. Babefuna ukukwazi ukuthuthukisa ngaphandle kokunikeza wonke umuntu ukufinyelela.
Mayelana ne-MySQL. Lolu hlelo lungasetshenziswa kunoma yini egcina isimo kudiski. Futhi njengoba senza i-Postgres, manje sesenza konke okuzenzakalelayo kwe-Postgres kuqala. Sifuna ukwenza ngokuzenzakalelayo ukuthola idatha kukhophi yasenqolobaneni. Silungiselela i-Postgres ngendlela efanele. Siyazi ukwenza izinhlelo zifane, njll.
Kodwa njengoba uhlelo lunwebeka, lungasetshenziswa naku-MySQL. Futhi zikhona izibonelo ezinjalo. I-Yandex inento efanayo, kodwa abayishicileli noma kuphi. Bayisebenzisa ngaphakathi kwe-Yandex.Metrica. Futhi kukhona nje indaba mayelana ne-MySQL. Kodwa ubuchwepheshe buyafana, ZFS.
Siyabonga ngombiko! Ngiphinde ngibe nemibuzo embalwa. Ushilo ukuthi i-cloning ingasetshenziselwa izibalo, isibonelo ukwakha izinkomba ezengeziwe lapho. Ungakwazi ukusho okwengeziwe mayelana nokuthi isebenza kanjani?
Futhi ngokushesha ngizobuza umbuzo wesibili mayelana nokufana kwezitendi, ukufana kwezinhlelo. Uhlelo luphinde luncike ezibalweni eziqoqwe yi-Postgres. Uyixazulula kanjani le nkinga?
Ngokusho kokuhlaziywa, awekho amacala athile, ngoba asikasebenzisi okwamanje, kodwa kukhona ithuba elinjalo. Uma sikhuluma ngezinkomba, cabanga-ke ukuthi umbuzo ujaha ithebula elinamakhulu ezigidi zamarekhodi kanye nekholomu evamise ukungeniswa ku-prod. Futhi sifuna ukubala idatha ethile lapho. Uma lesi sicelo sithunyelwa ku-prod, khona-ke kukhona okungenzeka ukuthi kuzoba lula ku-prod, ngoba isicelo sizocutshungulwa lapho umzuzu.
Kulungile, masenze i-clone elincanyana elingebi ukumisa imizuzu embalwa. Futhi ukuze sikwenze kukhululeke kakhudlwana ukufunda izibalo, sizofaka izinkomba zalawo makholomu esinentshisekelo kuwo kudatha.
Inkomba izodalwa isikhathi ngasinye?
Ungayenza ukuze sithinte idatha, senze izifinyezo, bese sizolulama kulesi sifinyezo futhi sishayele izicelo ezintsha. Okusho ukuthi, ungayenza ukuze ukwazi ukukhulisa ama-clone amasha anezinkomba ezivele zinamathiselwe.
Mayelana nombuzo omayelana nezibalo, uma sibuyisela kukhophi yasenqolobaneni, uma senza ukuphindaphinda, izibalo zethu zizofana ncamashi. Ngoba sinaso sonke isakhiwo sedatha ebonakalayo, okungukuthi, sizoletha idatha njengoba injalo nawo wonke ama-metrics ezibalo.
Nansi enye inkinga. Uma usebenzisa isisombululo sefu, khona-ke ukulahlwa okunengqondo kuphela kutholakala lapho, ngoba i-Google, i-Amazon ayikuvumeli ukuthi uthathe ikhophi ebonakalayo. Kuzoba nenkinga.
Siyabonga ngombiko. Kube nemibuzo emibili emihle lapha mayelana ne-MySQL nokwabelana ngezisetshenziswa. Kodwa, empeleni, konke kwehla eqinisweni lokuthi lesi akusona isihloko se-DBMS ethile, kodwa yesistimu yefayela iyonke. Futhi, ngokufanele, izindaba zokwabelana ngezinsiza kufanele futhi zixazululwe kusukela lapho, hhayi ekugcineni ukuthi yi-Postgres, kodwa ohlelweni lwefayela, kuseva, ngokwesibonelo.
Umbuzo wami uhlukile kancane. Iseduze nesizindalwazi esinezingqimba eziningi, lapho kunezendlalelo ezimbalwa. Isibonelo, simisa isibuyekezo sesithombe se-terabyte eyishumi, siyaphindaphinda. Futhi sisebenzisa lesi sixazululo kusizindalwazi. Ukuphindaphinda kuyaqhubeka, idatha iyabuyekezwa. Kunabasebenzi abayi-100 abasebenza ngokufana lapha, abahlale bethula lezi zithombe ezahlukahlukene. Okufanele ngikwenze? Indlela yokuqinisekisa ukuthi akukho ukungqubuzana, ukuthi baqalise eyodwa, bese uhlelo lwefayela lushintsha, futhi lezi zithombe zahamba zonke?
Ngeke bahambe ngoba iZFS isebenza kanjalo. Singagcina ngokuhlukene kuchungechunge olulodwa izinguquko zesistimu yefayela eziza ngenxa yokuphindaphinda. Futhi gcina ama-clones asetshenziswa onjiniyela ezinguqulweni ezindala zedatha. Futhi kuyasisebenzela, yonke into ihlelekile ngalokhu.
Kuvela ukuthi isibuyekezo sizokwenzeka njengesendlalelo esengeziwe, futhi zonke izithombe ezintsha zizohamba kakade, ngokusekelwe kulesi sendlalelo, akunjalo?
Kusuka kuzendlalelo zangaphambilini ebezisuka kuzimpinda zangaphambilini.
Izendlalelo zangaphambilini zizowa, kodwa zizobhekisela kusendlalelo esidala, futhi ingabe zizothatha izithombe ezintsha kusendlalelo sokugcina esamukelwe kusibuyekezo?
Ngokuvamile, yebo.
Khona-ke ngenxa yalokho sizoba nekhiwane lezingqimba. Futhi ngokuhamba kwesikhathi bazodinga ukucindezelwa?
Yebo konke kulungile. Kukhona iwindi. Sigcina izifinyezo zamasonto onke. Kuya ngokuthi unayiphi insiza. Uma unamandla okugcina idatha eningi, ungagcina izifinyezo isikhathi eside. Ngeke bazihambele. Ngeke kube khona ukonakala kwedatha. Uma izifinyezo zidlulelwe yisikhathi, njengoba kubonakala kithi, okungukuthi kuncike kunqubomgomo enkampanini, lapho-ke singavele sizisuse futhi sikhulule isikhala.
Sawubona, ngiyabonga ngombiko! Umbuzo ngoJoe. Uthe ikhasimende belingafuni ukunikeza wonke umuntu ukufinyelela kudatha. Uma sikhuluma ngokuqinile, uma umuntu enomphumela we-Chaza Hlaziya, khona-ke angakwazi ukuhlola idatha.
Kunjalo. Isibonelo, singabhala: "KHETHA KUSUKA LAPHO i-imeyili = kuya kulokho". Okusho ukuthi, ngeke sibone idatha ngokwayo, kodwa singabona ezinye izimpawu ezingaqondile. Lokhu kumele kuqondwe. Kodwa ngakolunye uhlangothi, konke kukhona. Sinokuhlolwa kwelogi, sinokulawula abanye ozakwethu nabo ababona ukuthi onjiniyela benzani. Futhi uma othile ezama ukwenza lokhu, khona-ke isevisi yezokuphepha izofika kubo futhi isebenze kulolu daba.
Sawubona Siyabonga ngombiko! Nginombuzo omfushane. Uma inkampani ingasebenzisi i-Slack, ingabe kukhona okuyibophezela kukho manje, noma kungenzeka yini ukuthi abathuthukisi basebenzise izimo ukuze baxhume uhlelo lokusebenza lokuhlola kusizindalwazi?
Manje sekunesixhumanisi esiya ku-Slack, okungukuthi asikho esinye isithunywa, kodwa ngifuna ngempela ukweseka nezinye izithunywa. Yini ongayenza? Ungakwazi ukusebenzisa i-DB Lab ngaphandle kuka-Joe, hamba ngosizo lwe-REST API noma ngosizo lweplatifomu yethu futhi udale ama-clones futhi uxhumane ne-PSQL. Kodwa lokhu kungenziwa uma usukulungele ukunikeza onjiniyela bakho ukufinyelela kudatha, ngoba ngeke kusaba khona isikrini.
Angiludingi lolu ungqimba, kodwa ngidinga ithuba elinjalo.
Khona-ke yebo, kungenziwa.
Source: www.habr.com