DBA bot Joe. Anatoly Stansler (Postgres.ai)

Kodi wopanga backend amamvetsetsa bwanji kuti funso la SQL lidzagwira ntchito bwino pa "prod"? M'makampani akuluakulu kapena omwe akukula mofulumira, si onse omwe ali ndi mwayi wopeza "chinthu". Ndipo ndi mwayi, si zopempha zonse zomwe zingathe kufufuzidwa mopweteka, ndipo kupanga kopi ya database nthawi zambiri kumatenga maola ambiri. Kuti tithetse mavutowa, tidapanga DBA yopangira - Joe. Zakhala zikugwiritsidwa ntchito bwino m'makampani angapo ndipo zimathandiza opitilira khumi ndi awiri.

Video:

Moni nonse! Dzina langa ndine Anatoly Stansler. Ndimagwira ntchito kukampani postgres.ai. Tadzipereka kufulumizitsa ntchito yachitukuko pochotsa kuchedwa komwe kumakhudzana ndi ntchito ya Postgres kuchokera kwa opanga, DBAs ndi QAs.

Tili ndi makasitomala abwino ndipo lero gawo la lipotilo lidzaperekedwa kumilandu yomwe tidakumana nayo tikugwira nawo ntchito. Ndikambirana mmene tinawathandizira kuthetsa mavuto aakulu.

Pamene tikupanga ndikuchita kusamuka kwakukulu kolemetsa, timadzifunsa tokha funso: "Kodi kusamukaku kudzatha?". Timagwiritsa ntchito ndemanga, timagwiritsa ntchito chidziwitso cha anzathu odziwa zambiri, akatswiri a DBA. Ndipo amatha kudziwa ngati idzawuluke kapena ayi.

Koma mwina zingakhale bwino ngati titha kudziyesa tokha pamakope akulu akulu. Ndipo lero tingolankhula za njira zoyesera tsopano komanso momwe zingachitikire bwino komanso ndi zida ziti. Tidzakambirananso za ubwino ndi kuipa kwa njira zoterezi, ndi zomwe tingakonze apa.

Ndani adapangapo ma index mwachindunji pa prod kapena kusintha? Pang'ono ndithu. Ndipo ndani adatsogolera kuti deta idatayika kapena panali nthawi yochepera? Ndiye umadziwa ululu umenewu. Zikomo Mulungu pali zosunga zobwezeretsera.

Njira yoyamba ndikuyesa mu prod. Kapena, pamene wopanga mapulogalamu akukhala pamakina apafupi, ali ndi deta yoyesera, pali mtundu wina wa kusankha kochepa. Ndipo timatuluka kuti tipange, ndipo timapeza izi.

Zimapweteka, ndizokwera mtengo. Mwina ndi bwino kuti asatero.

Ndipo njira yabwino yochitira izo ndi iti?

Tiyeni titengeko ndikusankha gawo lina la prod pamenepo. Kapena zabwino kwambiri, tiyeni titengepo njira yeniyeni, deta yonse. Ndipo titatha kuzipanga kwanuko, tidzayang'ananso momwe zimakhalira.

Izi zidzatithandiza kuchotsa zolakwika zina, mwachitsanzo, kuti zisakhale pa prod.

Mavuto ndi chiyani?

• Vuto ndiloti timagawana nawo sitejiyi ndi anzathu. Ndipo nthawi zambiri zimachitika kuti mumapanga kusintha kwina, bam - ndipo palibe deta, ntchitoyo imakhala pansi. Masitepe anali multi-terabyte. Ndipo muyenera kuyembekezera nthawi yayitali kuti iwukenso. Ndipo taganiza zomaliza mawa. Ndi zimenezo, tili ndi chitukuko.
• Ndipo, ndithudi, tili ndi anzathu ambiri ogwira ntchito kumeneko, magulu ambiri. Ndipo izo ziyenera kuchitidwa pamanja. Ndipo izi ndizosasangalatsa.

Ndipo ndizoyenera kunena kuti tili ndi kuyesa kumodzi kokha, kuwombera kumodzi, ngati tikufuna kusintha zina pa database, kukhudza deta, kusintha mawonekedwe. Ndipo ngati chinachake chalakwika, ngati panali cholakwika kusamuka, ndiye kuti sitidzafulumizitsa kubwerera mmbuyo.

Izi ndi zabwino kuposa njira yapitayi, komabe pali kuthekera kwakukulu kuti mtundu wina wa zolakwika upite kupanga.

Ndi chiyani chomwe chimatilepheretsa kupatsa wopanga aliyense benchi yoyesera, kopi yokwanira? Ndikuganiza kuti zikuwonekeratu zomwe zili m'njira.

Ndani ali ndi database yayikulu kuposa terabyte? Oposa theka la chipinda.

Ndipo n'zoonekeratu kuti kusunga makina kwa wopanga aliyense, pamene pali kupanga kwakukulu, ndi okwera mtengo kwambiri, ndipo pambali pake, zimatenga nthawi yaitali.

Tili ndi makasitomala omwe azindikira kuti ndikofunika kwambiri kuyesa kusintha konse pa makope akuluakulu, koma deta yawo ndi yochepa kuposa terabyte, ndipo palibe zothandizira kusunga benchi yoyesera kwa wopanga aliyense. Chifukwa chake, amayenera kutsitsa zotayira kwanuko kumakina awo ndikuyesa motere. Zimatenga nthawi yambiri.

Ngakhale mutachita izi mkati mwazomangamanga, ndiye kuti kutsitsa terabyte imodzi ya data pa ola ndikwabwino kwambiri. Koma amagwiritsa ntchito zotayira zomveka, amatsitsa kwanuko kuchokera pamtambo. Kwa iwo, liwiro ndi pafupifupi 200 gigabytes pa ola. Ndipo zimatengerabe nthawi kuti mutembenuke kuchoka pamalo otayika, kukweza ma index, ndi zina.

Koma amagwiritsa ntchito njira imeneyi chifukwa imawathandiza kuti pulojekitiyi ikhale yodalirika.

Titani apa? Tiyeni tipange mabenchi oyesera otchipa ndikupatsa wopanga aliyense benchi yake yoyeserera.

Ndipo izi ndi zotheka.

Ndipo mwanjira iyi, tikapanga ma clones owonda kwa wopanga aliyense, titha kugawana nawo pamakina amodzi. Mwachitsanzo, ngati muli ndi nkhokwe ya 10TB ndipo mukufuna kuipereka kwa opanga 10, simuyenera kukhala ndi XNUMX x XNUMXTB database. Mungofunika makina amodzi okha kuti mupange makope owonda akutali kwa wopanga aliyense pogwiritsa ntchito makina amodzi. Ndikuuzani momwe zimagwirira ntchito mtsogolo pang'ono.

Chitsanzo chenicheni:

• DB - 4,5 terabytes.

• Titha kupeza makope odziyimira pawokha mumasekondi 30.

Simuyenera kudikirira mayeso ndikutengera kukula kwake. Mutha kuzipeza mumasekondi. Zidzakhala malo akutali, koma omwe amagawana deta pakati pawo.

Izi ndizabwino. Apa tikukamba zamatsenga ndi chilengedwe chofanana.

Kwa ife, izi zimagwira ntchito pogwiritsa ntchito OpenZFS.

OpenZFS ndi fayilo yolemba-pa-kulemba yomwe imathandizira zithunzithunzi ndi ma clones kunja kwa bokosi. Ndi yodalirika komanso scalable. Iye ndi wosavuta kusamalira. Itha kugawidwa m'magulu awiri.

Palinso njira zina:

• lvm,

• Kusungirako (mwachitsanzo, Pure Storage).

Database Lab yomwe ndikunena ndi modular. Itha kukhazikitsidwa pogwiritsa ntchito njira izi. Koma pakadali pano, tayang'ana pa OpenZFS, chifukwa panali mavuto ndi LVM makamaka.

Zimagwira ntchito bwanji? M'malo molemba zambiri nthawi iliyonse yomwe tikusintha, timasunga pongolemba kuti deta yatsopanoyi ikuchokera pa nthawi yatsopano, chithunzi chatsopano.

Ndipo mtsogolomo, tikafuna kubweza kapena tikufuna kupanga chofananira chatsopano kuchokera ku mtundu wina wakale, timangonena kuti: "Chabwino, tipatseni midadada iyi ya data yomwe yalembedwa motere."

Ndipo wogwiritsa ntchitoyo azigwira ntchito ndi seti yotereyi. Adzawasintha pang'onopang'ono, adzipangire yekha zithunzi.

Ndipo tidzasintha. Wopanga aliyense m'malo mwathu adzakhala ndi mwayi wokhala ndi chojambula chake chomwe amakonza, ndipo deta yomwe imagawidwa idzagawidwa pakati pa aliyense.

Kuti mugwiritse ntchito makina otere kunyumba, muyenera kuthana ndi mavuto awiri:

• Choyamba ndi gwero la deta, kumene mudzatenge kuchokera. Mukhoza kukhazikitsa kubwereza ndi kupanga. Mutha kugwiritsa ntchito kale zosunga zobwezeretsera zomwe mwakonza, ndikhulupilira. WAL-E, WAL-G kapena Barman. Ndipo ngakhale mukugwiritsa ntchito njira ya Cloud ngati RDS kapena Cloud SQL, ndiye kuti mutha kugwiritsa ntchito zotayira zomveka. Koma tikukulangizanibe kuti mugwiritse ntchito zosunga zobwezeretsera, chifukwa ndi njirayi mudzasunganso mawonekedwe a mafayilo, zomwe zidzakuthandizani kuti mukhale pafupi kwambiri ndi ma metric omwe mungawone popanga kuti muthe kuthana ndi mavuto omwe alipo.

• Yachiwiri ndipamene mukufuna kuchititsa Database Lab. Ikhoza kukhala Mtambo, ikhoza kukhala Pamalo. Ndikofunikira kunena apa kuti ZFS imathandizira kupsinjika kwa data. Ndipo amachita bwino kwambiri.

Tangoganizani kuti pamtundu uliwonse wotere, kutengera momwe timagwirira ntchito ndi maziko, mtundu wina wa dev udzakula. Kwa izi, dev adzafunikanso malo. Koma chifukwa chakuti tinatenga maziko a 4,5 terabytes, ZFS idzayipanikiza mpaka 3,5 terabytes. Izi zingasiyane malinga ndi zoikamo. Ndipo tikadali ndi malo a dev.

Dongosolo lotereli lingagwiritsidwe ntchito pamilandu yosiyanasiyana.

• Awa ndi Madivelopa, ma DBA otsimikizira mafunso, kuti mukwaniritse bwino.

• Izi zitha kugwiritsidwa ntchito pakuyesa kwa QA kuyesa kusamuka kwina tisanazitulutse kuti tipange. Ndipo tikhoza kukwezanso malo apadera a QA ndi deta yeniyeni, komwe angayese ntchito zatsopano. Ndipo zitenga masekondi m'malo modikirira maola, ndipo mwina masiku nthawi zina pomwe makope owonda sagwiritsidwa ntchito.

• Ndipo mlandu wina. Ngati kampaniyo ilibe ma analytics okhazikitsidwa, ndiye kuti titha kupatula cholozera chochepa kwambiri chazogulitsa ndikuchipereka ku mafunso ataliatali kapena ma index apadera omwe angagwiritsidwe ntchito pofufuza.

Ndi njira iyi:

1. Kuthekera kochepa kwa zolakwika pa "prod", chifukwa tidayesa zosintha zonse pazambiri zazikuluzikulu.

2. Tili ndi chikhalidwe choyesera, chifukwa tsopano simuyenera kudikirira kwa maola kuti muyimire nokha.

3. Ndipo palibe chotchinga, palibe kudikira pakati pa mayeso. Mutha kupita kukafufuza. Ndipo zikhala bwino motere pamene tikufulumizitsa chitukuko.

• Padzakhala zochepa refactoring. Nsikidzi zocheperako zimatha kukhala prod. Tidzawafotokozeranso pang'ono.

• Titha kusintha zosintha zosasinthika. Iyi si njira yokhazikika.

1. Izi ndizopindulitsa chifukwa timagawana zothandizira za mabenchi oyesera.

Zabwino kale, koma ndi chiyani chinanso chomwe chingapititsidwe patsogolo?

Chifukwa cha dongosolo loterolo, tikhoza kuchepetsa kwambiri malo olowera kuyesedwa koteroko.

Tsopano pali bwalo loyipa pomwe wopanga mapulogalamu ayenera kukhala katswiri kuti athe kupeza chidziwitso chokwanira chokwanira. Ayenera kudaliridwa ndi mwayi wotero.

Koma mmene kukula ngati palibe. Koma bwanji ngati muli ndi kagawo kakang'ono ka mayeso komwe mungapeze? Ndiye simudzapeza zochitika zenizeni.

Kodi mungachoke bwanji mubwaloli? Monga mawonekedwe oyamba, osavuta kwa opanga mulingo uliwonse, tidasankha Slack bot. Koma ikhoza kukhala mawonekedwe ena aliwonse.

Kodi chimakulolani kuchita chiyani? Mutha kutenga funso linalake ndikulitumiza ku njira yapadera ya database. Tidzatumiza chojambula chopyapyala mumasekondi. Tiyeni tiyendetse pempholi. Timasonkhanitsa ma metrics ndi malingaliro. Tiyeni tiwonetse mawonekedwe. Ndiyeno chojambulachi chikhalabe kotero kuti funsoli likhoza kukonzedwa mwanjira ina, kuwonjezera ma index, ndi zina zotero.

Komanso Slack amatipatsa mwayi wogwirizana kunja kwa bokosi. Popeza iyi ndi njira chabe, mutha kuyamba kukambirana pempholi pomwepa mu ulusi wa pempho lotere, ping anzanu, ma DBA omwe ali mkati mwa kampaniyo.

Koma pali, ndithudi, mavuto. Chifukwa ili ndi dziko lenileni, ndipo tikugwiritsa ntchito seva yokhala ndi ma clones ambiri nthawi imodzi, tiyenera kukakamiza kuchuluka kwa kukumbukira ndi mphamvu ya CPU yomwe ikupezeka kwa ma clones.

Koma kuti mayeserowa akhale omveka, muyenera kuthetsa vutoli mwanjira ina.

N'zoonekeratu kuti mfundo yofunika ndi deta yomweyo. Koma ife tiri nazo kale. Ndipo tikufuna kukwaniritsa kasinthidwe komweko. Ndipo ife tikhoza kupereka chotero pafupifupi zofanana kasinthidwe.

Zingakhale zabwino kukhala ndi hardware yofanana ndi yomwe imapanga, koma zingakhale zosiyana.

Tikumbukire momwe Postgres imagwirira ntchito ndi kukumbukira. Tili ndi cache ziwiri. Imodzi kuchokera pamafayilo ndi Postgres m'modzi, mwachitsanzo, Shared Buffer Cache.

Ndikofunika kuzindikira kuti Cache Yogawidwa Yogawidwa imaperekedwa pamene Postgres ikuyamba, malingana ndi kukula kwake komwe mumatchula mu kasinthidwe.

Ndipo cache yachiwiri imagwiritsa ntchito malo onse omwe alipo.

Ndipo tikapanga ma clones angapo pa makina amodzi, zimakhala kuti timadzaza kukumbukira pang'onopang'ono. Ndipo m'njira yabwino, Shared Buffer Cache ndi 25% ya kuchuluka kwa kukumbukira komwe kumapezeka pamakina.

Ndipo zikuwoneka kuti ngati sitisintha izi, ndiye kuti titha kuyendetsa zochitika 4 zokha pamakina amodzi, ndiye kuti, 4 mwa ma clones oonda onse. Ndipo izi, ndithudi, ndi zoipa, chifukwa tikufuna kukhala ndi zambiri za izo.

Koma kumbali ina, Buffer Cache imagwiritsidwa ntchito poyankha mafunso a indexes, ndiye kuti, dongosololi limatengera kukula kwa ma cache athu. Ndipo ngati tingotenga parameter iyi ndikuyichepetsa, ndiye kuti mapulani athu amatha kusintha kwambiri.

Mwachitsanzo, ngati tili ndi chosungira chachikulu pa prod, ndiye kuti Postgres angakonde kugwiritsa ntchito index. Ndipo ngati sichoncho, ndiye kuti padzakhala SeqScan. Ndipo zikanakhala zotani ngati zomwe tikukonzekera sizinagwirizane?

Koma apa tikufika pomaliza kuti kwenikweni dongosolo mu Postgres silidalira kukula kwake komwe kufotokozedwa mu Shared Buffer mu dongosolo, zimatengera efficient_cache_size.

Effective_cache_size ndi kuchuluka kwa cache komwe kulipo kwa ife, mwachitsanzo, kuchuluka kwa Cache ya Buffer ndi cache system system. Izi zimakhazikitsidwa ndi config. Ndipo kukumbukira uku sikunapatsidwe.

Ndipo chifukwa cha izi, titha kunyenga Postgres, kunena kuti tili ndi zambiri zomwe zilipo, ngakhale tilibe izi. Ndipo motero, mapulaniwo adzagwirizana kwathunthu ndi kupanga.

Koma izi zingasokoneze nthawi. Ndipo timakulitsa mafunso potengera nthawi, koma ndikofunikira kuti nthawi imadalira zinthu zambiri:

• Zimatengera katundu amene panopa pa prod.

• Zimatengera mawonekedwe a makinawo.

Ndipo iyi ndi gawo losalunjika, koma kwenikweni titha kukulitsa ndendende ndi kuchuluka kwa deta yomwe funsoli liwerenge kuti tipeze zotsatira.

Ndipo ngati mukufuna kuti nthawiyo ikhale pafupi ndi zomwe tidzaziwona mu prod, ndiye kuti tifunika kutenga zida zofanana kwambiri ndipo, mwinamwake, makamaka kuti ma clones onse agwirizane. Koma uku ndi kunyengerera, mwachitsanzo, mudzalandira mapulani omwewo, mudzawona kuchuluka kwa deta yomwe funso linalake lidzawerengedwe ndipo mudzatha kunena ngati funso ili ndilabwino (kapena kusamuka) kapena loyipa, likufunikabe kukonzedwa. .

Tiyeni tiwone momwe Joe amakongoleredwa mwapadera.

Tiyeni titenge pempho kuchokera ku dongosolo lenileni. Pankhaniyi, database ndi 1 terabyte. Ndipo tikufuna kuwerengera ma post atsopano omwe anali ndi zokonda zopitilira 10.

Tikulemba uthenga ku tchanelo, wojambula watumizidwa kwa ife. Ndipo tiwona kuti pempho loterolo lidzatha mu mphindi 2,5. Ichi ndi chinthu choyamba chomwe timazindikira.

B Joe akuwonetsani malingaliro anu okhazikika potengera dongosolo ndi ma metric.

Tiwona kuti funsoli limapanga deta yochulukirapo kuti mupeze mizere yaying'ono. Ndipo mtundu wina wa mlozera wapadera ukufunika, popeza tawona kuti pali mizere yochulukira yosefedwa pafunso.

Tiyeni tione bwinobwino zimene zinachitika. Zowonadi, tikuwona kuti tawerenga pafupifupi gigabytes imodzi ndi theka ya data kuchokera pa cache yamafayilo kapenanso kuchokera pa disk. Ndipo izi sizabwino, chifukwa tili ndi mizere 142 yokha.

Ndipo, zikuwoneka, tili ndi cholozera apa ndipo tikadachita mwachangu, koma popeza tidasefa mizere yambiri (tinayenera kuiwerenga), funsolo lidayenda pang'onopang'ono.

Ndipo izi zidachitika mu dongosolo chifukwa chakuti zomwe zili mufunso ndi zomwe zili muzolozera sizikugwirizana.

Tiyeni tiyese kupanga index kukhala yolondola kwambiri ndikuwona momwe mafufuzidwe amasinthira pambuyo pake.

Kulengedwa kwa ndondomekoyi kunatenga nthawi yaitali, koma tsopano timayang'ana funsolo ndikuwona kuti nthawiyo m'malo mwa maminiti a 2,5 ndi ma milliseconds 156 okha, omwe ndi abwino. Ndipo timawerenga ma megabytes 6 okha a data.

Ndipo tsopano timagwiritsa ntchito index yokha.

Nkhani ina yofunika ndi yakuti tikufuna kufotokoza ndondomekoyi m'njira yomveka bwino. Takhazikitsa zowonera pogwiritsa ntchito Flame Graph.

Ili ndi pempho lina, lamphamvu kwambiri. Ndipo timapanga Flame Graphs molingana ndi magawo awiri: izi ndi kuchuluka kwa deta yomwe node ina idawerengedwa mu dongosolo ndi nthawi, i.e. nthawi yophedwa ya node.

Apa tikhoza kufananiza mfundo zenizeni wina ndi mzake. Ndipo zidzadziwikiratu kuti ndi ndani mwa iwo amene amatenga zambiri kapena zochepa, zomwe nthawi zambiri zimakhala zovuta kuchita m'njira zina zomasulira.

Inde, aliyense amadziwa explain.depesz.com. Ubwino wa chiwonetserochi ndikuti timasunga zolemba ndikuyikanso magawo ena patebulo kuti tithe kusankha.

Ndipo Madivelopa omwe sanawerengerebe mutuwu amagwiritsanso ntchito explain.depesz.com, chifukwa ndikosavuta kwa iwo kudziwa kuti ndi ma metric omwe ndi ofunikira komanso omwe alibe.

Pali njira yatsopano yowonera - iyi ndi explain.dalibo.com. Amapanga mawonekedwe amtengo, koma ndizovuta kwambiri kufananiza mfundo wina ndi mzake. Apa mutha kumvetsetsa bwino kapangidwe kake, komabe, ngati pali pempho lalikulu, ndiye kuti muyenera kusuntha mmbuyo ndi mtsogolo, komanso njira.

mgwirizano

Ndipo, monga ndidanenera, Slack amatipatsa mwayi wogwirizana. Mwachitsanzo, tikakumana ndi funso lovuta lomwe silikudziwika bwino momwe tingakwaniritsire, titha kufotokozera nkhaniyi ndi anzathu mu ulusi wa Slack.

Zikuwoneka kwa ife kuti ndikofunikira kuyesa pa data yayikulu. Kuti tichite izi, tidapanga chida cha Update Database Lab, chomwe chimapezeka poyera. Mutha kugwiritsanso ntchito Joe bot. Mutha kuzitenga pompano ndikuziyika pamalo anu. Maupangiri onse akupezeka pamenepo.

Ndikofunikiranso kuzindikira kuti yankho lokhalo silinasinthe, chifukwa pali Delphix, koma ndi yankho labizinesi. Yatsekedwa kwathunthu, ndiyokwera mtengo kwambiri. Timakhazikika makamaka ku Postgres. Zonsezi ndizinthu zotseguka. Titsatireni!

Apa ndi pamene ndikuthera. Zikomo!

Mafunso anu

Moni! Zikomo chifukwa cha lipoti! Zosangalatsa kwambiri, makamaka kwa ine, chifukwa ndidathetsa vuto lomwelo nthawi yapitayo. Ndiye ndili ndi mafunso angapo. Ndikukhulupirira kuti ndipeza gawo lake.

Ndikudabwa momwe mungawerengere malo a chilengedwechi? Ukadaulo umatanthauza kuti nthawi zina, ma clones anu amatha kukula mpaka kukula. Kunena zowona, ngati muli ndi nkhokwe ya terabyte khumi ndi ma clones 10, ndiye kuti ndizosavuta kutengera momwe choyimira chilichonse chimalemera 10 data yapadera. Kodi mumawerengera bwanji malo awa, ndiye kuti, matsinje aja omwe mudanenapo, momwe ma cloneswa azikhalamo?

Funso labwino. Ndikofunika kuti muzitsatira ma clones enieni apa. Ndipo ngati clone ili ndi kusintha kwakukulu, imayamba kukula, ndiye kuti titha kupereka chenjezo kwa wogwiritsa ntchito izi, kapena kuyimitsa nthawi yomweyo kuti tisakhale ndi vuto.

Inde, ndili ndi funso limodzi. Ndiye kuti, mumatsimikizira bwanji mayendedwe amoyo wa ma module awa? Tili ndi vuto ili ndi nkhani yosiyana. Kodi izi zimachitika bwanji?

Pali ttl pa clone iliyonse. Kwenikweni, tili ndi ttl yokhazikika.

Bwanji, ngati si chinsinsi?

1 ola, i.e. osagwira ntchito - 1 ora. Ngati sichigwiritsidwe, ndiye kuti timachiwombera. Koma palibe chodabwitsa apa, chifukwa titha kukweza chojambulacho mumasekondi. Ndipo ngati mukufuna kachiwiri, chonde.

Ndimakondanso kusankha kwaukadaulo, chifukwa, mwachitsanzo, timagwiritsa ntchito njira zingapo zofanana pazifukwa zina. Chifukwa chiyani ZFS? Chifukwa chiyani simunagwiritse ntchito LVM? Munanena kuti panali mavuto ndi LVM. Kodi mavuto anali otani? M'malingaliro anga, njira yabwino kwambiri ndiyosungirako, potengera magwiridwe antchito.

Vuto lalikulu ndi ZFS ndi chiyani? Mfundo yakuti muyenera kuthamanga pa wolandira yemweyo, mwachitsanzo, zochitika zonse zidzakhala mkati mwa OS yomweyo. Ndipo pankhani yosungirako, mukhoza kulumikiza zipangizo zosiyanasiyana. Ndipo botolo ndi midadada yokhayo yomwe ili pamakina osungira. Ndipo funso la kusankha matekinoloje ndilosangalatsa. Chifukwa chiyani LVM?

Makamaka, titha kukambirana za LVM pamisonkhano. Za kusungirako - ndizokwera mtengo basi. Titha kukhazikitsa dongosolo la ZFS kulikonse. Mutha kuziyika pa makina anu. Mutha kutsitsa chosungira ndikuyika. ZFS imayikidwa pafupifupi kulikonse ngati tikukamba za Linux. Ndiko kuti, timapeza yankho losinthika kwambiri. Ndipo ZFS yokha imapereka zambiri m'bokosi. Mutha kukweza zambiri momwe mukufunira, kulumikiza ma disks ambiri, pali zithunzithunzi. Ndipo, monga ndidanenera, ndizosavuta kuyendetsa. Ndiko kuti, zikuwoneka zosangalatsa kwambiri kugwiritsa ntchito. Wayesedwa, ali ndi zaka zambiri. Ali ndi gulu lalikulu kwambiri lomwe likukula. ZFS ndi yankho lodalirika kwambiri.

Nikolai Samokhvalov: Kodi ndingafotokoze zambiri? Dzina langa ndine Nikolay, timagwira ntchito limodzi ndi Anatoly. Ndikuvomereza kuti yosungirako ndi yabwino. Ndipo ena mwa makasitomala athu ali ndi Pure Storage etc.

Anatoly adazindikira kuti timayang'ana kwambiri modularity. Ndipo m'tsogolomu, mutha kugwiritsa ntchito mawonekedwe amodzi - tengani chithunzithunzi, kupanga chojambula, kuwononga chojambula. Zonse ndi zophweka. Ndipo kusungirako kumakhala kozizira, ngati kuli.

Koma ZFS imapezeka kwa aliyense. DelPhix ndiyokwanira kale, ali ndi makasitomala 300. Mwa izi, mwayi 100 uli ndi makasitomala a 50, mwachitsanzo, akuyang'ana NASA, ndi zina zotero. Ndi nthawi yoti aliyense apeze lusoli. Ndipo ndichifukwa chake tili ndi Core yotseguka. Tili ndi gawo la mawonekedwe lomwe silili lotseguka. Iyi ndiye nsanja yomwe tiwonetse. Koma tikufuna kuti aliyense athe kuzipeza. Tikufuna kupanga kusintha kotero kuti oyesa asiye kupeka pa laputopu. Tiyenera kulemba SELECT ndipo nthawi yomweyo tiwone kuti ikuchedwa. Siyani kudikirira kuti a DBA akuuzeni za izi. Pano pali cholinga chachikulu. Ndipo ndikuganiza kuti tonse tidzafika ku izi. Ndipo timapanga chinthu ichi kuti aliyense akhale nacho. Chifukwa chake ZFS, chifukwa ipezeka paliponse. Tithokoze anthu ammudzi pothetsa mavuto komanso kukhala ndi chilolezo chotsegula, ndi zina zambiri.*

Moni! Zikomo chifukwa cha lipoti! Dzina langa ndine Maxim. Ife tathana ndi nkhani zomwezo. Anasankha okha. Kodi mumagawana bwanji zothandizira pakati pa ma clones awa? Wojambula aliyense amatha kuchita zake nthawi iliyonse: wina amayesa chinthu china, china, wina amapanga index, wina ali ndi ntchito yolemetsa. Ndipo ngati mutha kugawabe ndi CPU, ndiye ndi IO, mumagawa bwanji? Ili ndi funso loyamba.

Ndipo funso lachiwiri ndi la kusiyanasiyana kwa maimidwe. Tinene kuti ndili ndi ZFS pano ndipo chilichonse ndichabwino, koma kasitomala pa prod alibe ZFS, koma ext4, mwachitsanzo. Nanga bwanji pamenepa?

Mafunso ndi abwino kwambiri. Ndinatchula vuto ili pang'ono ndi mfundo yakuti timagawana zothandizira. Ndipo yankho lake ndi ili. Tangoganizani kuti mukuyesa pa siteji. Mungakhalenso ndi mkhalidwe wotero panthaŵi imodzimodziyo kuti wina apatsa katundu wina, wina. Ndipo zotsatira zake, mumawona ma metric osamvetsetseka. Ngakhale vuto lomwelo lingakhale ndi prod. Pamene mukufuna kufufuza pempho lina ndikuwona kuti pali vuto ndi ilo - limagwira ntchito pang'onopang'ono, ndiye kuti vuto silinali mu pempho, koma chifukwa chakuti pali mtundu wina wa katundu wofanana.

Ndipo kotero, ndikofunikira pano kuti tiganizire za zomwe ndondomekoyo idzakhala, zomwe tidzatenge mu ndondomekoyi ndi kuchuluka kwa deta yomwe tidzakwezera izi. Mfundo yakuti ma disks athu, mwachitsanzo, adzadzazidwa ndi chinachake, zidzakhudza kwambiri nthawi. Koma tikhoza kulingalira momwe pempholi ladzaza ndi kuchuluka kwa deta. Sikofunikira kwambiri kuti nthawi yomweyo pakhale mtundu wina wakupha.

Ndili ndi mafunso awiri. Izi ndi zinthu zabwino kwambiri. Kodi pakhala pali zochitika pomwe zambiri zopanga ndizofunikira, monga manambala a kirediti kadi? Kodi pali china chake chomwe chakonzeka kale kapena ndi ntchito yosiyana? Ndipo funso lachiwiri - kodi pali china chonga ichi cha MySQL?

Za data. Tidzachita obfusation mpaka ife tichite. Koma ngati mutumiza ndendende Joe, ngati simupereka mwayi kwa opanga, ndiye kuti palibe mwayi wopeza deta. Chifukwa chiyani? Chifukwa Joe sawonetsa deta. Zimangowonetsa ma metric, mapulani ndipo ndi momwemo. Izi zidachitika mwadala, chifukwa ichi ndi chimodzi mwazofunikira za kasitomala wathu. Iwo ankafuna kuti athe kukhathamiritsa popanda kupereka aliyense mwayi.

Za MySQL. Dongosololi litha kugwiritsidwa ntchito pa chilichonse chomwe chimasungidwa pa disk. Ndipo popeza tikuchita Postgres, tsopano tikuchita zonse zokha za Postgres poyamba. Tikufuna kuti tipeze deta kuchokera ku zosunga zobwezeretsera. Tikukonzekera Postgres molondola. Timadziwa kupanga mapulani kugwirizana, etc.

Koma popeza dongosololi ndi lowonjezera, litha kugwiritsidwanso ntchito pa MySQL. Ndipo pali zitsanzo zoterozo. Yandex ili ndi zofanana, koma sizimasindikiza kulikonse. Amagwiritsa ntchito mkati mwa Yandex.Metrica. Ndipo pali nkhani chabe ya MySQL. Koma matekinoloje ndi omwewo, ZFS.

Zikomo chifukwa cha lipoti! Ndilinso ndi mafunso angapo. Mudanena kuti cloning itha kugwiritsidwa ntchito kusanthula, mwachitsanzo kupanga ma index owonjezera pamenepo. Kodi munganene zambiri za momwe zimagwirira ntchito?

Ndipo nthawi yomweyo ndifunsa funso lachiwiri lokhudza kufanana kwa maimidwe, kufanana kwa mapulaniwo. Ndondomekoyi imadaliranso ziwerengero zomwe zasonkhanitsidwa ndi Postgres. Kodi mumathetsa bwanji vutoli?

Malinga ndi ma analytics, palibe milandu yeniyeni, chifukwa sitinagwiritse ntchito, koma pali mwayi wotero. Ngati tikukamba za ma index, ndiye yerekezani kuti funso likuthamangitsa tebulo lomwe lili ndi ma rekodi mamiliyoni mazana ambiri ndi gawo lomwe nthawi zambiri silinalembedwe mu prod. Ndipo tikufuna kuwerengera deta pamenepo. Ngati pempholi litumizidwa ku prod, ndiye kuti pali kuthekera kuti lidzakhala losavuta pa prod, chifukwa pempholi lidzakonzedwa kumeneko kwa mphindi imodzi.

Chabwino, tiyeni tipange chojambula chopyapyala chomwe sichimayimitsa kwa mphindi zingapo. Ndipo kuti tikhale omasuka kuwerenga ma analytics, tidzawonjezera ma indices a ma columns omwe timakonda deta.

Mndandanda udzapangidwa nthawi iliyonse?

Mukhoza kupanga izo kuti ife kukhudza deta, kupanga zithunzithunzi, ndiye ife achire chithunzithunzi ichi ndi kuyendetsa zopempha zatsopano. Ndiko kuti, mutha kupanga kuti mutha kukweza ma clones atsopano okhala ndi ma indices omwe adayikidwa kale.

Ponena za funso lokhudza ziwerengero, ngati tibwezeretsa kuchokera ku zosunga zobwezeretsera, ngati tibwerezabwereza, ndiye kuti ziwerengero zathu zidzakhala chimodzimodzi. Chifukwa tili ndi dongosolo lonse la data, ndiye kuti, tidzabweretsa deta monga momwe zilili ndi ma metrics onse a ziwerengero.

Nali vuto lina. Ngati mugwiritsa ntchito njira yamtambo, ndiye kuti zotayira zomveka zopezeka pamenepo, chifukwa Google, Amazon samakulolani kuti mutenge kopi yakuthupi. Padzakhala vuto.

Zikomo chifukwa cha lipoti. Panali mafunso awiri abwino apa okhudza MySQL ndi kugawana zothandizira. Koma, kwenikweni, zonse zimatsikira ku mfundo yakuti iyi si mutu wa DBMS yeniyeni, koma ya fayilo yonse. Ndipo, moyenerera, nkhani zogawana zothandizira ziyeneranso kuthetsedwa kuchokera pamenepo, osati pamapeto pake kuti ndi Postgres, koma mu fayilo, mu seva, mwachitsanzo.

Funso langa ndi losiyana pang'ono. Ili pafupi ndi database yamitundu yambiri, pomwe pali zigawo zingapo. Mwachitsanzo, timakhazikitsa zosintha zazithunzi za ten-terabyte, tikubwereza. Ndipo timagwiritsa ntchito mwapadera yankho ili pama database. Kubwereza kuli mkati, deta ikusinthidwa. Pali antchito 100 omwe akugwira ntchito limodzi pano, omwe nthawi zonse akuyambitsa kuwombera kosiyanasiyana. Zoyenera kuchita? Kodi kuonetsetsa kuti palibe mkangano, kuti anapezerapo mmodzi, ndiyeno wapamwamba dongosolo anasintha, ndipo zithunzi zonsezi anapita?

Sangapite chifukwa ndi momwe ZFS imagwirira ntchito. Titha kusunga padera mu ulusi umodzi kusintha kwamafayilo komwe kumabwera chifukwa cha kubwereza. Ndipo sungani ma clones omwe opanga amagwiritsa ntchito mitundu yakale ya data. Ndipo izo zimagwira ntchito kwa ife, chirichonse chiri mu dongosolo ndi izi.

Zikuoneka kuti zosinthazi zidzachitika ngati zowonjezera zowonjezera, ndipo zithunzi zonse zatsopano zidzapita kale, kutengera gawo ili, sichoncho?

Kuchokera m'magawo am'mbuyomu omwe anali ochokera kubwereza koyambirira.

Zigawo zam'mbuyo zidzagwa, koma zidzatanthawuza zachikale zakale, ndipo kodi zidzatenga zithunzi zatsopano kuchokera kumalo otsiriza omwe adalandiridwa muzosintha?

Mwambiri, inde.

Ndiye monga chotsatira tidzakhala mpaka mkuyu wa zigawo. Ndipo m'kupita kwa nthawi adzafunika wothinikizidwa?

Inde zonse ndi zolondola. Pali zenera. Timasunga zowonera sabata iliyonse. Zimatengera zomwe muli nazo. Ngati muli ndi mwayi wosunga deta yambiri, mukhoza kusunga zithunzithunzi kwa nthawi yaitali. Sadzachoka paokha. Sipadzakhala katangale wa data. Ngati zojambulazo ndi zachikale, monga zikuwonekera kwa ife, mwachitsanzo, zimatengera ndondomeko ya kampaniyo, ndiye kuti tikhoza kuzichotsa ndikumasula malo.

Moni, zikomo chifukwa cha lipoti! Funso la Joe. Munanena kuti kasitomala sakufuna kupatsa aliyense mwayi wopeza deta. Kunena zowona, ngati munthu ali ndi zotsatira za Fotokozani Kusanthula, ndiye kuti amatha kuyang'ana zomwe zalembedwazo.

Ziri monga choncho. Mwachitsanzo, tikhoza kulemba: "SANKHA KUCHOKERA KUTI imelo = kuti". Ndiko kuti, sitidzawona deta yokha, koma tikhoza kuona zizindikiro zina zosalunjika. Izi ziyenera kumveka. Koma kumbali ina, zonse zilipo. Tili ndi log audit, timakhala ndi ulamuliro wa anzathu ena omwe amawonanso zomwe opanga amapanga. Ndipo ngati wina ayesa kuchita izi, ndiye kuti chitetezo chidzafika kwa iwo ndikugwira ntchito pankhaniyi.

Masana abwino Zikomo chifukwa cha lipoti! Ndili ndi funso lalifupi. Ngati kampaniyo sigwiritsa ntchito Slack, kodi pali chomangirira pakali pano, kapena ndizotheka kuti opanga agwiritse ntchito zochitika kuti alumikizitse pulogalamu yoyeserera ku database?

Tsopano pali ulalo wa Slack, mwachitsanzo, palibe mthenga wina, koma ndikufunanso kuthandizira amithenga ena. Kodi mungatani? Mutha kutumiza DB Lab popanda Joe, pitani mothandizidwa ndi REST API kapena mothandizidwa ndi nsanja yathu ndikupanga ma clones ndikulumikizana ndi PSQL. Koma izi zitha kuchitika ngati mwakonzeka kupatsa omwe akutukulani mwayi wopeza deta, chifukwa sipadzakhalanso chophimba chilichonse.

Sindikufuna wosanjikiza uyu, koma ndikufunika mwayi wotero.

Ndiye inde, zikhoza kuchitika.

Source: www.habr.com