Amafu afana nebhokisi lomlingo - ubuza ukuthi yini oyidingayo, futhi izinsiza zivele zivele nje. Imishini ebonakalayo, isizindalwazi, inethiwekhi - konke lokhu kungokwakho kuphela. Kukhona abanye abaqashi bamafu, kodwa ku-Universe yakho nguwe wedwa umbusi. Uyaqiniseka ukuthi uzohlala uthola izinsiza ezidingekayo, akudingeki ukuthi ucabangele noma ubani futhi unquma ngokuzimela ukuthi inethiwekhi izoba njani. Ngabe usebenza kanjani lo mlingo owenza ifu lihlukanise izinsiza ngokunwebeka futhi lihlukanise ngokuphelele abaqashi komunye nomunye?
Ifu le-AWS liwuhlelo oluyinkimbinkimbi kakhulu obeluvela ngokuziphendukela kwemvelo kusukela ngo-2006. Ingxenye yalokhu kuthuthukiswa yenzeka Vasily Pantyukhin - I-Amazon Web Services Architect. Njengomakhi wezakhiwo, uthola ukubukeka kwangaphakathi hhayi kuphela kumphumela wokugcina, kodwa nasezinseleleni ezinqotshwa yi-AWS. Ukuqonda okukhulu kokuthi isistimu isebenza kanjani, kulapho ukwethenjwa kukhulu. Ngakho-ke, uVasily uzokwabelana ngezimfihlo zezinsizakalo zamafu ze-AWS. Ngezansi ukwakheka kwamaseva e-AWS aphathekayo, ukukala kwedatha okunwebekayo, isizindalwazi sangokwezifiso se-Amazon nezindlela zokukhulisa ukusebenza kwemishini ebonakalayo ngenkathi kunciphisa inani layo ngasikhathi sinye. Ulwazi lwezindlela zokwakha ze-Amazon luzokusiza ukuthi usebenzise izinsiza ze-AWS ngempumelelo futhi lungakunikeza imibono emisha yokwakha ezakho izixazululo.
Mayelana nesikhulumi: Vasily Pantyukhin () waqala njengomphathi we-Unix ezinkampanini ze-.ru, wasebenza ku-Hardware enkulu ye-Sun Microsystem iminyaka engu-6, futhi washumayela umhlaba we-data-centric ku-EMC iminyaka engu-11. Ivele yashintsha yaba amafu angasese, kwathi ngo-2017 yathuthela emphakathini. Manje unikeza iseluleko sobuchwepheshe ukuze asize siphile futhi sithuthuke efwini le-AWS.
Ukuzihlangula: yonke into engezansi ingumbono womuntu siqu kaVasily futhi ingase ingahambelani nesikhundla se-Amazon Web Services. Umbiko okusekelwe kuwo i-athikili uyatholakala esiteshini sethu se-YouTube.
Kungani ngikhuluma ngedivayisi ye-Amazon?
Imoto yami yokuqala yayinomshini ozishintshayo. Kwakukuhle ngenxa yomuzwa wokuthi ngingakwazi ukushayela imoto futhi ngilawule ngokuphelele. Ngiphinde ngathanda ukuthi okungenani ngiqonde isimiso sokusebenza kwayo. Ngokwemvelo, ngangicabanga ukuthi ukwakheka kwebhokisi kwakungekudala kakhulu - into efana nebhokisi lebhayisikili ebhayisikilini.
Konke bekukuhle, ngaphandle kwento eyodwa - ukubhajwa esiminyaminyeni sezimoto. Kubonakala sengathi uhlezi futhi ungenzi lutho, kodwa ulokhu ushintsha amagiya, ucindezela i-clutch, igesi, ibhuleki - kukwenza ukhathele ngempela. Inkinga yesiminyaminya yaxazululeka kancane lapho umndeni uthola imoto ezishintshayo. Ngenkathi ngishayela, ngaba nesikhathi sokucabanga ngokuthile futhi ngilalele i-audiobook.
Enye imfihlakalo yavela empilweni yami, ngoba ngayeka ngokuphelele ukuqonda ukuthi imoto yami isebenza kanjani. Imoto yesimanje iyithuluzi eliyinkimbinkimbi. Imoto ijwayela ngesikhathi esisodwa inqwaba yemingcele ehlukene: ukucindezela igesi, ibhuleki, isitayela sokushayela, ikhwalithi yomgwaqo. Angisaqondi ukuthi isebenza kanjani.
Lapho ngiqala ukusebenza ngefu lase-Amazon, bekuyimpicabadala nakimi. Le mfihlakalo kuphela ingumyalelo wobukhulu obukhulu, ngoba kukhona umshayeli oyedwa emotweni, futhi ku-AWS kunezigidi zabo. Bonke abasebenzisi ngesikhathi esisodwa baqondise, cindezela igesi kanye namabhuleki. Kuyamangaza ukuthi baya lapho bethanda khona - kuyisimangaliso kimi! Uhlelo luzivumelanisa ngokuzenzakalelayo, lukala futhi lulungise ngokunwebekayo kumsebenzisi ngamunye ukuze kubonakale kuye sengathi uyedwa kulo Mkhathi.
Umlingo waphela kancane lapho ngifika ngizosebenza njengomakhi e-Amazon. Ngibone ukuthi yiziphi izinkinga esibhekana nazo, ukuthi sizixazulula kanjani, nokuthi sithuthukisa kanjani izinsiza. Ngokuqonda okukhulayo kokuthi isistimu isebenza kanjani, ukuqiniseka okwengeziwe kusevisi kubonakala. Ngakho-ke ngifuna ukwabelana ngesithombe salokho okungaphansi kwe-hood yefu le-AWS.
Sizokhuluma ngani
Ngikhethe indlela ehlukahlukene - ngikhethe izinsiza ezi-4 ezithokozisayo okufanele ngikhulume ngazo.
Ukulungiselela iseva. Amafu e-Ephemeral anomfanekiso womuntu ongokoqobo: izikhungo zedatha ebonakalayo lapho kukhona amaseva aphathekayo anomsindo, ashise futhi acwayize ngezibani.
Imisebenzi engenaseva (Lambda) cishe iyisevisi eyingozi kakhulu efwini.
Ukukala kwesizindalwazi. Ngizokutshela mayelana nendlela esizakha ngayo isizindalwazi sethu esingakala.
Ukukala kwenethiwekhi. Ingxenye yokugcina engizovula kuyo idivayisi yenethiwekhi yethu. Lokhu kuyinto emangalisayo - wonke umsebenzisi wamafu ukholelwa ukuthi uyedwa efwini futhi akababoni nhlobo abanye abaqashi.
Qaphela. Lesi sihloko sizodingida ukwenziwa ngcono kweseva kanye nokukalwa kwedathabheyisi. Sizobheka ukukala kwenethiwekhi esihlokweni esilandelayo. Ikuphi imisebenzi engenaseva? Kushicilelwe umbhalo ohlukile ngabo “" Ikhuluma ngezindlela ezimbalwa ezihlukene zokukala, futhi ixoxa ngokuningiliziwe ngesixazululo se-Firecracker - i-symbiosis yezimfanelo ezinhle kakhulu zomshini obonakalayo neziqukathi.
Amaseva
Ifu liyisikhashana. Kodwa le ephemerality isenomfanekiso ongokoqobo - amaseva. Ekuqaleni, izakhiwo zabo zaziyi-classical. I-chipset ye-x86 ejwayelekile, amakhadi enethiwekhi, i-Linux, i-Xen hypervisor lapho imishini ebonakalayo yethulwe khona.
Ngo-2012, lesi sakhiwo sabhekana kahle nemisebenzi yayo. I-Xen iyi-hypervisor enhle, kodwa ine-drawback eyodwa enkulu. Usekwanele phezulu okuphezulu kokulingiswa kwedivayisi. Njengoba amakhadi enethiwekhi amasha, asheshayo noma amadrayivu e-SSD atholakala, le nhloko iba phezulu kakhulu. Indlela yokubhekana nale nkinga? Sinqume ukusebenza ezinhlangothini ezimbili ngesikhathi esisodwa - thuthukisa kokubili ihadiwe kanye ne-hypervisor. Umsebenzi unzima kakhulu.
Ukuthuthukisa ihadiwe kanye ne-hypervisor
Ukwenza konke ngesikhathi esisodwa nokwenza kahle ngeke kusebenze. Yini "okuhle" bekungacaci ekuqaleni.
Sinqume ukuthatha indlela yokuziphendukela kwemvelo - sishintsha into eyodwa ebalulekile yezakhiwo bese siyiphonsa ekukhiqizeni.
Sinyathela kuwo wonke amareki, silalela izikhalo neziphakamiso. Bese sishintsha enye ingxenye. Ngakho-ke, ngokunyuka okuncane, sishintsha ngokuphelele yonke i-architecture ngokusekelwe kumpendulo evela kubasebenzisi nokusekelwa.
Ukuguqulwa kwaqala ngo-2013 ngento eyinkimbinkimbi kakhulu - inethiwekhi. IN С3 izimo, ikhadi elikhethekile le-Network Accelerator lengezwe ekhadini lenethiwekhi elijwayelekile. Yayixhunywe ngokoqobo ngentambo emfushane ye-loopback kuphaneli yangaphambili. Ayimuhle, kodwa ayibonakali emafini. Kodwa ukusebenzisana okuqondile nehadiwe kuyithuthukise ngokuyisisekelo i-jitter kanye nokuphuma kwenethiwekhi.
Okulandelayo sinqume ukuthuthukisa ukufinyelela ekuvimbeni isitoreji sedatha ye-EBS - I-Elastic Block Storage. Kuyinhlanganisela yenethiwekhi nesitoreji. Ubunzima ukuthi ngenkathi amakhadi e-Network Accelerator ekhona emakethe, ibingekho inketho yokumane uthenge ihadiwe ye-Storage Accelerator. Ngakho saphendukela kusiqalo I-Annapurna Labs, owasenzela ama-ASIC chips akhethekile. Bavumele amavolumu akude we-EBS ukuthi afakwe njengamadivayisi e-NVMe.
Ezimweni C4 sixazulule izinkinga ezimbili. Esokuqala ukuthi sisebenzise isisekelo sekusasa elithembisayo, kodwa esisha ngaleso sikhathi, ubuchwepheshe be-NVMe. Okwesibili, sehlise kakhulu iphrosesa emaphakathi ngokudlulisela ukucutshungulwa kwezicelo ku-EBS ekhadini elisha. Kuvele kahle, ngakho manje i-Annapurna Labs iyingxenye ye-Amazon.
NgoNovemba 2017, sabona ukuthi kwase kuyisikhathi sokushintsha i-hypervisor ngokwayo.
I-hypervisor entsha yathuthukiswa ngokususelwa kumamojula we-KVM kernel aguquliwe.
Kwenze kwaba nokwenzeka ukwehlisa ngokuyisisekelo okungaphezulu kokulingiswa kwedivayisi nokusebenza ngokuqondile nama-ASIC amasha. Izimo С5 bekuyimishini yokuqala ebonakalayo ene-hypervisor entsha esebenza ngaphansi kwe-hood. Simqambe igama I-Nitro.
Ukuvela kwezimo kumugqa wesikhathi.
Zonke izinhlobo ezintsha zemishini ebonakalayo evele kusukela ngoNovemba 2017 isebenza kule hypervisor. Izimo ze-Bare Metal azinayo i-hypervisor, kodwa zibizwa nangokuthi i-Nitro, njengoba zisebenzisa amakhadi e-Nitro akhethekile.
Phakathi neminyaka emibili eyalandela, inani lezinhlobo ze-Nitro zehlakalo ladlula ishumi nambili: A1, C5, M5, T3 nabanye.
Izinhlobo zezibonelo.
Isebenza kanjani imishini yesimanje ye-Nitro
Anezingxenye ezintathu eziyinhloko: i-Nitro hypervisor (okuxoxwe ngayo ngenhla), i-chip yezokuphepha namakhadi e-Nitro.
I-chip yokuphepha kuhlanganiswe ngqo ebhodini lomama. Ilawula imisebenzi eminingi ebalulekile, njengokulawula ukulayishwa kwe-OS yesikhungo.
Amakhadi e-Nitro - Kunezinhlobo ezine zazo. Wonke athuthukiswa yi-Annapurna Labs futhi asuselwa kuma-ASIC ajwayelekile. Ezinye ze-firmware yazo nazo zijwayelekile.
Izinhlobo ezine zamakhadi e-Nitro.
Elinye lamakhadi lakhelwe ukusebenza ngalo inethiwekhiI-VPC. Yilokhu okubonakala emishinini ebonakalayo njengekhadi lenethiwekhi ENA - I-adaptha yenethiwekhi ye-Elastic. Iphinde ihlanganise ithrafikhi lapho uyidlulisela ngenethiwekhi ebonakalayo (sizokhuluma ngalokhu engxenyeni yesibili yesihloko), ilawula i-firewall ye-Security Groups, futhi inesibopho somzila nezinye izinto zenethiwekhi.
Khetha amakhadi asebenza nge-block storage I-EBS namadiski akhelwe kuseva. Zivela emshinini wokubuka wezihambeli njengo Ama-adaptha we-NVMe. Futhi banesibopho sokubethelwa kwedatha nokuqapha kwediski.
Uhlelo lwamakhadi we-Nitro, i-hypervisor kanye ne-chip yokuphepha ihlanganiswe kunethiwekhi ye-SDN noma I-Software Defined Network. Unesibopho sokuphatha le nethiwekhi (Indiza Yokulawula) isilawuli ikhadi.
Yebo, siyaqhubeka nokuthuthukisa ama-ASIC amasha. Isibonelo, ekupheleni kuka-2018 bakhiphe i-chip ye-Inferentia, evumela ukuthi usebenze ngokuphumelelayo ngemisebenzi yokufunda yomshini.
I-Inferentia Machine Learning Processor chip.
Database Scalable
Isizindalwazi sendabuko sinesakhiwo esinezingqimba. Ukwenza lula kakhulu, amazinga alandelayo ahlukaniswa.
- SQL - amaklayenti kanye nama-dispatchers esicelo asebenza kuwo.
- Izinhlinzeko ukuthengiselana - konke kucacile lapha, i-ACID nakho konke lokho.
- Ukulondoloza inqolobane, ehlinzekwa ngamachibi okubhukuda.
- Ukugawula - inikeza umsebenzi ngamalogi wokwenza kabusha. Ku-MySQL abizwa ngokuthi ama-Bin Logs, ku-PosgreSQL - Bhala Ngaphambili Logs (WAL).
- Isitoreji – ukurekhoda okuqondile kudiski.
Isakhiwo sesizindalwazi esinezendlalelo.
Kunezindlela ezihlukene zokukala imininingwane yolwazi: ukwahlukanisa, ukwakheka kwe-Shared Nothing, amadiski abelwe.
Kodwa-ke, zonke lezi zindlela zigcina isakhiwo sedatha ye-monolithic efanayo. Lokhu kunciphisa kakhulu ukukala. Ukuxazulula le nkinga, sakha eyethu isizindalwazi − I-Amazon Aurora. Iyahambisana ne-MySQL ne-PostgreSQL.
I-Amazon Aurora
Umqondo oyinhloko wezakhiwo uwukuhlukanisa amazinga okugcina nokugawula kusizindalwazi esikhulu.
Uma ngibheka phambili, ngizosho ukuthi senze nezinga le-caching lizimele. I-architecture iyayeka ukuba yi-monolith, futhi sithola amadigri engeziwe enkululeko ekulinganiseni amabhulokhi angawodwana.
Amazinga wokuloga nawokugcina ahlukene kusizindalwazi.
I-DBMS yendabuko ibhala idatha ohlelweni lwesitoreji ngendlela yamabhulokhi. E-Amazon Aurora, sidale isitoreji esihlakaniphile esikwazi ukukhuluma ulimi yenza kabusha izingodo. Ngaphakathi, isitoreji siguqula izingodo zibe amabhulokhi wedatha, siqaphe ubuqotho bawo futhi sisekela ngokuzenzakalelayo.
Le ndlela ikuvumela ukuthi usebenzise izinto ezithakazelisayo njenge cloning. Isebenza ngokushesha futhi ngokonga okwengeziwe ngenxa yokuthi ayidingi ukudala ikhophi ephelele yayo yonke idatha.
Isendlalelo sesitoreji sisetshenziswa njengesistimu esabalalisiwe. Iqukethe inani elikhulu kakhulu lamaseva aphathekayo. Ilogi ngayinye yokwenza kabusha iyacutshungulwa futhi ilondolozwe kanyekanye amafindo ayisithupha. Lokhu kuqinisekisa ukuvikelwa kwedatha nokulinganisa kokulayisha.
Ukukala kokufunda kungafinyelelwa ngokusebenzisa izifaniso ezifanele. Isitoreji esabalalisiwe siqeda isidingo sokuvumelanisa phakathi kwesibonelo sesizindalwazi esiyinhloko, lapho sibhala khona idatha, nezifaniso ezisele. Idatha yakamuva iqinisekisiwe ukuthi izotholakala kuzo zonke izifaniso.
Inkinga kuphela ukugcina idatha endala kuma-replicas afundiwe. Kodwa le nkinga iyaxazululeka ukudluliswa kwawo wonke amalogi enziwe kabusha ukuphindaphinda ngenethiwekhi yangaphakathi. Uma ilogu ikunqolobane, imakwa njengengalungile futhi ibhalwe ngaphezulu. Uma ingekho kunqolobane, imane ilahlwe.
Silungise isitoreji.
Ungakala kanjani ama-DBMS tiers
Lapha, ukukala okuvundlile kunzima kakhulu. Ngakho ake sehle ngendlela eshayiwe ukukala okuqondile okujwayelekile.
Ake sicabange ukuthi sinohlelo lokusebenza oluxhumana ne-DBMS ngenodi eyinhloko.
Lapho sikala sibheka phezulu, sinikeza i-node entsha ezoba namaphrosesa amaningi nenkumbulo.
Okulandelayo, sishintsha uhlelo lokusebenza kusuka ku-master node endala kuya kwentsha. Izinkinga ziyavela.
- Lokhu kuzodinga isikhathi sokuphumula esibalulekile sohlelo lokusebenza.
- I-master node entsha izoba nesilondolozi esibandayo. Ukusebenza kwesizindalwazi kuzoba okuphezulu kuphela ngemuva kokuthi inqolobane isifudumale.
Indlela yokuthuthukisa isimo? Setha ummeleli phakathi kohlelo lokusebenza ne-master node.
Lokhu kuzosinikani? Manje zonke izinhlelo zokusebenza azidingi ukuthi ziqondiswe kabusha endaweni entsha. Ukushintsha kungenziwa ngaphansi kommeleli futhi kuyashesha kakhulu.
Kubonakala sengathi inkinga isixazululiwe. Kodwa cha, sisahlushwa isidingo sokufudumala i-cache. Ngaphezu kwalokho, sekuvele inkinga entsha - manje i-proxy iyindawo engase yehluleke.
Isixazululo sokugcina nge-Amazon Aurora serverless
Sizixazulule kanjani lezi zinkinga?
Ushiye ummeleli. Lesi akusona isenzakalo esihlukile, kodwa iqoqo lonke elisabalalisiwe lama-proxi lapho izinhlelo zokusebenza zixhuma kusizindalwazi. Uma kwenzeka ukwehluleka, noma yimaphi ama-node angashintshwa cishe ngokushesha.
Kwengezwe ichibi lama-node afudumele anosayizi abahlukahlukene. Ngakho-ke, uma kudingekile ukwaba i-node entsha yesayizi enkulu noma encane, itholakala ngokushesha. Asikho isidingo sokulinda ukuthi ilayishe.
Yonke inqubo yokukala ilawulwa uhlelo olukhethekile lokuqapha. Ukuqapha njalo kuqapha isimo se-master node yamanje. Uma ithola, isibonelo, ukuthi umthwalo weprosesa ufinyelele inani elibalulekile, yazisa inqwaba yezimo ezifudumele mayelana nesidingo sokwaba i-node entsha.
Ama-proxy asabalalisiwe, izimo ezifudumele nokuqapha.
I-node enamandla adingekayo iyatholakala. Ama-buffer pools akopishelwe kuwo, futhi isistimu iqala ukulinda isikhathi esiphephile ukuze ishintshe.
Ngokuvamile isikhathi sokushintsha sifika ngokushesha. Bese ukuxhumana phakathi kwe-proxy ne-master node endala kuyamiswa, zonke izikhathi zishintshelwa endaweni entsha.
Sebenza nge-database iqala kabusha.
Igrafu ikhombisa ukuthi ukumiswa kufushane impela. Igrafu eluhlaza okwesibhakabhaka ibonisa umthwalo, futhi izinyathelo ezibomvu zibonisa izikhathi zokukala. Ukucwiliswa kwesikhashana kugrafu eluhlaza okwesibhakabhaka yilokho kanye kubambezeleka okufushane.
Ngendlela, i-Amazon Aurora ikuvumela ukuthi ulondoloze imali ngokuphelele futhi uvale i-database uma ingasetshenziswa, isibonelo, ngezimpelasonto. Ngemva kokumisa umthwalo, i-DB kancane kancane inciphisa amandla ayo futhi icime isikhathi esithile. Lapho umthwalo ubuya, uzokhuphuka kahle futhi.
Engxenyeni elandelayo yendaba mayelana nedivayisi ye-Amazon, sizokhuluma ngokukalwa kwenethiwekhi. Bhalisa futhi uhlale ubukele ukuze ungaphuthelwa isihloko.
In UVasily Pantyukhin uzonikeza umbiko "" Imaphi amaphethini okuklama wezinhlelo ezisabalalisiwe asetshenziswa abathuthukisi be-Amazon, yiziphi izizathu zokuhluleka kwesevisi, yini i-Cell-based architecture, Constant Work, Shuffle Sharding - izothakazelisa. Kungakapheli nenyanga kuze kufike ingqungquthela - . Okthoba 24 ukukhuphuka kwentengo yokugcina.
Source: www.habr.com