Gajimare kamar akwatin sihiri ne - kuna tambayar abin da kuke buƙata, kuma albarkatun kawai suna fitowa daga babu inda. Na'ura mai mahimmanci, bayanan bayanai, cibiyar sadarwa - duk wannan na ku ne kawai. Akwai wasu masu hayan gajimare, amma a cikin sararin samaniya ku ne kaɗai mai mulki. Kuna da tabbacin cewa koyaushe za ku karɓi albarkatun da ake buƙata, ba ku la'akari da kowa ba kuma kuna ƙayyade yadda hanyar sadarwar za ta kasance da kanta. Ta yaya wannan sihirin ke aiki wanda ke sa gajimare ya keɓe albarkatu tare da ware masu haya gaba ɗaya daga juna?
Gajimare na AWS babban hadadden tsarin mega-super ne wanda ke tasowa ta hanyar juyin halitta tun daga 2006. Wani bangare na wannan ci gaban ya faru Vasily Pantyukhin - Gidan Yanar Gizon Yanar Gizo na Amazon. A matsayin mai zane-zane, yana samun kallon ciki ba kawai a sakamakon ƙarshe ba, har ma a kalubalen da AWS ya ci nasara. Mafi girman fahimtar yadda tsarin ke aiki, mafi girman amana. Saboda haka, Vasily zai raba asirin ayyukan girgije na AWS. Da ke ƙasa akwai ƙira na sabobin AWS na zahiri, haɓakar bayanai na roba, bayanan Amazon na al'ada da hanyoyin haɓaka aikin injuna masu kama da juna yayin rage farashin su lokaci guda. Sanin hanyoyin gine-gine na Amazon zai taimake ka ka yi amfani da sabis na AWS yadda ya kamata kuma yana iya ba ka sababbin ra'ayoyi don gina naka mafita.
Game da mai magana: Vasily Pantyukhin () ya fara ne a matsayin mai kula da Unix a kamfanonin .ru, ya yi aiki a kan manyan kayan aikin Sun Microsystem na shekaru 6, kuma ya yi wa'azin duniya mai mahimmanci a EMC na shekaru 11. Ta halitta ta samo asali zuwa gajimare masu zaman kansu, kuma a cikin 2017 ya koma na jama'a. Yanzu yana ba da shawarar fasaha don taimakawa rayuwa da haɓakawa a cikin girgijen AWS.
Disclaimer: duk abin da ke ƙasa ra'ayin Vasily ne na sirri kuma maiyuwa bazai dace da matsayin Sabis na Yanar Gizo na Amazon ba. Rahoton wanda labarin ya dogara da shi yana samuwa a tasharmu ta YouTube.
Me yasa nake magana game da na'urar Amazon?
Mota ta farko tana da watsawa da hannu. Ya yi kyau saboda jin cewa zan iya tuka motar kuma in sami cikakken iko akanta. Ina kuma son cewa aƙalla na fahimci ƙa'idar aikinta. A zahiri, na yi tunanin tsarin akwatin ya zama na farko - wani abu kamar akwatin gear akan keke.
Komai ya yi kyau, sai dai abu ɗaya - makale a cikin cunkoson ababen hawa. Da alama kuna zaune ba abin da kuke yi, amma koyaushe kuna canza kaya, danna kama, gas, birki - yana sa ku gaji sosai. An warware matsalar cunkoson ababen hawa lokacin da iyali suka sami mota mai sarrafa kanta. Yayin tuƙi, na sami lokaci don yin tunani game da wani abu kuma in saurari littafin mai jiwuwa.
Wani asiri ya bayyana a rayuwata, domin gaba daya na daina fahimtar yadda motata ke aiki. Mota na zamani wata na'ura ce mai rikitarwa. Motar tana daidaita lokaci guda zuwa ɗimbin sigogi daban-daban: latsa gas, birki, salon tuki, ingancin hanya. Ban fahimci yadda yake aiki ba kuma.
Lokacin da na fara aiki a kan gajimare na Amazon, shi ma wani asiri ne a gare ni. Kawai wannan asiri shine tsari mafi girma, saboda akwai direba ɗaya a cikin motar, kuma a cikin AWS akwai miliyoyin su. Duk masu amfani a lokaci guda suna tuƙi, danna gas da birki. Yana da ban mamaki cewa suna zuwa inda suke so - abin al'ajabi ne a gare ni! Tsarin yana daidaitawa ta atomatik, daidaitawa da daidaitawa ga kowane mai amfani ta yadda zai zama a gare shi cewa shi kaɗai ne a cikin wannan Duniyar.
sihirin ya ɗan ƙare lokacin da na zo aikin injiniyan gine-gine a Amazon. Na ga irin matsalolin da muke fuskanta, yadda muke magance su, da kuma yadda muke haɓaka ayyuka. Tare da ƙara fahimtar yadda tsarin ke aiki, ƙarin amincewa ga sabis yana bayyana. Don haka ina so in raba hoton abin da ke ƙarƙashin murfin girgijen AWS.
Me zamuyi magana akai
Na zaɓi hanyar da ta bambanta - Na zaɓi ayyuka 4 masu ban sha'awa waɗanda suka cancanci magana akai.
Haɓaka uwar garken. Gizagizai na Ephemeral tare da siffa ta jiki: cibiyoyin bayanan jiki inda akwai sabobin jiki waɗanda ke huɗa, zafi da kiftawa tare da fitilu.
Ayyuka mara amfani (Lambda) tabbas shine mafi girman sabis a cikin gajimare.
Sikelin bayanai. Zan gaya muku yadda muke gina namu ma'auni na bayanai.
Sikelin hanyar sadarwa. Kashi na karshe wanda zan bude na'urar sadarwar mu. Wannan abu ne mai ban mamaki - kowane mai amfani da girgije ya yi imanin cewa shi kadai ne a cikin gajimare kuma ba ya ganin sauran masu haya ko kadan.
Lura. Wannan labarin zai tattauna ingantawar uwar garken da kuma tantance bayanai. Za mu yi la'akari da sikelin cibiyar sadarwa a cikin labarin na gaba. Ina ayyuka marasa uwar garken? An buga wani rubutun daban game da su "" Yana magana game da hanyoyi daban-daban na sikelin, kuma yayi magana dalla-dalla dalla-dalla da mafita na Firecracker - symbiosis na ingantattun halaye na injin kama-da-wane da kwantena.
Sabar
Gizagizai na al'ada ne. Amma wannan ɓacin rai har yanzu yana da siffar jiki - sabobin. Da farko, gine-ginensu na gargajiya ne. Daidaitaccen chipset x86, katunan cibiyar sadarwa, Linux, Xen hypervisor wanda aka sarrafa injina.
A cikin 2012, wannan gine-ginen ya jimre da ayyukansa da kyau. Xen babban hypervisor ne, amma yana da babban koma baya. Ya isa babban sama don kwaikwayon na'urar. Kamar yadda sabbin katunan cibiyar sadarwa masu sauri ko faifan SSD ke samuwa, wannan abin hawa ya zama babba. Yadda za a magance wannan matsala? Mun yanke shawarar yin aiki a gaba biyu lokaci guda - inganta duka hardware da hypervisor. Aikin yana da tsanani sosai.
Inganta hardware da hypervisor
Yin komai a lokaci guda kuma yin shi da kyau ba zai yi aiki ba. Abin da "mai kyau" yake nufi shi ma ba a san shi da farko ba.
Mun yanke shawarar ɗaukar hanyar juyin halitta - mun canza wani muhimmin abu na gine-gine kuma mu jefa shi cikin samarwa.
Muna taka kowane rakiya, muna sauraron koke-koke da shawarwari. Sannan mu canza wani bangaren. Don haka, a cikin ƙananan haɓaka, muna canza tsarin gine-gine gabaɗaya bisa ga martani daga masu amfani da tallafi.
Canjin ya fara ne a cikin 2013 tare da abu mafi mahimmanci - hanyar sadarwa. IN С3 misali, an ƙara katin ƙararrakin cibiyar sadarwa na musamman zuwa daidaitaccen katin sadarwar. An haɗa shi a zahiri tare da ɗan gajeren kebul na loopback a gaban panel. Ba kyakkyawa ba ne, amma ba a gani a cikin gajimare. Amma hulɗar kai tsaye tare da hardware ingantaccen jitter da kayan aikin cibiyar sadarwa.
Bayan haka mun yanke shawarar haɓaka damar toshe bayanan EBS - Ma'ajiya na Ƙarfafawa. Yana da haɗin hanyar sadarwa da ajiya. Wahalar ita ce yayin da katunan Accelerator Network ke wanzu a kasuwa, babu wani zaɓi don siyan kayan Haɓaka Accelerator kawai. Don haka mun juya zuwa farawa Annapurna Labs, wanda ya haɓaka kwakwalwan ASIC na musamman a gare mu. Sun ba da izinin saka kundin EBS mai nisa azaman na'urorin NVMe.
A cikin misalai C4 mun warware matsaloli biyu. Na farko shi ne cewa mun aiwatar da tushe don makomar mai ban sha'awa, amma sabo a wancan lokacin, fasahar NVMe. Na biyu, mun zazzage mai sarrafa na'ura mai mahimmanci ta hanyar canja wurin sarrafa buƙatun zuwa EBS zuwa sabon kati. Ya zama da kyau, don haka yanzu Annapurna Labs wani yanki ne na Amazon.
By Nuwamba 2017, mun gane cewa lokaci ya yi da za a canza hypervisor kanta.
An haɓaka sabon hypervisor bisa gyare-gyaren kernel na KVM.
Ya ba da damar rage girman kan kwaikwayar na'urar da aiki kai tsaye tare da sabbin ASICs. Misali С5 sune na'urori na farko na kama-da-wane tare da sabon hypervisor wanda ke gudana a ƙarƙashin kaho. Muka sanya masa suna nitro.
Juyin Halitta akan tsarin lokaci.
Duk sabbin nau'ikan injunan kama-da-wane waɗanda suka bayyana tun Nuwamba 2017 suna gudana akan wannan hypervisor. Misalin Bare Metal ba su da hypervisor, amma kuma ana kiran su Nitro, tunda suna amfani da katunan Nitro na musamman.
A cikin shekaru biyu masu zuwa, adadin nau'ikan lokuta na Nitro sun zarce dozin biyu: A1, C5, M5, T3 da sauransu.
Nau'in misali.
Yadda injin Nitro na zamani ke aiki
Suna da manyan abubuwa guda uku: Nitro hypervisor (wanda aka tattauna a sama), guntun tsaro da katunan Nitro.
guntun tsaro hadedde kai tsaye a cikin motherboard. Yana sarrafa ayyuka masu mahimmanci da yawa, kamar sarrafa lodin OS mai watsa shiri.
Nitro katunan - Akwai nau'i hudu daga cikinsu. Dukkansu Annapurna Labs ne suka haɓaka kuma sun dogara ne akan ASICs gama gari. Wasu daga cikin firmware su ma na kowa ne.
Katunan Nitro iri huɗu.
An tsara ɗayan katunan don yin aiki da su hanyar sadarwaFarashin VPC. Wannan shi ne abin da ake iya gani a cikin injunan kama-da-wane azaman katin cibiyar sadarwa ENA - Adaftar hanyar sadarwa na roba. Hakanan yana ɗaukar zirga-zirgar zirga-zirga lokacin watsa shi ta hanyar hanyar sadarwa ta zahiri (za mu yi magana game da wannan a cikin kashi na biyu na labarin), yana sarrafa Tacewar Wuta ta Ƙungiyoyin Tsaro, kuma yana da alhakin zirga-zirga da sauran abubuwan hanyar sadarwa.
Zaɓi katunan aiki tare da toshe ajiya EBS da faifai da aka gina a cikin uwar garken. Suna bayyana ga injin kama-da-wane kamar Adaftar NVMe. Suna kuma da alhakin ɓoye bayanan da kuma sa ido kan faifai.
Tsarin katunan Nitro, hypervisor da guntu tsaro an haɗa su cikin hanyar sadarwar SDN ko Cibiyar Sadarwar Sadarwar Software. Mai alhakin sarrafa wannan hanyar sadarwa (Control Plane) katin sarrafawa.
Tabbas, muna ci gaba da haɓaka sabbin ASICs. Misali, a karshen 2018 sun fito da guntun Inferentia, wanda ke ba ka damar yin aiki da inganci tare da ayyukan koyon injin.
Inferentia Machine Learning Processor guntu.
Database mai daidaitawa
Rubutun bayanai na gargajiya yana da tsari mai tsari. Don sauƙaƙe sosai, ana rarrabe matakan masu zuwa.
- SQL - abokin ciniki da buƙatar masu aikawa suyi aiki akan shi.
- Tanadi ma'amaloli - komai ya bayyana a nan, ACID da duk wannan.
- caching, wanda aka samar ta wurin wuraren waha.
- Shiga - yana ba da aiki tare da redo logs. A cikin MySQL ana kiran su Bin Logs, a cikin PosgreSQL - Write Ahead Logs (WAL).
- Storage – yin rikodin kai tsaye zuwa faifai.
Tsarin bayanai mai shimfiɗa.
Akwai hanyoyi daban-daban don auna ma'auni na bayanai: sharding, Shared Nothing architecture, shared disks.
Duk da haka, duk waɗannan hanyoyin suna kiyaye tsarin bayanai guda ɗaya. Wannan mahimmanci yana iyakance ƙima. Don magance wannan matsala, mun ƙirƙira namu bayanan bayanai - Amazon Aurora. Ya dace da MySQL da PostgreSQL.
Amazon Aurora
Babban ra'ayin gine-gine shine raba matakan ajiya da matakan shiga daga babban bayanan bayanai.
Duba gaba, zan ce mun kuma sanya matakin caching mai zaman kansa. Gine-ginen ya daina zama monolith, kuma muna samun ƙarin digiri na 'yanci wajen zazzage tubalan mutum ɗaya.
Matakan shiga da ma'ajiya sun bambanta da bayanan bayanai.
DBMS na gargajiya yana rubuta bayanai zuwa tsarin ajiya a cikin nau'i na tubalan. A Amazon Aurora, mun ƙirƙiri wayayyun ajiya wanda zai iya magana da harshe sake yin rajistan ayyukan. A ciki, ma'ajiyar tana jujjuya rajistan ayyukan zuwa bayanan bayanai, tana lura da amincin su kuma ta atomatik tana adanawa.
Wannan tsarin yana ba ku damar aiwatar da abubuwa masu ban sha'awa kamar cloning. Yana aiki da sauri da kuma tattalin arziki saboda gaskiyar cewa baya buƙatar ƙirƙirar cikakken kwafin duk bayanai.
Ana aiwatar da Layer ajiya azaman tsarin rarrabawa. Ya ƙunshi adadi mai yawa na sabobin jiki. Ana sarrafa kowane log ɗin sake gyarawa kuma ana adana shi lokaci guda kulli shida. Wannan yana tabbatar da kariyar bayanai da daidaita nauyi.
Ana iya samun sikelin karatun ta amfani da kwafi masu dacewa. Ma'ajiyar da aka rarraba yana kawar da buƙatar aiki tare tsakanin babban misali na bayanai, ta inda muke rubuta bayanai, da sauran kwafi. Ana ba da tabbacin samun bayanan zamani ga duk kwafi.
Matsala ɗaya kawai shine adana tsoffin bayanai akan kwafin karantawa. Amma ana magance wannan matsalar canja wurin duk redo rajistan ayyukan don kwafi akan hanyar sadarwa na ciki. Idan log ɗin yana cikin cache, an yi masa alama a matsayin kuskure kuma an sake rubuta shi. Idan ba a cikin ma'ajin ba, ana jefar da shi kawai.
Mun tsara wurin ajiya.
Yadda ake daidaita matakan DBMS
Anan, sikelin kwance ya fi wahala. Don haka bari mu bi hanyar da aka buge classic a tsaye sikelin.
Bari mu ɗauka cewa muna da aikace-aikacen da ke sadarwa tare da DBMS ta hanyar babban kumburi.
Lokacin yin sikeli a tsaye, muna ware sabon kumburi wanda zai sami ƙarin na'urori masu sarrafawa da ƙwaƙwalwa.
Bayan haka, muna canza aikace-aikacen daga tsohuwar kumburin maigida zuwa sabon. Matsaloli sun taso.
- Wannan zai buƙaci gagarumin lokacin saukar aikace-aikacen.
- Sabuwar kumburin maigidan zata sami cache mai sanyi. Ayyukan rumbun adana bayanai zai zama mafi girma kawai bayan cache ɗin ya dumama.
Yadda za a inganta yanayin? Saita wakili tsakanin aikace-aikacen da kullin maigidan.
Menene wannan zai bamu? Yanzu duk aikace-aikacen ba sa buƙatar karkatar da su da hannu zuwa sabon kumburi. Ana iya yin sauyawa a ƙarƙashin wakili kuma yana da sauri sosai.
Da alama an warware matsalar. Amma a'a, har yanzu muna fama da buƙatar dumama cache. Bugu da kari, wata sabuwar matsala ta bayyana - yanzu wakili ne mai yuwuwar gazawar.
Magani na ƙarshe tare da Amazon Aurora mara amfani
Ta yaya muka magance wadannan matsalolin?
Hagu wakili. Wannan ba misali ba ne na daban, amma gabaɗayan ƙungiyar wakilai da aka rarraba ta inda aikace-aikacen ke haɗawa da bayanan bayanai. A cikin yanayin rashin nasara, ana iya maye gurbin kowane nodes kusan nan take.
An ƙara tafki na nodes masu dumi masu girma dabam dabam. Sabili da haka, idan ya zama dole don ware sabon kumburi na girma ko ƙarami, yana samuwa nan da nan. Babu buƙatar jira don ɗauka.
Dukkanin tsarin sikelin ana sarrafa shi ta tsarin kulawa na musamman. Saka idanu akai-akai yana lura da yanayin kullin maigidan na yanzu. Idan ta gano, alal misali, cewa nauyin mai sarrafawa ya kai ga ƙima mai mahimmanci, yana sanar da tafkin kyawawan lokuta game da buƙatar ware sabon kumburi.
Rarraba wakilai, yanayi mai dumi da saka idanu.
Akwai kumburi tare da ikon da ake buƙata. Ana kwafin wuraren tafkunan buffer zuwa gare shi, kuma tsarin ya fara jira na ɗan lokaci mai aminci don canzawa.
Yawancin lokaci lokacin canzawa yana zuwa da sauri. Sa'an nan kuma an dakatar da sadarwa tsakanin wakili da tsohuwar kullin maigidan, duk zaman an canza zuwa sabon kumburi.
Aiki tare da database ya ci gaba.
Jadawalin ya nuna cewa dakatarwar ta kasance gajeru sosai. Hoton shuɗin shuɗi yana nuna nauyin kaya, matakan ja kuma suna nuna lokutan ƙima. Tsuntsaye na ɗan gajeren lokaci a cikin jadawali mai shuɗi shine ainihin ɗan gajeren jinkiri.
Af, Amazon Aurora yana ba ku damar adana kuɗi gaba ɗaya kuma ku kashe bayanan lokacin da ba a amfani da shi, alal misali, a ƙarshen mako. Bayan dakatar da kaya, DB a hankali yana rage ƙarfinsa kuma ya kashe na ɗan lokaci. Lokacin da lodi ya dawo, zai sake tashi lafiya.
A cikin ɓangaren na gaba na labarin game da na'urar Amazon, za mu yi magana game da sikelin cibiyar sadarwa. Yi rijista kuma ku kasance da mu don kada ku rasa labarin.
a kan Vasily Pantyukhin zai ba da rahoto "" Waɗanne tsarin ƙirar ƙira don tsarin rarrabawa masu haɓaka Amazon ke amfani da su, menene dalilai na gazawar sabis, menene tsarin gine-ginen Cell, Ayyukan Constant, Shuffle Sharding - zai zama mai ban sha'awa. Kasa da wata guda kafin taron - . Ƙarshen farashin ƙarshe na Oktoba 24.
source: www.habr.com