Mitambo ili ngati bokosi lamatsenga - mumafunsa zomwe mukufuna, ndipo zothandizira zimangowoneka modzidzimutsa. Makina apakompyuta, nkhokwe, maukonde - zonsezi ndi zanu zokha. Palinso ena obwereketsa mitambo, koma mu Chilengedwe chanu ndinu nokha wolamulira. Muli otsimikiza kuti nthawi zonse mudzalandira zofunikira, simumaganizira aliyense ndipo mumadziwa nokha kuti maukondewo adzakhala otani. Kodi zamatsengazi zimagwira ntchito bwanji zomwe zimapangitsa mtambo kugawa chuma mokhazikika ndikulekanitsa anthu ochita lendi?
Mtambo wa AWS ndi mega-super complex system yomwe yakhala ikusintha kuyambira 2006. Mbali ina ya chitukukochi inachitika Vasily Pantyukhin - Wopanga Amazon Web Services Architect. Monga womangamanga, amawona mkati osati pazotsatira zomaliza, komanso zovuta zomwe AWS imagonjetsa. Kumvetsetsa bwino momwe dongosololi limagwirira ntchito, kudalirana kwakukulu. Chifukwa chake, Vasily adzagawana zinsinsi za ntchito zamtambo za AWS. Pansipa pali mapangidwe a maseva a AWS akuthupi, zotanuka database scalability, makonda a Amazon database ndi njira zowonjezerera magwiridwe antchito a makina enieni ndikuchepetsa mtengo wawo. Kudziwa njira zamamangidwe za Amazon kudzakuthandizani kugwiritsa ntchito bwino ntchito za AWS ndipo kungakupatseni malingaliro atsopano opangira mayankho anu.
Za wokamba: Vasily Pantyukhin () adayamba ngati woyang'anira Unix pamakampani a .ru, adagwira ntchito pazida zazikulu za Sun Microsystem kwa zaka 6, ndikulalikira dziko la data-centric ku EMC kwa zaka 11. Mwachilengedwe idasintha kukhala mitambo yachinsinsi, ndipo mu 2017 idasamukira kugulu la anthu. Tsopano amapereka upangiri waukadaulo kuti athandizire kukhala ndi moyo mumtambo wa AWS.
Chodzikanira: Zonse zomwe zili pansipa ndi malingaliro a Vasily ndipo mwina sizingafanane ndi momwe Amazon Web Services ilili. Lipoti lomwe nkhaniyi idachokera likupezeka panjira yathu ya YouTube.
Chifukwa chiyani ndikulankhula za chipangizo cha Amazon?
Galimoto yanga yoyamba inali ndi ma transmission pamanja. Zinali zabwino chifukwa chodzimva kuti ndimatha kuyendetsa galimoto ndikuyilamulira kotheratu. Ndinkakondanso kuti ndimamvetsetsa bwino mfundo za ntchito yake. Mwachilengedwe, ndimaganiza momwe bokosilo limapangidwira - chinthu chonga gearbox panjinga.
Chilichonse chinali chabwino, kupatula chinthu chimodzi - kutsekeka m'misewu. Zikuwoneka ngati mwakhala osachita kalikonse, koma mukusintha magiya nthawi zonse, kukanikiza cholumikizira, gasi, brake - zimakupangitsani kutopa. Vuto la kuchulukana kwa magalimoto linathetsedwa pang'ono banjali litapeza galimoto yodzichitira yokha. Ndikuyendetsa galimoto, ndinali ndi nthawi yoganizira zinazake ndikumvetsera audiobook.
Chinsinsi china chinawonekera m'moyo wanga, chifukwa ndinasiya kumvetsetsa momwe galimoto yanga imagwirira ntchito. Galimoto yamakono ndi chipangizo chovuta. Galimotoyo imasintha nthawi imodzi kukhala magawo angapo osiyanasiyana: kukanikiza gasi, brake, mawonekedwe oyendetsa, mtundu wamisewu. Sindikumvetsanso momwe zimagwirira ntchito.
Nditayamba kugwira ntchito pamtambo wa Amazon, zinalinso chinsinsi kwa ine. Chokhacho chinsinsi ichi ndi dongosolo la kukula kwake, chifukwa m'galimoto muli dalaivala mmodzi yekha, ndipo mu AWS muli mamiliyoni a iwo. Ogwiritsa ntchito nthawi imodzi amawongolera, kukanikiza gasi ndikuphwanya. Ndizodabwitsa kuti amapita kumene akufuna - ndi chozizwitsa kwa ine! Dongosololi limangosintha, masikelo ndikusintha mokhazikika kwa wogwiritsa ntchito aliyense kotero kuti ziwonekere kwa iye kuti ali yekha mu Chilengedwe ichi.
Matsengawo anatha pang'ono nditabwera kudzagwira ntchito yomanga ku Amazon. Ndinawona mavuto omwe timakumana nawo, momwe timawathetsera, ndi momwe timapangira ntchito. Ndi kumvetsetsa kowonjezereka kwa momwe dongosololi limagwirira ntchito, kudalira kwambiri muutumiki kumawonekera. Chifukwa chake ndikufuna kugawana chithunzi cha zomwe zili pansi pamtambo wa AWS.
Tikambirana chiyani
Ndinasankha njira zosiyanasiyana - ndinasankha mautumiki 4 osangalatsa omwe ndi ofunika kukambirana.
Kukhathamiritsa kwa seva. Mitambo ya Ephemeral yokhala ndi mawonekedwe owoneka bwino: malo opangira data pomwe pali maseva akuthupi omwe amang'ung'udza, kutentha ndi kuthwanima ndi magetsi.
Ntchito zopanda seva (Lambda) mwina ndiye ntchito yowopsa kwambiri pamtambo.
Kuchulukitsa kwa database. Ndikuwuzani momwe timapangira ma database athu omwe angasinthidwe.
Kuchulukitsa kwa netiweki. Gawo lomaliza lomwe nditsegule chipangizo cha maukonde athu. Ichi ndi chinthu chodabwitsa - aliyense wogwiritsa ntchito mitambo amakhulupirira kuti ali yekha mumtambo ndipo sawona alendi ena nkomwe.
Zindikirani. Nkhaniyi ifotokoza za kukhathamiritsa kwa seva komanso makulitsidwe a database. Tikambirana makulitsidwe a netiweki m'nkhani yotsatira. Kodi ntchito zopanda seva zili kuti? Nkhani ina inasindikizidwa ponena za iwo "" Imalankhula za njira zingapo zokulirapo, ndikukambirana mwatsatanetsatane yankho la Firecracker - symbiosis ya zabwino kwambiri zamakina ndi zotengera.
Ma seva
Mtambowu ndi wa ephemeral. Koma ephemerality iyi imakhalabe ndi mawonekedwe - ma seva. Poyamba, mapangidwe awo anali akale. Chipset chokhazikika cha x86, makhadi amtaneti, Linux, Xen hypervisor pomwe makina enieni adakhazikitsidwa.
Mu 2012, zomangamangazi zidakwanitsa ntchito zake bwino. Xen ndi hypervisor yabwino, koma ili ndi vuto limodzi lalikulu. Iye ali nazo zokwanira mkulu pamwamba kwa chipangizo kutsanzira. Pamene makhadi atsopano, othamanga kwambiri kapena ma drive a SSD akupezeka, mutuwu umakhala wokwera kwambiri. Kodi kuthana ndi vutoli? Tinaganiza zogwira ntchito ziwiri nthawi imodzi - kukhathamiritsa zonse za hardware ndi hypervisor. Ntchitoyi ndi yaikulu kwambiri.
Kukhathamiritsa kwa hardware ndi hypervisor
Kuchita zonse mwakamodzi ndikuchita bwino sikungagwire ntchito. Zomwe "zabwino" sizinali zodziwika poyamba.
Tinaganiza zotengera njira yosinthira - timasintha chinthu chimodzi chofunikira pazomangamanga ndikuchiponya pakupanga.
Timapondaponda chilichonse, kumvera madandaulo ndi malingaliro. Kenako timasintha chigawo china. Chifukwa chake, pang'onopang'ono, timasintha kwambiri zomanga zonse kutengera mayankho a ogwiritsa ntchito ndi chithandizo.
Kusintha kunayamba mu 2013 ndi chinthu chovuta kwambiri - maukonde. MU C3 Nthawi zina, khadi yapadera ya Network Accelerator idawonjezedwa pakhadi yokhazikika yamaneti. Zinalumikizidwa kwenikweni ndi chingwe chachifupi cha loopback kutsogolo. Sizokongola, koma sizikuwoneka mumtambo. Koma kuyanjana kwachindunji ndi hardware kunasintha kwambiri jitter ndi ma network.
Kenaka tinaganiza zopititsa patsogolo mwayi wosungirako deta EBS - Elastic Block Storage. Ndi kuphatikiza kwa maukonde ndi yosungirako. Chovuta ndichakuti ngakhale makhadi a Network Accelerator analipo pamsika, panalibe mwayi wongogula zida za Storage Accelerator. Kotero ife tinatembenukira ku chiyambi Annapurna Labs, omwe adapanga zida zapadera za ASIC kwa ife. Adalola kuti ma EBS akutali akhazikitsidwe ngati zida za NVMe.
Muzochitika C4 tinathetsa mavuto awiri. Choyamba ndi chakuti tinakhazikitsa maziko a tsogolo lolonjeza, koma latsopano panthawiyo, teknoloji ya NVMe. Kachiwiri, tidatsitsa kwambiri purosesa yapakati posamutsa zopempha ku EBS ku khadi latsopano. Zidakhala bwino, ndiye tsopano Annapurna Labs ndi gawo la Amazon.
Pofika November 2017, tinazindikira kuti inali nthawi yoti tisinthe hypervisor yokha.
Hypervisor yatsopano idapangidwa kutengera ma module a KVM kernel osinthidwa.
Zinapangitsa kuti zitheke kuchepetsa kutsanzira kwa chipangizocho ndikugwira ntchito mwachindunji ndi ma ASIC atsopano. Zochitika C5 anali makina oyamba okhala ndi hypervisor yatsopano yomwe ikuyenda pansi pa hood. Tinamutcha dzina Nitro.
Kusintha kwa zochitika pa nthawi yanthawi.
Mitundu yonse yatsopano yamakina omwe adawonekera kuyambira Novembala 2017 amayendetsa pa hypervisor iyi. Zitsanzo za Bare Metal zilibe hypervisor, koma amatchedwanso Nitro, chifukwa amagwiritsa ntchito makadi apadera a Nitro.
Pazaka ziwiri zotsatira, chiwerengero cha mitundu ya Nitro inaposa khumi ndi awiri: A1, C5, M5, T3 ndi ena.
Mitundu yachitsanzo.
Momwe makina amakono a Nitro amagwirira ntchito
Ali ndi zigawo zitatu zazikulu: Nitro hypervisor (yomwe takambirana pamwambapa), chipangizo chachitetezo ndi makhadi a Nitro.
Chip chitetezo zophatikizidwa molunjika mu boardboard. Imawongolera ntchito zambiri zofunika, monga kuwongolera kutsitsa kwa OS yomwe yakhala.
Makhadi a Nitro - Pali mitundu inayi ya izo. Zonsezi zimapangidwa ndi Annapurna Labs ndipo zimachokera ku ASIC wamba. Zina mwa firmware yawo ndizofala.
Mitundu inayi yamakhadi a Nitro.
Imodzi mwa makadiwo idapangidwa kuti izigwira ntchito maukondeZamgululi. Izi ndi zomwe zimawonekera m'makina enieni ngati khadi lamaneti ENA - Elastic Network Adapter. Zimaphatikizanso kuchuluka kwa magalimoto pamene mukuzitumiza kudzera pa intaneti (tidzakambirana za izi mu gawo lachiwiri la nkhaniyi), imayang'anira firewall ya Security Groups, ndipo imakhala ndi udindo woyendetsa ndi zinthu zina zapaintaneti.
Sankhani makhadi ntchito ndi chipika yosungirako EBS ndi ma disks omwe amapangidwa mu seva. Iwo amawonekera kwa mlendo makina pafupifupi ngati Ma adapter a NVMe. Amakhalanso ndi udindo wosunga deta komanso kuyang'anira disk.
Dongosolo la makhadi a Nitro, hypervisor ndi chip chitetezo zimaphatikizidwa mu netiweki ya SDN kapena Software Defined Network. Ndi udindo woyang'anira netiweki iyi (Control Plane) khadi yowongolera.
Zachidziwikire, tikupitiliza kupanga ma ASIC atsopano. Mwachitsanzo, kumapeto kwa 2018 adatulutsa chipangizo cha Inferentia, chomwe chimakulolani kuti mugwire ntchito bwino ndi ntchito zophunzirira makina.
Inferentia Machine Learning processor chip.
Scalable Database
Dongosolo lachidziwitso lachikhalidwe lili ndi mawonekedwe osanjikiza. Kuti muchepetse kwambiri, magawo otsatirawa amasiyanitsidwa.
- SQL - kasitomala ndi pempho dispatchers ntchito pa izo.
- Zopereka zochita - Chilichonse chikuwonekera apa, ACID ndi zonsezo.
- posungira, yomwe imaperekedwa ndi maiwe a buffer.
- Kudula mitengo - imapereka ntchito ndi zolemba zobwereza. Mu MySQL amatchedwa Bin Logs, mu PosgreSQL - Lembani Ahead Logs (WAL).
- Kusungirako - kujambula molunjika ku disk.
Layered database kapangidwe.
Pali njira zingapo zosinthira nkhokwe: sharding, Zomangamanga Zogawana, ma disks ogawana.
Komabe, njira zonsezi zimasunga mawonekedwe ofanana a database ya monolithic. Izi zimachepetsa kwambiri makulitsidwe. Kuti tithane ndi vutoli, tinapanga database yathuyathu - Amazon Aurora. Ndiwogwirizana ndi MySQL ndi PostgreSQL.
Amazon Aurora
Lingaliro lalikulu la zomangamanga ndikulekanitsa magawo osungira ndi kudula mitengo kuchokera ku database yayikulu.
Kuyang'ana m'tsogolo, ndikunena kuti tidapanganso gawo la caching palokha. Zomangamanga zimasiya kukhala monolith, ndipo timapeza magawo owonjezera a ufulu pakukweza midadada yamunthu payekha.
Miyezo yodula mitengo ndi yosungirako ndi yosiyana ndi database.
DBMS yachikhalidwe imalemba deta kumalo osungirako zinthu monga midadada. Ku Amazon Aurora, tapanga malo osungira anzeru omwe amatha kulankhula chilankhulo konzanso-zipika. Mkati, zosungirako zimatembenuza zipika kukhala midadada ya data, kuyang'anira kukhulupirika kwawo ndikuzisunga zokha.
Njirayi imakulolani kuti mugwiritse ntchito zinthu zosangalatsa monga cloning. Zimagwira ntchito mwachangu komanso mwachuma chifukwa sizifuna kupanga buku lathunthu lazinthu zonse.
Chigawo chosungirako chimagwiritsidwa ntchito ngati dongosolo logawidwa. Zimakhala ndi ma seva ambiri akuthupi. chipika chilichonse chobwereza chimakonzedwa ndikusungidwa nthawi imodzi mfundo zisanu. Izi zimatsimikizira chitetezo cha data ndi kusanja kwa katundu.
Werengani makulitsidwe amatha kutheka pogwiritsa ntchito zofananira zoyenera. Kusungidwa kogawidwa kumathetsa kufunika kwa kulunzanitsa pakati pa nkhokwe yayikulu, momwe timalembera deta, ndi zofananira zotsalira. Zambiri zaposachedwa ndizotsimikizika kuti zitha kupezeka kwa zofananira zonse.
Vuto lokhalo ndikusunga deta yakale pazithunzi zowerengera. Koma vutoli likuthetsedwa kusamutsa zipika zonse zobwereza kufanizira pa netiweki yamkati. Ngati chipikacho chili mu cache, chimalembedwa kuti ndicholakwika komanso chalembedwa. Ngati sichili mu cache, chimangotayidwa.
Tinakonza zosungirako.
Momwe mungakulitsire magawo a DBMS
Apa, kukweza kopingasa kumakhala kovuta kwambiri. Ndiye tiyeni tipite ku njira yopunthidwa classic ofukula makulitsidwe.
Tiyeni tiyerekeze kuti tili ndi pulogalamu yomwe imalumikizana ndi DBMS kudzera mu node yayikulu.
Tikamakula molunjika, timagawa node yatsopano yomwe idzakhala ndi mapurosesa ambiri ndi kukumbukira.
Kenako, timasintha pulogalamuyo kuchoka pa node yakale kupita ku yatsopano. Mavuto amabuka.
- Izi zidzafuna nthawi yochepa yogwiritsira ntchito.
- Node yatsopano ya master idzakhala ndi cache yozizira. Kuchita kwa database kudzakhala kokulirapo pokhapokha posungira ikatenthedwa.
Kodi kusintha zinthu? Khazikitsani proxy pakati pa pulogalamuyo ndi master node.
Kodi izi zidzatipatsa chiyani? Tsopano ntchito zonse siziyenera kutumizidwa pamanja ku node yatsopano. Kusinthaku kutha kuchitidwa pansi pa proxy ndipo kumathamanga kwambiri.
Zikuwoneka kuti vutoli lathetsedwa. Koma ayi, timavutikabe chifukwa chofuna kutentha cache. Kuonjezera apo, vuto latsopano lawonekera - tsopano wothandizira ndi wokhoza kulephera.
Yankho lomaliza ndi Amazon Aurora serverless
Tinathetsa bwanji mavuto amenewa?
Wasiya woyimira. Izi sizochitika zosiyana, koma gulu lonse la ma proxies omwe mapulogalamu amalumikizana ndi database. Zikalephera, node iliyonse imatha kusinthidwa nthawi yomweyo.
Anawonjezera dziwe la mfundo ofunda osiyanasiyana misinkhu. Choncho, ngati kuli kofunikira kugawa node yatsopano ya kukula kwakukulu kapena kocheperako, imapezeka nthawi yomweyo. Palibe chifukwa chodikirira kuti ikweze.
Njira yonse yochepetsera imayang'aniridwa ndi njira yapadera yowunikira. Kuyang'anira nthawi zonse kumayang'anira mkhalidwe wa node yamakono. Ngati izindikira, mwachitsanzo, kuti purosesa yonyamula yafika pamtengo wofunikira, imadziwitsa dziwe la zochitika zotentha zakufunika kogawa node yatsopano.
Ma proxies ogawidwa, zochitika zotentha ndi kuwunika.
Node yokhala ndi mphamvu yofunikira ilipo. Maiwe osungira amakopera kwa iwo, ndipo makina amayamba kudikirira mphindi yotetezeka kuti asinthe.
Nthawi zambiri nthawi yosinthira imabwera mwachangu. Kenako kulumikizana pakati pa proxy ndi master node yakale kumayimitsidwa, magawo onse amasinthidwa kupita ku node yatsopano.
Gwirani ntchito ndi database ikuyambiranso.
Grafu ikuwonetsa kuti kuyimitsidwa kulidi kwakanthawi kochepa. Grafu yabuluu ikuwonetsa katunduyo, ndipo masitepe ofiira akuwonetsa mphindi zokulira. Kulowetsedwa kwakanthawi kochepa mu graph ya buluu ndiko kuchedwa kwakanthawi kochepa.
Mwa njira, Amazon Aurora imakulolani kuti musunge ndalama zonse ndikuzimitsa nkhokwe pamene sichikugwiritsidwa ntchito, mwachitsanzo, kumapeto kwa sabata. Pambuyo poyimitsa katunduyo, DB imachepetsa pang'onopang'ono mphamvu yake ndikuzimitsa kwakanthawi. Pamene katunduyo abwerera, adzauka bwino kachiwiri.
Mu gawo lotsatira la nkhani ya chipangizo cha Amazon, tikambirana za kukulitsa maukonde. Lembetsani ndipo khalani maso kuti musaphonye nkhaniyo.
pa Vasily Pantyukhin adzapereka lipoti "" Mapangidwe otani a machitidwe ogawidwa omwe amagwiritsidwa ntchito ndi opanga Amazon, ndi zifukwa zotani zolephereka kwa utumiki, ndi zomangamanga zotani za Cell, Constant Work, Shuffle Sharding - zidzakhala zosangalatsa. Pasanathe mwezi umodzi kuti msonkhano uchitike - . October 24 kuwonjezeka mtengo komaliza.
Source: www.habr.com