{"id":39302,"date":"2019-10-31T22:29:40","date_gmt":"2019-10-31T19:29:40","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh\/"},"modified":"2019-10-31T22:29:40","modified_gmt":"2019-10-31T19:29:40","slug":"kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh","title":{"rendered":"Jak AWS \"va\u0159\u00ed\" sv\u00e9 elastick\u00e9 slu\u017eby. \u0160k\u00e1lov\u00e1n\u00ed server\u016f a datab\u00e1z\u00ed","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Clouds are like a magical box \u2014 you specify what you need, and resources simply appear out of nowhere. Virtual machines, databases, networks \u2014 all of these belong only to you. There are other cloud tenants, but in your universe, you are the sole ruler. You are confident that you will always receive the required resources, not accounting for anyone else, and you independently determine how the network will be configured. How does this magic work, allowing the cloud to elastically allocate resources and completely isolate tenants from each other?<\/p>\n

\"Jak
\n
\nAWS Cloud is a mega-super-complex system that has been evolving since 2006. Part of this evolution was witnessed Vasily Pantyuchin<\/strong> \u2014 by an architect of Amazon Web Services. As an architect, he sees not only the end result but also the challenges that AWS overcomes. The greater the understanding of how the system works, the more trust is built. Therefore, Vasily will share the secrets of AWS cloud services. Below, you will find the structure of AWS physical servers, elastic database scalability, custom Amazon databases, and methods to enhance virtual machine performance while simultaneously reducing costs. Knowledge of Amazon's architectural approaches will help you utilize AWS services more effectively and may provide new ideas for building your own solutions.
\n<\/a><\/noindex>
\nAbout the speaker: Vasily Pantyukhin (Hen<\/a><\/noindex>) started as a Unix admin in Russian companies, spent 6 years working with large hardware at Sun Microsystems, and devoted 11 years to advocating for a data-centric world at EMC. He naturally evolved into private clouds and in 2017 ventured into public clouds. He now offers technical advice to help one thrive and develop in AWS.<\/p>\n

Disclaimer: everything below is Vasily's personal opinion and may not coincide with Amazon Web Services' position. The video recording<\/a><\/noindex> of the report on which this article is based is available on our YouTube channel.<\/i><\/p>\n

Why I am talking about the structure of Amazon<\/h2>\n

\nMy first car had a \u2018stick\u2019 \u2014 it had a manual transmission. It was fantastic because I felt that I could drive the car and have full control over it. I also liked that I understood at least roughly how it worked. Naturally, I envisioned the mechanism of the gearbox quite primitively \u2014 somewhat like a bicycle transmission.<\/p>\n

\"Jak
\n
\nV\u0161e bylo skv\u011bl\u00e9, krom\u011b jedn\u00e9 v\u011bci \u2014 st\u00e1n\u00ed v z\u00e1cp\u00e1ch. Sice sed\u00edte a nic ned\u011bl\u00e1te, ale neust\u00e1le \u0159ad\u00edte, ma\u010dk\u00e1te spojku, plyn, brzdu \u2014 t\u00edm se opravdu unav\u00edte. Probl\u00e9m z\u00e1cp se \u010d\u00e1ste\u010dn\u011b vy\u0159e\u0161il, kdy\u017e jsme si v rodin\u011b po\u0159\u00eddili automat. Za volantem se objevilo \u010das na p\u0159em\u00fd\u0161len\u00ed, poslouch\u00e1n\u00ed audioknihy.<\/p>\n

Dal\u0161\u00ed z\u00e1hada se objevila v m\u00e9m \u017eivot\u011b, proto\u017ee jsem p\u0159estal rozum\u011bt tomu, jak moje auto funguje. Modern\u00ed automobil je slo\u017eit\u00e9 za\u0159\u00edzen\u00ed. Auto se p\u0159izp\u016fsobuje z\u00e1rove\u0148 des\u00edtk\u00e1m r\u016fzn\u00fdch parametr\u016f: stisknut\u00ed plynu, brzdy, stylu j\u00edzdy, kvalit\u011b silnice. U\u017e \u00fapln\u011b nerozum\u00edm, jak to funguje.<\/p>\n

Kdy\u017e jsem za\u010dal pracovat s Amazonem, byla to pro m\u011b tak\u00e9 tajemstv\u00ed. Jen\u017ee toto tajemstv\u00ed je na zcela jin\u00e9 \u00farovni, proto\u017ee v aut\u011b je jen jeden \u0159idi\u010d, zat\u00edmco v AWS jich jsou miliony. V\u0161ichni u\u017eivatel\u00e9 sou\u010dasn\u011b \u0159\u00edd\u00ed, ma\u010dkaj\u00ed plyn a brzdu. Je p\u0159ekvapiv\u00e9, \u017ee jedou tam, kam cht\u011bj\u00ed \u2014 pro m\u011b je to z\u00e1zrak! Syst\u00e9m se automaticky p\u0159izp\u016fsobuje, \u0161k\u00e1luje a elasticky upravuje pro ka\u017ed\u00e9ho u\u017eivatele tak, \u017ee m\u00e1 pocit, \u017ee je v t\u00e9to vesm\u00edru s\u00e1m.<\/p>\n

M\u00e1gie se trochu rozpt\u00fdlila, kdy\u017e jsem pozd\u011bji za\u010dal pracovat jako architekt v Amazonu. Vid\u011bl jsem, s jak\u00fdmi probl\u00e9my se pot\u00fdk\u00e1me, jak je \u0159e\u0161\u00edme, jak vyv\u00edj\u00edme slu\u017eby. S rostouc\u00edm porozum\u011bn\u00edm fungov\u00e1n\u00ed syst\u00e9mu nar\u016fst\u00e1 d\u016fv\u011bra ve slu\u017ebu. Proto se chci pod\u011blit o obr\u00e1zek toho, co se skr\u00fdv\u00e1 pod kapotou cloudov\u00e9ho \u0159e\u0161en\u00ed AWS.<\/p>\n

O \u010dem se budeme bavit<\/h2>\n

\nZvolil jsem diverzifikovan\u00fd p\u0159\u00edstup \u2014 vybral jsem \u010dty\u0159i zaj\u00edmav\u00e9 slu\u017eby, o kter\u00fdch stoj\u00ed za to hovo\u0159it.<\/p>\n

Optimalizace server\u016f<\/strong>. Efem\u00e9rn\u00ed cloudy s fyzickou podobou: fyzick\u00e1 datacentra, kde stoj\u00ed fyzick\u00e9 servery, kter\u00e9 bzu\u010d\u00ed, zah\u0159\u00edvaj\u00ed se a blikaj\u00ed kontrolkami.<\/p>\n

Serverless funkce <\/strong>(Lambda) \u2014 pravd\u011bpodobn\u011b nej\u0161k\u00e1lovateln\u011bj\u0161\u00ed slu\u017eba v cloudu.<\/p>\n

\u0160k\u00e1lov\u00e1n\u00ed datab\u00e1ze<\/strong>. \u0158eknu v\u00e1m, jak budujeme na\u0161e vlastn\u00ed \u0161k\u00e1lovateln\u00e9 datab\u00e1ze.<\/p>\n

\u0160k\u00e1lov\u00e1n\u00ed s\u00edt\u011b<\/strong>. Posledn\u00ed \u010d\u00e1st, ve kter\u00e9 odhal\u00edm strukturu na\u0161\u00ed s\u00edt\u011b. Je to \u00fa\u017easn\u00e1 v\u011bc \u2014 ka\u017ed\u00fd u\u017eivatel cloudu si mysl\u00ed, \u017ee je v cloudu s\u00e1m a v\u016fbec nevid\u00ed ostatn\u00ed n\u00e1jemce.<\/p>\n

Pozn\u00e1mka. V tomto \u010dl\u00e1nku se budeme v\u011bnovat optimalizaci server\u016f a \u0161k\u00e1lov\u00e1n\u00ed datab\u00e1z\u00ed. \u0160k\u00e1lov\u00e1n\u00ed s\u00edt\u011b probereme v dal\u0161\u00edm \u010dl\u00e1nku. Kde jsou serverless funkce? Ty byly podrobn\u011b pops\u00e1ny v samostatn\u00e9m \u010dl\u00e1nku \u201eMal\u00fd, ale velmi schopn\u00fd. Unboxing mikrovirtualiza\u010dn\u00edch prost\u0159edk\u016f Firecracker<\/a><\/noindex>\u201c. V n\u011bm je pops\u00e1no n\u011bkolik r\u016fzn\u00fdch zp\u016fsob\u016f \u0161k\u00e1lov\u00e1n\u00ed a podrobn\u011b rozebr\u00e1no \u0159e\u0161en\u00ed Firecracker \u2013 spojen\u00ed nejlep\u0161\u00edch vlastnost\u00ed virtu\u00e1ln\u00edch stroj\u016f a kontejner\u016f.<\/i><\/p><\/blockquote>\n

<\/p>\n

Servery<\/h2>\n

\nCloud je efem\u00e9rn\u00ed. Ale tato efem\u00e9rnost m\u00e1 p\u0159esto fyzickou podobu \u2013 servery. Jejich architektura byla p\u016fvodn\u011b klasick\u00e1. Standardn\u00ed \u010dipov\u00e1 sada x86, s\u00ed\u0165ov\u00e9 karty, Linux, hypervizor Xen, na kter\u00e9m b\u011b\u017eely virtu\u00e1ln\u00ed stroje.<\/p>\n

\"Jak
\n
\nV roce 2012 takov\u00e1 architektura zcela splnila sv\u00e9 \u00fakoly. Xen je skv\u011bl\u00fd hypervizor, ale m\u00e1 jeden z\u00e1sadn\u00ed nedostatek. M\u00e1 dost vysok\u00e9 re\u017eijn\u00ed n\u00e1klady na emulaci za\u0159\u00edzen\u00ed<\/strong>. S p\u0159\u00edchodem nov\u011bj\u0161\u00edch, rychlej\u0161\u00edch s\u00ed\u0165ov\u00fdch karet nebo SSD disk\u016f se tyto re\u017eijn\u00ed n\u00e1klady st\u00e1vaj\u00ed p\u0159\u00edli\u0161 vysok\u00fdmi. Jak se s t\u00edmto probl\u00e9mem vypo\u0159\u00e1dat? Rozhodli jsme se pracovat hned na dvou front\u00e1ch \u2013 optimalizovat jak hardware, tak hypervizor<\/strong>. \u00dakol je velmi v\u00e1\u017en\u00fd.<\/p>\n

Optimalizace hardware a hypervizoru<\/h3>\n

\nD\u011blat v\u0161echno najednou a dob\u0159e se nepoda\u0159\u00ed. Co znamen\u00e1 \u201edob\u0159e\u201c, p\u016fvodn\u011b tak\u00e9 nebylo jasn\u00e9.<\/p>\n

Rozhodli jsme se pou\u017e\u00edt evolu\u010dn\u00ed p\u0159\u00edstup \u2013 m\u011bn\u00edme jednu d\u016fle\u017eitou sou\u010d\u00e1st architektury a zav\u00e1d\u00edme ji do produkce.<\/p><\/blockquote>\n

Stup\u0148ujeme kroky, naslouch\u00e1me st\u00ed\u017enostem a n\u00e1vrh\u016fm. Pak m\u011bn\u00edme dal\u0161\u00ed komponentu. Tak, mal\u00fdmi inkrementy, radik\u00e1ln\u011b m\u011bn\u00edme celou architekturu na z\u00e1klad\u011b zp\u011btn\u00e9 vazby od u\u017eivatel\u016f a podpory.<\/p>\n

Transformace za\u010daly v roce 2013 s t\u00edm nejn\u00e1ro\u010dn\u011bj\u0161\u00edm \u2013 s\u00edt\u00ed. V C3<\/strong> instanc\u00edch byla ke standardn\u00ed s\u00ed\u0165ov\u00e9 kart\u011b p\u0159id\u00e1na speci\u00e1ln\u00ed karta Network Accelerator. Ta byla p\u0159ipojena doslova kr\u00e1tk\u00fdm loopback kabelem na p\u0159edn\u00edm panelu. Nen\u00ed to estetick\u00e9, ale v cloudu to nen\u00ed vid\u011bt. P\u0159\u00edm\u00e1 interakce s hardwarem z\u00e1sadn\u011b zlep\u0161ila jitter a propustnost s\u00edt\u011b.<\/p>\n

D\u00e1le jsme se rozhodli zlep\u0161it p\u0159\u00edstup k blokov\u00e9mu \u00falo\u017ei\u0161ti dat EBS \u2013 Elastic Block Storage. To je kombinace s\u00edt\u011b a \u00falo\u017ei\u0161t\u011b. Slo\u017eitost spo\u010d\u00edv\u00e1 v tom, \u017ee zat\u00edmco na trhu existovaly karty Network Accelerator, mo\u017enosti po\u0159\u00eddit hardware Storage Accelerator nebyly. Proto jsme se obr\u00e1tili na startup Annapurna Labs<\/strong>, kter\u00fd vyvinul speci\u00e1ln\u00ed \u010dipy ASIC pro n\u00e1s. Ty umo\u017enily p\u0159ipojit vzd\u00e1len\u00e9 objemy EBS jako za\u0159\u00edzen\u00ed NVMe.<\/p>\n

Ve instanc\u00edch C4<\/strong> jsme vy\u0159e\u0161ili dva probl\u00e9my. Prvn\u00ed \u2014 p\u0159ipravili jsme se na budoucnost perspektivn\u00ed, ale v t\u00e9 dob\u011b novou technologii NVMe. Druh\u00e1 \u2014 v\u00fdrazn\u011b jsme uvolnili centr\u00e1ln\u00ed procesor p\u0159esunut\u00edm zpracov\u00e1n\u00ed po\u017eadavk\u016f na EBS na novou kartu. Dopadlo to \u00fasp\u011b\u0161n\u011b, a proto je nyn\u00ed Annapurna Labs sou\u010d\u00e1st\u00ed Amazonu.<\/p>\n

Do listopadu 2017 jsme pochopili, \u017ee nastal \u010das zm\u011bnit i samotn\u00fd hypervizor.<\/p>\n

Nov\u00fd hypervizor byl vyvinut na z\u00e1klad\u011b upraven\u00fdch modul\u016f j\u00e1dra KVM.<\/p><\/blockquote>\n

Umo\u017enil z\u00e1sadn\u011b sn\u00ed\u017eit re\u017eijn\u00ed n\u00e1klady na emulaci za\u0159\u00edzen\u00ed a pracovat p\u0159\u00edmo s nov\u00fdmi ASIC. Instance C5<\/strong> byly prvn\u00edmi virtu\u00e1lkami, pod kapotou kter\u00fdch b\u011b\u017e\u00ed nov\u00fd hypervizor. Pojmenovali jsme ho Nitro<\/strong>.<\/p>\n

\"JakEvoluce instanc\u00ed na \u010dasov\u00e9 ose.<\/em><\/p>\n

V\u0161echny nov\u00e9 typy virtu\u00e1ln\u00edch stroj\u016f, kter\u00e9 vznikly od listopadu 2017, pracuj\u00ed na tomto hypervizoru. Bare Metal instance nemaj\u00ed hypervizor,<\/strong>, ale tak\u00e9 se naz\u00fdvaj\u00ed Nitro, proto\u017ee pou\u017e\u00edvaj\u00ed specializovan\u00e9 Nitro karty.<\/p>\n

B\u011bhem n\u00e1sleduj\u00edc\u00edch dvou let se po\u010det typ\u016f Nitro instanc\u00ed zv\u00fd\u0161il na v\u00edce ne\u017e dv\u011b des\u00edtky: A1, C5, M5, T3 a dal\u0161\u00ed.<\/p>\n

\"Jak
\nTypy instanc\u00ed.<\/em><\/p>\n

Jak funguj\u00ed modern\u00ed Nitro stroje<\/h3>\n

\nMaj\u00ed t\u0159i hlavn\u00ed komponenty: Nitro hypervizor (o n\u011bm jsme hovo\u0159ili v\u00fd\u0161e), bezpe\u010dnostn\u00ed \u010dip a Nitro karty.<\/p>\n

Bezpe\u010dnostn\u00ed \u010dip<\/strong> je integrov\u00e1n p\u0159\u00edmo na z\u00e1kladn\u00ed desku. Kontroluje \u0159adu d\u016fle\u017eit\u00fdch funkc\u00ed, nap\u0159\u00edklad kontrolu spu\u0161t\u011bn\u00ed hostitelsk\u00e9ho opera\u010dn\u00edho syst\u00e9mu.<\/p>\n

Nitro karty<\/strong> existuj\u00ed ve \u010dty\u0159ech typech. V\u0161echny byly vyvinuty Annapurna Labs a jsou zalo\u017eeny na spole\u010dn\u00fdch ASIC. \u010c\u00e1st jejich firmwaru je tak\u00e9 spole\u010dn\u00e1.<\/p>\n

\"Jak
\n\u010cty\u0159i typy Nitro karet.<\/em><\/p>\n

Jedna z karet je ur\u010dena pro pr\u00e1ci s s\u00ed\u0165ov\u00fdmi<\/strong>VPC<\/strong>za\u0159\u00edzen\u00edmi. Ta je viditeln\u00e1 ve virtu\u00e1lk\u00e1ch jako s\u00ed\u0165ov\u00e1 karta ENA \u2014 Elastic Network Adaptor<\/strong>. Ta tak\u00e9 inkapskuluje provoz p\u0159i jeho p\u0159enosu p\u0159es fyzickou s\u00ed\u0165 (o tom si pov\u00edme ve druh\u00e9 \u010d\u00e1sti \u010dl\u00e1nku), kontroluje firewall Security Groups, zodpov\u00edd\u00e1 za routov\u00e1n\u00ed a dal\u0161\u00ed s\u00ed\u0165ov\u00e9 \u00fakoly.<\/p>\n

Samostatn\u00e9 karty pracuj\u00ed s blokov\u00fdm \u00falo\u017ei\u0161t\u011bm EBS<\/strong> a disky, kter\u00e9 jsou integrov\u00e1ny do serveru. Pro hostitelskou virtu\u00e1ln\u00ed ma\u0161inu jsou prezentov\u00e1ny jako NVMe adapt\u00e9ry<\/strong>. Tak\u00e9 se pod\u00edlej\u00ed na \u0161ifrov\u00e1n\u00ed dat a monitorov\u00e1n\u00ed disk\u016f.<\/p>\n

Syst\u00e9m Nitro karet, hypervizoru a bezpe\u010dnostn\u00edho \u010dipu je spojen do s\u00edt\u011b SDN nebo Software Defined Network<\/strong>. Za spr\u00e1vu t\u00e9to s\u00edt\u011b (Control Plane) odpov\u00edd\u00e1 karta-ovlada\u010d<\/strong>.<\/p>\n

Samoz\u0159ejm\u011b, pokra\u010dujeme ve v\u00fdvoji nov\u00fdch ASIC. Nap\u0159\u00edklad na konci roku 2018 jsme vydali \u010dip Inferentia, kter\u00fd umo\u017e\u0148uje efektivn\u011bj\u0161\u00ed zpracov\u00e1n\u00ed \u00faloh strojov\u00e9ho u\u010den\u00ed.<\/p>\n

\"Jak
\n\u010cip Inferentia pro zpracov\u00e1n\u00ed strojov\u00e9ho u\u010den\u00ed.<\/em><\/p>\n

\u0160k\u00e1lovateln\u00e1 datab\u00e1ze<\/h2>\n

\nTradi\u010dn\u00ed datab\u00e1ze m\u00e1 vrstevnatou strukturu. Pokud to zjednodu\u0161\u00edme, vymezujeme n\u00e1sleduj\u00edc\u00ed \u00farovn\u011b.<\/p>\n