{"id":84876,"date":"2020-06-11T13:43:18","date_gmt":"2020-06-11T11:43:18","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno"},"modified":"2020-06-11T13:43:18","modified_gmt":"2020-06-11T11:43:18","slug":"uskoryaem-internet-zaprosy-i-spim-spokojno","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sk\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno","title":{"rendered":"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/07e2e32d406b8b5d6d6cb3f627a31c3a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNetflix je l\u00eddrom na trhu internetovej telev\u00edzie, spolo\u010dnos\u0165, ktor\u00e1 vytvorila a akt\u00edvne rozv\u00edja tento segment. Netflix je zn\u00e1my nielen rozsiahlym katal\u00f3gom filmov a seri\u00e1lov dostupn\u00fdch takmer z ak\u00e9hoko\u013evek miesta na svete a z ak\u00e9hoko\u013evek zariadenia s displejom, ale aj spo\u013eahlivou infra\u0161trukt\u00farou a jedine\u010dnou kult\u00farou in\u017einierstva. <\/p>\n<p>N\u00e1zorn\u00fd pr\u00edklad pr\u00edstupu Netflixu k v\u00fdvoju a podpore zlo\u017eit\u00fdch syst\u00e9mov na DevOops 2019 predstavil <noindex><a rel=\"nofollow\" href=\"https:\/\/sfedov.com\">Sergej Fedorov<\/a><\/noindex> \u2014 riadite\u013e pre v\u00fdvoj v Netflixe. Absolvent Fakulty matematick\u00fdch a fyzik\u00e1lnych vied NNU im. Lobachevsk\u00e9ho, Sergej je jedn\u00fdm z prv\u00fdch in\u017einierov v Open Connect \u2014 CDN t\u00edmu v Netflixe. Vytvoril syst\u00e9my monitorovania a anal\u00fdzy video d\u00e1t, spustil popul\u00e1rnu slu\u017ebu na hodnotenie r\u00fdchlosti internetov\u00e9ho pripojenia FAST.com a posledn\u00e9 roky pracuje na optimaliz\u00e1cii internetov\u00fdch po\u017eiadaviek, aby aplik\u00e1cia Netflix fungovala \u010do najr\u00fdchlej\u0161ie pre pou\u017e\u00edvate\u013eov.<\/p>\n<p>Predn\u00e1\u0161ka z\u00edskala najlep\u0161ie recenzie od \u00fa\u010dastn\u00edkov konferencie a pripravili sme pre v\u00e1s textov\u00fa verziu.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\n<center><div class=\"youtube-placeholder\" data-id=\"n7Te9WIz1ho\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/n7Te9WIz1ho\/hqdefault.jpg\" alt=\"Prehrajte video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<h2>V predn\u00e1\u0161ke Sergej podrobne hovoril<\/h2>\n<p><\/p>\n<ul>\n<li>o tom, \u010do ovplyv\u0148uje oneskorenie internetov\u00fdch po\u017eiadaviek medzi klientom a serverom;<\/li>\n<li>ako toto oneskorenie zn\u00ed\u017ei\u0165;<\/li>\n<li>ako navrhova\u0165, udr\u017eiava\u0165 a monitorova\u0165 odoln\u00e9 vo\u010di chyb\u00e1m syst\u00e9my;<\/li>\n<li>ako dosahova\u0165 v\u00fdsledky v kr\u00e1tkych term\u00ednoch a s minim\u00e1lnym rizikom pre podnikanie;<\/li>\n<li>ako analyzova\u0165 v\u00fdsledky a u\u010di\u0165 sa na chyb\u00e1ch.<\/li>\n<\/ul>\n<p>\nOdpovede na tieto ot\u00e1zky potrebuj\u00fa nielen t\u00ed, ktor\u00ed pracuj\u00fa vo ve\u013ek\u00fdch korpor\u00e1ci\u00e1ch. <\/p>\n<p>Predlo\u017een\u00e9 princ\u00edpy a techniky by mal pozna\u0165 a praktikova\u0165 ka\u017ed\u00fd, kto vyv\u00edja a udr\u017eiava internetov\u00e9 produkty.<\/p>\n<p><b>\u010ealej \u2014 rozpr\u00e1vanie z poh\u013eadu re\u010dn\u00edka.<\/b><\/p>\n<h2>D\u00f4le\u017eitos\u0165 r\u00fdchlosti internetu<\/h2>\n<p>\nR\u00fdchlos\u0165 internetov\u00fdch po\u017eiadaviek je priamo spojen\u00e1 s podnikan\u00edm. Zv\u00e1\u017ete oblas\u0165 nakupovania: spolo\u010dnos\u0165 Amazon v roku 2009 <noindex><a rel=\"nofollow\" href=\"https:\/\/www.gigaspaces.com\/blog\/amazon-found-every-100ms-of-latency-cost-them-1-in-sales\/\">uviedla<\/a><\/noindex>, \u017ee oneskorenie 100 ms vedie k strate 1% predaja.<\/p>\n<p>St\u00e1va sa \u010doraz viac mobiln\u00fdch zariaden\u00ed, a spolu s nimi aj mobiln\u00e9 str\u00e1nky a aplik\u00e1cie. Ak sa va\u0161a str\u00e1nka na\u010d\u00edta dlh\u0161ie ako 3 sekundy, prich\u00e1dzate o pribli\u017ene polovicu pou\u017e\u00edvate\u013eov. Od <noindex><a rel=\"nofollow\" href=\"https:\/\/webmasters.googleblog.com\/2018\/01\/using-page-speed-in-mobile-search.html\">j\u00fala 2018<\/a><\/noindex> Google zoh\u013ead\u0148uje r\u00fdchlos\u0165 na\u010d\u00edtania va\u0161ej str\u00e1nky vo vyh\u013ead\u00e1van\u00ed: \u010d\u00edm r\u00fdchlej\u0161ia je str\u00e1nka, t\u00fdm vy\u0161\u0161ia je jej poz\u00edcia v Google.<\/p>\n<p>R\u00fdchlos\u0165 pripojenia je tie\u017e d\u00f4le\u017eit\u00e1 vo finan\u010dn\u00fdch in\u0161tit\u00faci\u00e1ch, kde s\u00fa oneskorenia kritick\u00e9. V roku 2015 spolo\u010dnos\u0165 Hibernia Networks <noindex><a rel=\"nofollow\" href=\"https:\/\/www.submarinenetworks.com\/en\/systems\/trans-atlantic\/project-express\/hibernia-express-connects-new-york-to-london-in-under-58-95ms\">uzavrela<\/a><\/noindex> kabel medzi New Yorkom a Lond\u00fdnom v hodnote 400 mili\u00f3nov dol\u00e1rov, aby sa zn\u00ed\u017eila oneskorenie medzi mestami o 6 ms. Predstavte si, 66 mili\u00f3nov dol\u00e1rov za 1 ms zn\u00ed\u017eenia oneskorenia!<\/p>\n<p>Pod\u013ea <noindex><a rel=\"nofollow\" href=\"https:\/\/hpbn.co\/primer-on-web-performance\/\">v\u00fdskumu<\/a><\/noindex>, r\u00fdchlos\u0165 pripojenia nad 5 Mbit\/s prest\u00e1va priamo ovplyv\u0148ova\u0165 r\u00fdchlos\u0165 na\u010d\u00edtania be\u017enej webovej str\u00e1nky. Av\u0161ak medzi oneskoren\u00edm pripojenia a r\u00fdchlos\u0165ou na\u010d\u00edtania str\u00e1nky existuje line\u00e1rna z\u00e1vislos\u0165:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/340ca1f2c5bdd6ca95a5caf02f67e3fd.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNetflix v\u0161ak nie je be\u017en\u00fd produkt. Vplyv oneskorenia a r\u00fdchlosti na pou\u017e\u00edvate\u013ea je akt\u00edvnou oblas\u0165ou anal\u00fdzy a v\u00fdvoja. Existuje na\u010d\u00edtanie aplik\u00e1cie a v\u00fdber obsahu, ktor\u00e9 z\u00e1visia od oneskorenia, ale na\u010d\u00edtanie statick\u00fdch prvkov a streamovanie tie\u017e z\u00e1vis\u00ed od r\u00fdchlosti pripojenia. Anal\u00fdza a optimaliz\u00e1cia k\u013e\u00fa\u010dov\u00fdch faktorov, ktor\u00e9 ovplyv\u0148uj\u00fa kvalitu slu\u017eby pre pou\u017e\u00edvate\u013ea, je akt\u00edvna oblas\u0165 v\u00fdvoja nieko\u013ek\u00fdch t\u00edmov v Netflixe. Jedn\u00fdm z \u00faloh je zn\u00ed\u017eenie oneskorenia po\u017eiadaviek medzi zariadeniami Netflix a cloudovou infra\u0161trukt\u00farou.<\/p>\n<p>V spr\u00e1ve sa zameriame pr\u00e1ve na zn\u00ed\u017eenie oneskorenia (latency) na pr\u00edklade infra\u0161trukt\u00fary Netflix. Pozrieme sa prakticky, ako pristupova\u0165 k procesom navrhovania, v\u00fdvoja a prev\u00e1dzkovania zlo\u017eit\u00fdch distribuovan\u00fdch syst\u00e9mov a tr\u00e1vi\u0165 \u010das na inov\u00e1ci\u00e1ch a v\u00fdsledkoch, nie na diagnostike prev\u00e1dzkov\u00fdch probl\u00e9mov a por\u00fach.<\/p>\n<h2>Vo vn\u00fatri Netflixu<\/h2>\n<p>\nTis\u00edcky r\u00f4znych zariaden\u00ed podporuj\u00fa aplik\u00e1cie Netflix. Ich v\u00fdvojom sa zaoberaj\u00fa \u0161tyri r\u00f4zne t\u00edmy, ktor\u00e9 vytv\u00e1raj\u00fa samostatn\u00e9 verzie klienta pre Android, iOS, TV a webov\u00e9 prehliada\u010de. A vynaklad\u00e1me ve\u013ea \u00fasilia na vylep\u0161enie a personaliz\u00e1ciu pou\u017e\u00edvate\u013esk\u00e9ho rozhrania. Na tento \u00fa\u010del sp\u00fa\u0161\u0165ame paralelne stovky A\/B testov.<\/p>\n<p>Personaliz\u00e1cia je podporovan\u00e1 v\u010faka stovk\u00e1m mikroslu\u017eieb v cloude AWS, ktor\u00e9 poskytuj\u00fa personalizovan\u00e9 \u00fadaje pre pou\u017e\u00edvate\u013ea, spracovanie po\u017eiadaviek, telemetriu, Big Data a Encoding. Vizualiz\u00e1cia prev\u00e1dzky vyzer\u00e1 takto:<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/youtu.be\/n7Te9WIz1ho?t=364\">Odkaz na video s demon\u0161tr\u00e1ciou (6:04-6:23)<\/a><\/noindex><\/p>\n<p>Na\u013eavo je vstupn\u00fd bod a n\u00e1sledne sa doprava rozde\u013euje medzi nieko\u013eko stoviek mikroslu\u017eieb, ktor\u00e9 podporuj\u00fa r\u00f4zne backendov\u00e9 t\u00edmy.<\/p>\n<p>\u010eal\u0161ou d\u00f4le\u017eitou s\u00fa\u010das\u0165ou na\u0161ej infra\u0161trukt\u00fary je Open Connect CDN, ktor\u00e1 dod\u00e1va statick\u00fd obsah - vide\u00e1, obr\u00e1zky, k\u00f3d pre klientov at\u010f. CDN je umiestnen\u00e1 na vlastn\u00fdch serveroch (OCA - Open Connect Appliance). Vo vn\u00fatri sa nach\u00e1dzaj\u00fa matice SSD a HDD diskov riaden\u00e9 optimalizovan\u00fdm FreeBSD, s NGINX a s\u00faborom slu\u017eieb. Navrhujeme a optimalizujeme hardv\u00e9rov\u00e9 a softv\u00e9rov\u00e9 komponenty tak, aby tento CDN server mohol posiela\u0165 \u010do najviac \u00fadajov pou\u017e\u00edvate\u013eom. <\/p>\n<p>\u201eStena\u201c z t\u00fdchto serverov na bod\u011b v\u00fdmeny internetov\u00e9ho trafiku (Internet eXchange - IX) vyzer\u00e1 takto:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/8155601c344acb1eb05b925193af8f9a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nInternet Exchange poskytuje mo\u017enos\u0165 internetov\u00fdm poskytovate\u013eom a dod\u00e1vate\u013eom obsahu \u201epripoji\u0165\u201c sa navz\u00e1jom pre priamu v\u00fdmenu \u00fadajov na internete. Na celom svete je pribli\u017ene 70-80 Internet Exchange bodov, kde s\u00fa umiestnen\u00e9 na\u0161e servery, a my sa sami zaober\u00e1me ich in\u0161tal\u00e1ciou a \u00fadr\u017ebou:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/77a2465314e47c0b2c90dbf7a2fb5e6b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nOkrem toho tie\u017e poskytujeme servery priamo internetov\u00fdm poskytovate\u013eom, ktor\u00e9 si in\u0161taluj\u00fa do svojej siete, \u010d\u00edm zlep\u0161ujeme lokaliz\u00e1ciu Netflix trafficu a kvalitu streamovania pre pou\u017e\u00edvate\u013eov:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/63e077d2649bfafea5c7af43a0faaf85.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nS\u00fabor slu\u017eieb AWS zodpoved\u00e1 za spracovanie video po\u017eiadaviek od klientov k CDN serverom, ako aj konfigur\u00e1ciu samotn\u00fdch serverov - aktualizovanie obsahu, softv\u00e9rov\u00e9ho k\u00f3du, nastaven\u00ed at\u010f. Na to sme tie\u017e vybudovali backbone network, ktor\u00e1 sp\u00e1ja servery v Internet Exchange bodoch s AWS. Backbone network predstavuje glob\u00e1lnu sie\u0165 optick\u00fdch k\u00e1blov a routerov, ktor\u00e9 m\u00f4\u017eeme navrhova\u0165 a konfigurova\u0165 pod\u013ea na\u0161ich potrieb.<\/p>\n<p>V porovnan\u00ed <noindex><a rel=\"nofollow\" href=\"https:\/\/www.sandvine.com\/hubfs\/Sandvine_Redesign_2019\/Downloads\/Internet%20Phenomena\/Internet%20Phenomena%20Report%20Q32019%2020190910.pdf\">odhadom Sandvine<\/a><\/noindex>, na\u0161a CDN infra\u0161trukt\u00fara dod\u00e1va v \u0161pi\u010dk\u00e1ch pribli\u017ene \u215b svetov\u00e9ho internetov\u00e9ho trafiku a \u2153 trafiku v Severnej Amerike, kde Netflix existuje najdlh\u0161ie. P\u00f4sobiv\u00e9 \u010d\u00edsla, ale pre m\u0148a jedn\u00fdm z naj\u00fa\u017easnej\u0161\u00edch \u00faspechov je to, \u017ee cel\u00fd syst\u00e9m CDN sa vyv\u00edja a udr\u017eiava t\u00edmom menej ako 150 \u013eud\u00ed.<\/p>\n<p>P\u00f4vodne bola infra\u0161trukt\u00fara CDN navrhnut\u00e1 na dod\u00e1vanie video d\u00e1t. Av\u0161ak, s \u010dasom sme si uvedomili, \u017ee ju m\u00f4\u017eeme pou\u017ei\u0165 aj na optimaliz\u00e1ciu dynamick\u00fdch po\u017eiadaviek od klientov v AWS cloude.<\/p>\n<h2>O ur\u00fdchlen\u00ed internetu<\/h2>\n<p>\nDnes m\u00e1 Netflix 3 regi\u00f3ny AWS a latencia po\u017eiadaviek na cloud bude z\u00e1visie\u0165 od toho, ak\u00e1 \u010faleko sa klient nach\u00e1dza od najbli\u017e\u0161ieho regi\u00f3nu. M\u00e1me mno\u017estvo CDN serverov, ktor\u00e9 sa pou\u017e\u00edvaj\u00fa na distrib\u00faciu statick\u00e9ho obsahu. M\u00f4\u017eeme nejako vyu\u017ei\u0165 t\u00fato infra\u0161trukt\u00faru na ur\u00fdchlenie dynamick\u00fdch po\u017eiadaviek? Pri tom nie je mo\u017en\u00e9 tieto po\u017eiadavky uklada\u0165 do cache \u2014 API s\u00fa personalizovan\u00e9 a ka\u017ed\u00fd v\u00fdsledok je unik\u00e1tny.<\/p>\n<p>Po\u010fme vytvori\u0165 proxy na CDN serveri a za\u010dnime cez neho prev\u00e1dza\u0165 dopravu. Bude to r\u00fdchlej\u0161ie?<\/p>\n<h2>Z\u00e1klady<\/h2>\n<p>\nPripome\u0148me si, ako funguj\u00fa sie\u0165ov\u00e9 protokoly. Dnes v\u00e4\u010d\u0161ina prem\u00e1vky na internete pou\u017e\u00edva HTTPs, ktor\u00fd z\u00e1vis\u00ed od ni\u017e\u0161\u00edch protokolov TCP a TLS. Aby sa klient pripojil k serveru, vykon\u00e1 handshake, a na vytvorenie zabezpe\u010den\u00e9ho spojenia mus\u00ed klient vymeni\u0165 spr\u00e1vy so serverom trikr\u00e1t a e\u0161te aspo\u0148 raz, aby preniesol d\u00e1ta. Pri latencii na jedno kolo (RTT) 100 ms n\u00e1m bude trva\u0165 400 ms, aby sme z\u00edskali prv\u00fd bit d\u00e1t:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ab12024f5a9940ca470f27821ca3c631.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAk umiestnime certifik\u00e1ty na CDN server, \u010das 'r\u00fakou' medzi klientom a serverom m\u00f4\u017eeme zna\u010dne skr\u00e1ti\u0165, ak je CDN bli\u017e\u0161ie. Predpokladajme, \u017ee latencia do CDN servera je 30 ms. Potom na z\u00edskanie prv\u00e9ho bitu u\u017e bude potrebn\u00fdch len 220 ms:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/d63134bb51592aaa6a5e0c86f1a070e0.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAle v\u00fdhody t\u00fdmto nekon\u010dia. Po tom, ako je spojenie u\u017e vytvoren\u00e9, TCP zvy\u0161uje congestion window (po\u010det inform\u00e1ci\u00ed, ktor\u00e9 m\u00f4\u017ee pren\u00e1\u0161a\u0165 cez toto spojenie paralelne). Ak sa strat\u00ed paket d\u00e1t, klasick\u00e9 implement\u00e1cie TCP protokolu (ako TCP New Reno) zni\u017euj\u00fa otvoren\u00e9 'okno' na polovicu. R\u00fdchlos\u0165 rastu congestion window a jeho obnovenia po strate op\u00e4\u0165 z\u00e1vis\u00ed od latencie (RTT) do servera. Ak toto spojenie ide len do CDN servera, t\u00e1to obnova bude r\u00fdchlej\u0161ia. Pri tom strata paketov je \u0161tandardn\u00fd jav, najm\u00e4 pre bezdr\u00f4tov\u00e9 siete.<\/p>\n<p>R\u00fdchlos\u0165 internetu m\u00f4\u017ee klesa\u0165, najm\u00e4 po\u010das \u0161pi\u010dkov\u00fdch hod\u00edn kv\u00f4li pou\u017e\u00edvate\u013esk\u00e9mu trafficu, \u010do m\u00f4\u017ee vies\u0165 k \"z\u00e1cp\u00e1m\". V internete neexistuje sp\u00f4sob, ako da\u0165 prioritu jedn\u00fdm po\u017eiadavk\u00e1m pred in\u00fdmi. Napr\u00edklad, aby sa prioritizovali mal\u00e9 a citliv\u00e9 na oneskorenie po\u017eiadavky v porovnan\u00ed s \"\u0165a\u017ek\u00fdmi\" tokmi d\u00e1t, ktor\u00e9 za\u0165a\u017euj\u00fa sie\u0165. V na\u0161om pr\u00edpade v\u0161ak vlastn\u00edctvo vlastnej backbone siete umo\u017e\u0148uje spravi\u0165 to na \u010dasti cesty po\u017eiadavky \u2014 medzi CDN a cloudom, a m\u00f4\u017eeme ju \u00faplne konfigurova\u0165. M\u00f4\u017eeme nastavi\u0165, aby mal\u00e9 a oneskoren\u00e9 pakety mali prioritu a v\u00e4\u010d\u0161ie toky d\u00e1t i\u0161li o nie\u010do nesk\u00f4r. \u010c\u00edm bli\u017e\u0161ie je CDN k klientovi, t\u00fdm je vy\u0161\u0161ia efekt\u00edvnos\u0165.<\/p>\n<p>Na oneskorenie maj\u00fa vplyv aj aplika\u010dn\u00e9 protokoly (\u00farove\u0148 OSI 7). Nov\u00e9 protokoly, ako HTTP\/2, umo\u017e\u0148uj\u00fa optimalizova\u0165 v\u00fdkon paraleln\u00fdch po\u017eiadaviek. Av\u0161ak m\u00e1me klientov Netflixu so star\u00fdmi zariadeniami, ktor\u00e9 nepodporuj\u00fa nov\u00e9 protokoly. Nie v\u0161etk\u00fdch klientov je mo\u017en\u00e9 aktualizova\u0165 alebo optim\u00e1lne nastavi\u0165. Medzi CDN proxy a cloudom je v\u0161ak pln\u00e1 kontrola, ako aj mo\u017enos\u0165 vyu\u017e\u00edva\u0165 nov\u00e9, optim\u00e1lne protokoly a nastavenia. Neefekt\u00edvna \u010das\u0165 so star\u00fdmi protokolmi bude fungova\u0165 len medzi klientom a CDN serverom. Navy\u0161e, m\u00f4\u017eeme multiplexova\u0165 po\u017eiadavky na u\u017e vytvoren\u00e9 spojenie medzi CDN a cloudom, \u010d\u00edm zlep\u0161\u00edme vyu\u017eitie spojenia na \u00farovni TCP.<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/a05b49680734670dcfce9111f3ba1919.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h2>Meranie<\/h2>\n<p>\nNapriek tomu, \u017ee te\u00f3ria s\u013eubuje zlep\u0161enia, nepon\u00e1h\u013eame sa okam\u017eite spusti\u0165 syst\u00e9m do produkcie. Namiesto toho mus\u00edme najprv dok\u00e1za\u0165, \u017ee n\u00e1pad bude fungova\u0165 v praxi. Na to mus\u00edme odpoveda\u0165 na nieko\u013eko ot\u00e1zok:<\/p>\n<ul>\n<li><b>R\u00fdchlos\u0165<\/b>Bude proxy r\u00fdchlej\u0161ia?<\/li>\n<li><b>Spo\u013eahlivos\u0165<\/b>Bude sa \u010dastej\u0161ie rozb\u00edja\u0165?<\/li>\n<li><b>Zlo\u017eitost<\/b>Ako integrova\u0165 s aplik\u00e1ciami?<\/li>\n<li><b>Cena<\/b>Ko\u013eko stoj\u00ed nasadenie dodato\u010dnej infra\u0161trukt\u00fary?<\/li>\n<\/ul>\n<p>\nPresk\u00famame podrobne n\u00e1\u0161 pr\u00edstup k hodnoteniu prvej ot\u00e1zky. Ostatn\u00e9 sa sk\u00famaj\u00fa podobn\u00fdm sp\u00f4sobom.<\/p>\n<p>Na anal\u00fdzu r\u00fdchlosti po\u017eiadaviek chceme z\u00edska\u0165 \u00fadaje pre v\u0161etk\u00fdch pou\u017e\u00edvate\u013eov, nezabera\u0165 ve\u013ea \u010dasu na v\u00fdvoj a nezhodi\u0165 produkciu. Preto existuje nieko\u013eko pr\u00edstupov:<\/p>\n<ol>\n<li>RUM, alebo pas\u00edvne meranie po\u017eiadaviek. Meriame \u010das vykon\u00e1vania aktu\u00e1lnych po\u017eiadaviek od pou\u017e\u00edvate\u013eov a zabezpe\u010dujeme \u00fapln\u00e9 pokrytie pou\u017e\u00edvate\u013eov. Nev\u00fdhodou je, \u017ee sign\u00e1l nie je ve\u013emi stabiln\u00fd z d\u00f4vodu mnoh\u00fdch faktorov, ako s\u00fa r\u00f4zne ve\u013ekosti po\u017eiadaviek, \u010das spracovania na serveri a kliente. Okrem toho nie je mo\u017en\u00e9 otestova\u0165 nov\u00fa konfigur\u00e1ciu bez vplyvu na produkciu.<\/li>\n<li>Laborat\u00f3rne testy. \u0160peci\u00e1lne servery a infra\u0161trukt\u00fara, ktor\u00e9 imituj\u00fa z\u00e1kazn\u00edkov. S ich pomocou vykon\u00e1vame potrebn\u00e9 testy. Tak z\u00edskavame \u00fapln\u00fa kontrolu nad v\u00fdsledkami meran\u00ed a jasn\u00fd sign\u00e1l. Av\u0161ak nem\u00e1me \u00fapln\u00e9 pokrytie zariaden\u00ed a polohy pou\u017e\u00edvate\u013eov (najm\u00e4 s slu\u017ebou po celom svete a podporou tis\u00edc modelov zariaden\u00ed).<\/li>\n<\/ol>\n<p>\nAko je mo\u017en\u00e9 spoji\u0165 v\u00fdhody oboch met\u00f3d?<\/p>\n<p>N\u00e1\u0161 t\u00edm na\u0161iel rie\u0161enie. Nap\u00edsali sme mal\u00fd k\u00fasok k\u00f3du \u2014 vzorok \u2014 ktor\u00fd sme integrovali do na\u0161ej aplik\u00e1cie. Vzorci n\u00e1m umo\u017e\u0148uj\u00fa vykon\u00e1va\u0165 plne kontrolovan\u00e9 sie\u0165ov\u00e9 testy z na\u0161ich zariaden\u00ed. Funguje to takto: <\/p>\n<ol>\n<li>Hne\u010f po na\u010d\u00edtan\u00ed aplik\u00e1cie a dokon\u010den\u00ed \u00favodnej aktivity sp\u00fa\u0161\u0165ame na\u0161e vzorci. <\/li>\n<li>Klient vykon\u00e1va po\u017eiadavku na server a dost\u00e1va \"recept\" testu. Recept je zoznam URL adries, na ktor\u00e9 je potrebn\u00e9 vykona\u0165 HTTP(s) po\u017eiadavku. Okrem toho recept konfiguruje parametre po\u017eiadaviek: oneskorenia medzi po\u017eiadavkami, objem po\u017eadovan\u00fdch \u00fadajov, HTTP(s) hlavi\u010dky a pod. Pri tom m\u00f4\u017eeme paralelne testova\u0165 nieko\u013eko r\u00f4znych receptov \u2014 pri po\u017eiadavke na konfigur\u00e1ciu n\u00e1hodne ur\u010dujeme, ktor\u00fd recept poskytn\u00fa\u0165. <\/li>\n<li>\u010cas spustenia vzorky sa vyber\u00e1 tak, aby nezr\u00e1\u017eal s akt\u00edvnym pou\u017e\u00edvan\u00edm sie\u0165ov\u00fdch zdrojov na klientovi. V podstate sa vyber\u00e1 \u010das, ke\u010f nie je klient akt\u00edvny.<\/li>\n<li>Po obdr\u017ean\u00ed receptu klient vykon\u00e1va po\u017eiadavky na ka\u017ed\u00fa z URL adries paralelne. Po\u017eiadavka na ka\u017ed\u00fa z adries m\u00f4\u017ee by\u0165 opakovan\u00e1 \u2014 tzv. \u201epulzy\u201c. Na prvom pulze meriame, ko\u013eko \u010dasu trvalo na nadviazanie spojenia a stiahnutie \u00fadajov. Na druhom pulze meriame \u010das na\u010d\u00edtania \u00fadajov cez u\u017e nadviazan\u00e9 spojenie. Pred tret\u00edm m\u00f4\u017eeme nastavi\u0165 oneskorenie a zmera\u0165 r\u00fdchlos\u0165 nadviazania opakovan\u00e9ho spojenia a pod.\n<p>Po\u010das testu mer\u00e1me v\u0161etky parametre, ktor\u00e9 m\u00f4\u017ee zariadenie z\u00edska\u0165:<\/p>\n<ul>\n<li>\u010das DNS dopytu;<\/li>\n<li>\u010das na nadviazanie spojenia cez TCP;<\/li>\n<li>\u010das na nadviazanie spojenia cez TLS;<\/li>\n<li>\u010das na prijatie prv\u00e9ho bajtu d\u00e1t;<\/li>\n<li>celkov\u00fd \u010das na\u010d\u00edtania;<\/li>\n<li>status k\u00f3d v\u00fdsledku.<\/li>\n<\/ul>\n<\/li>\n<li> Po skon\u010den\u00ed v\u0161etk\u00fdch pulsov vzorka na\u010d\u00edta v\u00fdsledky v\u0161etk\u00fdch meran\u00ed na anal\u00fdzu.<\/li>\n<\/ol>\n<p>\n<img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/fff11d487b7c7725707cdeeca0734296.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u013e\u00fa\u010dov\u00fdmi bodmi s\u00fa minim\u00e1lna z\u00e1vislos\u0165 od logiky na klientovi, spracovanie d\u00e1t na serveri a meranie paraleln\u00fdch dopytov. Takto z\u00edskavame mo\u017enos\u0165 izolova\u0165 a testova\u0165 vplyv r\u00f4znych faktorov na v\u00fdkonnos\u0165 dopytov, variabilizova\u0165 ich v r\u00e1mci jedn\u00e9ho receptu, a z\u00edskava\u0165 v\u00fdsledky z re\u00e1lnych klientov.<\/p>\n<p>Tak\u00e1to infra\u0161trukt\u00fara sa uk\u00e1zala ako u\u017eito\u010dn\u00e1 nielen pre anal\u00fdzu v\u00fdkonnosti dopytov. Moment\u00e1lne m\u00e1me 14 akt\u00edvnych receptov, viac ako 6000 vzoriek za sekundu, ktor\u00e9 z\u00edskavaj\u00fa d\u00e1ta zo v\u0161etk\u00fdch k\u00fatov sveta a s pln\u00fdm pokryt\u00edm zariaden\u00ed. Keby si Netflix kupoval podobn\u00fa slu\u017ebu od extern\u00fdch spolo\u010dnost\u00ed, st\u00e1lo by to mili\u00f3ny dol\u00e1rov ro\u010dne, s ove\u013ea hor\u0161\u00edm pokryt\u00edm.<\/p>\n<h2>Overujeme te\u00f3riu v praxi: prototyp<\/h2>\n<p>\nS takouto s\u00fastavou sme z\u00edskali mo\u017enos\u0165 pos\u00fadi\u0165 \u00fa\u010dinnos\u0165 CDN proxy na oneskorenie dopytov. Teraz je potrebn\u00e9:<\/p>\n<ul>\n<li>vytvori\u0165 prototyp proxy;<\/li>\n<li>umiestni\u0165 prototyp na CDN;<\/li>\n<li>ur\u010di\u0165, ako nasmerova\u0165 klientov na proxy na konkr\u00e9tnom CDN serveri;<\/li>\n<li>porovna\u0165 v\u00fdkonnos\u0165 s dopytmi v AWS bez proxy.<\/li>\n<\/ul>\n<p>\n\u00dalohou je \u010do najr\u00fdchlej\u0161ie pos\u00fadi\u0165 \u00fa\u010dinnos\u0165 navrhnut\u00e9ho rie\u0161enia. Na implement\u00e1ciu prototypu sme vybrali Go, v\u010faka kvalitn\u00fdm sie\u0165ov\u00fdm kni\u017eniciam. Na ka\u017edom CDN serveri sme nain\u0161talovali prototyp proxy ako statick\u00fd bin\u00e1rny s\u00fabor, aby sme minimalizovali z\u00e1vislosti a zjednodu\u0161ili integr\u00e1ciu. V po\u010diato\u010dnej implement\u00e1cii sme maxim\u00e1lne vyu\u017eili \u0161tandardn\u00e9 komponenty a mal\u00e9 modifik\u00e1cie pre HTTP\/2 connection pooling a request multiplexing.<\/p>\n<p>Na vyv\u00e1\u017eenie medzi regi\u00f3nmi AWS sme pou\u017eili geografick\u00fa datab\u00e1zu DNS, t\u00fa ist\u00fa, ktor\u00e1 sa pou\u017e\u00edva na vyv\u00e1\u017eenie klientov. Na v\u00fdber CDN servera pre klienta pou\u017e\u00edvame TCP Anycast pre servery v Internet Exchange (IX). V tomto variante pou\u017e\u00edvame jednu IP adresu pre v\u0161etky CDN servery, pri\u010dom klient bude smerovan\u00fd na CDN server s najmenej IP hops. Na CDN serveroch umiestnen\u00fdch u internetov\u00fdch poskytovate\u013eov (ISP) nem\u00e1me kontrolu nad routerom pre nastavenie TCP Anycast, preto vyu\u017e\u00edvame <noindex><a rel=\"nofollow\" href=\"https:\/\/www.infoq.com\/presentations\/netflix-streaming-arch\/\">t\u00fa ist\u00fa logiku<\/a><\/noindex>, pod\u013ea ktorej s\u00fa klienti smerovan\u00ed k internetov\u00fdm poskytovate\u013eom pre streamovanie videa.<\/p>\n<p>Tak\u017ee m\u00e1me tri typy ciest pre po\u017eiadavku: do cloudu cez otvoren\u00fd internet, cez CDN server v IX alebo cez CDN server umiestnen\u00fd u internetov\u00e9ho poskytovate\u013ea. Na\u0161\u00edm cie\u013eom je pochopi\u0165, ktor\u00e1 cesta je najlep\u0161ia, a ak\u00e9 v\u00fdhody prin\u00e1\u0161a proxy, v porovnan\u00ed s t\u00fdm, ako s\u00fa po\u017eiadavky smerovan\u00e9 do produkcie. Na tento \u00fa\u010del pou\u017e\u00edvame syst\u00e9m testovania nasleduj\u00facim sp\u00f4sobom:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/51b64d5be0aaf0f141484ee0fd373396.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKa\u017ed\u00e1 z ciest sa st\u00e1va samostatn\u00fdm cie\u013eom a sledujeme \u010das, ktor\u00fd sme dostali. Na anal\u00fdzu sp\u00e1jame proxy v\u00fdsledky do jednej skupiny (vyber\u00e1me najlep\u0161\u00ed \u010das medzi IX a ISP proxy) a porovn\u00e1vame ho s \u010dasom po\u017eiadaviek do cloudu bez proxy:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ec01690f6a312e61649282b0e6208778.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAko vidno, v\u00fdsledky boli nejednozna\u010dn\u00e9 \u2014 vo v\u00e4\u010d\u0161ine pr\u00edpadov proxy poskytuje dobr\u00e9 ur\u00fdchlenie, ale existuje aj dostato\u010dn\u00e9 mno\u017estvo klientov, pre ktor\u00fdch sa situ\u00e1cia v\u00fdrazne zhor\u0161\u00ed. <\/p>\n<p>Nakoniec sme urobili nieko\u013eko d\u00f4le\u017eit\u00fdch vec\u00ed:<\/p>\n<ol>\n<li>Odhadli sme o\u010dak\u00e1van\u00fd v\u00fdkon po\u017eiadaviek od klientov do cloudu cez CDN proxy.<\/li>\n<li>Z\u00edskali sme \u00fadaje od skuto\u010dn\u00fdch klientov, zo v\u0161etk\u00fdch typov zariaden\u00ed.<\/li>\n<li>Pochopili sme, \u017ee te\u00f3ria nebola potvrden\u00e1 na 100 % a po\u010diato\u010dn\u00fd n\u00e1vrh s CDN proxy pre n\u00e1s nefunguje.<\/li>\n<li>Nebr\u00e1\u0148ili sme sa \u2014 nemenili sme konfigura\u010dn\u00e9 nastavenia produkcie pre klientov.<\/li>\n<li>Ni\u010d sme nezlomili. <\/li>\n<\/ol>\n<p><\/p>\n<h2>Prototyp 2.0<\/h2>\n<p>\nTak sa vraciame k kresliacej doske a znova opakujeme proces.<\/p>\n<p>My\u0161lienka \u2014 namiesto 100 % proxy ur\u010d\u00edme pre ka\u017ed\u00e9ho klienta najr\u00fdchlej\u0161iu cestu a budeme tam smerova\u0165 po\u017eiadavky \u2014 teda budeme robi\u0165 to, \u010do sa naz\u00fdva client steering.<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/780817f48a4d2b292d5545e0aa1ccc50.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAko to zrealizova\u0165? Nem\u00f4\u017eeme pou\u017ei\u0165 logiku na serverovej strane, preto\u017ee cie\u013eom je pripoji\u0165 sa k tomuto serveru. Mus\u00edme to nejako spravi\u0165 na klientskej strane. A ide\u00e1lne, aby sme to urobili s minim\u00e1lnym mno\u017estvom zlo\u017eitej logiky, aby sme sa nemuseli zaobera\u0165 integr\u00e1ciou s rozsiahlym mno\u017estvom klientskej platformy. <\/p>\n<p>Odpove\u010f \u2014 vyu\u017eitie DNS. V na\u0161om pr\u00edpade m\u00e1me svoju DNS infra\u0161trukt\u00faru, a m\u00f4\u017eeme nastavi\u0165 dom\u00e9nov\u00fa z\u00f3nu, pre ktor\u00fa bud\u00fa na\u0161e servery autoritat\u00edvne. Funguje to takto:<\/p>\n<ol>\n<li>Klient rob\u00ed po\u017eiadavku na DNS server pomocou hostite\u013ea, napr\u00edklad api.netflix.com.<\/li>\n<li>Po\u017eiadavka sa dostane na n\u00e1\u0161 DNS server.<\/li>\n<li>DNS server vie, ak\u00e1 cesta je pre tohto klienta najr\u00fdchlej\u0161ia a vyd\u00e1va zodpovedaj\u00facu IP adresu. <\/li>\n<\/ol>\n<p>\nV rie\u0161en\u00ed je \u010fal\u0161ia zlo\u017eitos\u0165: autoritat\u00edvni DNS poskytovatelia nevidia IP adresu klienta a m\u00f4\u017eu zoh\u013eadni\u0165 iba IP adresu rekurz\u00edvneho resolvera, ktor\u00fdm klient pou\u017e\u00edva. <\/p>\n<p>Nakoniec n\u00e1\u0161 autoritat\u00edvny resolver mus\u00ed prij\u00edma\u0165 rozhodnutie nie pre jednotliv\u00e9ho klienta, ale pre skupinu klientov na z\u00e1klade rekurz\u00edvneho resolvera. <\/p>\n<p>Na rie\u0161enie pou\u017e\u00edvame rovnak\u00e9 vzorky, agregujeme v\u00fdsledky meran\u00ed od klientov k ka\u017ed\u00e9mu z rekurz\u00edvnych resolverov a rozhodujeme, kam t\u00fato skupinu nasmerova\u0165 \u2014 cez proxy cez IX pomocou TCP Anycast, cez ISP proxy alebo priamo do cloudu.<\/p>\n<p>Dost\u00e1vame tak\u00fdto syst\u00e9m:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ad23219938d1cb7b6eef498671c3e151.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDostan\u00e1 model DNS steering umo\u017e\u0148uje nasmerova\u0165 klientov na z\u00e1klade historick\u00fdch pozorovan\u00ed o r\u00fdchlosti pripojen\u00ed od klientov do cloudu. <\/p>\n<p>Op\u00e4\u0165 sa p\u00fdtame \u2014 ako efekt\u00edvne bude tento pr\u00edstup fungova\u0165? Na odpove\u010f op\u00e4\u0165 pou\u017e\u00edvame n\u00e1\u0161 syst\u00e9m vzoriek. Preto nastavujeme konfigur\u00e1ciu ned\u00e1vnosti, kde jeden z cie\u013eov nasleduje smerovanie od DNS steering, druh\u00fd \u2014 ide priamo do cloudu (aktu\u00e1lny produk\u010dn\u00fd syst\u00e9m).<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ceb2ded9367ebd0aa0ede46191a89a88.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNakoniec porovn\u00e1vame v\u00fdsledky a z\u00edskavame hodnotenie efektivity:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ca0db88461d5f2cb0f3f7c07eae14f10.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNakoniec sme sa dozvedeli nieko\u013eko d\u00f4le\u017eit\u00fdch vec\u00ed:<\/p>\n<ol>\n<li>Vyhodnotili sme o\u010dak\u00e1van\u00fd v\u00fdkon po\u017eiadaviek od klientov do cloudu s vyu\u017eit\u00edm DNS Steering.<\/li>\n<li>Z\u00edskali sme \u00fadaje od skuto\u010dn\u00fdch klientov, zo v\u0161etk\u00fdch typov zariaden\u00ed.<\/li>\n<li>Dok\u00e1zali sme \u00fa\u010dinnos\u0165 navrhnutej my\u0161lienky.<\/li>\n<li>Nebr\u00e1\u0148ili sme sa \u2014 nemenili sme konfigura\u010dn\u00e9 nastavenia produkcie pre klientov.<\/li>\n<li>Ni\u010d sme nezlomili.<\/li>\n<\/ol>\n<p><\/p>\n<h2>Teraz k zlo\u017eitosti \u2014 sp\u00fa\u0161\u0165ame do produkcie.<\/h2>\n<p>\nNaj\u013eah\u0161ie je teraz za nami \u2013 m\u00e1me funk\u010dn\u00fd prototyp. Teraz prich\u00e1dza zlo\u017eit\u00e1 \u010das\u0165 \u2013 spusti\u0165 rie\u0161enie pre cel\u00fd Netflix traffic, roz\u0161\u00edri\u0165 ho na 150 mili\u00f3nov pou\u017e\u00edvate\u013eov, tis\u00edce zariaden\u00ed, stovky mikroslu\u017eieb a neust\u00e1le meniaci sa produkt a infra\u0161trukt\u00faru. Na servery Netflix prich\u00e1dzaj\u00fa mili\u00f3ny po\u017eiadaviek za sekundu, a neostr\u00e9 jednanie m\u00f4\u017ee \u013eahko rozbi\u0165 slu\u017ebu. Z\u00e1rove\u0148 chceme dynamicky smerova\u0165 traffic cez tis\u00edcky serverov CDN v prostred\u00ed, kde sa nie\u010do neust\u00e1le men\u00ed a rozb\u00edja v najnevhodnej\u0161\u00edch momentoch. <\/p>\n<p>A napriek tomu, v t\u00edme s\u00fa 3 in\u017einieri, ktor\u00ed s\u00fa zodpovedn\u00ed za v\u00fdvoj, nasadenie a \u00fapln\u00fa podporu syst\u00e9mu.<\/p>\n<p>Preto sa \u010falej budeme zaobera\u0165 pokojnym a zdrav\u00fdm sp\u00e1nkom.<\/p>\n<p>Ako pokra\u010dova\u0165 vo v\u00fdvoji a str\u00e1vi\u0165 v\u0161etok \u010das na podpore? Na\u0161im pr\u00edstupom s\u00fa 3 princ\u00edpy:<\/p>\n<ol>\n<li>Zmen\u0161ujeme potenci\u00e1lny rozsah zlyhan\u00ed (blast radius). <\/li>\n<li>Pripravujeme sa na prekvapenia \u2013 o\u010dak\u00e1vame, \u017ee sa nie\u010do rozbije, bez oh\u013eadu na testovanie a osobn\u00e9 sk\u00fasenosti.<\/li>\n<li>Postupn\u00e9 zhor\u0161ovanie (graceful degradation) \u2013 ak nie\u010do nefunguje spr\u00e1vne, malo by sa automaticky opravi\u0165, aj ke\u010f nie najefekt\u00edvnej\u0161\u00edm sp\u00f4sobom.<\/li>\n<\/ol>\n<p>\nUk\u00e1zalo sa, \u017ee v na\u0161om pr\u00edpade, s t\u00fdmto pr\u00edstupom k probl\u00e9mu, m\u00f4\u017eeme n\u00e1js\u0165 jednoduch\u00e9 a efekt\u00edvne rie\u0161enie a zna\u010dne zjednodu\u0161i\u0165 podporu syst\u00e9mu. Pochopili sme, \u017ee m\u00f4\u017eeme prida\u0165 mal\u00fd k\u00fasok k\u00f3du do klienta a sledova\u0165 chyby sie\u0165ov\u00fdch po\u017eiadaviek sp\u00f4soben\u00fdch probl\u00e9mami s pripojen\u00edm. Pri sie\u0165ov\u00fdch chyb\u00e1ch rob\u00edme fallback priamo do cloudu. Toto rie\u0161enie nevy\u017eaduje zna\u010dn\u00e9 \u00fasilie od klientskych t\u00edmov, ale v\u00fdrazne zni\u017euje riziko ne\u010dakan\u00fdch por\u00fach a prekvapen\u00ed pre n\u00e1s.<\/p>\n<p>Samozrejme, napriek fallbacku dodr\u017eiavame jasn\u00fa discipl\u00ednu po\u010das v\u00fdvoja:<\/p>\n<ol>\n<li>Test na vzork\u00e1ch.<\/li>\n<li>A\/B testovanie alebo Canaries.<\/li>\n<li>Postupn\u00e9 vydanie (progressive rollout).<\/li>\n<\/ol>\n<p>\nPr\u00edstup k testovaniu bol pop\u00edsan\u00fd \u2013 zmeny sa najprv testuj\u00fa pomocou nastaven\u00e9ho receptu.<\/p>\n<p>Pre canary testovanie potrebujeme z\u00edska\u0165 porovnate\u013en\u00e9 p\u00e1ry serverov, na ktor\u00fdch m\u00f4\u017eeme porovna\u0165, ako funguje syst\u00e9m pred a po zmen\u00e1ch. Na tento \u00fa\u010del z na\u0161ich po\u010detn\u00fdch CDN str\u00e1nok vytv\u00e1rame vzorku p\u00e1rov serverov, ktor\u00e9 dost\u00e1vaj\u00fa porovnate\u013en\u00fd traffic:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/eef504f5c81aa985b78339fd5f913d14.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPotom nasad\u00edme zostavenie s \u00fapravami na servery Canary. Na hodnotenie v\u00fdsledkov sp\u00fa\u0161\u0165ame syst\u00e9m, ktor\u00fd porovn\u00e1va pribli\u017ene 100-150 metr\u00edk s kontroln\u00fdm vzorkom serverov:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/898edb0a5bd493d6ef228cd116c1a939.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAk testovanie Canary prebehlo \u00faspe\u0161ne, rob\u00edme vydanie postupne, v vln\u00e1ch. Na ka\u017edom z webov neaktualizujeme servery naraz \u2014 strata cel\u00e9ho webu v pr\u00edpade probl\u00e9mov m\u00e1 ove\u013ea v\u00e4\u010d\u0161\u00ed vplyv na slu\u017eby pre pou\u017e\u00edvate\u013eov, ako strata rovnak\u00e9ho mno\u017estva serverov, ale na r\u00f4znych miestach.<\/p>\n<p>Celkovo, efekt\u00edvnos\u0165 a bezpe\u010dnos\u0165 tak\u00e9hoto pr\u00edstupu z\u00e1vis\u00ed od mno\u017estva a kvality zhroma\u017eden\u00fdch metr\u00edk. Pre n\u00e1\u0161 syst\u00e9m ur\u00fdch\u013eovania po\u017eiadaviek zhroma\u017e\u010fujeme metriky zo v\u0161etk\u00fdch mo\u017en\u00fdch komponentov: <\/p>\n<ul>\n<li>od klientov \u2014 po\u010det rel\u00e1ci\u00ed a po\u017eiadaviek, fallback rates; <\/li>\n<li>proxy \u2014 \u0161tatistika o po\u010dte a \u010dase po\u017eiadaviek;<\/li>\n<li>DNS \u2014 po\u010det a v\u00fdsledky po\u017eiadaviek;<\/li>\n<li>cloud edge \u2014 po\u010det a \u010das na spracovanie po\u017eiadaviek v cloude.<\/li>\n<\/ul>\n<p>\nV\u0161etko toto sa zhroma\u017e\u010fuje do jedn\u00e9ho pipeline a v z\u00e1vislosti od potrieb sa rozhodujeme, ktor\u00e9 metriky posiela\u0165 na real-time analytiku a ktor\u00e9 do Elasticsearch alebo Big Data na podrobnej\u0161iu diagnostiku.<\/p>\n<h2>Monitorujeme<\/h2>\n<p>\n<img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/ff8b00b1239a20f85e96ce23c2d7c2d0.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nV na\u0161om pr\u00edpade rob\u00edme zmeny na kritickej ceste po\u017eiadaviek medzi klientom a serverom. Pri tom je mno\u017estvo r\u00f4znych komponentov na klientovi, na serveri, a na ceste cez internet \u2014 obrovsk\u00e9. Zmeny na klientovi a serveri prebiehaj\u00fa neust\u00e1le \u2014 po\u010das pr\u00e1ce desiatok t\u00edmov a prirodzen\u00fdch zmien v ekosyst\u00e9me. Sme uprostred \u2014 pri diagnostike probl\u00e9mov je ve\u013ek\u00e1 \u0161anca, \u017ee sa na tom budeme podie\u013ea\u0165. Preto mus\u00edme jasne ch\u00e1pa\u0165, ako ur\u010di\u0165, zhroma\u017e\u010fova\u0165 a analyzova\u0165 metriky na r\u00fdchlu lokaliz\u00e1ciu probl\u00e9mov. <\/p>\n<p>Ide\u00e1lne \u2014 pln\u00fd pr\u00edstup ku v\u0161etk\u00fdm typom metr\u00edk a filtrom v re\u00e1lnom \u010dase. Ale metr\u00edk je ve\u013emi ve\u013ea, preto vyvst\u00e1va ot\u00e1zka n\u00e1kladov. V na\u0161om pr\u00edpade rozde\u013eujeme metriky a v\u00fdvojov\u00e9 n\u00e1stroje nasledovne:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/0b4a15776f3b44331652adc14ea87390.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNa objavovanie a tri\u00e1\u017e probl\u00e9mov pou\u017e\u00edvame vlastn\u00fd real-time syst\u00e9m s otvoren\u00fdm zdrojov\u00fdm k\u00f3dom <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Netflix\/atlas\">Atlas<\/a><\/noindex> a <noindex><a rel=\"nofollow\" href=\"https:\/\/netflixtechblog.com\/lumen-custom-self-service-dashboarding-for-netflix-8c56b541548c\">Lumen<\/a><\/noindex> \u2014 na vizualiz\u00e1ciu. Uklad\u00e1 agregovan\u00e9 metriky do pam\u00e4te, je spo\u013eahliv\u00fd a integruje sa s alerting syst\u00e9mom. Na lokaliz\u00e1ciu a diagnostiku m\u00e1me pr\u00edstup k logom s Elasticsearch a Kibana. Pre \u0161tatistick\u00fa anal\u00fdzu a modelovanie vyu\u017e\u00edvame big data a vizualiz\u00e1ciu v Tableau.<\/p>\n<p>Zd\u00e1 sa, \u017ee s tak\u00fdmto pr\u00edstupom je ve\u013emi \u0165a\u017ek\u00e9 pracova\u0165. Av\u0161ak pri hierarchickej organiz\u00e1cii metr\u00edk a n\u00e1strojov m\u00f4\u017eeme r\u00fdchlo analyzova\u0165 probl\u00e9m, ur\u010di\u0165 jeho typ a n\u00e1sledne sa hlb\u0161ie zaobera\u0165 podrobn\u00fdmi metrikami. Na identifik\u00e1ciu zdroja poruchy zvy\u010dajne potrebujeme pribli\u017ene 1-2 min\u00faty. N\u00e1sledne u\u017e spolupracujeme s konkr\u00e9tnym t\u00edmom na diagnostike \u2014 od desiatok min\u00fat do nieko\u013ek\u00fdch hod\u00edn.<\/p>\n<p>Aj ke\u010f sa diagnostika rob\u00ed r\u00fdchlo, nechceme, aby sa to deje \u010dasto. V ide\u00e1lnom pr\u00edpade dostaneme kritick\u00e9 upozornenie iba v pr\u00edpade, \u017ee m\u00e1 v\u00fdznamn\u00fd dopad na slu\u017ebu. Pre n\u00e1\u0161 syst\u00e9m zr\u00fdchlen\u00fdch po\u017eiadaviek m\u00e1me iba 2 upozornenia, ktor\u00e9 n\u00e1s bud\u00fa informova\u0165:<\/p>\n<ul>\n<li>percento Client Fallback \u2014 hodnotenie spr\u00e1vania z\u00e1kazn\u00edkov;<\/li>\n<li>percento Probe errors \u2014 \u00fadaje o stabilite sie\u0165ov\u00fdch komponentov.<\/li>\n<\/ul>\n<p>\nTieto kritick\u00e9 upozornenia sleduj\u00fa, \u010di syst\u00e9m funguje pre v\u00e4\u010d\u0161inu pou\u017e\u00edvate\u013eov. Sledujeme, ko\u013eko z\u00e1kazn\u00edkov vyu\u017eilo fallback, ak sa im nepodarilo z\u00edska\u0165 zr\u00fdchlenie po\u017eiadaviek. Priemerne m\u00e1me menej ako 1 kritick\u00e9 upozornenie za t\u00fd\u017ede\u0148, hoci v syst\u00e9me prebieha obrovsk\u00e9 mno\u017estvo zmien. Pre\u010do n\u00e1m to sta\u010d\u00ed?<\/p>\n<ol>\n<li>Existuje client fallback v pr\u00edpade, \u017ee na\u0161a proxy nefunguje.<\/li>\n<li>Existuje automatick\u00fd syst\u00e9m spr\u00e1vy, ktor\u00fd reaguje na probl\u00e9my.<\/li>\n<\/ol>\n<p>\nO poslednom viac podrobne. N\u00e1\u0161 syst\u00e9m prob, a syst\u00e9m automatick\u00e9ho ur\u010denia optim\u00e1lnej cesty po\u017eiadaviek od klienta do cloudu, umo\u017e\u0148uj\u00fa automaticky zvl\u00e1da\u0165 niektor\u00e9 probl\u00e9my. <\/p>\n<p>Vr\u00e1\u0165me sa k na\u0161ej konfigur\u00e1cii prob a 3 kateg\u00f3ri\u00e1m ciest. Okrem \u010dasu na\u010d\u00edtania m\u00f4\u017eeme sledova\u0165 aj samotn\u00fd fakt doru\u010denia. Ak sa \u00fadaje nepodar\u00ed na\u010d\u00edta\u0165, m\u00f4\u017eete sa pozrie\u0165 na v\u00fdsledky z r\u00f4znych ciest a ur\u010di\u0165, kde a \u010do sa pokazilo a \u010di to m\u00f4\u017eeme automaticky opravi\u0165 zmenou cesty po\u017eiadavky.<\/p>\n<p>Pr\u00edklady:<\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/7d935da82aaca53af87f05fbdf215c4c.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/f74ef9d3de953099921a68fbc65cb187.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/3c77cbd47813d3320efbf339a0690c53.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTento proces je mo\u017en\u00e9 automatizova\u0165. Zapracova\u0165 ho do syst\u00e9mu spr\u00e1vy. A nau\u010di\u0165 ho reagova\u0165 na probl\u00e9my s v\u00fdkonom a spo\u013eahlivos\u0165ou. Ak sa nie\u010do za\u010dne pokazi\u0165 \u2014 reagova\u0165, ak existuje lep\u0161ia mo\u017enos\u0165. Pri tom okam\u017eit\u00e1 reakcia nie je kritick\u00e1, v\u010faka fallbacku na klientoch.<\/p>\n<p>Tak\u017ee, z\u00e1sady podpory syst\u00e9mu m\u00f4\u017eeme formulova\u0165 takto:<\/p>\n<ul>\n<li>zmen\u0161ujeme rozsah por\u00fach;<\/li>\n<li>zhroma\u017e\u010fujeme metriky;<\/li>\n<li>automaticky opravujeme poruchy, ak m\u00f4\u017eeme;<\/li>\n<li>ak nem\u00f4\u017eeme \u2014 upozorn\u00edme;<\/li>\n<li>Pracujeme na dashboards a triage toolset pre r\u00fdchlu reakciu.<\/li>\n<\/ul>\n<p><\/p>\n<h2>Z\u00edskan\u00e9 ponau\u010denia<\/h2>\n<p>\nNa vytvorenie prototypu netreba ve\u013ea \u010dasu. V na\u0161om pr\u00edpade bol hotov\u00fd u\u017e za 4 mesiace. S n\u00edm sme z\u00edskali nov\u00e9 metriky a po 10 mesiacoch od za\u010diatku v\u00fdvoja sme dostali prv\u00fd produk\u010dn\u00fd pr\u00fad. Potom nasledovala zd\u013ahav\u00e1 a ve\u013emi zlo\u017eit\u00e1 pr\u00e1ca: postupne produktizova\u0165 a \u0161k\u00e1lova\u0165 syst\u00e9m, migrova\u0165 hlavn\u00fd pr\u00fad a u\u010di\u0165 sa na chyb\u00e1ch. Tento efekt\u00edvny proces v\u0161ak nebude line\u00e1rny \u2014 napriek v\u0161etk\u00fdm snah\u00e1m sa ned\u00e1 v\u0161etko predv\u00edda\u0165. Mnoho efekt\u00edvnej\u0161ie je r\u00fdchla iter\u00e1cia a reakcia na nov\u00e9 d\u00e1ta. <\/p>\n<p><img decoding=\"async\" alt=\"Zr\u00fdch\u013eujeme internetov\u00e9 po\u017eiadavky a sp\u00edme pokojne\" src=\"\/wp-content\/uploads\/2020\/06\/c9c182a1a4fa048b1f1b2a82a4db65e1.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNa z\u00e1klade na\u0161ich sk\u00fasenost\u00ed m\u00f4\u017eeme odporu\u010di\u0165 nasleduj\u00face:<\/p>\n<ol>\n<li>Neverte intu\u00edcii.\n<p>Na\u0161a intu\u00edcia n\u00e1s neust\u00e1le sklamala, napriek obrovsk\u00e9mu sk\u00fasenostn\u00e9mu z\u00e1zemiu \u010dlenu t\u00edmu. Napr\u00edklad, nespr\u00e1vne sme predpovedali o\u010dak\u00e1van\u00e9 ur\u00fdchlenie pri pou\u017eit\u00ed CDN proxy, alebo spr\u00e1vanie TCP Anycast.<\/li>\n<li>Z\u00edskajte d\u00e1ta z produkcie.\n<p>Je d\u00f4le\u017eit\u00e9 \u010do najsk\u00f4r z\u00edska\u0165 pr\u00edstup aspo\u0148 k mal\u00e9mu mno\u017estvu produk\u010dn\u00fdch d\u00e1t. Po\u010det unik\u00e1tnych pr\u00edpadov, konfigur\u00e1ci\u00ed, nastaven\u00ed v laborat\u00f3rnych podmienkach je prakticky nemo\u017en\u00e9 z\u00edska\u0165. R\u00fdchly pr\u00edstup k v\u00fdsledkom umo\u017en\u00ed r\u00fdchlej\u0161ie zisti\u0165 potenci\u00e1lne probl\u00e9my a zoh\u013eadni\u0165 ich v architekt\u00fare syst\u00e9mu.<\/li>\n<li>Nenasledujte rady a v\u00fdsledky in\u00fdch \u2014 zbierajte svoje d\u00e1ta.\n<p>Dbajte na princ\u00edpy zbierania a analyzovania d\u00e1t, ale neberte slepo v\u00fdsledky a tvrdenia in\u00fdch. Iba vy m\u00f4\u017eete presne vedie\u0165, \u010do funguje pre va\u0161ich pou\u017e\u00edvate\u013eov. Va\u0161e syst\u00e9my a va\u0161i klienti sa m\u00f4\u017eu v\u00fdrazne l\u00ed\u0161i\u0165 od in\u00fdch firiem. Na\u0161\u0165astie s\u00fa n\u00e1stroje na anal\u00fdzu teraz dostupn\u00e9 a \u013eahko pou\u017eite\u013en\u00e9. Va\u0161e z\u00edskan\u00e9 v\u00fdsledky sa m\u00f4\u017eu l\u00ed\u0161i\u0165 od toho, \u010do tvrd\u00ed Netflix, Facebook, Akamai a in\u00e9 spolo\u010dnosti. V na\u0161om pr\u00edpade sa v\u00fdkonnos\u0165 TLS, HTTP2 alebo \u0161tatistika DNS dotazov l\u00ed\u0161i od v\u00fdsledkov Facebooku, Uberu, Akamai \u2014 preto\u017ee m\u00e1me in\u00e9 zariadenia, klientov a toky d\u00e1t.<\/li>\n<li>Nesna\u017ete sa bezd\u00f4vodne sledova\u0165 m\u00f3dne trendy a hodnoti\u0165 ich efekt\u00edvnos\u0165.\n<p>Za\u010dnite s jednoduch\u00fdm. Lep\u0161ie je vytvori\u0165 jednoduch\u00fd funk\u010dn\u00fd syst\u00e9m za kr\u00e1tky \u010das, ne\u017e str\u00e1vi\u0165 obrovsk\u00e9 mno\u017estvo \u010dasu v\u00fdvojom nepotrebn\u00fdch komponentov. Rie\u0161te \u00falohy a probl\u00e9my, ktor\u00e9 s\u00fa d\u00f4le\u017eit\u00e9 na z\u00e1klade va\u0161ich meran\u00ed a v\u00fdsledkov. <\/li>\n<li>Bu\u010fte pripraven\u00ed na nov\u00e9 aplik\u00e1cie.\n<p>Rovnako ako je \u0165a\u017ek\u00e9 predv\u00edda\u0165 v\u0161etky probl\u00e9my, je \u0165a\u017ek\u00e9 aj vopred predpoveda\u0165 v\u00fdhody a aplik\u00e1cie. Berte si pr\u00edklad zo startupov \u2014 ich schopnos\u0165 prisp\u00f4sobi\u0165 sa potreb\u00e1m z\u00e1kazn\u00edkov. Vo va\u0161om pr\u00edpade m\u00f4\u017eete objavi\u0165 nov\u00e9 probl\u00e9my a ich rie\u0161enia. V na\u0161om projekte sme si dali za cie\u013e zn\u00ed\u017ei\u0165 latenciu po\u017eiadaviek. Av\u0161ak po\u010das anal\u00fdzy a diskusi\u00ed sme si uvedomili, \u017ee proxy servery m\u00f4\u017eeme vyu\u017e\u00edva\u0165 aj:<\/p>\n<ul>\n<li>na vyv\u00e1\u017eenie prev\u00e1dzky medzi regi\u00f3nmi AWS a zni\u017eovanie n\u00e1kladov;<\/li>\n<li>na modelovanie stability CDN;<\/li>\n<li>na konfigur\u00e1ciu DNS;<\/li>\n<li>na konfigur\u00e1ciu TLS\/TCP.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p><\/p>\n<h2>Z\u00e1ver<\/h2>\n<p>\nV spr\u00e1ve som op\u00edsal, ako Netflix rie\u0161i probl\u00e9m zr\u00fdchlenia internetov\u00fdch po\u017eiadaviek medzi klientmi a cloudom. Ako zbierame d\u00e1ta pomocou syst\u00e9mov s v\u00fdberom na klientoch a vyu\u017e\u00edvame zhroma\u017eden\u00e9 historick\u00e9 d\u00e1ta na smerovanie produk\u010dn\u00fdch po\u017eiadaviek od klientov najr\u00fdchlej\u0161ou cestou na internete. Ako vyu\u017e\u00edvame princ\u00edpy siete, na\u0161u CDN infra\u0161trukt\u00faru, backbone sie\u0165 a DNS servery na dosiahnutie tohto cie\u013ea.<\/p>\n<p>Av\u0161ak na\u0161e rie\u0161enie je len pr\u00edkladom toho, ako sme v Netflixe implementovali tak\u00fdto syst\u00e9m. \u010co fungovalo pre n\u00e1s. Aplika\u010dn\u00e1 \u010das\u0165 mojej spr\u00e1vy pre v\u00e1s \u2014 princ\u00edpy v\u00fdvoja a \u00fadr\u017eby, ktor\u00fdmi sa riadime a dosahujeme dobr\u00e9 v\u00fdsledky.<\/p>\n<p>Na\u0161e rie\u0161enie probl\u00e9mu v\u00e1m nemus\u00ed vyhovova\u0165. Av\u0161ak te\u00f3ria a princ\u00edpy v\u00fdvoja zost\u00e1vaj\u00fa, aj ke\u010f nem\u00e1te svoju vlastn\u00fa CDN infra\u0161trukt\u00faru, alebo ak sa t\u00e1 va\u0161a v\u00fdrazne l\u00ed\u0161i od na\u0161ej. <\/p>\n<p>Na\u010falej zost\u00e1va d\u00f4le\u017eitos\u0165 r\u00fdchlosti po\u017eiadaviek pre podnikanie. A dokonca aj pre jednoduch\u00fa slu\u017ebu treba urobi\u0165 vo\u013ebu: medzi \u201ecloudov\u00fdmi\u201c poskytovate\u013emi, polohou serverov, CDN a DNS poskytovate\u013emi. V\u00e1\u0161 v\u00fdber bude ma\u0165 vplyv na efektivitu internetov\u00fdch po\u017eiadaviek pre va\u0161ich z\u00e1kazn\u00edkov. A pre v\u00e1s je d\u00f4le\u017eit\u00e9 tento vplyv mera\u0165 a ch\u00e1pa\u0165.<\/p>\n<p>Za\u010dnite s jednoduch\u00fdmi rie\u0161eniami, d\u00e1vajte si pozor na to, ako men\u00edte produkt. U\u010dte sa v procese a vylep\u0161ujte syst\u00e9m na z\u00e1klade d\u00e1t od va\u0161ich klientov, va\u0161ej infra\u0161trukt\u00fary a v\u00e1\u0161ho podnikania. Myslite na mo\u017enos\u0165 neo\u010dak\u00e1van\u00fdch por\u00fach v priebehu n\u00e1vrhu. A potom budete m\u00f4c\u0165 ur\u00fdchli\u0165 v\u00e1\u0161 proces v\u00fdvoja, zlep\u0161i\u0165 efektivitu rie\u0161enia, vyhn\u00fa\u0165 sa nadmern\u00e9mu za\u0165a\u017eeniu na podporu a pokojne spa\u0165.<\/p>\n<blockquote><p>V tomto roku <noindex><a rel=\"nofollow\" href=\"http:\/\/devoops-moscow.ru\/?utm_source=habr&amp;utm_medium=506106\">sa konferencia uskuto\u010dn\u00ed od 6. do 10. j\u00fala<\/a><\/noindex> online form\u00e1te. M\u00f4\u017eete polo\u017ei\u0165 ot\u00e1zky jedn\u00e9mu z otcov DevOps, samotn\u00e9mu Johnovi Willisovi!<\/p><\/blockquote>\n<p>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/jugru\/blog\/506106\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>Netflix \u2014 \u043b\u0438\u0434\u0435\u0440 \u0440\u044b\u043d\u043a\u0430 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0442\u0435\u043b\u0435\u0432\u0438\u0434\u0435\u043d\u0438\u044f \u2014 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f, \u0441\u043e\u0437\u0434\u0430\u0432\u0448\u0430\u044f \u0438 \u0430\u043a\u0442\u0438\u0432\u043d\u043e \u0440\u0430\u0437\u0432\u0438\u0432\u0430\u044e\u0449\u0430\u044f \u044d\u0442\u043e\u0442 \u0441\u0435\u0433\u043c\u0435\u043d\u0442. Netflix \u0438\u0437\u0432\u0435\u0441\u0442\u0435\u043d \u043d\u0435 \u0442\u043e\u043b\u044c\u043a\u043e \u043e\u0431\u0448\u0438\u0440\u043d\u044b\u043c \u043a\u0430\u0442\u0430\u043b\u043e\u0433\u043e\u043c \u043a\u0438\u043d\u043e \u0438 \u0441\u0435\u0440\u0438\u0430\u043b\u043e\u0432, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 \u0441 \u043f\u043e\u0447\u0442\u0438 \u043b\u044e\u0431\u043e\u0433\u043e \u0443\u0433\u043e\u043b\u043a\u0430 \u043f\u043b\u0430\u043d\u0435\u0442\u044b \u0438 \u043b\u044e\u0431\u043e\u0433\u043e \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432\u0430 \u0441 \u0434\u0438\u0441\u043f\u043b\u0435\u0435\u043c, \u043d\u043e \u0438 \u043d\u0430\u0434\u0435\u0436\u043d\u043e\u0439 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043e\u0439 \u0438 \u0443\u043d\u0438\u043a\u0430\u043b\u044c\u043d\u043e\u0439 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043d\u043e\u0439 \u043a\u0443\u043b\u044c\u0442\u0443\u0440\u043e\u0439. \u041d\u0430\u0433\u043b\u044f\u0434\u043d\u044b\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 Netflix \u043f\u043e\u0434\u0445\u043e\u0434\u0430 \u043a \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0435 \u0438 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0435 \u0441\u043b\u043e\u0436\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u043d\u0430 DevOops 2019 \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u043b [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":84877,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-84876","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.4 - aioseo.com -->\n\t<meta name=\"description\" content=\"Netflix \u2014 \u043b\u0438\u0434\u0435\u0440 \u0440\u044b\u043d\u043a\u0430 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0442\u0435\u043b\u0435\u0432\u0438\u0434\u0435\u043d\u0438\u044f \u2014.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sk\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.4\" \/>\n\t\t<meta property=\"og:locale\" content=\"sk_SK\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u044f\u0435\u043c \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0437\u0430\u043f\u0440\u043e\u0441\u044b \u0438 \u0441\u043f\u0438\u043c \u0441\u043f\u043e\u043a\u043e\u0439\u043d\u043e | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"Netflix \u2014 \u043b\u0438\u0434\u0435\u0440 \u0440\u044b\u043d\u043a\u0430 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0442\u0435\u043b\u0435\u0432\u0438\u0434\u0435\u043d\u0438\u044f \u2014.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sk\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-06-11T11:43:18+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-06-11T11:43:18+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Zr\u00fdchlime internetov\u00e9 po\u017eiadavky a pokojne sp\u00edme | ProHoster","description":"Netflix je l\u00eddrom na trhu internetov\u00e9ho telev\u00edzneho vysielania -.","canonical_url":"https:\/\/prohoster.info\/sk\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sk_SK","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u044f\u0435\u043c \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0437\u0430\u043f\u0440\u043e\u0441\u044b \u0438 \u0441\u043f\u0438\u043c \u0441\u043f\u043e\u043a\u043e\u0439\u043d\u043e | ProHoster","og:description":"Netflix \u2014 \u043b\u0438\u0434\u0435\u0440 \u0440\u044b\u043d\u043a\u0430 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u0442\u0435\u043b\u0435\u0432\u0438\u0434\u0435\u043d\u0438\u044f \u2014.","og:url":"https:\/\/prohoster.info\/sk\/blog\/administrirovanie\/uskoryaem-internet-zaprosy-i-spim-spokojno","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-06-11T11:43:18+00:00","article:modified_time":"2020-06-11T11:43:18+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"84876","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 14:48:22","updated":"2022-09-27 23:34:06","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/posts\/84876","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/comments?post=84876"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/posts\/84876\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/media\/84877"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/media?parent=84876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/categories?post=84876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sk\/wp-json\/wp\/v2\/tags?post=84876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}