{"id":36754,"date":"2019-10-31T22:13:37","date_gmt":"2019-10-31T19:13:37","guid":{"rendered":"https:\/\/prohoster.info\/blog\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada\/"},"modified":"2019-10-31T22:13:37","modified_gmt":"2019-10-31T19:13:37","slug":"werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada","title":{"rendered":"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>27. kv\u011btna v hlavn\u00edm s\u00e1le konference DevOpsConf 2019, kter\u00e1 se kon\u00e1 v r\u00e1mci festivalu <noindex><a rel=\"nofollow\" href=\"http:\/\/ritfest.ru\/2019\/\">RIT++ 2019<\/a><\/noindex>, v r\u00e1mci sekce \u201eNep\u0159etr\u017eit\u00e9 dod\u00e1v\u00e1n\u00ed\u201c, zazn\u011bla p\u0159edn\u00e1\u0161ka \u201ewerf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes\u201c. Ta se zab\u00fdv\u00e1 t\u011bmi <b>probl\u00e9my a v\u00fdzvami, kter\u00fdm \u010del\u00ed ka\u017ed\u00fd p\u0159i nasazov\u00e1n\u00ed do Kubernetes<\/b>, i nuancemi, kter\u00e9 si nemus\u00edme hned uv\u011bdomit. P\u0159i rozboru mo\u017en\u00fdch \u0159e\u0161en\u00ed ukazujeme, jak je to realizov\u00e1no v Open Source n\u00e1stroji <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/werf\">werf<\/a><\/noindex>.<\/p>\n<p>Od doby vystoupen\u00ed na\u0161e utilita (d\u0159\u00edve zn\u00e1m\u00e1 jako dapp) p\u0159ekonala historickou hranici <b>1000 hv\u011bzd na GitHubu<\/b> \u2014 douf\u00e1me, \u017ee rostouc\u00ed komunita jej\u00edch u\u017eivatel\u016f usnadn\u00ed \u017eivot mnoha DevOps in\u017een\u00fdr\u016fm.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/c2d1ad5133c0de944b60ae37e3dbe598.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTak\u017ee p\u0159edstavujeme <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=cK3ackGUTLw\"><b>video z p\u0159edn\u00e1\u0161ky<\/b><\/a><\/noindex> (~47 minut, mnohem informativn\u011bj\u0161\u00ed ne\u017e \u010dl\u00e1nek) a hlavn\u00ed shrnut\u00ed z n\u00ed v textov\u00e9 podob\u011b. Jdeme na to!<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Dod\u00e1v\u00e1n\u00ed k\u00f3du do Kubernetes<\/h2>\n<p>\nV p\u0159edn\u00e1\u0161ce se ji\u017e nebudeme zab\u00fdvat werf, ale CI\/CD v Kubernetes, s p\u0159edpokladem, \u017ee n\u00e1\u0161 software je zabalen v Docker kontejnerech <i>(o tom jsem mluvil v <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/322686\/\">p\u0159edn\u00e1\u0161ce z roku 2016<\/a><\/noindex>)<\/i>, a K8s bude pou\u017eit pro jeho spu\u0161t\u011bn\u00ed v produkci <i>(o tom \u2014 v <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/331188\/\">roce 2017<\/a><\/noindex>)<\/i>.<\/p>\n<p>Jak vypad\u00e1 dod\u00e1v\u00e1n\u00ed do Kubernetes?<\/p>\n<ul>\n<li> Existuje Git repozit\u00e1\u0159 s k\u00f3dem a instrukcemi pro jeho sestaven\u00ed. Aplikace se sestavuje do Docker obrazu a publikuje se v Docker Registry.<\/li>\n<li> Ve stejn\u00e9m repozit\u00e1\u0159i jsou tak\u00e9 instrukce o tom, jak aplikaci nasadit a spustit. V f\u00e1zi nasazen\u00ed se tyto instrukce pos\u00edlaj\u00ed do Kubernetes, kter\u00fd z\u00edsk\u00e1v\u00e1 pot\u0159ebn\u00fd obraz z registry a spou\u0161t\u00ed ho.<\/li>\n<li> Nav\u00edc jsou obvykle k dispozici testy. N\u011bkter\u00e9 z nich lze prov\u00e1d\u011bt p\u0159i publikaci obrazu. Tak\u00e9 je mo\u017en\u00e9 (podle t\u011bchto instrukc\u00ed) nasadit kopii aplikace (v odd\u011blen\u00e9m prostoru n\u00e1zv\u016f K8s nebo v odd\u011blen\u00e9m clusteru) a testy spou\u0161t\u011bt tam.<\/li>\n<li> Nakonec pot\u0159ebujeme CI syst\u00e9m, kter\u00fd z\u00edsk\u00e1v\u00e1 ud\u00e1losti z Gitu (nebo stisknut\u00ed tla\u010d\u00edtek) a spou\u0161t\u00ed v\u0161echny definovan\u00e9 f\u00e1ze: build, publish, deploy, test.<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/ec0d1fd1bf1a68cca92badf7cf1affe1.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJe zde n\u011bkolik d\u016fle\u017eit\u00fdch pozn\u00e1mek:<\/p>\n<ol>\n<li> Proto\u017ee m\u00e1me nem\u011bnnou infrastrukturu <i>(immutable infrastructure)<\/i>, obraz aplikace, kter\u00fd se pou\u017e\u00edv\u00e1 ve v\u0161ech f\u00e1z\u00edch (staging, production atd.), <b>mus\u00ed b\u00fdt jeden<\/b>. <i>Podrobnosti o tom a s p\u0159\u00edklady jsem popisoval <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/324274\/\">zde<\/a><\/noindex>.<\/i><\/li>\n<li> Proto\u017ee se \u0159\u00edd\u00edme p\u0159\u00edstupem infrastruktura jako k\u00f3d <i>(IaC)<\/i>, k\u00f3d aplikace, instrukce pro jeho sestaven\u00ed a spu\u0161t\u011bn\u00ed mus\u00ed le\u017eet <b>p\u0159esn\u011b v jednom repozit\u00e1\u0159i<\/b>. <i>Podrobnosti o tom \u2014 viz v <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/324274\/\">t\u00e9\u017ee p\u0159edn\u00e1\u0161ce<\/a><\/noindex>.<\/i><\/li>\n<li> Dod\u00e1vkov\u00fd \u0159et\u011bzec <i>(delivery)<\/i> Obvykle vid\u00edme takto: aplikaci sestav\u00edme, otestujeme, vyd\u00e1me. <i>(f\u00e1ze vyd\u00e1n\u00ed)<\/i> A to je v\u0161e \u2014 distribuce byla dokon\u010dena. Ale ve skute\u010dnosti u\u017eivatel dost\u00e1v\u00e1 to, co jste vydali, <b>ne<\/b> v okam\u017eiku, kdy jste to dodali do produkce, a kdy do n\u011bj mohl vstoupit a tato produkce fungovala. Proto si mysl\u00edm, \u017ee dodac\u00ed \u0159et\u011bzec kon\u010d\u00ed <b>a\u017e ve f\u00e1zi provozu<\/b> <i>(b\u011bh)<\/i>, a pokud to vezmeme p\u0159esn\u011bji, dokonce v okam\u017eiku, kdy byl k\u00f3d odstran\u011bn z produkce (nahradil ho nov\u00fd).<\/li>\n<\/ol>\n<p>\nVra\u0165me se k v\u00fd\u0161e uveden\u00e9mu sch\u00e9matu dod\u00e1n\u00ed v Kubernetes: nevynalezli jsme ho jen my, ale doslova ka\u017ed\u00fd, kdo se dan\u00fdm probl\u00e9mem zab\u00fdval. V podstat\u011b se tento vzor nyn\u00ed naz\u00fdv\u00e1 GitOps <i>(v\u00edce o pojmu a my\u0161lenk\u00e1ch, kter\u00e9 za n\u00edm stoj\u00ed, se m\u016f\u017eete do\u010d\u00edst <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/458878\/\">zde<\/a><\/noindex>)<\/i>. Pod\u00edvejme se na f\u00e1ze sch\u00e9matu.<\/p>\n<h2>F\u00e1ze sestaven\u00ed<\/h2>\n<p>\nMohlo by se zd\u00e1t, co lze v roce 2019 \u0159\u00edct o sestavov\u00e1n\u00ed Docker obraz\u016f, kdy\u017e v\u0161ichni um\u00ed ps\u00e1t Dockerfile a spou\u0161t\u011bt <code>docker build<\/code>?.. \u0412\u043e\u0442 \u043d\u044e\u0430\u043d\u0441\u044b, \u043d\u0430 \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0445\u043e\u0442\u0435\u043b\u043e\u0441\u044c \u0431\u044b \u043e\u0431\u0440\u0430\u0442\u0438\u0442\u044c \u0432\u043d\u0438\u043c\u0430\u043d\u0438\u0435:<\/p>\n<ol>\n<li> <b>Hmotnost obrazu<\/b> m\u00e1 v\u00fdznam, proto pou\u017eijte <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.docker.com\/develop\/develop-images\/multistage-build\/\">v\u00edcef\u00e1zov\u00e9<\/a><\/noindex>, abyste v obrazu ponechali pouze to, co je skute\u010dn\u011b pot\u0159ebn\u00e9 pro fungov\u00e1n\u00ed aplikace.<\/li>\n<li> <b>Po\u010det vrstev<\/b> je t\u0159eba minimalizovat slou\u010den\u00edm \u0159et\u011bzc\u016f p\u0159\u00edkaz\u016f podle smyslu. <code>RUN<\/code>Nicm\u00e9n\u011b to p\u0159id\u00e1v\u00e1 probl\u00e9my<\/li>\n<li> p\u0159i lad\u011bn\u00ed <b>, proto\u017ee p\u0159i selh\u00e1n\u00ed sestaven\u00ed je pot\u0159eba naj\u00edt ten spr\u00e1vn\u00fd p\u0159\u00edkaz v \u0159et\u011bzci, kter\u00fd probl\u00e9m zp\u016fsobil.<\/b>Rychlost sestaven\u00ed<\/li>\n<li> <b>je d\u016fle\u017eit\u00e1, proto\u017ee chceme rychle vyd\u00e1vat zm\u011bny a sledovat v\u00fdsledek. Nap\u0159\u00edklad, nechceme p\u0159i ka\u017ed\u00e9m sestaven\u00ed aplikace znovu sestavovat z\u00e1vislosti v jazykov\u00fdch knihovn\u00e1ch.<\/b> \u010casto je z jednoho Git repozit\u00e1\u0159e vy\u017eadov\u00e1no<\/li>\n<li> v\u00edce obraz\u016f, <b>co\u017e lze vy\u0159e\u0161it sadou Dockerfile (nebo pojmenovan\u00fdmi f\u00e1zemi v jednom souboru) a Bash skriptem pro jejich sekven\u010dn\u00ed sestaven\u00ed.<\/b>, co lze vy\u0159e\u0161it sadou Dockerfile (nebo pojmenovan\u00fdmi f\u00e1zemi v jednom souboru) a Bash skriptem s jejich sekven\u010dn\u00edm sestavov\u00e1n\u00edm.<\/li>\n<\/ol>\n<p>\n\u010casto v f\u00e1zi sestaven\u00ed pot\u0159ebujeme n\u011bco<\/p>\n<ol>\n<li> p\u0159ipojit <b>(nap\u0159\u00edklad mezipam\u011b\u0165 v\u00fdsledku p\u0159\u00edkazu typu apt do extern\u00edho adres\u00e1\u0159e).<\/b> (nap\u0159\u00edklad ulo\u017eit v\u00fdsledek p\u0159\u00edkazu typu apt do extern\u00edho adres\u00e1\u0159e).<\/li>\n<li> m\u00edsto psan\u00ed v shellu. <b>Ansible<\/b> m\u00edsto psan\u00ed v shellu.<\/li>\n<li> m\u00edsto psan\u00ed v shellu. <b>(pro\u010d bychom pot\u0159ebovali dal\u0161\u00ed virtu\u00e1ln\u00ed stroj, na kter\u00e9m bychom to v\u0161echno nastavovali, kdy\u017e u\u017e m\u00e1me klastr Kubernetes, kde m\u016f\u017eeme spou\u0161t\u011bt kontejnery?).<\/b> Paraleln\u00ed sestavov\u00e1n\u00ed<\/li>\n<li> <b>Paraleln\u00ed sestaven\u00ed<\/b>, kter\u00e9 lze ch\u00e1pat r\u016fzn\u00fdmi zp\u016fsoby: r\u016fzn\u00e9 p\u0159\u00edkazy z Dockerfile (pokud se pou\u017e\u00edv\u00e1 multi-stage), n\u011bkolik commit\u016f z jednoho repozit\u00e1\u0159e, n\u011bkolik Dockerfile.<\/li>\n<li> <b>Distribuovan\u00e1 sestava<\/b>: chceme n\u011bco sestavovat v podech, kter\u00e9 jsou \"efem\u00e9rn\u00ed\", proto\u017ee ztr\u00e1cej\u00ed ke\u0161, co\u017e znamen\u00e1, \u017ee ji mus\u00edme uchov\u00e1vat n\u011bkde jinde.<\/li>\n<li> Nakonec jsem vrchol p\u0159\u00e1n\u00ed nazval <b>automagie<\/b>: bylo by ide\u00e1ln\u00ed vstoupit do repozit\u00e1\u0159e, napsat n\u011bjak\u00fd p\u0159\u00edkaz a obdr\u017eet hotov\u00fd obraz, sestaven\u00fd s porozum\u011bn\u00edm, jak a co spr\u00e1vn\u011b ud\u011blat. Nicm\u00e9n\u011b osobn\u011b nejsem si jist\u00fd, zda lze v\u0161echny nuance takto p\u0159edpokl\u00e1dat.<\/li>\n<\/ol>\n<p>\nA tak jsou projekty:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/moby\/buildkit\">moby\/buildkit<\/a><\/noindex> \u2014 tv\u016frce od spole\u010dnosti Docker Inc (ji\u017e integrovan\u00fd v aktu\u00e1ln\u00edch verz\u00edch Dockeru), kter\u00fd se sna\u017e\u00ed vy\u0159e\u0161it v\u0161echny tyto probl\u00e9my;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/GoogleContainerTools\/kaniko\">kaniko<\/a><\/noindex> \u2014 tv\u016frce od Googlu, kter\u00fd umo\u017e\u0148uje sestavovat bez Dockeru;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/buildpacks.io\/\">Buildpacks.io<\/a><\/noindex> \u2014 pokus CNCF ud\u011blat automagii a konkr\u00e9tn\u011b zaj\u00edmav\u00e9 \u0159e\u0161en\u00ed s rebase pro vrstvy;<\/li>\n<li> a je\u0161t\u011b spousta dal\u0161\u00edch utilit, jako nap\u0159\u00edklad <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/containers\/buildah\">buildah<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/genuinetools\/img\">genuinetools\/img<\/a><\/noindex>\u2026<\/li>\n<\/ul>\n<p>\n\u2026 a pod\u00edvejte se, kolik maj\u00ed hv\u011bzdi\u010dek na GitHubu. To znamen\u00e1, \u017ee na jedn\u00e9 stran\u011b <code>docker build<\/code> existuj\u00ed a mohou n\u011bco ud\u011blat, ale ve skute\u010dnosti <b>ot\u00e1zka nen\u00ed vy\u0159e\u0161ena<\/b> \u2014 d\u016fkazem toho je paraleln\u00ed rozvoj alternativn\u00edch tv\u016frc\u016f, z nich\u017e ka\u017ed\u00fd \u0159e\u0161\u00ed \u010d\u00e1st probl\u00e9m\u016f.<\/p>\n<h2>Sestaven\u00ed v werf<\/h2>\n<p>\nTak jsme se dostali k <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/werf\">werf<\/a><\/noindex> <i>(d\u0159\u00edve <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/333682\/\">zn\u00e1m\u00e9<\/a><\/noindex> jak dapp)<\/i> \u2014 Open Source n\u00e1stroj spole\u010dnosti \"Flant\", kter\u00fd ji\u017e d\u011bl\u00e1me mnoho let. V\u0161echno to za\u010dalo p\u0159ed asi 5 lety s Bash skripty optimalizuj\u00edc\u00edmi sestavov\u00e1n\u00ed Dockerfile, a posledn\u00ed 3 roky prob\u00edh\u00e1 plnohodnotn\u00fd v\u00fdvoj v r\u00e1mci jednoho projektu se sv\u00fdm Git repozit\u00e1\u0159em <i>(p\u016fvodn\u011b na Ruby, a pak <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/437044\/\">p\u0159epsan\u00e9<\/a><\/noindex> na Go, a z\u00e1rove\u0148 p\u0159ejmenovan\u00e9)<\/i>. Jak\u00e9 ot\u00e1zky sestaven\u00ed jsou vy\u0159e\u0161eny ve werf?<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/ab6aa8831b49977a419fd4cc3543eb83.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nMod\u0159e ozna\u010den\u00e9 probl\u00e9my ji\u017e byly realizov\u00e1ny, paraleln\u00ed sestaven\u00ed bylo provedeno v r\u00e1mci jednoho hostitele, a ot\u00e1zky ozna\u010den\u00e9 \u017elut\u011b pl\u00e1nujeme dokon\u010dit do konce l\u00e9ta.<\/p>\n<h2>F\u00e1ze publikace v registry (publish)<\/h2>\n<p>\nNabrali jsme <code>docker push<\/code>\u2026 \u2014 co m\u016f\u017ee b\u00fdt obt\u00ed\u017en\u00e9ho na tom, abychom nahr\u00e1li obraz do registry? A pak se objevuje ot\u00e1zka: \u201eJak\u00fd tag d\u00e1t obrazu?\u201c Vznik\u00e1 z toho d\u016fvodu, \u017ee m\u00e1me <b>Gitflow<\/b> (nebo jin\u00e1 strategie Gitu) a Kubernetes, a odv\u011btv\u00ed usiluje o to, aby se to, co se d\u011bje v Kubernetes, \u0159\u00eddilo t\u00edm, co se d\u011bl\u00e1 v Gitu. V\u017edy\u0165 Git je n\u00e1\u0161 jedin\u00fd zdroj pravdy.<\/p>\n<p>Co na tom je slo\u017eit\u00e9? <b>Zaru\u010dit reprodukovatelnost<\/b>: od commitu v Gitu, kter\u00fd je ze sv\u00e9 podstaty nem\u011bnn\u00fd <i>(immutable)<\/i>, a\u017e po Docker obraz, kter\u00fd by m\u011bl z\u016fstat stejn\u00fd.<\/p>\n<p>Je pro n\u00e1s tak\u00e9 d\u016fle\u017eit\u00e9 <b>ur\u010dovat p\u016fvod<\/b>, proto\u017ee chceme rozum\u011bt tomu, z jak\u00e9ho commitu byla sestavena aplikace spu\u0161t\u011bn\u00e1 v Kubernetes (potom budeme moci prov\u00e1d\u011bt diffy a podobn\u00e9 v\u011bci).<\/p>\n<h3>Strategie ozna\u010dov\u00e1n\u00ed<\/h3>\n<p>\nPrvn\u00ed je jednoduch\u00e9 <b>git tag<\/b>. M\u00e1me registry s obrazem, kter\u00fd je ozna\u010den jako <code>1.0<\/code>. V Kubernetes existuj\u00ed f\u00e1ze a produkce, kam byl tento obraz nahr\u00e1n. V Gitu prov\u00e1d\u00edme commity a v n\u011bjak\u00e9m okam\u017eiku nastav\u00edme tag <code>2.0<\/code>. Zpracov\u00e1v\u00e1me ho podle pokyn\u016f z repozit\u00e1\u0159e a ukl\u00e1d\u00e1me do registru s tagem <code>2.0<\/code>. Nahr\u00e1v\u00e1me na stage a, pokud je v\u0161e v po\u0159\u00e1dku, pak na produkci.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/8545b84bfa63a91773f0d7dc1c2bd49f.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nProbl\u00e9m s t\u00edmto p\u0159\u00edstupem je, \u017ee jsme nejprve nastavili tag a teprve potom testovali a nasadili. Pro\u010d? Jednak to prost\u011b ned\u00e1v\u00e1 smysl: vyd\u00e1v\u00e1me verzi softwaru, kter\u00fd jsme je\u0161t\u011b ani neprov\u011b\u0159ili (nem\u016f\u017eeme ud\u011blat jinak, proto\u017ee k prov\u011b\u0159en\u00ed je pot\u0159eba nastavit tag). Za druh\u00e9, takov\u00e1 cesta se neslu\u010duje s Gitflow.<\/p>\n<p>Druh\u00e1 varianta je <b>git commit + tag<\/b>. V master v\u011btvi je tag <code>1.0<\/code>; pro n\u011bj v registru \u2014 obraz, nasazen\u00fd na produkci. Krom\u011b toho m\u00e1 Kubernetes klastr preview a staging obvody. D\u00e1le dodr\u017eujeme Gitflow: v hlavn\u00ed v\u011btvi pro v\u00fdvoj (<code>develop<\/code>) d\u011bl\u00e1me nov\u00e9 funkce, co\u017e m\u00e1 za n\u00e1sledek commit s identifik\u00e1torem <code>#c1<\/code>. Ten zpracov\u00e1v\u00e1me a publikujeme do registru pomoc\u00ed tohoto identifik\u00e1toru (<code>#c1<\/code>). Se stejn\u00fdm identifik\u00e1torem nahr\u00e1v\u00e1me na preview. Stejn\u011b tak postupujeme s commity <code>#c2<\/code> a <code>#c3<\/code>.<\/p>\n<p>Kdy\u017e jsme pochopili, \u017ee funkc\u00ed je dost, za\u010d\u00edn\u00e1me v\u0161e stabilizovat. V Gitu vytv\u00e1\u0159\u00edme v\u011btev <code>release_1.1<\/code> ), pou\u017e\u00edv\u00e1 se v RAPIDS. BlazingSQL je dodate\u010dn\u00e1 vrstva, kter\u00e1 pracuje nad cuDF a vyu\u017e\u00edv\u00e1 knihovnu cuIO pro \u010dten\u00ed dat z disku. SQL dotazy jsou p\u0159evedeny na vol\u00e1n\u00ed funkc\u00ed cuUDF, kter\u00e9 umo\u017e\u0148uj\u00ed na\u010d\u00edt\u00e1n\u00ed dat do GPU a prov\u00e1d\u011bn\u00ed nad nimi operac\u00ed slou\u010den\u00ed, agregace a filtrov\u00e1n\u00ed. Podporuje se vytv\u00e1\u0159en\u00ed distribuovan\u00fdch konfigurac\u00ed zahrnuj\u00edc\u00edch tis\u00edce GPU. <code>#c3<\/code> z <code>develop<\/code>). Sestavit tuto verzi nebude t\u0159eba, proto\u017ee to bylo ud\u011bl\u00e1no v p\u0159edchoz\u00ed f\u00e1zi. Proto ji m\u016f\u017eeme jednodu\u0161e nasadit na staging. Opraven\u00e9 bugy v <code>#c4<\/code> a podobn\u011b nasazujeme na staging. Sou\u010dasn\u011b prob\u00edh\u00e1 v\u00fdvoj v <code>develop<\/code>, kam se ob\u010das p\u0159eb\u00edraj\u00ed zm\u011bny z <code>release_1.1<\/code>. V n\u011bjak\u00e9m okam\u017eiku z\u00edsk\u00e1me zkompilovan\u00fd a nahran\u00fd commit na staging, se kter\u00fdm jsme spokojeni (<code>#c25<\/code>).<\/p>\n<p>Tak\u017ee d\u011bl\u00e1me merge (s fast-forwardem) vyd\u00e1vac\u00ed v\u011btve (<code>release_1.1<\/code>) do master. Na tento commit nastav\u00edme tag s novou verz\u00ed (<code>1.1<\/code>). Ale tento obraz u\u017e byl sestaven v registru, proto, abychom ho znovu nesestavovali, jednodu\u0161e p\u0159id\u00e1me druh\u00fd tag na existuj\u00edc\u00ed obraz (nyn\u00ed m\u00e1 v registru tagy <code>#c25<\/code> a <code>1.1<\/code>). Po tom ho nasazujeme na produkci.<\/p>\n<p>Existuje nev\u00fdhoda, \u017ee na staging byl nahr\u00e1n jeden obraz (<code>#c25<\/code>), a na produkci \u2014 jakoby jin\u00fd (<code>1.1<\/code>), ale v\u00edme, \u017ee \u201efyzicky\u201c je to stejn\u00fd obraz z registru.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/b3fa2c34442aa401d1e7b30eb593be9a.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSkute\u010dn\u00fdm m\u00ednusem v\u0161ak je, \u017ee neexistuje podpora pro merge commity, mus\u00edme prov\u00e9st fast-forward.<\/p>\n<p>M\u016f\u017eeme zaj\u00edt d\u00e1l a ud\u011blat trik\u2026 Zva\u017eme p\u0159\u00edklad jednoduch\u00e9ho Dockerfile:<\/p>\n<pre><code class=\"plaintext\">FROM ruby:2.3 as assets\nRUN mkdir -p \/app\nWORKDIR \/app\nCOPY . .\/\nRUN gem install bundler &amp;&amp; bundle install\nRUN bundle exec rake assets:precompile\nCMD bundle exec puma -C config\/puma.rb\n\nFROM nginx:alpine\nCOPY --from=assets \/app\/public \/usr\/share\/nginx\/www\/public<\/code><\/pre>\n<p>\nVytvo\u0159\u00edme soubor podle tohoto principu, \u017ee vezmeme:<\/p>\n<ul>\n<li> SHA256 z identifik\u00e1tor\u016f pou\u017eit\u00fdch obraz\u016f (<code>ruby:2.3<\/code> a <code>nginx:alpine<\/code>), kter\u00e9 jsou kontroln\u00edmi sou\u010dty jejich obsahu;<\/li>\n<li> v\u0161echny p\u0159\u00edkazy (<code>RUN<\/code>, <code>CMD<\/code> atd.);<\/li>\n<li> SHA256 z p\u0159idan\u00fdch soubor\u016f.<\/li>\n<\/ul>\n<p>\n\u2026 a vezmeme kontroln\u00ed sou\u010det (op\u011bt SHA256) z takov\u00e9ho souboru. To je <b>podpis<\/b> v\u0161ech, co ur\u010duje obsah Docker obrazu.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/b2af0b7952eefe26ddbb145955e778e8.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nVra\u0165me se ke sch\u00e9matu a <b>nam\u00edsto commit\u016f budeme pou\u017e\u00edvat takov\u00e9 podpisy<\/b>, tedy tagovat obrazy podpisy.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/115f290bd5951614c041b3d510fae38e.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTe\u010f, kdy\u017e budeme pot\u0159ebovat nap\u0159\u00edklad slou\u010dit zm\u011bny z vyd\u00e1n\u00ed do masteru, m\u016f\u017eeme ud\u011blat skute\u010dn\u00fd merge commit: ten bude m\u00edt jin\u00fd identifik\u00e1tor, ale stejn\u00fd podpis. S takov\u00fdm identifik\u00e1torem nasad\u00edme obraz tak\u00e9 na produkci.<\/p>\n<p>Nev\u00fdhodou je, \u017ee nyn\u00ed nebude mo\u017en\u00e9 ur\u010dit, jak\u00fd commit byl nasazen na produkci \u2014 kontroln\u00ed sou\u010dty funguj\u00ed pouze jedn\u00edm sm\u011brem. Tento probl\u00e9m se \u0159e\u0161\u00ed dodate\u010dnou vrstvou metadat \u2014 podrobn\u011bji pov\u00edm pozd\u011bji.<\/p>\n<h3>Tagov\u00e1n\u00ed ve werf<\/h3>\n<p>\nVe werf jsme za\u0161li je\u0161t\u011b d\u00e1l a p\u0159ipravujeme se na distribuovanou v\u00fdstavbu s cachem, kter\u00fd nen\u00ed ulo\u017een na jednom stroji\u2026 Tak\u017ee, sestavujeme Docker obrazy dvou typ\u016f, naz\u00fdv\u00e1me je <i>stage<\/i> a <i>obr\u00e1zek<\/i>.<\/p>\n<p>V Git repozit\u00e1\u0159i werf jsou ulo\u017eeny specifick\u00e9 instrukce pro stavbu, popisuj\u00edc\u00ed r\u016fzn\u00e9 f\u00e1ze v\u00fdstavby (<i>beforeInstall<\/i>, <i>install<\/i>, <i>beforeSetup<\/i>, <i>setup<\/i>). Prvn\u00ed stage-obraz sestavujeme s podpisem definovan\u00fdm jako kontroln\u00ed sou\u010det prvn\u00edch krok\u016f. Pak p\u0159id\u00e1v\u00e1me zdrojov\u00fd k\u00f3d, pro nov\u00fd stage-obraz po\u010d\u00edt\u00e1me jeho kontroln\u00ed sou\u010det\u2026 Tyto operace se opakuj\u00ed pro v\u0161echny f\u00e1ze, co\u017e n\u00e1m poskytuje sadu stage-obraz\u016f. Pot\u00e9 vytvo\u0159\u00edme fin\u00e1ln\u00ed image-obraz, kter\u00fd tak\u00e9 obsahuje metadata o jeho p\u016fvodu. A tento obraz pak tagujeme r\u016fzn\u00fdmi zp\u016fsoby (podrobnosti pozd\u011bji).<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/22df8b6347b45be19ecb889c102b92b5.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPokud se objev\u00ed nov\u00fd commit, ve kter\u00e9m byl zm\u011bn\u011bn pouze k\u00f3d aplikace, co se stane? Bude vytvo\u0159en patch pro zm\u011bny k\u00f3du, p\u0159ipraven nov\u00fd stage-image. Jeho podpis bude ur\u010den jako kontroln\u00ed sou\u010det star\u00e9ho stage-image a nov\u00e9ho patch. Z tohoto obrazu bude vytvo\u0159en nov\u00fd fin\u00e1ln\u00ed image.<\/p>\n<p>Stage-image tedy p\u0159edstavuj\u00ed cache, kterou je mo\u017en\u00e9 uchov\u00e1vat distribuovan\u011b, a z nich vytvo\u0159en\u00e9 image budou nahr\u00e1ny do Docker Registry.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/35805eca81605bd64c6910b7965b567e.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>\u00daklid registry<\/h3>\n<p>\nNep\u016fjde o maz\u00e1n\u00ed vrstev, kter\u00e9 z\u016fstaly viset po odstran\u011bn\u00fdch taz\u00edch \u2014 to je standardn\u00ed mo\u017enost samotn\u00e9ho Docker Registry. Jde o situaci, kdy se hromad\u00ed velk\u00e9 mno\u017estv\u00ed Docker-tags a my v\u00edme, \u017ee n\u011bkter\u00e9 z nich ji\u017e nepot\u0159ebujeme, a zab\u00edraj\u00ed m\u00edsto (a\/nebo za n\u011b plat\u00edme).<\/p>\n<p>Jak\u00e9 jsou strategie \u00faklidu?<\/p>\n<ol>\n<li> M\u016f\u017eete klidn\u011b nic <b>ne\u010distit.<\/b>N\u011bkdy je opravdu jednodu\u0161\u0161\u00ed zaplatit za nadbyte\u010dn\u00fd prostor, ne\u017e se sna\u017eit rozpl\u00e9st obrovsk\u00fd klubko tag\u016f. Ale to funguje jen do ur\u010dit\u00e9ho okam\u017eiku.<\/li>\n<li> <b>\u00dapln\u00fd reset<\/b>. Pokud odstran\u00edte v\u0161echny obrazy a znovu sestav\u00edte pouze aktu\u00e1ln\u00ed v CI syst\u00e9mu, m\u016f\u017ee nastat probl\u00e9m. Pokud se na produkci restartuje kontejner, na\u010dte se nov\u00fd obraz \u2014 takov\u00fd, kter\u00fd je\u0161t\u011b nikdo netestoval. To zab\u00edj\u00ed my\u0161lenku immutable infrastructure.<\/li>\n<li> <b>Blue-green<\/b>. Kdy\u017e se jeden registry za\u010dne plnit \u2014 nahr\u00e1v\u00e1me obrazy do jin\u00e9ho. Stejn\u00fd probl\u00e9m jako v p\u0159edchoz\u00edm zp\u016fsobu: kdy m\u016f\u017eeme za\u010d\u00edt \u010distit ten registry, kter\u00fd se za\u010dal plnit?<\/li>\n<li> <b>\u010casov\u011b<\/b>. Odstranit v\u0161echny obrazy star\u0161\u00ed ne\u017e 1 m\u011bs\u00edc? Ale jist\u011b se najde slu\u017eba, kter\u00e1 nebyla aktualizov\u00e1na cel\u00fd m\u011bs\u00edc\u2026<\/li>\n<li> <b>Manu\u00e1ln\u011b<\/b> ur\u010dit, co ji\u017e lze odstranit.<\/li>\n<\/ol>\n<p>\nOpravdu \u017eivotaschopn\u00e9 varianty jsou dv\u011b: ne\u010distit nebo kombinace blue-green + ru\u010dn\u011b. V posledn\u00edm p\u0159\u00edpad\u011b jde o to, \u017ee kdy\u017e v\u00edte, \u017ee je \u010das vy\u010distit registry, vytvo\u0159\u00edte novou a b\u011bhem, nap\u0159\u00edklad, m\u011bs\u00edce p\u0159id\u00e1te v\u0161echny nov\u00e9 obrazy do n\u00ed. A po m\u011bs\u00edci se pod\u00edv\u00e1te, kter\u00e9 podech v Kubernetes st\u00e1le pou\u017e\u00edvaj\u00ed star\u00fd registry, a tak\u00e9 je p\u0159esunete do nov\u00e9ho registry.<\/p>\n<p>K \u010demu jsme dosp\u011bli v <b>werf<\/b>? \u041c\u044b \u0441\u043e\u0431\u0438\u0440\u0430\u0435\u043c:<\/p>\n<ol>\n<li> Git head: v\u0161echny tagy, v\u0161echny v\u011btve \u2014 p\u0159edpokl\u00e1d\u00e1me, \u017ee v\u0161e, co bylo ozna\u010deno v Gitu, pot\u0159ebujeme tak\u00e9 v obrazech (a pokud ne, je t\u0159eba odstranit to v samotn\u00e9m Gitu);<\/li>\n<li> v\u0161echny pod'y, kter\u00e9 jsou nyn\u00ed sta\u017eeny v Kubernetes;<\/li>\n<li> star\u00e9 ReplicaSet'y (to, co bylo recentn\u011b sta\u017eeno), a tak\u00e9 pl\u00e1nujeme skenovat Helm releasy a vybrat posledn\u00ed obrazy tam.<\/li>\n<\/ol>\n<p>\n\u2026 a vytv\u00e1\u0159\u00edme z t\u00e9to sady whitelist \u2014 seznam obraz\u016f, kter\u00e9 nebudeme odstra\u0148ovat. V\u0161echno ostatn\u00ed vy\u010dist\u00edme, a pot\u00e9 najdeme osi\u0159el\u00e9 stage-obrazy a tak\u00e9 je odstran\u00edme.<\/p>\n<h2>F\u00e1ze nasazen\u00ed (deploy)<\/h2>\n<p><\/p>\n<h3>Spolehliv\u00e1 deklarativnost<\/h3>\n<p>\nPrvn\u00ed v\u011bc, na kterou bychom cht\u011bli upozornit p\u0159i nasazen\u00ed, je nasazen\u00ed aktualizovan\u00e9 konfigurace zdroj\u016f, ozn\u00e1men\u00e9 deklarativn\u011b. P\u016fvodn\u00ed YAML dokument s popisem Kubernetes zdroj\u016f se v\u017edy siln\u011b li\u0161\u00ed od v\u00fdsledku, kter\u00fd skute\u010dn\u011b funguje v clusteru. Proto\u017ee Kubernetes p\u0159id\u00e1v\u00e1 do konfigurace:<\/p>\n<ol>\n<li> identifik\u00e1tory;<\/li>\n<li> slu\u017eebn\u00ed informace;<\/li>\n<li> mnoho v\u00fdchoz\u00edch hodnot;<\/li>\n<li> sekci s aktu\u00e1ln\u00edm stavem;<\/li>\n<li> zm\u011bny proveden\u00e9 v r\u00e1mci pr\u00e1ce admission webhook;<\/li>\n<li> v\u00fdsledek pr\u00e1ce r\u016fzn\u00fdch kontrol\u00e9r\u016f (a pl\u00e1nova\u010de).<\/li>\n<\/ol>\n<p>\nProto, kdy\u017e se objev\u00ed nov\u00e1 konfigurace zdroje (<i>nov\u00fd<\/i>), nem\u016f\u017eeme prost\u011b vz\u00edt a p\u0159epsat j\u00ed aktu\u00e1ln\u00ed, \u201e\u017eivou\u201c konfiguraci (<i>live<\/i>). K tomu budeme muset porovnat <i>nov\u00fd<\/i> s p\u0159edchoz\u00ed aplikovanou konfigurac\u00ed (<i>last-applied<\/i>) a aplikovat <i>live<\/i> z\u00edskanou opravu.<\/p>\n<p>Takov\u00fd p\u0159\u00edstup se naz\u00fdv\u00e1 <b>2-way merge<\/b>. Pou\u017e\u00edv\u00e1 se nap\u0159\u00edklad v Helmu.<\/p>\n<p>Je\u0161t\u011b existuje <b>3-way merge<\/b>, kter\u00fd se li\u0161\u00ed t\u00edm, \u017ee:<\/p>\n<ul>\n<li> p\u0159i porovn\u00e1v\u00e1n\u00ed <i>last-applied<\/i> a <i>nov\u00fd<\/i>, se d\u00edv\u00e1me, co bylo odstran\u011bno;<\/li>\n<li> p\u0159i porovn\u00e1v\u00e1n\u00ed <i>nov\u00fd<\/i> a <i>live<\/i>, d\u00edv\u00e1me se, co bylo p\u0159id\u00e1no nebo zm\u011bn\u011bno;<\/li>\n<li> souhrnnou opravu aplikujeme na <i>live<\/i>.<\/li>\n<\/ul>\n<p>\nNasazujeme 1000+ aplikac\u00ed s Helm, tak\u017ee fakticky \u017eijeme s 2-way merge. Nicm\u00e9n\u011b m\u00e1 to \u0159adu probl\u00e9m\u016f, kter\u00e9 jsme vy\u0159e\u0161ili sv\u00fdmi patche, kter\u00e9 pom\u00e1haj\u00ed Helmu fungovat spr\u00e1vn\u011b.<\/p>\n<h3>Skute\u010dn\u00fd stav nasazen\u00ed<\/h3>\n<p>\nPo tom, co na\u0161e CI-syst\u00e9m vygeneroval nov\u00e9 konfigurace pro Kubernetes podle ud\u00e1losti, p\u0159en\u00e1\u0161\u00ed ji na aplikaci <i>(apply)<\/i> do clusteru \u2014 pomoc\u00ed Helmu nebo <code>kubectl apply<\/code>. D\u00e1le se prov\u00e1d\u00ed ji\u017e popsan\u00fd N-way merge, na co\u017e Kubernetes API souhlasn\u011b odpov\u00edd\u00e1 CI-syst\u00e9mu, a ten \u2014 sv\u00e9mu u\u017eivateli.<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/fcc8251525f32c913f30fbcdc4296198.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJe v\u0161ak obrovsk\u00fd probl\u00e9m: proto\u017ee <b>\u00fasp\u011b\u0161n\u00e1 aplikace neznamen\u00e1 \u00fasp\u011b\u0161n\u00e9 nasazen\u00ed<\/b>. Pokud Kubernetes pochopil, jak\u00e9 zm\u011bny je t\u0159eba aplikovat, aplikuje je \u2014 zat\u00edm nev\u00edme, co z toho vzejde. Nap\u0159\u00edklad aktualizace a restart pod\u016f ve frontend\u011b mohou prob\u011bhnout \u00fasp\u011b\u0161n\u011b, zat\u00edmco v backend\u011b ne, a dostaneme r\u016fzn\u00e9 verze spu\u0161t\u011bn\u00fdch obraz\u016f aplikace.<\/p>\n<p>Aby v\u0161e prob\u00edhalo spr\u00e1vn\u011b, v tomto sch\u00e9matu se nab\u00edz\u00ed dal\u0161\u00ed prvek \u2014 speci\u00e1ln\u00ed tracker, kter\u00fd bude p\u0159ij\u00edmat informace od Kubernetes API o stavu a p\u0159ed\u00e1vat je k dal\u0161\u00edmu anal\u00fdze skute\u010dn\u00e9 situace. Vytvo\u0159ili jsme knihovnu Open Source v Go \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/kubedog\"><b>kubedog<\/b><\/a><\/noindex> <i>(viz jej\u00ed ozn\u00e1men\u00ed <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/434160\/\">zde<\/a><\/noindex>)<\/i>, \u2014 kter\u00e1 tento probl\u00e9m \u0159e\u0161\u00ed a je integrovan\u00e1 ve werf.<\/p>\n<p>Chov\u00e1n\u00ed tohoto trackera na \u00farovni werf se konfiguruje pomoc\u00ed anotac\u00ed, kter\u00e9 se umis\u0165uj\u00ed na Deployments nebo StatefulSets. Hlavn\u00ed anotace \u2014 <code>fail-mode<\/code> \u2014 rozum\u00ed n\u00e1sleduj\u00edc\u00edm hodnot\u00e1m:<\/p>\n<ul>\n<li> <code>IgnoreAndContinueDeployProcess<\/code> \u2014 ignorovat probl\u00e9my s nasazen\u00edm t\u00e9to komponenty a pokra\u010dovat v deployi;<\/li>\n<li> <code>FailWholeDeployProcessImmediately<\/code> \u2014 chyba v t\u00e9to komponent\u011b okam\u017eit\u011b zastav\u00ed proces deploye;<\/li>\n<li> <code>HopeUntilEndOfDeployProcess<\/code> \u2014 douf\u00e1me, \u017ee tato komponenta za\u010dne fungovat do konce deploye.<\/li>\n<\/ul>\n<p>\nNap\u0159\u00edklad tato kombinace zdroj\u016f a hodnot anotace <code>fail-mode<\/code>:<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/40e161710e8a535cd95f1eb66ca8407b.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKdy\u017e nasazujeme poprv\u00e9, datab\u00e1ze (MongoDB) je\u0161t\u011b nemus\u00ed b\u00fdt p\u0159ipravena \u2014 Deploymenty spadnou. Ale m\u016f\u017eeme po\u010dkat, ne\u017e se spust\u00ed, a nasazen\u00ed st\u00e1le \u00fasp\u011b\u0161n\u011b prob\u011bhne.<\/p>\n<p>D\u00e1le existuj\u00ed dv\u011b anotace pro kubedog ve werf:<\/p>\n<ul>\n<li> <code>failures-allowed-per-replica<\/code> \u2014 po\u010det povolen\u00fdch selh\u00e1n\u00ed na ka\u017edou replikaci;<\/li>\n<li> <code>show-logs-until<\/code> \u2014 reguluje moment, do kter\u00e9ho werf zobrazuje (v stdout) logy ze v\u0161ech nasazovan\u00fdch pod\u016f. Ve v\u00fdchoz\u00edm nastaven\u00ed je to <code>PodIsReady<\/code> (abychom ignorovali zpr\u00e1vy, kter\u00e9 pravd\u011bpodobn\u011b nepot\u0159ebujeme, kdy\u017e na pod za\u010d\u00edn\u00e1 p\u0159ich\u00e1zet provoz), ale tak\u00e9 jsou povolen\u00e9 hodnoty <code>ControllerIsReady<\/code> a <code>EndOfDeploy<\/code>.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Co dal\u0161\u00edho chceme od nasazen\u00ed?<\/h3>\n<p>\nKrom\u011b ji\u017e popsan\u00fdch dvou bod\u016f bychom cht\u011bli:<\/p>\n<ul>\n<li> vid\u011bt <b>logy<\/b> \u2014 a to pouze pot\u0159ebn\u00e9, ne v\u0161echny;<\/li>\n<li> sledovat <b>pokrok<\/b>, proto\u017ee pokud job \u201eti\u0161e\u201c vis\u00ed n\u011bkolik minut, je d\u016fle\u017eit\u00e9 pochopit, co se tam d\u011bje;<\/li>\n<li> m\u00edt <b>automatick\u00fd rollback<\/b> v p\u0159\u00edpad\u011b, \u017ee n\u011bco neprob\u011bhne podle pl\u00e1nu (a proto je kriticky d\u016fle\u017eit\u00e9 zn\u00e1t skute\u010dn\u00fd stav nasazen\u00ed). Nasazen\u00ed mus\u00ed b\u00fdt atom\u00e1rn\u00ed: bu\u010f prob\u011bhne do konce, nebo se v\u0161e vr\u00e1t\u00ed do p\u0159edchoz\u00edho stavu.<\/li>\n<\/ul>\n<p><\/p>\n<h2>Z\u00e1v\u011bry<\/h2>\n<p>\nNa\u0161e spole\u010dnost pot\u0159ebuje k implementaci v\u0161ech popsan\u00fdch nuanc\u00ed na r\u016fzn\u00fdch f\u00e1z\u00edch dod\u00e1vky (build, publish, deploy) pouze CI syst\u00e9m a n\u00e1stroj <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/werf\">werf<\/a><\/noindex>.<\/p>\n<p>Na z\u00e1v\u011br:<\/p>\n<p><img decoding=\"async\" alt=\"werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video p\u0159edn\u00e1\u0161ky)\" src=\"\/wp-content\/uploads\/2019\/08\/bafba54f2df8740a1a12c93b3476e49a.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nS pomoc\u00ed werf jsme ud\u011blali zna\u010dn\u00fd pokrok v \u0159e\u0161en\u00ed mnoha probl\u00e9m\u016f DevOps in\u017een\u00fdr\u016f a budeme r\u00e1di, pokud \u0161ir\u0161\u00ed komunita tuto utilitu alespo\u0148 vyzkou\u0161\u00ed. Spole\u010dn\u011b bude snaz\u0161\u00ed dos\u00e1hnout dobr\u00e9ho v\u00fdsledku.<\/p>\n<h2>Video a slajdy<\/h2>\n<p>\nVideo z prezentace (~47 minut):<\/p>\n<p><center><div class=\"youtube-placeholder\" data-id=\"cK3ackGUTLw\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/cK3ackGUTLw\/hqdefault.jpg\" alt=\"P\u0159ehr\u00e1t video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<p>Prezentace zpr\u00e1vy:<\/p>\n<p><center><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"\/\/speakerdeck.com\/player\/2033277984c04900b18940588edf1161\" frameborder=\"0\" allowfullscreen><\/iframe><\/center><\/p>\n<h2>P.S.<\/h2>\n<p>\nDal\u0161\u00ed zpr\u00e1vy o Kubernetes na na\u0161em blogu:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/459326\/\">Automatick\u00e9 \u0161k\u00e1lov\u00e1n\u00ed a spr\u00e1va zdroj\u016f v Kubernetes<\/a><\/noindex>\u00bb <i>(Dmitrij Stoljarov; 27. dubna 2019 na \"Stachka\")<\/i>;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/449096\/\">Roz\u0161i\u0159ujeme a dopl\u0148ujeme Kubernetes<\/a><\/noindex>\u00bb <i>(Andrej Polovov; 8. dubna 2019 na Saint HighLoad++)<\/i>;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/431500\/\">Datab\u00e1ze a Kubernetes<\/a><\/noindex>\u00bb <i>(Dmitrij Stoljarov; 8. listopadu 2018 na HighLoad++)<\/i>;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/412901\/\">Monitorov\u00e1n\u00ed a Kubernetes<\/a><\/noindex>\u00bb <i>(Dmitrij Stoljarov; 28. kv\u011btna 2018 na RootConf)<\/i>;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/345116\/\">Nejlep\u0161\u00ed praktiky CI\/CD s Kubernetes a GitLab<\/a><\/noindex>\u00bb <i>(Dmitrij Stoljarov; 7. listopadu 2017 na HighLoad++)<\/i>;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/331188\/\">Na\u0161e zku\u0161enosti s Kubernetes v mal\u00fdch projektech<\/a><\/noindex>\u00bb <i>(Dmitrij Stoljarov; 6. \u010dervna 2017 na RootConf)<\/i>.<\/li>\n<\/ul>\n<p>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/460351\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>27 \u043c\u0430\u044f \u0432 \u0433\u043b\u0430\u0432\u043d\u043e\u043c \u0437\u0430\u043b\u0435 \u043a\u043e\u043d\u0444\u0435\u0440\u0435\u043d\u0446\u0438\u0438 DevOpsConf 2019, \u043f\u0440\u043e\u0445\u043e\u0434\u044f\u0449\u0435\u0439 \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0444\u0435\u0441\u0442\u0438\u0432\u0430\u043b\u044f \u0420\u0418\u0422++ 2019, \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0441\u0435\u043a\u0446\u0438\u0438 \u00ab\u041d\u0435\u043f\u0440\u0435\u0440\u044b\u0432\u043d\u0430\u044f \u043f\u043e\u0441\u0442\u0430\u0432\u043a\u0430\u00bb, \u043f\u0440\u043e\u0437\u0432\u0443\u0447\u0430\u043b \u0434\u043e\u043a\u043b\u0430\u0434 \u00abwerf \u2014 \u043d\u0430\u0448 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442 \u0434\u043b\u044f CI\/CD \u0432 Kubernetes\u00bb. \u0412 \u043d\u0451\u043c \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u043e \u0442\u0435\u0445 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0430\u0445 \u0438 \u0432\u044b\u0437\u043e\u0432\u0430\u0445, \u0441 \u043a\u043e\u0442\u043e\u0440\u044b\u043c\u0438 \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u043a\u0430\u0436\u0434\u044b\u0439 \u043f\u0440\u0438 \u0434\u0435\u043f\u043b\u043e\u0435 \u0432 Kubernetes, \u0430 \u0442\u0430\u043a\u0436\u0435 \u043e \u043d\u044e\u0430\u043d\u0441\u0430\u0445, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0433\u0443\u0442 \u0431\u044b\u0442\u044c \u0437\u0430\u043c\u0435\u0442\u043d\u044b \u043d\u0435 \u0441\u0440\u0430\u0437\u0443. [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":27531,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-36754","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"27 \u043c\u0430\u044f \u0432 \u0433\u043b\u0430\u0432\u043d\u043e\u043c \u0437\u0430\u043b\u0435 \u043a\u043e\u043d\u0444\u0435\u0440\u0435\u043d\u0446\u0438\u0438 DevOpsConf 2019, \u043f\u0440\u043e\u0445\u043e\u0434\u044f\u0449\u0435\u0439 \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0444\u0435\u0441\u0442\u0438\u0432\u0430\u043b\u044f \u0420\u0418\u0422++ 2019, \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0441\u0435\u043a\u0446\u0438\u0438 \u00ab\u041d\u0435\u043f\u0440\u0435\u0440\u044b\u0432\u043d\u0430\u044f \u043f\u043e\u0441\u0442\u0430\u0432\u043a\u0430\u00bb, \u043f\u0440\u043e\u0437\u0432\u0443\u0447\u0430\u043b.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"cs_CZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47werf \u2014 \u043d\u0430\u0448 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442 \u0434\u043b\u044f CI\/CD \u0432 Kubernetes (\u043e\u0431\u0437\u043e\u0440 \u0438 \u0432\u0438\u0434\u0435\u043e \u0434\u043e\u043a\u043b\u0430\u0434\u0430) | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"27 \u043c\u0430\u044f \u0432 \u0433\u043b\u0430\u0432\u043d\u043e\u043c \u0437\u0430\u043b\u0435 \u043a\u043e\u043d\u0444\u0435\u0440\u0435\u043d\u0446\u0438\u0438 DevOpsConf 2019, \u043f\u0440\u043e\u0445\u043e\u0434\u044f\u0449\u0435\u0439 \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0444\u0435\u0441\u0442\u0438\u0432\u0430\u043b\u044f \u0420\u0418\u0422++ 2019, \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0441\u0435\u043a\u0446\u0438\u0438 \u00ab\u041d\u0435\u043f\u0440\u0435\u0440\u044b\u0432\u043d\u0430\u044f \u043f\u043e\u0441\u0442\u0430\u0432\u043a\u0430\u00bb, \u043f\u0440\u043e\u0437\u0432\u0443\u0447\u0430\u043b.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:13:37+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:13:37+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47werf \u2014 n\u00e1\u0161 n\u00e1stroj pro CI\/CD v Kubernetes (p\u0159ehled a video zpr\u00e1vy) | ProHoster","description":"27. kv\u011btna v hlavn\u00edm s\u00e1le konference DevOpsConf 2019, kter\u00e1 se kon\u00e1 v r\u00e1mci festivalu RIT++ 2019, v sekci \"Nep\u0159etr\u017eit\u00e1 dod\u00e1vka\".","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47werf \u2014 \u043d\u0430\u0448 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442 \u0434\u043b\u044f CI\/CD \u0432 Kubernetes (\u043e\u0431\u0437\u043e\u0440 \u0438 \u0432\u0438\u0434\u0435\u043e \u0434\u043e\u043a\u043b\u0430\u0434\u0430) | ProHoster","og:description":"27 \u043c\u0430\u044f \u0432 \u0433\u043b\u0430\u0432\u043d\u043e\u043c \u0437\u0430\u043b\u0435 \u043a\u043e\u043d\u0444\u0435\u0440\u0435\u043d\u0446\u0438\u0438 DevOpsConf 2019, \u043f\u0440\u043e\u0445\u043e\u0434\u044f\u0449\u0435\u0439 \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0444\u0435\u0441\u0442\u0438\u0432\u0430\u043b\u044f \u0420\u0418\u0422++ 2019, \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0441\u0435\u043a\u0446\u0438\u0438 \u00ab\u041d\u0435\u043f\u0440\u0435\u0440\u044b\u0432\u043d\u0430\u044f \u043f\u043e\u0441\u0442\u0430\u0432\u043a\u0430\u00bb, \u043f\u0440\u043e\u0437\u0432\u0443\u0447\u0430\u043b.","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/werf-nash-instrument-dlya-ci-cd-v-kubernetes-obzor-i-video-doklada","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:13:37+00:00","article:modified_time":"2019-10-31T19:13:37+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"36754","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 04:44:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:39:23","updated":"2026-01-22 04:44:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/36754","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=36754"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/36754\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/27531"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=36754"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=36754"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=36754"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}