{"id":94977,"date":"2020-09-24T07:43:00","date_gmt":"2020-09-24T05:43:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes"},"modified":"2020-09-24T07:43:00","modified_gmt":"2020-09-24T05:43:00","slug":"nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes","title":{"rendered":"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><i><b>Nota traduc\u0103torului.<\/b>: adaptarea Kubernetes \u00een GitLab este considerat\u0103 unul dintre cele dou\u0103 factori principali care contribuie la cre\u0219terea companiei. Cu toate acestea, p\u00e2n\u0103 de cur\u00e2nd, infrastructura serviciului online GitLab.com a fost construit\u0103 pe ma\u0219ini virtuale, iar migrarea sa \u00een K8s a \u00eenceput acum aproximativ un an \u0219i nu este \u00eenc\u0103 finalizat\u0103. Suntem \u00eenc\u00e2nta\u021bi s\u0103 prezent\u0103m traducerea unui articol recent al inginerului SRE de la GitLab despre cum decurge acest proces \u0219i ce concluzii trag inginerii implica\u021bi \u00een proiect.<\/i><\/p>\n<p><img decoding=\"async\" alt=\"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/58429abf60cd19a49c5ce593051c2fca.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDe aproximativ un an, departamentul nostru de infrastructur\u0103 se ocup\u0103 cu migrarea tuturor serviciilor de pe GitLab.com \u00een Kubernetes. Pe parcursul acestei perioade, ne-am confruntat cu probleme legate nu doar de mutarea serviciilor \u00een Kubernetes, ci \u0219i de gestionarea unui deployment hibrid \u00een timpul tranzi\u021biei. Lec\u021biile valoroase \u00eenv\u0103\u021bate de noi sunt subiectul acestei articole.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>\u00cenc\u0103 de la \u00eenceput, serverele GitLab.com func\u021bionau \u00een cloud pe ma\u0219ini virtuale. Aceste ma\u0219ini virtuale sunt gestionate de Chef, iar instalarea lor se efectueaz\u0103 cu ajutorul <noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/install\/#ubuntu\">pachetului oficial Linux<\/a><\/noindex>. <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-org\/release\/docs\/-\/blob\/master\/general\/deploy\/gitlab-com-deployer.md\">Strategia de implementare<\/a><\/noindex> \u00een cazul \u00een care este necesar\u0103 actualizarea aplica\u021biei, presupune pur \u0219i simplu actualizarea parcului de servere \u00eentr-un mod coordonat \u0219i secven\u021bial prin intermediul CI pipeline-ului. Aceast\u0103 metod\u0103 \u2014 de\u0219i lent\u0103 \u0219i pu\u021bin <noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/handbook\/values\/#boring-solutions\">plictisitoare<\/a><\/noindex> \u2014 garanteaz\u0103 c\u0103 GitLab.com aplic\u0103 acelea\u0219i metode de instalare \u0219i configurare ca \u0219i utilizatorii de <i>(autogestionate)<\/i> instala\u021bii GitLab, care folosesc pentru aceasta pachetele noastre Linux.<\/p>\n<p>Folosim aceast\u0103 metod\u0103 deoarece este extrem de important s\u0103 tr\u0103im toate bucuriile \u0219i triste\u021bile pe care le resimt utilizatorii comunit\u0103\u021bii atunci c\u00e2nd instaleaz\u0103 \u0219i configureaz\u0103 propriile copii GitLab. Aceast\u0103 abordare a func\u021bionat bine pentru o vreme, dar c\u00e2nd num\u0103rul proiectelor pe GitLab a trecut de 10 milioane, ne-am dat seama c\u0103 nu mai satisface nevoile noastre de scalare \u0219i implementare.<\/p>\n<h2>Primii pa\u0219i c\u0103tre Kubernetes \u0219i GitLab cloud-native<\/h2>\n<p>\n\u00cen 2017, a fost creat proiectul <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-org\/charts\">GitLab Charts<\/a><\/noindex> pentru a preg\u0103ti GitLab pentru desf\u0103\u0219urarea \u00een cloud, precum \u0219i pentru a oferi utilizatorilor posibilitatea de a instala GitLab \u00een clustere Kubernetes. Atunci \u0219tiam c\u0103 mutarea GitLab \u00een Kubernetes va extinde capacit\u0103\u021bile de scalare ale platformei SaaS, va simplifica desf\u0103\u0219ur\u0103rile \u0219i va \u00eembun\u0103t\u0103\u021bi eficien\u021ba utiliz\u0103rii resurselor de calcul. \u00cen acela\u0219i timp, multe func\u021bii ale aplica\u021biei noastre depindeau de p\u0103r\u021bi NFS montate, ceea ce \u00eencetinea tranzi\u021bia de la ma\u0219inile virtuale.<\/p>\n<p>Dorin\u021ba de a deveni cloud native \u0219i utilizarea Kubernetes le-a permis inginerilor no\u0219tri s\u0103 planifice o tranzi\u021bie treptat\u0103, \u00een cadrul c\u0103reia am renun\u021bat la anumite dependen\u021be ale aplica\u021biei de stocarea \u00een re\u021bea, continu\u00e2nd \u00een acela\u0219i timp s\u0103 dezvolt\u0103m noi func\u021bionalit\u0103\u021bi. De c\u00e2nd am \u00eenceput s\u0103 planific\u0103m migrarea \u00een vara anului 2019, multe dintre aceste restric\u021bii au fost eliminate, iar procesul de transfer al GitLab.com pe Kubernetes este acum \u00een plin\u0103 desf\u0103\u0219urare!<\/p>\n<h2>Caracteristicile func\u021bion\u0103rii GitLab.com \u00een Kubernetes<\/h2>\n<p>\nPentru GitLab.com, utiliz\u0103m un singur cluster regional GKE, care gestioneaz\u0103 tot traficul aplica\u021biei. Pentru a minimiza complexitatea (deja complicat\u0103) migra\u021biei, ne concentr\u0103m pe servicii care nu depind de stocarea local\u0103 sau de NFS. GitLab.com utilizeaz\u0103 predominant o baz\u0103 de cod monolitic\u0103 pe Rails \u0219i direc\u021bion\u0103m traficul \u00een func\u021bie de caracteristicile sarcinilor de lucru c\u0103tre diferite endpoint-uri, izolate \u00een propriile grupuri de noduri.<\/p>\n<p>\u00cen ceea ce prive\u0219te frontendul, aceste tipuri se \u00eempart \u00een cereri c\u0103tre web, API, Git SSH\/HTTPS \u0219i Registry. \u00cen cazul backendului, \u00eemp\u0103r\u021bim joburile din cozi \u00een func\u021bie de diferite caracteristici, \u00een func\u021bie de <noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/blog\/2020\/06\/24\/scaling-our-use-of-sidekiq\/\">limitele de resurse predefinite<\/a><\/noindex>, care ne permit s\u0103 stabilim obiective de nivel de serviciu (Service-Level Objectives, SLOs) pentru diferitele sarcini.<\/p>\n<p>Toate aceste servicii GitLab.com sunt configurate folosind chart-uri Helm nemodificate GitLab. Configurarea se face \u00een sub-chart-uri care pot fi incluse selectiv pe m\u0103sur\u0103 ce ne mut\u0103m gradual serviciile \u00een cluster. Chiar \u0219i av\u00e2nd \u00een vedere c\u0103 s-a decis s\u0103 nu fie incluse \u00een migra\u021bie unele dintre serviciile noastre stateful, cum ar fi Redis, Postgres, GitLab Pages \u0219i Gitaly, utilizarea Kubernetes permite o reducere radical\u0103 a num\u0103rului de VM-uri gestionate \u00een prezent de Chef.<\/p>\n<h2>Transparen\u021b\u0103 \u0219i gestionare a configura\u021biei Kubernetes<\/h2>\n<p>\nToate set\u0103rile sunt gestionate de c\u0103tre GitLab. Pentru aceasta, folosim trei proiecte de configurare bazate pe Terraform \u0219i Helm. Ne str\u0103duim s\u0103 folosim, acolo unde este posibil, GitLab \u00eensu\u0219i pentru a rula GitLab, dar pentru sarcinile opera\u021bionale avem o instalare separat\u0103 GitLab care func\u021bioneaz\u0103. Aceasta este necesar\u0103 pentru a nu depinde de disponibilitatea GitLab.com \u00een timpul desf\u0103\u0219ur\u0103rii \u0219i actualiz\u0103rilor GitLab.com.<\/p>\n<p>De\u0219i pipeline-urile noastre pentru clusterul Kubernetes func\u021bioneaz\u0103 pe o instalare separat\u0103 de GitLab, exist\u0103 oglinzi pentru repositoarele de cod, disponibile public la urm\u0103toarele adrese:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gl-infra\/k8s-workloads\/gitlab-com\">k8s-workloads\/gitlab-com<\/a><\/noindex> \u2014 un binding de configurare GitLab.com pentru chart-ul Helm GitLab;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gl-infra\/k8s-workloads\/gitlab-helmfiles\/\">k8s-workloads\/gitlab-helmfiles<\/a><\/noindex> \u2014 con\u021bine configura\u021bii pentru servicii care nu sunt direct legate de aplica\u021bia GitLab. Acestea includ configura\u021bii pentru logare \u0219i monitorizarea clusterului, precum \u0219i pentru uneltele integrate precum PlantUML;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gitlab-com-infrastructure\">Gitlab-com-infrastructure<\/a><\/noindex> \u2014 configura\u021bia Terraform pentru Kubernetes \u0219i infrastructura VM veche (legacy). Aici se configureaz\u0103 toate resursele necesare pentru a rula clusterul, inclusiv clusterul \u00een sine, pool-urile de noduri, conturile de servicii, rezervarea adreselor IP.<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/612125403171d73106a081bf4244a52b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>C\u00e2nd se fac modific\u0103ri, se prezint\u0103 un <\/i><noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gl-infra\/k8s-workloads\/gitlab-com\/-\/merge_requests\/315#note_390180361\"><i>rezumat public<\/i><\/a><\/noindex><i> cu un link la un diff detaliat, pe care SRE-ul \u00eel analizeaz\u0103 \u00eenainte de a face modific\u0103ri \u00een cluster.<\/i><\/p>\n<p>Pentru SRE, linkul duce la un diffs detaliat \u00een instalarea GitLab, care este utilizat\u0103 pentru opera\u021biuni \u0219i accesul c\u0103reia este restric\u021bionat. Acest lucru permite angaja\u021bilor \u0219i comunit\u0103\u021bii, care nu au acces la proiectul opera\u021bional (care este deschis doar pentru SRE), s\u0103 examineze modific\u0103rile propuse \u00een configurare. Combin\u00e2nd o instan\u021b\u0103 public\u0103 a GitLab pentru cod cu o instan\u021b\u0103 privat\u0103 pentru pipeline-urile CI, p\u0103str\u0103m un flux de lucru unificat, asigur\u00e2nd \u00een acela\u0219i timp independen\u021ba fa\u021b\u0103 de GitLab.com \u00een timpul actualiz\u0103rilor de configurare.<\/p>\n<h2>Ce am descoperit \u00een timpul migra\u021biei<\/h2>\n<p>\nPe parcursul mut\u0103rii, am acumulat experien\u021b\u0103 pe care o aplic\u0103m \u00een noile migra\u021bii \u0219i desf\u0103\u0219ur\u0103ri \u00een Kubernetes.<\/p>\n<h3>1. Cre\u0219terea costurilor din cauza traficului \u00eentre zonele de disponibilitate<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/3dce44b3f803ffcea13e0101e7343d91.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Statistica de egress zilnic\u0103 (bai\u021bi pe zi) pentru parcul de repositoare Git pe GitLab.com<\/i><\/p>\n<p>Google \u00ee\u0219i \u00eemparte re\u021beaua \u00een regiuni. Acestea sunt \u00eemp\u0103r\u021bite \u00een zone de disponibilitate (AZ). G\u0103zduirea Git este asociat\u0103 cu volume mari de date, a\u0219a c\u0103 este important s\u0103 control\u0103m egressul de re\u021bea. \u00cen cazul traficului intern, egressul este gratuit doar dac\u0103 r\u0103m\u00e2ne \u00een cadrul unei singure zone de disponibilitate. La momentul redact\u0103rii acestui articol, livr\u0103m aproximativ 100 TB de date \u00eentr-o zi obi\u0219nuit\u0103 de lucru (\u0219i aceasta este doar pentru repozitoarele Git). Serviciile care \u00een vechea noastr\u0103 topologie, bazat\u0103 pe VM, erau pe acelea\u0219i ma\u0219ini virtuale, acum func\u021bioneaz\u0103 \u00een poduri diferite Kubernetes. Aceasta \u00eenseamn\u0103 c\u0103 o parte din traficul care \u00eenainte era local pentru VM poate ie\u0219i poten\u021bial din limitele zonelor de disponibilitate.<\/p>\n<p>Clusterile regionale GKE permit acoperirea mai multor zone de disponibilitate pentru redundan\u021b\u0103. Lu\u0103m \u00een considerare <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gl-infra\/delivery\/-\/issues\/1175\">\u00eemp\u0103r\u021birea clusterului regional GKE \u00een clustere unizone<\/a><\/noindex> pentru serviciile care genereaz\u0103 volume mari de trafic. Acest lucru va reduce costurile pentru egress \u0219i va men\u021bine redundan\u021ba la nivel de cluster.<\/p>\n<h3>2. Limit\u0103ri, solicit\u0103ri de resurse \u0219i scalare<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/6e2e1ca4d37666b49358d94cd8660c56.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Num\u0103rul de replici care gestioneaz\u0103 traficul de produc\u021bie pe registry.gitlab.com. Traficul atinge v\u00e2rful la aproximativ 15:00 UTC.<\/i><\/p>\n<p>Povestea noastr\u0103 de migra\u021bie a \u00eenceput \u00een august 2019, c\u00e2nd am mutat primul serviciu \u2014 Registrul de containere GitLab (GitLab Container Registry) \u2014 \u00een Kubernetes. Acest serviciu critic, cu un trafic mare, a fost potrivit pentru prima migra\u021bie, deoarece reprezenta o aplica\u021bie stateless cu pu\u021bine dependen\u021be externe. Prima problem\u0103 cu care ne-am confruntat a fost num\u0103rul mare de poduri scoase din serviciu din cauza lipsei de memorie pe noduri. A\u0219adar, a trebuit s\u0103 ajust\u0103m solicit\u0103rile \u0219i limit\u0103rile.<\/p>\n<p>S-a constatat c\u0103, \u00een cazul unei aplica\u021bii a c\u0103rei consum de memorie cre\u0219te \u00een timp, valori sc\u0103zute pentru solicit\u0103ri (rezerv\u00e2nd memorie pentru fiecare pod) \u00eempreun\u0103 cu un 'limit' 'generos' pentru utilizare duceau la saturare <i>(saturation)<\/i> nodurilor \u0219i la un nivel ridicat de expulzare. Pentru a face fa\u021b\u0103 acestei probleme, s-a <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/gl-infra\/delivery\/-\/issues\/998#note_388983696\">s-a decis s\u0103 cre\u0219tem solicit\u0103rile \u0219i s\u0103 sc\u0103dem limit\u0103rile<\/a><\/noindex>. Acest lucru a u\u0219urat presiunea pe noduri \u0219i a asigurat un ciclu de via\u021b\u0103 al podurilor care nu exercita o presiune prea mare pe nod. Acum \u00eencepem migra\u021biile cu valori 'generoase' (\u0219i aproape identice) pentru solicit\u0103ri \u0219i limit\u0103ri, ajust\u00e2ndu-le dup\u0103 cum este necesar.<\/p>\n<h3>3. Metrici \u0219i jurnale<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/66d1fd47b57d5826f13defa6e6a7fe3c.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Divizia de infrastructur\u0103 se concentreaz\u0103 pe \u00eent\u00e2rzieri, procentul de erori \u0219i satura\u021bia stabilit\u0103 <\/i><noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Service-level_objective\"><i>obiectivelor de nivel de serviciu<\/i><\/a><\/noindex><i> (SLO), legate de <\/i><noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-com\/dashboards-gitlab-com\/-\/metrics\/sla-dashboard.yml?environment=1790496&amp;duration_seconds=86400\"><i>disponibilitatea general\u0103 a sistemului nostru<\/i><\/a><\/noindex><i>.<\/i><\/p>\n<p>\u00cen ultimul an, unul dintre evenimentele cheie din cadrul diviziei de infrastructur\u0103 a fost \u00eembun\u0103t\u0103\u021birea monitoriz\u0103rii \u0219i gestion\u0103rii SLO. SLO-urile ne-au permis s\u0103 stabilim obiective pentru servicii individuale, pe care le-am monitorizat cu aten\u021bie \u00een timpul migra\u021biei. Dar chiar \u0219i cu o astfel de observabilitate \u00eembun\u0103t\u0103\u021bit\u0103, nu este \u00eentotdeauna posibil s\u0103 vedem imediat problemele utiliz\u00e2nd metrici \u0219i alerte. De exemplu, concentr\u00e2ndu-ne pe \u00eent\u00e2rzieri \u0219i pe procentul de erori, nu acoperim pe deplin toate scenariile de utilizare a serviciului \u00een migrare.<\/p>\n<p>Aceast\u0103 problem\u0103 a fost identificat\u0103 aproape imediat dup\u0103 transferul unei p\u0103r\u021bi din sarcinile de lucru \u00een cluster. A devenit deosebit de evident\u0103 atunci c\u00e2nd a fost necesar\u0103 verificarea func\u021biilor a c\u0103ror num\u0103r de solicit\u0103ri este mic, dar care au dependen\u021be de configurare foarte specifice. Una dintre lec\u021biile cheie \u00eenv\u0103\u021bate \u00een urma migra\u021biei a fost necesitatea de a lua \u00een considerare \u00een timpul monitoriz\u0103rii nu doar metricile, ci \u0219i jurnalele \u0219i \u201ecoada lung\u0103\u201d <i>(referindu-se la <\/i><noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Long_tail\"><i>aceast\u0103 distribu\u021bie<\/i><\/a><\/noindex><i> pe grafic \u2014 nota traductoarei)<\/i> de erori. Acum, pentru fiecare migra\u021bie, includem o list\u0103 detaliat\u0103 de solicit\u0103ri pentru jurnale <i>(log queries)<\/i> \u0219i planific\u0103m proceduri clare de restore, care pot fi transmise de la o echip\u0103 la alta \u00een cazul apari\u021biei problemelor.<\/p>\n<p>Servirea paralel\u0103 a acelorasi solicit\u0103ri pe vechea infrastructur\u0103 VM \u0219i pe cea nou\u0103, bazat\u0103 pe Kubernetes, a reprezentat o sarcin\u0103 unic\u0103. Spre deosebire de migrarea de tip lift-and-shift <i>(transfer rapid al aplica\u021biilor \u201ea\u0219a cum sunt\u201d \u00een noua infrastructur\u0103; mai multe detalii pot fi citite, de exemplu, <\/i><noindex><a rel=\"nofollow\" href=\"https:\/\/www.ibm.com\/cloud\/learn\/lift-and-shift\"><i>aici<\/i><\/a><\/noindex><i> \u2014 nota trad.)<\/i>, munca paralel\u0103 pe VM-urile \u201evechi\u201d \u0219i Kubernetes necesit\u0103 ca instrumentele de monitorizare s\u0103 fie compatibile cu ambele medii \u0219i s\u0103 poat\u0103 combina metricile \u00eentr-o singur\u0103 form\u0103. Este important s\u0103 folosim acelea\u0219i dashboard-uri \u0219i interog\u0103ri la loguri pentru a realiza o observabilitate consistent\u0103 \u00een timpul perioadei de tranzi\u021bie.<\/p>\n<h3>4. Comutarea traficului pe noul cluster<\/h3>\n<p>\nPentru GitLab.com, o parte din servere este alocat\u0103 <noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/handbook\/engineering\/#canary-testing\">etapei canary<\/a><\/noindex>. Parcul canary deserveste proiectele noastre interne \u0219i poate <noindex><a rel=\"nofollow\" href=\"https:\/\/next.gitlab.com\/\">fi activat de utilizatori<\/a><\/noindex>. Dar, \u00een primul r\u00e2nd, este destinat verific\u0103rii modific\u0103rilor aduse infrastructurii \u0219i aplica\u021biei. Primul serviciu migrat a \u00eenceput prin a accepta un volum limitat de trafic intern, \u0219i continu\u0103m s\u0103 folosim aceast\u0103 metod\u0103 pentru a ne asigura de respectarea SLO \u00eenainte de a direc\u021biona tot traficul c\u0103tre cluster.<\/p>\n<p>\u00cen cazul migr\u0103rii, aceasta \u00eenseamn\u0103 c\u0103 mai \u00eent\u00e2i cererile pentru proiectele interne sunt direc\u021bionate c\u0103tre Kubernetes, iar apoi comut\u0103m treptat restul traficului c\u0103tre cluster prin modificarea greut\u0103\u021bii pentru backend prin HAProxy. \u00cen timpul tranzi\u021biei de la VM la Kubernetes, a devenit clar c\u0103 este foarte util s\u0103 avem o modalitate simpl\u0103 de redirec\u021bionare a traficului \u00eentre infrastructura veche \u0219i cea nou\u0103 \u0219i, \u00een consecin\u021b\u0103, s\u0103 \u021binem infrastructura veche preg\u0103tit\u0103 pentru un rollback \u00een primele zile dup\u0103 migrare.<\/p>\n<h3>5. Capacit\u0103\u021bile de rezerv\u0103 ale podurilor \u0219i utilizarea acestora<\/h3>\n<p>\nAproape imediat a fost identificat\u0103 urm\u0103toarea problem\u0103: podurile pentru serviciul Registry porneau rapid, \u00eens\u0103 pornirea podurilor pentru Sidekiq dura p\u00e2n\u0103 la <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-org\/charts\/gitlab\/-\/issues\/1775\">dou\u0103 minute<\/a><\/noindex>. Pornirea \u00eendelungat\u0103 a podurilor pentru Sidekiq a devenit o problem\u0103 atunci c\u00e2nd am \u00eenceput migrarea \u00een Kubernetes a sarcinilor de lucru pentru lucr\u0103torii care trebuie s\u0103 proceseze rapid sarcini \u0219i s\u0103 se scaleze rapid.<\/p>\n<p>\u00cen acest caz, lec\u021bia a fost c\u0103, de\u0219i Horizontal Pod Autoscaler (HPA) \u00een Kubernetes gestioneaz\u0103 bine cre\u0219terea traficului, este important s\u0103 lu\u0103m \u00een considerare caracteristicile sarcinilor de lucru \u0219i s\u0103 rezerv\u0103m resurse pentru pod-uri (mai ales \u00een condi\u021bii de cerere inegal distribuit\u0103). \u00cen cazul nostru, am observat o cre\u0219tere brusc\u0103 a job-urilor, ceea ce a dus la o scalare rapid\u0103, epuiz\u00e2nd resursele CPU \u00eenainte de a reu\u0219i s\u0103 scal\u0103m grupul de noduri.<\/p>\n<p>Exist\u0103 \u00eentotdeauna tenta\u021bia de a \u201estoarce\u201d c\u00e2t mai mult din cluster, totu\u0219i, dup\u0103 ce ne-am confruntat ini\u021bial cu probleme de performan\u021b\u0103, acum \u00eencepem cu un buget generos de pod-uri \u0219i \u00eel reducem ulterior, monitoriz\u00e2nd atent SLO. Rularea pod-urilor pentru serviciul Sidekiq s-a accelerat semnificativ \u0219i acum dureaz\u0103 \u00een medie aproximativ 40 de secunde. <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/gitlab-org\/charts\/gitlab\/-\/issues\/1775\">Reducerea timpului de lansare a pod-urilor<\/a><\/noindex> au beneficiat at\u00e2t GitLab.com, c\u00e2t \u0219i utilizatorii no\u0219tri de instal\u0103ri self-managed care folosesc chartul Helm oficial GitLab.<\/p>\n<h2>Concluzie<\/h2>\n<p>\nDup\u0103 mutarea fiec\u0103rui serviciu, ne-am bucurat de avantajele utiliz\u0103rii Kubernetes \u00een produc\u021bie: o desf\u0103\u0219urare mai rapid\u0103 \u0219i mai sigur\u0103 a aplica\u021biei, scalabilitate \u0219i o distribu\u021bie mai eficient\u0103 a resurselor. Iar beneficiile migra\u021biei dep\u0103\u0219esc serviciul GitLab.com. Fiecare \u00eembun\u0103t\u0103\u021bire a chartului Helm oficial aduce beneficii \u0219i utilizatorilor s\u0103i.<\/p>\n<p>Sper c\u0103 v-a pl\u0103cut povestea despre aventurile noastre cu migrarea \u00een Kubernetes. Continu\u0103m s\u0103 mut\u0103m noi servicii \u00een cluster. Informa\u021bii suplimentare pot fi g\u0103site \u00een publica\u021biile urm\u0103toare:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/handbook\/engineering\/infrastructure\/production\/kubernetes\/gitlab-com\/\">De ce ne migr\u0103m la Kubernetes?<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/handbook\/engineering\/infrastructure\/production\/architecture\/#gitlab-com-on-kubernetes\">GitLab.com pe Kubernetes<\/a><\/noindex>\u00bb;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/gitlab.com\/groups\/gitlab-com\/gl-infra\/-\/epics\/112\">Epic pentru migrarea GitLab.com pe Kubernetes<\/a><\/noindex>.<\/li>\n<\/ul>\n<p><\/p>\n<h2>P.S. de la traduc\u0103tor<\/h2>\n<p>\nCiti\u021bi \u0219i \u00een blogul nostru:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/519962\/\">3 ani cu Kubernetes \u00een produc\u021bie: iat\u0103 ce am \u00eenv\u0103\u021bat<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/504396\/\">10 gre\u0219eli tipice \u00een utilizarea Kubernetes<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/335814\/\">Pove\u0219ti de succes Kubernetes \u00een produc\u021bie. Partea 3: GitHub<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/440278\/\">Pove\u0219ti de succes Kubernetes \u00een produc\u021bie. Partea 10: Reddit<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/520150\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0430\u0434\u0430\u043f\u0442\u0430\u0446\u0438\u044e Kubernetes \u0432 GitLab \u0441\u0447\u0438\u0442\u0430\u044e\u0442 \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0434\u0432\u0443\u0445 \u0433\u043b\u0430\u0432\u043d\u044b\u0445 \u0444\u0430\u043a\u0442\u043e\u0440\u043e\u0432, \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0445 \u0440\u043e\u0441\u0442\u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438. \u0422\u0435\u043c \u043d\u0435 \u043c\u0435\u043d\u0435\u0435, \u0434\u043e \u043d\u0435\u0434\u0430\u0432\u043d\u0435\u0433\u043e \u0432\u0440\u0435\u043c\u0435\u043d\u0438 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u0441\u0435\u0440\u0432\u0438\u0441\u0430 GitLab.com \u0431\u044b\u043b\u0430 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0430 \u043d\u0430 \u0432\u0438\u0440\u0442\u0443\u0430\u043b\u044c\u043d\u044b\u0445 \u043c\u0430\u0448\u0438\u043d\u0430\u0445, \u0438 \u0442\u043e\u043b\u044c\u043a\u043e \u043e\u043a\u043e\u043b\u043e \u0433\u043e\u0434\u0430 \u043d\u0430\u0437\u0430\u0434 \u043d\u0430\u0447\u0430\u043b\u0430\u0441\u044c \u0435\u0451 \u043c\u0438\u0433\u0440\u0430\u0446\u0438\u044f \u0432 K8s, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u0434\u043e \u0441\u0438\u0445 \u043f\u043e\u0440 \u043d\u0435 \u0437\u0430\u0432\u0435\u0440\u0448\u0435\u043d\u0430. \u0420\u0430\u0434\u044b \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u043d\u0435\u0434\u0430\u0432\u043d\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0438 SRE-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430 GitLab \u043e \u0442\u043e\u043c, \u043a\u0430\u043a [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":94978,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-94977","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0430\u0434\u0430\u043f\u0442\u0430\u0446\u0438\u044e Kubernetes \u0432 GitLab \u0441\u0447\u0438\u0442\u0430\u044e\u0442 \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0434\u0432\u0443\u0445 \u0433\u043b\u0430\u0432\u043d\u044b\u0445 \u0444\u0430\u043a\u0442\u043e\u0440\u043e\u0432, \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0445 \u0440\u043e\u0441\u0442\u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041d\u0430\u0448\u0438 \u0432\u044b\u0432\u043e\u0434\u044b \u0437\u0430 \u0433\u043e\u0434 \u043c\u0438\u0433\u0440\u0430\u0446\u0438\u0438 GitLab.com \u043d\u0430 Kubernetes | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0430\u0434\u0430\u043f\u0442\u0430\u0446\u0438\u044e Kubernetes \u0432 GitLab \u0441\u0447\u0438\u0442\u0430\u044e\u0442 \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0434\u0432\u0443\u0445 \u0433\u043b\u0430\u0432\u043d\u044b\u0445 \u0444\u0430\u043a\u0442\u043e\u0440\u043e\u0432, \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0445 \u0440\u043e\u0441\u0442\u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-09-24T05:43:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-09-24T05:43:00+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Concluziile noastre dup\u0103 un an de migrare a GitLab.com pe Kubernetes | ProHoster","description":"Not\u0103 de traducere: adaptarea Kubernetes \u00een GitLab este considerat\u0103 unul dintre cele dou\u0103 factori principali care contribuie la cre\u0219terea companiei.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041d\u0430\u0448\u0438 \u0432\u044b\u0432\u043e\u0434\u044b \u0437\u0430 \u0433\u043e\u0434 \u043c\u0438\u0433\u0440\u0430\u0446\u0438\u0438 GitLab.com \u043d\u0430 Kubernetes | ProHoster","og:description":"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0430\u0434\u0430\u043f\u0442\u0430\u0446\u0438\u044e Kubernetes \u0432 GitLab \u0441\u0447\u0438\u0442\u0430\u044e\u0442 \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0434\u0432\u0443\u0445 \u0433\u043b\u0430\u0432\u043d\u044b\u0445 \u0444\u0430\u043a\u0442\u043e\u0440\u043e\u0432, \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0445 \u0440\u043e\u0441\u0442\u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438.","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/nashi-vyvody-za-god-migraczii-gitlab-com-na-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-24T05:43:00+00:00","article:modified_time":"2020-09-24T05:43:00+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"94977","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:14:45","updated":"2022-10-03 07:39:43","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/94977","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=94977"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/94977\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/94978"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=94977"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=94977"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=94977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}