{"id":41051,"date":"2020-02-05T21:28:58","date_gmt":"2020-02-05T18:28:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/consul-iptables-3"},"modified":"2020-02-05T21:28:58","modified_gmt":"2020-02-05T18:28:58","slug":"consul-iptables-3","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/consul-iptables-3","title":{"rendered":"Consul + iptables = :3","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>\u00cen 2010, compania <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/wargaming\/\">Wargaming<\/a><\/noindex> avea 50 de servere \u0219i un model de re\u021bea simplu: backend, frontend \u0219i firewall. Num\u0103rul serverelor a crescut, iar modelul s-a complicat: staging-uri, VLAN-uri izolate cu ACL, apoi VPN-uri cu VRF, VLAN-uri cu ACL pe L2, VRF cu ACL pe L3. Te sim\u021bi confuz? De aici devine mai interesant.<\/p>\n<p>C\u00e2nd num\u0103rul serverelor a ajuns la 16.000, a devenit imposibil s\u0103 lucrezi f\u0103r\u0103 lacrimi cu at\u00e2t de multe segmente diverse. A\u0219a c\u0103 au venit cu o alt\u0103 solu\u021bie. Au luat stiva Netfilter, au ad\u0103ugat Consul ca surs\u0103 de date, rezult\u00e2nd un firewall rapid \u0219i distribuit. Acesta a \u00eenlocuit ACL-urile de pe routere \u0219i a fost utilizat ca firewall extern \u0219i intern. Pentru gestionarea dinamic\u0103 a instrumentului, au dezvoltat un sistem BEFW, pe care l-au aplicat peste tot: de la gestionarea accesului utilizatorilor \u00een re\u021beaua de produc\u021bie p\u00e2n\u0103 la izolarea segmentelor de re\u021bea unele de altele.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/246e74278f413749a676599b5ce986a5.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nCum func\u021bioneaz\u0103 toate acestea \u0219i de ce ar trebui s\u0103 acorzi aten\u021bie acestui sistem, va explica <strong>Ivan Agarakov<\/strong> (<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/users\/annmuor\/\" class=\"user_link\">annmuor<\/a><\/noindex>) \u2014 \u0219eful grupului de securitate infrastructural\u0103 din divizia Maintenance a centrului de dezvoltare din Minsk al companiei. Ivan este un fan al SELinux, \u00eei place Perl, scrie cod. Ca \u0219ef al grupului ISC, lucreaz\u0103 regulat cu jurnalele, backup-urile \u0219i R&amp;D, pentru a proteja Wargaming de hackeri \u0219i pentru a asigura func\u021bionarea tuturor serverelor de jocuri din companie.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\n<center><div class=\"youtube-placeholder\" data-id=\"4zP67uYnsR4\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/4zP67uYnsR4\/hqdefault.jpg\" alt=\"Reda\u021bi video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<h2>Context istoric<\/h2>\n<p>\n\u00cenainte s\u0103 explic cum am f\u0103cut asta, voi povesti cum am ajuns aici \u0219i de ce a fost nevoie de asta. Pentru a ilustra, s\u0103 ne \u00eentoarcem cu 9 ani \u00een urm\u0103: 2010, c\u00e2nd au ap\u0103rut pentru prima dat\u0103 World of Tanks. Compania Wargaming avea aproximativ 50 de servere.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/64e1c2839363dbdf59ba5f11bb13530e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Graficul cre\u0219terii serverelor companiei.<\/em><\/p>\n<p>Aveam un model de re\u021bea. Pentru acea perioad\u0103, acesta era optim.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/c0509324e2a83882a9792ba75612ceea.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Modelul de re\u021bea \u00een 2010.<\/em><\/p>\n<p>\u00cen frontend locuiesc r\u0103uf\u0103c\u0103torii care vor s\u0103 ne distrug\u0103, dar exist\u0103 un firewall acolo. \u00cen backend nu exist\u0103 firewall, dar sunt 50 de servere, le \u0219tim pe toate. Totul func\u021bioneaz\u0103 bine.<\/p>\n<p>\u00cen 4 ani, parcul de servere a crescut de 100 de ori, p\u00e2n\u0103 la 5000. Au ap\u0103rut primele re\u021bele izolate \u2014 staging-uri: acestea nu pot accesa produc\u021bia \u0219i acolo rulau frecvent lucruri care ar fi putut fi periculoase.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/08e1d7670ce1a1346e49d87c67c4be1b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Modelul de re\u021bea \u00een 2014.<\/em><\/p>\n<p>Din iner\u021bie am folosit acelea\u0219i echipamente, iar toat\u0103 munca s-a desf\u0103\u0219urat pe VLAN-uri izolate: pe VLAN-uri sunt scrise ACL-uri care permit sau interzic anumite conexiuni.<\/p>\n<p>\u00cen 2016, num\u0103rul serverelor a ajuns la 8000. Wargaming a achizi\u021bionat alte studiouri, iar re\u021belele partenerilor s-au extins. Ele par a fi ale noastre, dar nu chiar: pentru parteneri, VLAN-ul adesea nu func\u021biona, fiind nevoie de VPN cu VRF; izol\u0103rile s-au complicat. Mixul de izol\u0103ri ACL a crescut.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/a6580c9551730447e16d7881eb102275.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Modelul de re\u021bea \u00een 2016.<\/em><\/p>\n<p>P\u00e2n\u0103 la \u00eenceputul anului 2018, parcul de ma\u0219ini a crescut la 16.000. Existau 6 segmente, iar celelalte nu au fost num\u0103rate, inclusiv cele \u00eenchise, \u00een care erau stocate datele financiare. Au ap\u0103rut re\u021bele de containere (Kubernetes), DevOps, re\u021bele cloud conectate prin VPN, de exemplu, din IWS. Regulile erau foarte multe \u2014 era dureros.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/ec44e6a60da98daa4bbf2bca2e3b2e1a.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Modelul de re\u021bea \u0219i metodele de izolare \u00een 2018.<\/em><\/p>\n<p>Pentru izolare, am folosit: VLAN cu ACL pe L2, VRF cu ACL pe L3, VPN \u0219i multe altele. Prea multe.<\/p>\n<h2>Probleme<\/h2>\n<p>\nToat\u0103 lumea folose\u0219te ACL \u0219i VLAN. Ceva este \u00een neregul\u0103? La aceast\u0103 \u00eentrebare va r\u0103spunde Harold, ascunz\u00e2nd durerea.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/4df5105af3bce515d80e8e5f3e14c027.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAu existat multe probleme, dar majore \u2014 cinci.<\/p>\n<ul>\n<li><strong>Cre\u0219terea geometric\u0103 a pre\u021bului pentru noile reguli<\/strong>. Fiecare nou\u0103 regul\u0103 se ad\u0103uga mai greu dec\u00e2t cea anterioar\u0103, pentru c\u0103 trebuia \u00eent\u00e2i v\u0103zut dac\u0103 nu exista deja o astfel de regul\u0103.<\/li>\n<li><strong>Nu exist\u0103 firewall \u00een interiorul segmentelor<\/strong>. Segmentele s-au separatat cumva, iar \u00een interior deja nu erau suficiente resurse.<\/li>\n<li><strong>Regulile se aplicau cu \u00eent\u00e2rziere. <\/strong>Cu m\u00e2na, un operator putea scrie o regul\u0103 local\u0103 \u00een aproximativ o or\u0103. Una global\u0103 dura c\u00e2teva zile.<\/li>\n<li><strong>Dificult\u0103\u021bi \u00een auditarea regulilor<\/strong>. Mai exact, nu putea fi realizat. Primele regulile au fost scrise \u00eenc\u0103 din 2010, iar majoritatea autorilor lor nu mai lucrau \u00een companie.<\/li>\n<li><strong>Nivel sc\u0103zut de control asupra infrastructurii<\/strong>. Aceasta este principala problem\u0103 \u2014 nu \u0219tiam bine ce se petrece cu adev\u0103rat.<\/li>\n<\/ul>\n<p>\nA\u0219a ar\u0103ta inginerul de re\u021bea \u00een 2018 c\u00e2nd auzea: \u201eMai avem nevoie de ceva ACL\u201d.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/8366d3092558ebb15a4ba6d8e5702032.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h2>Solu\u021bii<\/h2>\n<p>\nLa \u00eenceputul anului 2018 s-a decis s\u0103 se fac\u0103 ceva \u00een aceast\u0103 privin\u021b\u0103.<\/p>\n<p><strong>Pre\u021bul integr\u0103rilor cre\u0219te constant. <\/strong>Punctul de plecare a fost c\u0103 centrele de date mari au \u00eencetat s\u0103 mai sus\u021bin\u0103 VLAN-uri \u0219i ACL-uri izolate, deoarece memoria dispozitivelor s-a epuizat.<\/p>\n<p>Solu\u021bia: am eliminat factorul uman \u0219i am automatizat c\u00e2t mai mult accesul.<\/p>\n<p><strong>Noile reguli se aplic\u0103 lent. <\/strong>Solu\u021bia: accelerarea aplic\u0103rii regulilor, f\u0103c\u00e2nd-o distribuit\u0103 \u0219i paralel\u0103. Pentru aceasta, este nevoie de un sistem distribuit, astfel \u00eenc\u00e2t regulile s\u0103 fie livrate singure, f\u0103r\u0103 rsync sau SFTP pe o mie de sisteme.<\/p>\n<p><strong>Lipsa unui firewall \u00een interiorul segmentelor. <\/strong>Firewall-ul din segmente a \u00eenceput s\u0103 ne afecteze c\u00e2nd \u00een cadrul acelea\u0219i re\u021bele au ap\u0103rut diferite servicii. Solu\u021bia: utilizarea firewall-ului la nivel de gazd\u0103 \u2014 firewalls bazate pe gazd\u0103. Practic, avem Linux peste tot, \u0219i peste tot exist\u0103 iptables, deci nu este o problem\u0103.<\/p>\n<p><strong>Dificult\u0103\u021bi \u00een auditurile regulilor.<\/strong> Solu\u021bia: p\u0103strarea tuturor regulilor \u00eentr-un singur loc pentru revizuire \u0219i gestionare, astfel \u00eenc\u00e2t s\u0103 putem audita totul.<\/p>\n<p><strong>Un nivel sc\u0103zut de control asupra infrastructurii. <\/strong>Solu\u021bia: realizarea unei inventariz\u0103ri a tuturor serviciilor \u0219i acceselor \u00eentre acestea.<\/p>\n<p>Aceasta este mai mult un proces administrativ dec\u00e2t tehnic. Uneori avem 200-300 de versiuni noi pe s\u0103pt\u0103m\u00e2n\u0103, mai ales \u00een timpul promo\u021biilor \u0219i s\u0103rb\u0103torilor. Acestea sunt doar pentru o echip\u0103 din DevOps. Cu un astfel de num\u0103r de versiuni este imposibil s\u0103 vedem \u0219i s\u0103 \u00een\u021belegem ce porturi, IP-uri, integra\u021bii sunt necesare. De aceea, am avut nevoie de manageri de servicii special instrui\u021bi, care s\u0103 \u00eentrebe echipele: \u201eCe ave\u021bi \u0219i de ce a\u021bi ridicat acest serviciu?\u201d<\/p>\n<p>Dup\u0103 tot ce am lansat, inginerul re\u021belistic din 2019 ar\u0103ta deja a\u0219a.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/ccca25d7f7bb7f70638725addc6d99ed.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h2>Consul<\/h2>\n<p>\nAm decis c\u0103 tot ce am g\u0103sit cu ajutorul managerilor de servicii va fi stocat \u00een Consul \u0219i de acolo vom scrie regulile iptables.<\/p>\n<p>Cum am decis s\u0103 facem asta?<\/p>\n<ul>\n<li>Vom colecta toate serviciile, re\u021belele \u0219i utilizatorii.<\/li>\n<li>Vom crea reguli iptables pe baza acestora.<\/li>\n<li>Vom automatiza controlul.<\/li>\n<li>\u2026.<\/li>\n<li>PROFIT.<\/li>\n<\/ul>\n<p>\nConsul nu este un API distant, poate func\u021biona pe fiecare nod \u0219i scrie \u00een iptables. R\u0103m\u00e2ne doar s\u0103 ne g\u00e2ndim la mijloacele automate de control care vor cur\u0103\u021ba surplusul, iar cea mai mare parte a problemelor va fi rezolvat\u0103! Restul le vom rafina pe parcurs.<\/p>\n<h3>De ce Consul?<\/h3>\n<p>\n<strong>S-a dovedit a fi fiabil. <\/strong>\u00centre 2014-15 l-am folosit ca backend pentru Vault, unde stoc\u0103m parolele.<\/p>\n<p><strong>Nu pierde date.<\/strong>. De-a lungul utiliz\u0103rii Consul nu a pierdut date niciodat\u0103 \u00een urma unei avarii. Aceasta este o mare calitate pentru sistemul de gestionare a firewall-ului.<\/p>\n<p><strong>Conexiunile P2P accelereaz\u0103 r\u0103sp\u00e2ndirea modific\u0103rilor.<\/strong>. Cu P2P toate modific\u0103rile vin rapid, nu trebuie s\u0103 a\u0219tept\u0103m ore \u00eentregi.<\/p>\n<p><strong>API REST convenabil.<\/strong> Am considerat \u0219i Apache ZooKeeper, dar nu are API REST, va trebui s\u0103 implement\u0103m solu\u021bii improvizate.<\/p>\n<p><strong>Func\u021bioneaz\u0103 at\u00e2t ca un stocare de chei (KV), c\u00e2t \u0219i ca un catalog (Descoperirea serviciilor).<\/strong>. Putem stoca serviciile, catalogul, centrele de date simultan. Acest lucru este convenabil nu doar pentru noi, ci \u0219i pentru echipele \u00eenvecinate, deoarece construind un serviciu global, ne g\u00e2ndim la scalabilitate.<\/p>\n<p><strong>Scris \u00een Go, care face parte din stiva Wargaming. <\/strong>Ne place aceast\u0103 limb\u0103, avem mul\u021bi dezvoltatori Go.<\/p>\n<p><strong>Un sistem ACL puternic. <\/strong>\u00cen Consul, prin intermediul ACL, se poate gestiona cine poate scrie \u0219i ce. Garant\u0103m c\u0103 regulile firewall-ului nu se vor intersecta cu nimic altceva \u0219i nu vom avea probleme \u00een acest sens.<\/p>\n<p>Dar Consul are \u0219i dezavantaje.<\/p>\n<ul>\n<li><strong>Nu se scaleaz\u0103 \u00een cadrul centrului de date, dac\u0103 nu ave\u021bi versiunea business. <\/strong>Se scaleaz\u0103 doar prin federare.<\/li>\n<li><strong>Este foarte dependent de calitatea re\u021belei \u0219i de \u00eenc\u0103rcarea serverelor. <\/strong>Consul nu va func\u021biona corespunz\u0103tor pe un server aglomerat dac\u0103 exist\u0103 \u00eent\u00e2rzieri \u00een re\u021bea, de exemplu, viteze instabile. Aceasta se datoreaz\u0103 conexiunilor P2P \u0219i modelelor de distribu\u021bie a actualiz\u0103rilor.<\/li>\n<li><strong>Dificult\u0103\u021bi cu monitorizarea disponibilit\u0103\u021bii<\/strong>. \u00cen statusul Consul poate indica c\u0103 totul este \u00een regul\u0103, dar de fapt a murit de mult.<\/li>\n<\/ul>\n<p>\nAm rezolvat majoritatea acestor probleme \u00een timpul utiliz\u0103rii Consul, de aceea l-am ales. Compania are planuri pentru un backend alternativ, dar am \u00eenv\u0103\u021bat s\u0103 facem fa\u021b\u0103 problemelor \u0219i momentan tr\u0103im cu Consul.<\/p>\n<h3>Cum func\u021bioneaz\u0103 Consul<\/h3>\n<p>\n\u00centr-un centru de date conven\u021bional vom instala servere \u2014 de la trei la cinci. Unul sau dou\u0103 servere nu sunt suficiente: nu vor putea organiza un consens \u0219i s\u0103 decid\u0103 cine are dreptate c\u00e2nd datele nu coincid. Mai mult de cinci nu are sens, performan\u021ba va sc\u0103dea.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/96153c4e9075b067b34a63ceba5b80f0.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nClien\u021bii se conecteaz\u0103 la servere \u00een orice ordine: acelea\u0219i agen\u021bi, doar cu flag-ul <code>server = false<\/code>.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/0854d1286c18bb5e7c1130c6734154a9.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDup\u0103 aceasta, clien\u021bii primesc o list\u0103 de conexiuni P2P \u0219i construiesc leg\u0103turi \u00eentre ei.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/2457cb7707c27cc23f6e13415119b91d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nLa nivel global, conect\u0103m mai multe centre de date. Acestea se conecteaz\u0103, de asemenea, P2P \u0219i comunic\u0103 \u00eentre ele.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/3b4ae922eb8d2a6a9a79425e769505d7.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nC\u00e2nd dorim s\u0103 ob\u021binem date de la un alt centru de date, cererea merge de la server la server. Aceast\u0103 schem\u0103 se nume\u0219te <strong>protocolul Serf<\/strong>. Protocolul Serf, la fel ca \u0219i Consul, este o dezvoltare a HashiCorp.<\/p>\n<h3>C\u00e2teva fapte importante despre Consul<\/h3>\n<p>\nConsul are documenta\u021bie care descrie modul s\u0103u de func\u021bionare. Voi oferi doar fapte selectate care merit\u0103 s\u0103 fie cunoscute.<\/p>\n<p><strong>Serverele Consul aleg un lider dintre votan\u021bi<\/strong>. Consul alege un lider din lista serverelor pentru fiecare centru de date, iar toate cererile merg doar la el, indiferent de num\u0103rul de servere. Deconectarea liderului nu duce la alegeri suplimentare. Dac\u0103 nu este ales un lider, cererile nu sunt gestionate de nimeni.<\/p>\n<blockquote><p>A\u021bi dorit scalare orizontal\u0103? Ne pare r\u0103u, nu.<\/p><\/blockquote>\n<p> O cerere c\u0103tre un alt centru de date merge de la maestru la maestru, indiferent de serverul pe care a fost primit\u0103. Maestrul selectat prime\u0219te 100% din sarcin\u0103, cu excep\u021bia sarcinii pentru redirec\u021bionarea cererilor. O copie actualizat\u0103 a datelor exist\u0103 pe toate serverele din centrul de date, dar doar unul r\u0103spunde.<\/p>\n<blockquote><p>Singura modalitate de scalare este activarea modului stale pe client.<\/p><\/blockquote>\n<p>\u00cen modul stale se poate r\u0103spunde f\u0103r\u0103 quorum. Este un mod \u00een care renun\u021b\u0103m la consisten\u021ba datelor, dar citim pu\u021bin mai repede dec\u00e2t de obicei, iar orice server r\u0103spunde. Normal, scrierea se face doar prin maestru.<\/p>\n<p><strong>Consul nu copiaz\u0103 datele \u00eentre centrele de date<\/strong>. \u00cen cazul construirii unei federa\u021bii, fiecare server va avea doar propriile sale date. Pentru celelalte, se va adresa \u00eentotdeauna c\u0103tre altcineva.<\/p>\n<p><strong>Atomicitatea opera\u021biunilor nu este garantat\u0103 \u00een afara unei tranzac\u021bii<\/strong>. Aminti\u021bi-v\u0103 c\u0103 nu doar voi pute\u021bi modifica ceva. Dac\u0103 dori\u021bi altfel, efectua\u021bi o tranzac\u021bie cu blocare.<\/p>\n<p><strong>Opera\u021biile care blocheaz\u0103 nu garanteaz\u0103 blocarea<\/strong>. Cererea merge de la maestru la maestru, nu direct, a\u0219a c\u0103 nu exist\u0103 garan\u021bii c\u0103 blocarea va func\u021biona atunci c\u00e2nd efectua\u021bi o blocare, de exemplu, \u00eentr-un alt centru de date.<\/p>\n<p><strong>ACL-ul nu garanteaz\u0103 accesul (\u00een multe cazuri)<\/strong>. ACL-ul poate s\u0103 nu func\u021bioneze deoarece este stocat \u00eentr-un singur centru de date al federa\u021biei \u2014 \u00een centrul de date al ACL-ului (Primary DC). Dac\u0103 DC-ul nu v\u0103 r\u0103spunde, ACL-ul nu va func\u021biona.<\/p>\n<p><strong>Un maestru blocat va duce la blocarea \u00eentregii federa\u021bii<\/strong>. De exemplu, \u00eentr-o federa\u021bie cu 10 centre de date, iar \u00eentr-unul este o re\u021bea proast\u0103, iar un maestru se pr\u0103bu\u0219e\u0219te. To\u021bi cei care comunic\u0103 cu el se vor bloca \u00een cerc: cererea vine, nu exist\u0103 r\u0103spuns, firul se blocheaz\u0103. Nu ve\u021bi putea \u0219ti c\u00e2nd se va \u00eent\u00e2mpla, pur \u0219i simplu dup\u0103 o or\u0103 sau dou\u0103, \u00eentreaga federa\u021bie va c\u0103dea. Nu ve\u021bi putea face nimic \u00een leg\u0103tur\u0103 cu asta.<\/p>\n<p>Starea, quorum-ul \u0219i alegerile sunt procesate de un fir separat. Realegerile nu vor avea loc, starea nu va ar\u0103ta nimic. Crede\u021bi c\u0103 ave\u021bi un Consul activ, solicita\u021bi, \u0219i nimic nu se \u00eent\u00e2mpl\u0103 \u2014 nu exist\u0103 r\u0103spuns. Totu\u0219i, starea arat\u0103 c\u0103 totul este \u00een regul\u0103.<\/p>\n<p>Ne-am confruntat cu aceast\u0103 problem\u0103, a trebuit s\u0103 reconstruc\u0103m p\u0103r\u021bi specifice ale centrelor de date pentru a o evita.<\/p>\n<p><strong>\u00cen versiunea de afaceri a Consul Enterprise nu exist\u0103 unele dintre dezavantajele men\u021bionate mai sus.<\/strong>Are multe func\u021bii utile: selec\u021bia votan\u021bilor, distribu\u021bia, scalarea. Exist\u0103 un singur \u201edar\u201d \u2013 sistemul de licen\u021biere pentru sistemul distribuit este foarte scump.<\/p>\n<p>Sfaturi utile: <code>rm -rf \/var\/lib\/consul<\/code> \u2013 medicamentul universal pentru toate bolile agentului. Dac\u0103 ceva nu func\u021bioneaz\u0103, pur \u0219i simplu \u0219terge\u021bi datele dvs. \u0219i \u00eenc\u0103rca\u021bi datele dintr-o copie de rezerv\u0103. Cel mai probabil, Consul va \u00eencepe s\u0103 func\u021bioneze.<\/p>\n<h2>BEFW<\/h2>\n<p>\nAcum s\u0103 vorbim despre ce am ad\u0103ugat la Consul.<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/wgnet\/befw\">BEFW<\/a><\/noindex> \u2013 este un acronim pentru <strong>B<\/strong>ack<strong>E<\/strong>nd<strong>F<\/strong>ire<strong>W<\/strong>all. Trebuia s\u0103 numesc produsul \u00eentr-un mod oarecare c\u00e2nd am creat depozitul pentru a pune primele teste. Aceast\u0103 denumire a r\u0103mas.<\/p>\n<h3>\u0218abloane de reguli<\/h3>\n<p>\nRegulile sunt scrise \u00een sintaxa iptables.<\/p>\n<ul>\n<li>-N BEFW<\/li>\n<li>-P INPUT DROP<\/li>\n<li>-A INPUT -m state\u2014state RELATED,ESTABLISHED -j ACCEPT<\/li>\n<li>-A INPUT -i lo -j ACCEPT<\/li>\n<li>-A INPUT -j BEFW<\/li>\n<\/ul>\n<p>\nTot ce merge pe calea BEFW, cu excep\u021bia <code>ESTABLISHED<\/code>, <code>RELATED<\/code> \u0219i localhost. \u0218ablonul poate fi oricare, este doar un exemplu.<\/p>\n<p>Ce utilitate are BEFW?<\/p>\n<h3>Servicii<\/h3>\n<p>\nAvem un serviciu, are \u00eentotdeauna un port, nodul pe care func\u021bioneaz\u0103. De la nodul nostru putem \u00eentreba local agentul \u0219i afla c\u0103 avem un anumit serviciu. De asemenea, putem ad\u0103uga etichete.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/6d8c74db37e2e1eed971cfa6947cc5c8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nOrice serviciu care este pornit \u0219i \u00eenregistrat \u00een Consul se transform\u0103 \u00eentr-o regul\u0103 iptables. Avem SSH \u2013 deschidem portul 22. Scriptul Bash este simplu: curl \u0219i iptables, nimic mai mult.<\/p>\n<h3>Clien\u021bi<\/h3>\n<p>\nCum se deschide accesul nu tuturor, ci selectiv? Ad\u0103ug\u00e2nd listele de IP \u00een magazinul KV \u00een func\u021bie de numele serviciului.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/11a5af5aad9e848565458f168a1eee64.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDe exemplu, dorim ca to\u021bi din re\u021beaua zece s\u0103 poat\u0103 accesa serviciul SSH_TCP_22. Ad\u0103ug\u0103m un mic c\u00e2mp TTL? \u0219i acum avem permisiuni temporare, de exemplu, pe o zi.<\/p>\n<h3>Accesuri<\/h3>\n<p>\nConect\u0103m serviciile \u0219i clien\u021bii: avem un serviciu, fiecare are un magazin KV preg\u0103tit. Acum d\u0103m acces nu tuturor, ci selectiv.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/d1119fbb226595184cac8d9ce7a924e3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Grupuri<\/h3>\n<p>\nDac\u0103 de fiecare dat\u0103 am scrie mii de IP-uri pentru accesuri, ne-am epuiza. Vom inventa grupuri \u2013 un subset separat \u00een KV. \u00cel vom numi Alias (sau grupuri) \u0219i vom stoca acolo grupuri pe acela\u0219i principiu.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/67c6ab76cc25a3973739bdd44b69cef9.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nConect\u0103m: acum putem deschide SSH nu specific pe P2P, ci pe un \u00eentreg grup sau mai multe grupuri. De asemenea, exist\u0103 TTL \u2013 putem ad\u0103uga temporar \u00een grup \u0219i elimina din grup.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/a00b7db5087a96730c2725090f33e2b0.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Integrare<\/h3>\n<p>\nProblema noastr\u0103\u2014factorul uman \u0219i automatizarea. P\u00e2n\u0103 acum am rezolvat-o astfel.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/ce7a0c16543d0624453e6bdada3b8f03.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nLucr\u0103m cu Puppet \u0219i transfer\u0103m tot ce \u021bine de sistem (codul aplica\u021biilor). \u00cen puppetdb (un PostgreSQL obi\u0219nuit) se afl\u0103 o list\u0103 cu serviciile care sunt lansate acolo, iar acestea pot fi g\u0103site dup\u0103 tipul de resurs\u0103. Acolo se poate verifica \u0219i cine face cereri c\u0103tre ce. De asemenea, avem un sistem de pull request \u0219i merge request pentru aceasta.<\/p>\n<p>Am scris befw-sync \u2013 o solu\u021bie simpl\u0103 care ajut\u0103 la transferul datelor. La \u00eenceput, sincroniz\u0103m cookie-urile care se adreseaz\u0103 puppetdb. Acolo este configurat un API HTTP: cerem ce servicii avem, ce trebuie s\u0103 facem. Apoi facem o cerere \u00een Consul.<\/p>\n<p>Exist\u0103 integrare? Da: am scris reguli, am permis acceptarea Pull Request-ului. Este nevoie de un anumit port sau de a ad\u0103uga un host \u00eentr-un grup? Pull Request, revizuire \u2013 f\u0103r\u0103 \u201eCaut\u0103 200 de alte ACL-uri \u0219i \u00eencearc\u0103 s\u0103 faci ceva cu asta\u201d.<\/p>\n<h3>Optimizare<\/h3>\n<p>\nPing-ul localhost cu o interfa\u021b\u0103 de reguli goal\u0103 dureaz\u0103 0,075 ms.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/39263844f79c4795af05dc3ecb734394.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nVom ad\u0103uga \u00een aceast\u0103 interfa\u021b\u0103 10.000 de adrese iptables. Drept rezultat, ping-ul va cre\u0219te de 5 ori: iptables este complet liniar, procesarea fiec\u0103rei adrese dureaz\u0103 un anumit timp.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/e6c314155c3a34fdf61e40479ecb26c3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPentru firewall-ul \u00een care migr\u0103m mii de ACL-uri, avem multe reguli, ceea ce introduce \u00eent\u00e2rzieri. Pentru protocoalele de joc, acest lucru este nepl\u0103cut.<\/p>\n<p>Dar dac\u0103 plas\u0103m <strong>10.000 de adrese \u00een ipset<\/strong> ping-ul se va diminua chiar.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/2dd844d9324d057ba9ab098e501a1e51.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIdeea este c\u0103 \u201eO\u201d (complexitatea algoritmului) pentru ipset este \u00eentotdeauna egal\u0103 cu 1, indiferent c\u00e2te reguli sunt acolo. Totu\u0219i, exist\u0103 o limitare \u2013 nu pot fi mai mult de 65535 de reguli. P\u00e2n\u0103 acum ne descurc\u0103m cu asta: le putem combina, extinde, face dou\u0103 ipset-uri \u00eentr-unul.<\/p>\n<h3>Stocare<\/h3>\n<p>\nO continuare logic\u0103 a procesului de itera\u021bii este p\u0103strarea informa\u021biilor despre clien\u021bi pentru serviciul \u00een ipset.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/126069799d95594be8611237e935189d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAcum avem acela\u0219i SSH, \u0219i nu scriem imediat 100 de IP-uri, ci definim un nume de ipset cu care trebuie s\u0103 interac\u021bion\u0103m, iar urm\u0103toarea regul\u0103 <code>\u0218TERGE<\/code>. Se poate reformula \u00eentr-o regul\u0103 \u201eCine nu se afl\u0103 aici, acela DROP\u201d, dar a\u0219a este mai vizibil.<\/p>\n<p>Acum avem reguli \u0219i seturi. Principala sarcin\u0103 este s\u0103 cre\u0103m setul \u00eenainte de a scrie regula, deoarece altfel iptables nu va salva regula.<\/p>\n<h3>Schema general\u0103<\/h3>\n<p>\n\u00cen form\u0103 schematic\u0103, tot ce am spus arat\u0103 a\u0219a.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/4046e921cb5f466f8af5a6af1fcd151c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nCommit\u0103m \u00een Puppet, totul se trimite pe gazd\u0103, serviciile sunt aici, ipset-ul acolo, iar cine nu este acolo nu este l\u0103sat s\u0103 treac\u0103.<\/p>\n<h3>Permite\u021bi &amp; refuza\u021bi<\/h3>\n<p>\nPentru a salva rapid lumea sau pentru a deconecta rapid pe cineva, la \u00eenceputul tuturor lan\u021burilor am creat dou\u0103 ipset-uri: <code>rules_allow<\/code> \u0219i <code>rules_deny<\/code>. Cum func\u021bioneaz\u0103 asta?<\/p>\n<p>De exemplu, cineva genereaz\u0103 o \u00eenc\u0103rcare pe web cu boturi. Mai demult trebuia s\u0103 g\u0103sim IP-ul s\u0103u \u00een jurnale, s\u0103-l trimitem inginerilor de re\u021bea pentru a g\u0103si sursa traficului \u0219i a o bloca. Acum, lucrurile stau diferit.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/6e7f906919536cab2c283ba6ceb0472d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTrimitem \u00een Consul, a\u0219tept\u0103m 2,5 secunde, \u0219i este gata. Deoarece Consul distribuie rapid prin P2P, func\u021bioneaz\u0103 peste tot, \u00een orice col\u021b al lumii.<\/p>\n<p>Odat\u0103, am oprit complet WOT, gre\u0219ind cu firewall-ul. <code>rules_allow<\/code> \u2014 este asigurarea noastr\u0103 \u00eempotriva unor astfel de cazuri. Dac\u0103 am gre\u0219it undeva cu firewall-ul, iar ceva este blocat, putem \u00eentotdeauna s\u0103 trimitem un <code>0.0\/0<\/code>, pentru a ridica rapid totul. Apoi, vom repara manual toate problemele.<\/p>\n<h3>Seturi alternative<\/h3>\n<p>\nSe pot ad\u0103uga orice alte seturi \u00een spa\u021biu <code>$IPSETS$<\/code>.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/4e1e337b35da81cd7324b2adc106b085.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDe ce? Uneori, cineva are nevoie de ipset, de exemplu, pentru a emula oprirea unei p\u0103r\u021bi a clusterei. Fiecare poate aduce orice seturi, le poate numi \u0219i acestea vor fi preluate din Consul. Seturile pot participa at\u00e2t \u00een regulile iptables, c\u00e2t \u0219i func\u021biona ca o comand\u0103 <code>NOOP<\/code>: consisten\u021ba va fi men\u021binut\u0103 de demon.<\/p>\n<h3>Utilizatori<\/h3>\n<p>\nMai demult era a\u0219a: utilizatorul se conecta la re\u021bea \u0219i ob\u021binea parametrii prin domeniu. P\u00e2n\u0103 la apari\u021bia firewall-urilor de nou\u0103 genera\u021bie, Cisco nu putea \u00een\u021belege unde era utilizatorul \u0219i unde se afla IP-ul. De aceea, accesul era acordat doar prin hostname-ul ma\u0219inii.<\/p>\n<p>Ce am f\u0103cut noi? Ne-am intercalat \u00een momentul ob\u021binerii adresei. De obicei, este vorba despre dot1x, Wi-Fi sau VPN \u2014 totul trece prin RADIUS. Pentru fiecare utilizator cre\u0103m un grup dup\u0103 numele de utilizator \u0219i plas\u0103m \u00een el IP-ul cu TTL, care este egal cu dhcp.lease \u2014 imediat ce expire, regula dispare.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/04550475c6af49d094c18b1d5b89123a.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAcum putem deschide accesul la servicii, la fel ca \u0219i \u00een alte grupuri, pe baza numelui de utilizator. Am sc\u0103pat de problemele cu hostname-urile, c\u00e2nd acestea se schimb\u0103, \u0219i am ridicat povara de pe umerii inginerilor de re\u021bea, pentru c\u0103 nu mai au nevoie de Cisco. Acum inginerii \u00ee\u0219i noteaz\u0103 singuri accesurile pe serverele lor.<\/p>\n<h3>Izolare<\/h3>\n<p>\n\u00cen paralel, am \u00eenceput s\u0103 analiz\u0103m izola\u021bia. Managerii de servicii au realizat un inventar, iar noi am analizat toate re\u021belele noastre. Le vom \u00eemp\u0103r\u021bi \u00een grupuri similare, iar pe serverele necesare am ad\u0103ugat grupuri, de exemplu, \u00een deny. Acum, aceea\u0219i izola\u021bie de staging ajunge \u00een rules_deny la produc\u021bie, dar nu \u00een produc\u021bia \u00een sine.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/60fe7407013aa64b853791f8def627cc.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSchema func\u021bioneaz\u0103 rapid \u0219i simplu: elimin\u0103m toate ACL-urile de pe servere, desc\u0103rc\u0103m echipamentele, reduc\u00e2nd num\u0103rul VLAN-urilor izolate.<\/p>\n<h3>Controlul integrit\u0103\u021bii<\/h3>\n<p>\n\u00cen trecut, aveam un trigger special care anun\u021ba c\u00e2nd cineva modifica manual regula firewall-ului. Am scris un lintern imens pentru a verifica regulile firewall-ului, ceea ce era complicat. Acum, integritatea este controlat\u0103 de BEFW. Acesta se asigur\u0103 cu stricte\u021be c\u0103 regulile pe care le execut\u0103 nu sunt schimbate. Dac\u0103 cineva modific\u0103 regulile firewall-ului, le va restabili pe cele originale. \u201eAm ridicat rapid un proxy ca s\u0103 lucrez de acas\u0103\u201d \u2014 astfel de op\u021biuni nu mai exist\u0103.<\/p>\n<p>BEFW controleaz\u0103 ipset din servicii \u0219i lista din befw.conf, regulile serviciilor \u00een lan\u021bul BEFW. Dar nu supravegheaz\u0103 alte lan\u021buri \u0219i reguli \u0219i alte ipset.<\/p>\n<h3>Protec\u021bie \u00eempotriva erorilor<\/h3>\n<p>\nBEFW salveaz\u0103 \u00eentotdeauna ultima stare de succes direct \u00een structura binar\u0103 state.bin. Dac\u0103 ceva nu merge bine, se reapuc\u0103 \u00eentotdeauna de acest state.bin.<\/p>\n<p><img decoding=\"async\" alt=\"Consul + iptables = :3\" src=\"\/wp-content\/uploads\/2020\/02\/724f3537c06a7135fa6397f9ddd5630f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAceasta este o asigurare \u00eempotriva func\u021bion\u0103rii instabile a Consul, c\u00e2nd nu trimite date sau cineva gre\u0219e\u0219te \u0219i folose\u0219te reguli care nu pot fi aplicate. Pentru a nu r\u0103m\u00e2ne f\u0103r\u0103 firewall, BEFW se va \u00eentoarce la ultima stare, dac\u0103 se va \u00eent\u00e2mpla o eroare \u00een orice moment.<\/p>\n<p>\u00cen situa\u021bii critice, aceasta garanteaz\u0103 c\u0103 vom r\u0103m\u00e2ne cu un firewall func\u021bional. Deschidem toate re\u021belele gri \u00een speran\u021ba c\u0103 admin-ul va veni \u0219i le va remedia. C\u00e2ndva voi muta asta \u00een configura\u021bii, dar \u00een prezent avem doar trei re\u021bele gri: 10\/8, 172\/12 \u0219i 192.168\/16. \u00cen cadrul Consul-ului nostru, aceasta este o caracteristic\u0103 important\u0103 care ajut\u0103 la dezvoltarea ulterioar\u0103.<\/p>\n<p><em>Demo: \u00een timpul prezent\u0103rii, Ivan demonstreaz\u0103 modul de func\u021bionare al BEFW \u00een modul demo. Demonstra\u021bia este mai u\u0219or de vizionat pe <noindex><a rel=\"nofollow\" href=\"https:\/\/youtu.be\/4zP67uYnsR4?t=1663\">video<\/a><\/noindex>. Codul surs\u0103 pentru demo este disponibil <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/wgnet\/befw\/tree\/master\/demo\">de pe GitHub<\/a><\/noindex>.<\/em><\/p>\n<h2>Capcane<\/h2>\n<p>\nVoi vorbi despre bug-urile cu care ne-am confruntat.<\/p>\n<p><strong>ipset add set 0.0.0.0\/0. <\/strong>Ce se va \u00eent\u00e2mpla dac\u0103 adaug \u00een ipset 0.0.0.0\/0? Vor fi ad\u0103ugate toate IP-urile? Se va deschide accesul la internet? <\/p>\n<p>Nu, vom ob\u021bine un bug care ne-a costat dou\u0103 ore de nefunc\u021bionare. \u00cen plus, bug-ul nu func\u021bioneaz\u0103 din 2016, se afl\u0103 \u00een RedHat Bugzilla sub num\u0103rul #1297092, iar noi l-am g\u0103sit \u00eent\u00e2mpl\u0103tor \u2014 din raportul dezvoltatorului.<\/p>\n<p>Acum \u00een BEFW exist\u0103 o regul\u0103 strict\u0103, c\u0103 <code>0.0.0.0\/0<\/code> se transform\u0103 \u00een dou\u0103 adrese: <code>0.0.0.0\/1<\/code> \u0219i <code>128.0.0.0\/1<\/code>.<\/p>\n<p><strong>ipset restore set &lt; file. <\/strong>Ce face ipset c\u00e2nd \u00eei spui <code>restore<\/code>? \u0412\u044b \u0434\u0443\u043c\u0430\u0435\u0442\u0435, \u043e\u043d \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u0442 \u0442\u0430\u043a\u0436\u0435, \u043a\u0430\u043a iptables? \u0412\u043e\u0441\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442 \u0434\u0430\u043d\u043d\u044b\u0435?<\/p>\n<p>Nimic de genul \u2014 face un merge, iar vechile adrese nu dispar, accesul nu este \u00eenchis.<\/p>\n<p>Am g\u0103sit bug-ul c\u00e2nd testam izola\u021bia. Acum exist\u0103 un sistem destul de complex \u2014 \u00een loc de <code>restore<\/code> se efectueaz\u0103 <code>create temp<\/code>, apoi <code>restore flush temp<\/code> \u0219i <code>restore temp<\/code>. La final swap: pentru atomicitate, pentru c\u0103 dac\u0103 se realizeaz\u0103 mai \u00eent\u00e2i <code>flush<\/code> \u0218i \u00een acel moment, dac\u0103 vine un anumit pachet, acesta va fi respins \u0219i ceva va merge prost. A\u0219adar, exist\u0103 pu\u021bin\u0103 magie neagr\u0103 acolo.<\/p>\n<p><strong>consul kv get -datacenter=other. <\/strong>A\u0219a cum am spus deja, ne g\u00e2ndim c\u0103 solicit\u0103m anumite date, dar vom ob\u021bine fie date, fie o eroare. Putem face asta prin Consul local, dar chiar \u0219i \u00een acest caz va bloca ambele.<\/p>\n<p>Clientul local Consul este o wrapper peste API-ul HTTP. Dar pur \u0219i simplu se blocheaz\u0103 \u0219i nu r\u0103spunde nici la Ctrl+C, nici la Ctrl+Z, nimic, doar la <code>kill -9<\/code> \u00een consola al\u0103turat\u0103. Ne-am confruntat cu asta c\u00e2nd construiam un cluster mare. Dar nu avem \u00een prezent o solu\u021bie, ne preg\u0103tim s\u0103 corect\u0103m aceast\u0103 eroare \u00een Consul.<\/p>\n<p><strong>Leader-ul Consul nu r\u0103spunde. <\/strong>Nu avem un master care s\u0103 r\u0103spund\u0103 \u00een data center, ne g\u00e2ndim: \u201eProbabil, algoritmul de realegere va func\u021biona acum?\u201d <\/p>\n<p>Nu, nu va func\u021biona, iar monitorizarea nu va ar\u0103ta nimic: Consul va spune c\u0103 indicele de angajament exist\u0103, liderul a fost g\u0103sit, totul este bine.<\/p>\n<p>Cum ne lupt\u0103m cu asta? <code>service consul restart<\/code> \u00een cron de fiecare or\u0103. Dac\u0103 ave\u021bi 50 de servere - nu este nicio problem\u0103. C\u00e2nd vor fi 16 000, ve\u021bi \u00een\u021belege cum func\u021bioneaz\u0103 asta.<\/p>\n<h2>Concluzie<\/h2>\n<p>\n\u00cen cele din urm\u0103, am ob\u021binut urm\u0103toarele avantaje:<\/p>\n<ul>\n<li>100% acoperire a tuturor ma\u0219inilor Linux.<\/li>\n<li>Viteza.<\/li>\n<li>Automatizare.<\/li>\n<li>Am eliberat hardware-ul \u0219i inginerii de re\u021bea de munc\u0103 repetitiv\u0103.<\/li>\n<li>Au ap\u0103rut oportunit\u0103\u021bi de integrare care sunt practic nelimitate: fie cu Kubernetes, fie cu Ansible, fie cu Python.<\/li>\n<\/ul>\n<p>\n<strong>Dezavantaje<\/strong>: Consul, cu care trebuie s\u0103 tr\u0103im acum, \u0219i un pre\u021b foarte ridicat al erorii. Ca exemplu, o dat\u0103 am modificat ceva \u00een listele de re\u021bea la ora 18:00 (v\u00e2rf \u00een Rusia). Exact atunci construia izolarea pe BEFW. Am gre\u0219it undeva, cred c\u0103 am indicat o masc\u0103 gre\u0219it\u0103, dar totul s-a pr\u0103bu\u0219it \u00een dou\u0103 secunde. Monitorizarea se activeaz\u0103, vine tehnicianul de suport: \u201eTotul este c\u0103zut!\u201d \u0218eful departamentului a \u00eemb\u0103tr\u00e2nit c\u00e2nd a explicat afacerii de ce s-a \u00eent\u00e2mplat asta.<\/p>\n<p>Pre\u021bul erorii este at\u00e2t de ridicat \u00eenc\u00e2t am conceput o procedur\u0103 complex\u0103 de prevenire. Dac\u0103 ve\u021bi implementa asta \u00eentr-o produc\u021bie mare, nu trebuie s\u0103 oferi\u021bi token-ul master peste Consul tuturor. Se va termina prost.<\/p>\n<p><strong>Costul. <\/strong>Am scris cod timp de 400 de ore singur. La suport, echipa mea de 4 persoane cheltuie 10 ore pe lun\u0103 pentru to\u021bi. Comparativ cu pre\u021bul oric\u0103rui firewall de nou\u0103 genera\u021bie, este gratuit.<\/p>\n<p><strong>Planuri. <\/strong>Planul pe termen lung este c\u0103utarea unui transport alternativ \u00een locul sau pe l\u00e2ng\u0103 Consul. Poate c\u0103 va fi Kafka sau ceva similar. Dar \u00een urm\u0103torii ani vom tr\u0103i pe Consul.<\/p>\n<p>Planurile imediate: integrarea cu Fail2ban, cu monitorizare, cu nftables, posibil, cu alte distribu\u021bii, metrici, monitorizare extins\u0103, optimizare. Suportul pentru Kubernetes este, de asemenea, \u00een planuri, deoarece deocamdat\u0103 avem c\u00e2teva clustere \u0219i dorin\u021b\u0103.<\/p>\n<p>\u00cenc\u0103 din planuri:<\/p>\n<ul>\n<li>c\u0103utarea anomaliilor \u00een trafic;<\/li>\n<li>gestionarea hart\u0103i re\u021belei;<\/li>\n<li>suport pentru Kubernetes;<\/li>\n<li>compilarea pachetelor pentru toate sistemele;<\/li>\n<li>Interfa\u021ba Web.<\/li>\n<\/ul>\n<p>\nLucr\u0103m constant la extinderea configura\u021biei, cre\u0219terea metricilor \u0219i optimizarea.<\/p>\n<p><i>Al\u0103tura\u021bi-v\u0103 proiectului. Proiectul a ie\u0219it grozav, dar, din p\u0103cate, este deocamdat\u0103 un proiect al unei singure persoane. Veniti la <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/wgnet\/befw\">GitHub <\/a><\/noindex>\u0219i \u00eencerca\u021bi s\u0103 face\u021bi ceva: s\u0103 face\u021bi un commit, s\u0103 testa\u021bi, s\u0103 propune\u021bi ceva, s\u0103 oferi\u021bi evaluarea dumneavoastr\u0103.<\/i><\/p>\n<blockquote><p>\u00centre timp, ne preg\u0103tim pentru <noindex><a rel=\"nofollow\" href=\"https:\/\/www.highload.ru\/spb\/2020\">Saint HighLoad++<\/a><\/noindex>, care va avea loc pe 6 \u0219i 7 aprilie la Sankt Petersburg, \u0219i invit\u0103m dezvoltatorii de sisteme cu \u00eenc\u0103rc\u0103turi mari <noindex><a rel=\"nofollow\" href=\"https:\/\/conf.ontico.ru\/lectures\/propose?conference=hl2020-spb\">s\u0103 depun\u0103 o cerere pentru prezentare<\/a><\/noindex>. Vorbitorii cu experien\u021b\u0103 \u0219tiu deja ce s\u0103 fac\u0103, iar \u00eencep\u0103torilor le recomand\u0103m m\u0103car <noindex><a rel=\"nofollow\" href=\"https:\/\/www.highload.ru\/spb\/2020\/speakers\">testeaz\u0103<\/a><\/noindex>. Participarea la conferin\u021b\u0103 ca vorbitor are o serie de avantaje. Despre ce avantaje, pute\u021bi citi, de exemplu, la final <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/oleg-bunin\/blog\/486142\/\">acestui articol<\/a><\/noindex>.<\/p><\/blockquote>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/oleg-bunin\/blog\/486842\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 2010 \u0433\u043e\u0434\u0443 \u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Wargaming \u0431\u044b\u043b\u043e 50 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0438 \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0441\u0435\u0442\u0435\u0432\u0430\u044f \u043c\u043e\u0434\u0435\u043b\u044c: \u0431\u044d\u043a\u0435\u043d\u0434, \u0444\u0440\u043e\u043d\u0442\u0435\u043d\u0434 \u0438 \u0444\u0430\u0439\u0440\u0432\u043e\u043b. \u041a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0440\u043e\u0441\u043b\u043e, \u043c\u043e\u0434\u0435\u043b\u044c \u0443\u0441\u043b\u043e\u0436\u043d\u044f\u043b\u0430\u0441\u044c: \u0441\u0442\u0435\u0439\u0434\u0436\u0438\u043d\u0433\u0438, \u0438\u0437\u043e\u043b\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0435 VLAN \u0441 ACL, \u043f\u043e\u0442\u043e\u043c VPN \u0441 VRF, VLAN c ACL \u043d\u0430 L2, VRF \u0441 ACL \u043d\u0430 L3. \u0417\u0430\u043a\u0440\u0443\u0436\u0438\u043b\u0430\u0441\u044c \u0433\u043e\u043b\u043e\u0432\u0430? \u0414\u0430\u043b\u044c\u0448\u0435 \u0431\u0443\u0434\u0435\u0442 \u0432\u0435\u0441\u0435\u043b\u0435\u0435. \u041a\u043e\u0433\u0434\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0441\u0442\u0430\u043b\u043e 16 000 \u0440\u0430\u0431\u043e\u0442\u0430\u0442\u044c \u0431\u0435\u0437 \u0441\u043b\u0435\u0437 \u0441 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":41052,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-41051","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 2010 \u0433\u043e\u0434\u0443 \u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Wargaming \u0431\u044b\u043b\u043e 50 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0438 \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0441\u0435\u0442\u0435\u0432\u0430\u044f \u043c\u043e\u0434\u0435\u043b\u044c: \u0431\u044d\u043a\u0435\u043d\u0434, \u0444\u0440\u043e\u043d\u0442\u0435\u043d\u0434 \u0438 \u0444\u0430\u0439\u0440\u0432\u043e\u043b.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/consul-iptables-3\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47Consul + iptables = :3 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 2010 \u0433\u043e\u0434\u0443 \u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Wargaming \u0431\u044b\u043b\u043e 50 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0438 \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0441\u0435\u0442\u0435\u0432\u0430\u044f \u043c\u043e\u0434\u0435\u043b\u044c: \u0431\u044d\u043a\u0435\u043d\u0434, \u0444\u0440\u043e\u043d\u0442\u0435\u043d\u0434 \u0438 \u0444\u0430\u0439\u0440\u0432\u043e\u043b.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/consul-iptables-3\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-05T18:28:58+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-05T18:28:58+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Consul + iptables = :3 | ProHoster","description":"\u00cen 2010, compania Wargaming avea 50 de servere \u0219i un model de re\u021bea simplu: backend, frontend \u0219i firewall.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/consul-iptables-3","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Consul + iptables = :3 | ProHoster","og:description":"\u0412 2010 \u0433\u043e\u0434\u0443 \u0443 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Wargaming \u0431\u044b\u043b\u043e 50 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0438 \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0441\u0435\u0442\u0435\u0432\u0430\u044f \u043c\u043e\u0434\u0435\u043b\u044c: \u0431\u044d\u043a\u0435\u043d\u0434, \u0444\u0440\u043e\u043d\u0442\u0435\u043d\u0434 \u0438 \u0444\u0430\u0439\u0440\u0432\u043e\u043b.","og:url":"https:\/\/prohoster.info\/ro\/blog\/consul-iptables-3","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-05T18:28:58+00:00","article:modified_time":"2020-02-05T18:28:58+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"41051","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:24:42","updated":"2022-10-01 10:00:06","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/41051","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=41051"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/41051\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/41052"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=41051"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=41051"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=41051"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}