{"id":55467,"date":"2020-01-21T00:00:00","date_gmt":"2020-01-20T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/postgres-vtornik-5-postgresql-i-kubernetes-ci-cd-avtomatizatsiya-testirovaniya"},"modified":"2020-02-18T14:03:35","modified_gmt":"2020-02-18T11:03:35","slug":"postgres-vtornik-5-postgresql-i-kubernetes-ci-cd-avtomatizatsiya-testirovaniya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/postgres-vtornik-5-postgresql-i-kubernetes-ci-cd-avtomatizatsiya-testirovaniya","title":{"rendered":"Postgres-e mart\u00eb \u21165: \"PostgreSQL dhe Kubernetes. CI\/CD. Automatizimi i testimit\"","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Postgres-e
\n
\nN\u00eb fund t\u00eb vitit t\u00eb kaluar, u mbajt nj\u00eb transmetim tjet\u00ebr t\u00eb drejtp\u00ebrdrejt\u00eb t\u00eb komunitetit rus t\u00eb PostgreSQL. #RuPostgres<\/a><\/noindex>, n\u00eb kuad\u00ebr t\u00eb s\u00eb cil\u00ebs bashk\u00ebthemeluesi i tij, Nikolai Samokhvalov, bisedoi me drejtorin teknik t\u00eb \"Flant\u00ebs\" Dmitry Stolyarov mbi k\u00ebt\u00eb DBMS n\u00eb kontekstin e Kubernetes.<\/p>\n

Ne publikojm\u00eb transkriptin e pjes\u00ebs kryesore t\u00eb k\u00ebsaj diseksioni, dhe n\u00eb kanalin YouTube t\u00eb komunitetit<\/a><\/noindex> \u00ebsht\u00eb publikuar regjistrimi i plot\u00eb i videos:<\/a><\/noindex><\/p>\n

\r\n \"Luaj\r\n
<\/div>\r\n <\/div><\/center><\/p>\n

Baza t\u00eb dh\u00ebnash dhe Kubernetes<\/h2>\n

\nNS<\/b>: Nuk e do t\u00eb flasim sot p\u00ebr VACUUM dhe CHECKPOINT’t. Duam t\u00eb flasim p\u00ebr Kubernetes. E di q\u00eb ke shum\u00eb vite p\u00ebrvoj\u00eb. Kam par\u00eb videot e tua dhe disa madje i kam ripar\u00eb\u2026 Le t\u00eb fillojm\u00eb direkt: p\u00ebrse Postgres apo MySQL n\u00eb K8s?<\/i><\/p>\n

DS<\/b>: Nuk ka nj\u00eb p\u00ebrgjigje t\u00eb qart\u00eb p\u00ebr k\u00ebt\u00eb pyetje dhe nuk mund t\u00eb ket\u00eb. Por n\u00eb p\u00ebrgjith\u00ebsi, \u00ebsht\u00eb thjesht\u00ebsia dhe leht\u00ebsia... potenciale. T\u00eb gjith\u00eb duan sh\u00ebrbime t\u00eb menaxhuara.<\/p>\n

NS<\/b>: Si RDS<\/a><\/noindex>, vet\u00ebm n\u00eb serverin tuaj?<\/i><\/p>\n

DS<\/b>: Po: q\u00eb si RDS, vet\u00ebm kudo.<\/p>\n

NS<\/b>: \"Kudo\" - kjo \u00ebsht\u00eb nj\u00eb v\u00ebrejtje e mir\u00eb. N\u00eb kompanit\u00eb e m\u00ebdha, gjith\u00e7ka \u00ebsht\u00eb e shp\u00ebrndar\u00eb n\u00eb vende t\u00eb ndryshme. Pse at\u00ebher\u00eb, n\u00ebse kjo \u00ebsht\u00eb nj\u00eb kompani e madhe, t\u00eb mos marrim nj\u00eb zgjidhje t\u00eb gatshme? P\u00ebr shembull, Nutanix ka zhvillimet e veta, kompanit\u00eb e tjera (VMware...) kan\u00eb \"RDS, vet\u00ebm n\u00eb serverin tuaj\".<\/i><\/p>\n

DS<\/b>: Por ne po flasim p\u00ebr nj\u00eb zbatim t\u00eb ve\u00e7ant\u00eb, i cili do t\u00eb funksionoj\u00eb vet\u00ebm n\u00eb kushte t\u00eb caktuara. N\u00ebse flasim p\u00ebr Kubernetes, k\u00ebtu ka nj\u00eb shum\u00ebllojshm\u00ebri t\u00eb madhe infrastrukturore (e cila mund t\u00eb jet\u00eb n\u00eb K8s). N\u00eb thelb, kjo \u00ebsht\u00eb nj\u00eb standard p\u00ebr API n\u00eb re...<\/p>\n

NS<\/b>: Dhe gjithashtu falas!<\/i><\/p>\n

DS<\/b>: Kjo nuk \u00ebsht\u00eb aq e r\u00ebnd\u00ebsishme. Falas \u00ebsht\u00eb e r\u00ebnd\u00ebsishme p\u00ebr nj\u00eb segment t\u00eb vog\u00ebl t\u00eb tregut. E r\u00ebnd\u00ebsishmja \u00ebsht\u00eb tjet\u00ebr... Sigurisht, ti e kujton fjalimin \"Baza t\u00eb dh\u00ebnash dhe Kubernetes<\/a><\/noindex>\u00bb?<\/p>\n

NS<\/b>: Po.<\/i><\/p>\n

DS<\/b>: E kuptova se u perceptua shum\u00eb ndryshe. Disa njer\u00ebz menduan se un\u00eb thash: \u201cO njer\u00ebz, t\u00eb shkojm\u00eb me t\u00eb gjitha DB-t\u00eb n\u00eb Kubernetes!\u201d, \u2014 nd\u00ebrsa t\u00eb tjer\u00ebt menduan se k\u00ebto jan\u00eb gjithsej bi\u00e7ikleta t\u00eb frikshme. Por un\u00eb doja t\u00eb flisja p\u00ebr di\u00e7ka ndryshe: \u201cShikoni, \u00e7far\u00eb po ndodh, cilat jan\u00eb probleme dhe si mund t\u2019i zgjidhim. A \u00ebsht\u00eb p\u00ebr t\u00eb shkuar me bazat n\u00eb Kubernetes? N\u00eb Production? Vet\u00ebm n\u00ebse ju p\u00eblqen\u2026 t\u00eb b\u00ebni disa gj\u00ebra t\u00eb caktuara. Por p\u00ebr dev’in \u2014 mund t\u00eb them se rekomandoj. P\u00ebr dev’in \u00ebsht\u00eb shum\u00eb e r\u00ebnd\u00ebsishme dinamika e krijimit\/zhvendosjes s\u00eb mjediseve.<\/p>\n

NS: N\u00ebn dev’in e ke n\u00eb mendje t\u00eb gjitha mjediset q\u00eb nuk jan\u00eb prodhues? Staging, QA...<\/i><\/p>\n

DS<\/b>: N\u00ebse flasim p\u00ebr perf-stenda, ndoshta jo, sepse k\u00ebrkesat atje jan\u00eb specifike. N\u00ebse flasim p\u00ebr raste t\u00eb ve\u00e7anta, ku n\u00eb staging nevojitet nj\u00eb DB shum\u00eb e madhe, gjithashtu ndoshta jo... N\u00ebse kjo \u00ebsht\u00eb nj\u00eb mjedis statik, q\u00eb jeton p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb, \u00e7far\u00eb \u00ebsht\u00eb dobi q\u00eb baza \u00ebsht\u00eb e vendosur n\u00eb K8s?<\/p>\n

NS<\/b>: Asnj\u00eb. Por ku e shohim mjedisin statik? Mjedisi statik \u00ebsht\u00eb i vjetruar q\u00eb nes\u00ebr.<\/i><\/p>\n

DS<\/b>: Staging mund t\u00eb jet\u00eb statik. Ne kemi klient\u00eb...<\/p>\n

NS<\/b>: Po, edhe un\u00eb kam. \u00cbsht\u00eb nj\u00eb problem i madh n\u00ebse ke nj\u00eb baz\u00eb prej 10 TB, nd\u00ebrsa staging \u00ebsht\u00eb 200 GB...<\/i><\/p>\n

DS<\/b>: Kam nj\u00eb rast shum\u00eb t\u00eb mir\u00eb! N\u00eb staging ndodhet baza e prodhimit, n\u00eb t\u00eb cil\u00ebn b\u00ebhen ndryshime. Dhe ka nj\u00eb buton: \u201cnxjerr n\u00eb production\u201d. K\u00ebto ndryshime \u2014 delta \u2014 p\u00ebrfshihen (duket, thjesht sinkronizohen p\u00ebrmes API’sh) n\u00eb production. Ky \u00ebsht\u00eb nj\u00eb variant shum\u00eb ekzotik.<\/p>\n

NS<\/b>: Kam pare startup-e n\u00eb Lugin\u00ebn e Silikonit q\u00eb jan\u00eb n\u00eb RDS’e ose madje n\u00eb Heroku akoma \u2014 k\u00ebto jan\u00eb histori 2-3 vje\u00e7are \u2014 dhe ata shkarkojn\u00eb dump-in n\u00eb laptop. Sepse baza p\u00ebr momentin \u00ebsht\u00eb vet\u00ebm 80 GB, dhe n\u00eb laptop ka hap. Pastaj ata blejn\u00eb disqe p\u00ebr \u00e7do nj\u00ebri, p\u00ebr t\u00eb pasur nga 3 baza, p\u00ebr t\u00eb par\u00eb zhvillime t\u00eb ndryshme. K\u00ebshtu ndodh gjithashtu. Kam pare gjithashtu q\u00eb nuk kan\u00eb frik\u00eb t\u00eb kopjojn\u00eb prodhimin n\u00eb staging \u2014 shum\u00eb varietetet e kompanis\u00eb jan\u00eb t\u00eb ndryshme. Por kam pare gjithashtu q\u00eb kan\u00eb frik\u00eb, dhe shpesh nuk kan\u00eb koh\u00eb dhe duar. Por p\u00ebrpara se t\u00eb kalojm\u00eb n\u00eb k\u00ebt\u00eb tem\u00eb, doja t\u00eb d\u00ebgjoj p\u00ebr Kubernetes. Po kuptoj sakt\u00ebsisht, se n\u00eb prodhim nuk ka askush?<\/i><\/p>\n

DS<\/b>: Ne kemi disa baza t\u00eb vogla n\u00eb prodhim. B\u00ebhet fjal\u00eb p\u00ebr volume t\u00eb dhjet\u00ebra gigabajt dhe sh\u00ebrbime q\u00eb nuk jan\u00eb kritike, p\u00ebr t\u00eb cilat ishte len\u00eb t\u00eb b\u00ebhen replika (as nuk ka ndonj\u00eb nevoj\u00eb). Dhe n\u00ebse n\u00ebn Kubernetes ka nj\u00eb ruajtje t\u00eb mir\u00eb. Kjo baz\u00eb punoi n\u00eb nj\u00eb makin\u00eb virtuale \u2014 n\u00eb kushtet e VMware, mbi SAN. E vendos\u00ebm n\u00eb PV<\/a><\/noindex> dhe tani mund ta transferojm\u00eb nga makina n\u00eb machin\u00eb.<\/p>\n

NS<\/b>: Baza t\u00eb tilla, deri n\u00eb 100 GB, n\u00eb disqe t\u00eb mira dhe me nj\u00eb rrjet t\u00eb mir\u00eb mund t\u00eb operohen p\u00ebr disa minuta, e drejt\u00eb? Shpejt\u00ebsia prej 1 GB n\u00eb sekond\u00eb \u2014 kjo nuk \u00ebsht\u00eb m\u00eb ekzotike.<\/i><\/p>\n

DS<\/b>: Po, p\u00ebr nj\u00eb operacion linear kjo s\u2019\u00ebsht\u00eb problem.<\/p>\n

NS<\/b>: Okej, p\u00ebr prodhimin duhet t\u00eb mendojm\u00eb vet\u00ebm. Dhe n\u00ebse e shqyrtojm\u00eb Kubernetes p\u00ebr mjediset jo-prodhuese \u2014 si t\u00eb veprojm\u00eb? Shoh q\u00eb n\u00eb Zalando b\u00ebjn\u00eb operator<\/a><\/noindex>, n\u00eb Crunchy punojn\u00eb<\/a><\/noindex>, ka disa mund\u00ebsi t\u00eb tjera. Dhe ka OnGres<\/a><\/noindex> \u2014 ky \u00ebsht\u00eb shoku yn\u00eb i mir\u00eb Alvaro nga Spanja: ata b\u00ebjn\u00eb n\u00eb thelb jo vet\u00ebm operator<\/a><\/noindex>, por nj\u00eb distribucion t\u00eb t\u00ebr\u00eb (StackGres<\/a><\/noindex>), n\u00eb t\u00eb cilin p\u00ebrve\u00e7 Postgres’it vendos\u00ebm gjithashtu nj\u00eb backup, proxy Envoy\u2026<\/i><\/p>\n

DS<\/b>: Pse Envoy? P\u00ebr balancimin e trafikut t\u00eb Postgres-it n\u00eb ve\u00e7anti?<\/p>\n

NS<\/b>: Po. Pra ata e shohin k\u00ebshtu: n\u00ebse e marrim distribucionin e Linux-it dhe b\u00ebrtham\u00ebn, PostgreSQL-i i zakonsh\u00ebm \u00ebsht\u00eb b\u00ebrthama, dhe ata duan t\u00eb krijojn\u00eb nj\u00eb distribucion q\u00eb do t\u00eb jet\u00eb miq\u00ebsor me re dhe do t\u00eb funksionoj\u00eb n\u00eb Kubernetes. Ata po bashkojn\u00eb komponent\u00ebt (backup-et etj.) dhe po e rregullojn\u00eb q\u00eb t\u00eb funksionojn\u00eb mir\u00eb.<\/i><\/p>\n

DS<\/b>: Shum\u00eb e bukur! N\u00eb thelb ky \u00ebsht\u00eb nj\u00eb soft q\u00eb b\u00ebn q\u00eb PostgreSQL-i t\u00eb menaxhohet.<\/p>\n

NS<\/b>: Distribucionet e Linux-it kan\u00eb probleme t\u00eb p\u00ebrhershme: si t\u00eb krijojn\u00eb shofer\u00eb q\u00eb t\u00eb mb\u00ebshtesin gjith\u00eb harduerin. Dhe ata kan\u00eb iden\u00eb q\u00eb do t\u00eb punojn\u00eb n\u00eb Kubernetes. E di q\u00eb n\u00eb operatorin e Zalando-s s\u00eb fundmi kemi par\u00eb lidhjen me AWS dhe kjo nuk \u00ebsht\u00eb shum\u00eb mir\u00eb. Nuk duhet t\u00eb ket\u00eb lidhje me nj\u00eb infrastruktur\u00eb specifike - \u00e7far\u00eb ka kuptim at\u00ebher\u00eb?<\/i><\/p>\n

DS<\/b>: Nuk di n\u00eb cil\u00ebn situat\u00eb konkretisht Zalando \u00ebsht\u00eb lidhur, por n\u00eb Kubernetes tani ruajtja \u00ebsht\u00eb b\u00ebr\u00eb n\u00eb m\u00ebnyr\u00eb q\u00eb nuk mund t\u00eb merret nj\u00eb backup disku n\u00eb m\u00ebnyr\u00eb generike. S\u00eb fundmi n\u00eb standard - n\u00eb versionin m\u00eb t\u00eb ri specifikimeve CSI<\/a><\/noindex> \u2013 \u00ebsht\u00eb b\u00ebr\u00eb e mundur t\u00eb merret snapshot, por ku \u00ebsht\u00eb implementuar? Sinqerisht, ende \u00ebsht\u00eb kaq e papjekur\u2026 Ne po e provojm\u00eb CSI mbi AWS, GCE, Azure, vSphere, por sapo fillon ta p\u00ebrdor\u00ebsh, duket se ende nuk \u00ebsht\u00eb gati.<\/p>\n

NS<\/b>: Prandaj ndonj\u00ebher\u00eb duhet t\u00eb lidhemi me infrastruktur\u00ebn. Mendoj se kjo \u00ebsht\u00eb ende nj\u00eb faz\u00eb e hershme - problemet e rritjes. Pyetje: \u00e7far\u00eb do t\u00eb rekomandoje p\u00ebr fillestar\u00ebt q\u00eb duan t\u00eb provojn\u00eb PgSQL n\u00eb K8s? Cilin operator, ndoshta?<\/i><\/p>\n

DS<\/b>: Problemi \u00ebsht\u00eb se Postgres p\u00ebr ne \u00ebsht\u00eb vet\u00ebm 3%. Ne kemi nj\u00eb list\u00eb t\u00eb madhe t\u00eb softver\u00ebve t\u00eb ndrysh\u00ebm n\u00eb Kubernetes, nuk do ta rendis t\u00eb gjith\u00eb. P\u00ebr shembull, Elasticsearch. Ka shum\u00eb operator\u00eb: disa zhvillohen aktivisht, t\u00eb tjer\u00eb jo. Ne kemi p\u00ebrgatitur k\u00ebrkesat se \u00e7far\u00eb duhet t\u00eb ket\u00eb nj\u00eb operator p\u00ebr ta p\u00ebrmir\u00ebsuar imazhin e tij. Nj\u00eb operator p\u00ebr Kubernetes \u2014 jo n\u00eb \"operatorin, q\u00eb b\u00ebn di\u00e7ka n\u00eb kushte t\u00eb Amazon'it\"... N\u00eb fakt, ne p\u00ebrdorim mjaft masivisht (= pothuajse t\u00eb gjith\u00eb klient\u00ebt) nj\u00eb operator t\u00eb vet\u00ebm \u2014 p\u00ebr Redis<\/a><\/noindex> (shpejt do t\u00eb publikojm\u00eb nj\u00eb artikull dhe p\u00ebr t\u00eb)<\/i>.<\/p>\n

NS<\/b>: Po p\u00ebr MySQL nuk ka edhe? E di q\u00eb Percona\u2026 pasi tani ata jan\u00eb duke u marr\u00eb me MySQL, MongoDB dhe Postgres, ata duhet t\u00eb krijojn\u00eb di\u00e7ka universale: p\u00ebr t\u00eb gjitha bazat, p\u00ebr t\u00eb gjith\u00eb ofruesit e sh\u00ebrbimeve cloud.<\/i><\/p>\n

DS<\/b>: Nuk e kemi par\u00eb operator\u00ebt p\u00ebr MySQL. P\u00ebr ne, kjo nuk \u00ebsht\u00eb fokus kryesor tani. MySQL funktionon mir\u00eb n\u00eb m\u00ebnyr\u00eb independente. P\u00ebrse operator, n\u00ebse mund t\u00eb fillosh thjesht DB-n\u00eb... Mund t\u00eb fillosh nj\u00eb konteiner Docker me Postgres, dhe mund ta nis\u00ebsh at\u00eb n\u00eb nj\u00eb m\u00ebnyr\u00eb m\u00eb t\u00eb thjesht\u00eb.<\/p>\n

NS<\/b>: Kjo ishte nj\u00eb pyetje gjithashtu. N\u00eb t\u00eb v\u00ebrtet\u00eb pa operator?<\/i><\/p>\n

DS<\/b>: Po, 100% ne e kemi PostgreSQL-in t\u00eb nisur pa operator. P\u00ebr momentin k\u00ebshtu \u00ebsht\u00eb. Ne e p\u00ebrdorim aktivisht operatorin p\u00ebr Prometheus, p\u00ebr Redis. Kemi plane p\u00ebr t\u00eb gjetur nj\u00eb operator p\u00ebr Elasticsearch \u2014 ai \u00ebsht\u00eb m\u00eb urgjenti, sepse e duam p\u00ebr ta vendosur n\u00eb 100% t\u00eb rasteve n\u00eb Kubernetes. Si\u00e7 duam t\u00eb arrijm\u00eb q\u00eb MongoDB gjithashtu t\u00eb vendoset gjithmon\u00eb n\u00eb Kubernetes. K\u00ebtu dalin disa d\u00ebshira \u2014 ka nj\u00eb ndjenj\u00eb se n\u00eb k\u00ebto raste mund t\u00eb b\u00ebjm\u00eb di\u00e7ka. P\u00ebr Postgres nuk kemi par\u00eb. Sigurisht, e dim\u00eb p\u00ebr ekzistenc\u00ebn e varianteve t\u00eb ndryshme, por n\u00eb fakt ne kemi q\u00ebndruar me standalone.<\/p>\n

DB p\u00ebr testim n\u00eb Kubernetes<\/h2>\n

\nNS<\/b>: Le t\u00eb kalojm\u00eb n\u00eb tem\u00ebn e testimit. Si t\u00eb zgjerohen ndryshimet n\u00eb baz\u00eb \u2014 nga k\u00ebndv\u00ebshtrimi DevOps. Ka mikrosh\u00ebrbime, shum\u00eb baza, gjithmon\u00eb ndodhin ndryshime diku. Si t\u00eb sigurohet nj\u00eb CI\/CD i mir\u00eb, q\u00eb nga pozita e DB-s\u00eb gjith\u00e7ka t\u00eb jet\u00eb n\u00eb rregull. Cili \u00ebsht\u00eb qasja jote?<\/i><\/p>\n

DS<\/b>: Nuk mund t\u00eb ket\u00eb nj\u00eb p\u00ebrgjigje t\u00eb vetme. Ka disa parametra. I pari \u2014 \u00ebsht\u00eb madh\u00ebsia e baz\u00ebs q\u00eb duam t\u00eb shp\u00ebrndajm\u00eb. Ti e theve se n\u00eb kompani lidhen ndryshe me faktin q\u00eb kopja e baz\u00ebs prodhuese t\u00eb jet\u00eb n\u00eb dev dhe stage.<\/p>\n

NS<\/b>: Po n\u00eb kushtet e GDPR, mendoj se ata e trajtojn\u00eb gjithnj\u00eb e m\u00eb me kujdes... Mund t\u00eb them se n\u00eb Europ\u00eb jan\u00eb filluar t\u00eb nd\u00ebshkohen.<\/i><\/p>\n

DS<\/b>: Por shpeshher\u00eb mund t\u00eb shkruash nj\u00eb soft, i cili b\u00ebn dump nga produksioni dhe e obfuskon at\u00eb. K\u00ebshtu marrim t\u00eb dh\u00ebna prodhuese (snapshot, dump, kopje binare\u2026), por ato jan\u00eb anonimizuar. N\u00eb vend t\u00eb k\u00ebsaj mund t\u00eb jen\u00eb edhe skripta gjeneruese: ato mund t\u00eb jen\u00eb fiksturat ose thjesht nj\u00eb skripti, i cili gjeneron nj\u00eb baz\u00eb t\u00eb madhe. Problemi \u00ebsht\u00eb se sa koh\u00eb ndihmohet p\u00ebr t\u00eb krijuar imazhin baz\u00eb? Dhe sa koh\u00eb merr p\u00ebr ta shp\u00ebrndar\u00eb at\u00eb n\u00eb ambientin e nevojsh\u00ebm?<\/p>\n

Ne erdh\u00ebm n\u00eb nj\u00eb skem\u00eb: n\u00ebse klienti ka nj\u00eb grup t\u00eb dh\u00ebnash fikstura (version minimale t\u00eb baz\u00ebs), at\u00ebher\u00eb automatikisht i p\u00ebrdorim ato. N\u00ebse flasim p\u00ebr ambientet e rishikimit, kur ne krijuam branch-in, u hap nj\u00eb instanc\u00eb e aplikacionit \u2014 ne aty shp\u00ebrndajm\u00eb nj\u00eb baz\u00eb t\u00eb vog\u00ebl. Por doli mir\u00eb dhe nj\u00eb variant<\/a><\/noindex>, kur nga prodhimi marrim nj\u00eb dump \u00e7do nat\u00eb (n\u00eb mesnat\u00eb) dhe krijojm\u00eb n\u00eb baz\u00eb t\u00eb tij nj\u00eb kontejner Docker me PostgreSQL dhe MySQL me k\u00ebto t\u00eb dh\u00ebna t\u00eb ngarkuara. N\u00ebse nga ky imazh duhet t\u00eb shp\u00ebrndajm\u00eb baz\u00ebn 50 her\u00eb, kjo b\u00ebhet mjaft thjesht dhe shpejt.<\/p>\n

NS<\/b>: Thjesht me kopjim?<\/i><\/p>\n

DS<\/b>: T\u00eb dh\u00ebnat ruhen direkt n\u00eb imazhin Docker. Dometh\u00ebn\u00eb, kemi nj\u00eb imazh t\u00eb gatsh\u00ebm, le t\u00eb themi 100 GB. Fal\u00eb shtresave n\u00eb Docker, ne mund t\u00eb ndajm\u00eb k\u00ebt\u00eb imazh n\u00eb numrin e nevojsh\u00ebm her\u00ebsh. Metoda \u00ebsht\u00eb e thjesht\u00eb, por funksionon mir\u00eb.<\/p>\n

NS<\/b>: M\u00eb pas, kur e testoni, ndryshon direkt brenda Docker'it, apo jo? Copy-on-write brenda Docker'it \u2014 e hedhim dhe fillojm\u00eb nga e para, gjith\u00e7ka shkon mir\u00eb. Fantastic! Dhe ju tashm\u00eb e p\u00ebrdorni k\u00ebt\u00eb aktivisht?<\/i><\/p>\n

DS<\/b>: Q\u00eb t\u00eb nj\u00ebjt\u00eb.<\/p>\n

NS<\/b>: Ne merremi me gj\u00ebra mjaft t\u00eb ngjashme. Vet\u00ebm se ne nuk p\u00ebrdorim copy-on-write t\u00eb Docker'it, por ndonj\u00eb tjet\u00ebr.<\/i><\/p>\n

DS<\/b>: Ai nuk \u00ebsht\u00eb i p\u00ebrgjithsh\u00ebm. Nd\u00ebrsa ai i Docker'it funksionon kudo.<\/p>\n

NS<\/b>: Teoretikisht, po. Por ne gjithashtu kemi module atje, mund t\u00eb b\u00ebjm\u00eb module t\u00eb ndryshme dhe t\u00eb punojm\u00eb me sisteme t\u00eb ndryshme skedari. Tani, k\u00ebtu \u00ebsht\u00eb nj\u00eb moment. Ne e shohim t\u00eb gjith\u00eb k\u00ebt\u00eb ndryshe nga perspektiva e Postgres'it. Tani e shikova nga k\u00ebndv\u00ebshtrimi i Docker'it dhe pash\u00eb se gjith\u00e7ka funksionon. Por n\u00ebse baza \u00ebsht\u00eb e madhe, p\u00ebr shembull, 1 TB, at\u00ebher\u00eb gjith\u00e7ka shkon ngadal\u00eb: dhe operacionet nat\u00ebn, dhe ngarkimi i gjith\u00e7kaje n\u00eb Docker\u2026 E n\u00ebse duhet t\u00eb ngarkojm\u00eb 5 TB n\u00eb Docker\u2026 Ose gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull?<\/i><\/p>\n

DS<\/b>: \u00c7far\u00eb ka r\u00ebnd\u00ebsi: ato jan\u00eb blobs, thjesht bita dhe byte.<\/p>\n

NS<\/b>: R\u00ebnd\u00ebsia \u00ebsht\u00eb se a e b\u00ebni k\u00ebt\u00eb p\u00ebrmes dump dhe restore?<\/i><\/p>\n

DS<\/b>: Krejt\u00ebsisht e panevojshme. M\u00ebnyrat p\u00ebr t\u00eb gjeneruar k\u00ebt\u00eb imazh mund t\u00eb jen\u00eb t\u00eb ndryshme.<\/p>\n

NS<\/b>: P\u00ebr disa klient\u00eb, ne kemi b\u00ebr\u00eb q\u00eb, n\u00eb vend t\u00eb gjenerimit t\u00eb rregullt t\u00eb imazhit baz\u00eb, ta mbajm\u00eb at\u00eb vazhdimisht n\u00eb gjendje t\u00eb azhurnuar. N\u00eb thelb, ai \u00ebsht\u00eb nj\u00eb kopje, por t\u00eb dh\u00ebnat nuk i merr direkt nga masteri, por p\u00ebrmes arkivimit. Nj\u00eb arkiv binar, ku WAL-t\u00eb e aplikohen \u00e7do dit\u00eb, atje merren edhe backup-et\u2026 K\u00ebto WAL-e pastaj arrijn\u00eb \u2014 me nj\u00eb vones\u00eb t\u00eb vog\u00ebl (n\u00eb fakt 1-2 sekonda) \u2014 te imazhi baz\u00eb. Nga aty ne e klonojm\u00eb n\u00eb \u00e7do m\u00ebnyr\u00eb \u2014 tani p\u00ebr tani ne p\u00ebrdorim ZFS si parazgjedhje.<\/i><\/p>\n

DS<\/b>: Por me ZFS jeni t\u00eb kufizuar n\u00eb nj\u00eb nyje.<\/p>\n

NS<\/b>: Po. Por ZFS ka gjithashtu nj\u00eb magji tjet\u00ebr d\u00ebrgo<\/a><\/noindex>: me t\u00eb mund t\u00eb d\u00ebrgoni nj\u00eb snapshot dhe madje (k\u00ebt\u00eb nuk e kam testuar shum\u00eb, por...) mund t\u00eb d\u00ebrgoni diferenc\u00ebn mes dy PGDATA<\/code>. N\u00eb t\u00eb v\u00ebrtet\u00eb, ne kemi edhe nj\u00eb mjet tjet\u00ebr, t\u00eb cilin nuk e kemi shqyrtuar shum\u00eb p\u00ebr k\u00ebto detyra. N\u00eb PostgreSQL ka pg_rewind<\/a><\/noindex>, funksionon si nj\u00eb \"rsync\" i men\u00e7ur, duke kaluar shum\u00eb prej atij q\u00eb nuk ka nevoj\u00eb t\u00eb shikohet, sepse aty nuk ka ndryshuar asgj\u00eb. Ne mund t\u00eb b\u00ebjm\u00eb nj\u00eb sinkronizim t\u00eb shpejt\u00eb midis dy server\u00ebve dhe t\u00eb kthehemi mbrapa ashtu si\u00e7 \u00ebsht\u00eb.<\/i><\/p>\n

Pra, ne p\u00ebrpiqemi nga ana e DBA-s\u00eb p\u00ebr t\u00eb krijuar nj\u00eb mjet q\u00eb lejon t\u00eb b\u00ebjm\u00eb at\u00eb q\u00eb the, kemi nj\u00eb baz\u00eb, por duam t\u00eb testojm\u00eb di\u00e7ka 50 her\u00eb, gati n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb.<\/i><\/p>\n

DS<\/b>: 50 her\u00eb do t\u00eb thot\u00eb q\u00eb ju nevojitet t\u00eb porositni 50 instancat Spot.<\/p>\n

NS<\/b>: Jo, gjith\u00e7ka b\u00ebhet n\u00eb nj\u00eb makin\u00eb.<\/i><\/p>\n

DS<\/b>: Por si do t\u00eb shp\u00ebrndani 50 her\u00eb, n\u00ebse kjo nj\u00eb baz\u00eb, le t\u00eb themi, \u00ebsht\u00eb nj\u00eb terabajt? Ndoshta iu nevojitet 256 GB RAM?<\/p>\n

NS<\/b>: Po, ndonj\u00ebher\u00eb nevojitet shum\u00eb Memorje \u2014 \u00ebsht\u00eb normale. Por ky \u00ebsht\u00eb nj\u00eb shembull nga jeta. N\u00eb makin\u00ebn e production-it ka 96 b\u00ebrtham\u00eb dhe 600 GB. Nd\u00ebrsa p\u00ebr DB p\u00ebrdoren 32 b\u00ebrtham\u00eb (madje ndonj\u00ebher\u00eb edhe 16 b\u00ebrtham\u00eb) dhe memoria 100-120 GB.<\/i><\/p>\n

DS<\/b>: Dhe atje futen 50 kopje?<\/p>\n

NS<\/b>: Epo, kopja \u00ebsht\u00eb nj\u00eb, m\u00eb pas punon copy-on-write (ZFS)\u2026 Do t'ju tregoj m\u00eb shum\u00eb.<\/i><\/p>\n

Ne, p\u00ebr shembull, kemi nj\u00eb baz\u00eb prej 10 TB. Disku p\u00ebr t\u00eb \u00ebsht\u00eb b\u00ebr\u00eb, ZFS e ka kompresuar madh\u00ebsin\u00eb e saj me rreth 30-40%. Pasi ne nuk b\u00ebjm\u00eb teste ngarkese, koha e sakt\u00eb e p\u00ebrgjigjes nuk \u00ebsht\u00eb e r\u00ebnd\u00ebsishme: le t\u00eb jet\u00eb deri n\u00eb 2 her\u00eb m\u00eb e ngadalt\u00eb \u2014 \u00ebsht\u00eb n\u00eb rregull.<\/i><\/p>\n

Ne ofrojm\u00eb mund\u00ebsin\u00eb p\u00ebr zhvilluesit, QA, DBA dhe t\u00eb tjer\u00eb t\u00eb kryejn\u00eb testimin n\u00eb 1-2 rrjedha. P\u00ebr shembull, ata mund t\u00eb fillojn\u00eb nj\u00eb migrim. Ajo nuk k\u00ebrkon menj\u00ebher\u00eb 10 b\u00ebrthama \u2014 i nevojitet 1 backend-i i Postgres-it, 1 b\u00ebrtham\u00eb. Migrimi do t\u00eb nis\u00eb \u2014 ndoshta, autovacuum<\/a><\/noindex> do t\u00eb startoj\u00eb gjithashtu, pastaj aktivizohet b\u00ebrthama e dyt\u00eb. Ne kemi t\u00eb rezervuar 16-32 b\u00ebrthama, k\u00ebshtu q\u00eb 10 njer\u00ebz mund t\u00eb punojn\u00eb p\u00ebr nj\u00ebkoh\u00ebsish, nuk ka asnj\u00eb problem.<\/i><\/p>\n

Pasi fizikisht PGDATA<\/code> \u00ebsht\u00eb nj\u00ebsoj, rezulton k\u00ebshtu q\u00eb ne e mashtrojm\u00eb Postgres-in n\u00eb t\u00eb v\u00ebrtet\u00eb. Mrekullia \u00ebsht\u00eb kjo: p\u00ebr shembull, nisin 10 Postgres n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb. Problemi zakonisht \u00ebsht\u00eb? I vendosin shared_buffers<\/a><\/noindex>, le t\u00eb themi, n\u00eb 25%. Pra, kjo \u00ebsht\u00eb 200 GB. M\u00eb shum\u00eb se tre t\u00eb tilla nuk mund t\u00eb startohen, sepse do t\u00eb mbaroj\u00eb memoria.<\/i><\/p>\n

Por ne n\u00eb nj\u00eb moment e kuptuam se kjo nuk \u00ebsht\u00eb e nevojshme: ne vendosim shared_buffers n\u00eb 2 GB. PostgreSQL ka effective_cache_size<\/a><\/noindex>, dhe n\u00eb t\u00eb v\u00ebrtet\u00eb vet\u00ebm ai ndikon n\u00eb planet<\/a><\/noindex>. At\u00eb e vendosim n\u00eb 0,5 TB. Dhe nuk ka r\u00ebnd\u00ebsi q\u00eb n\u00eb t\u00eb v\u00ebrtet\u00eb nuk jan\u00eb atje: ai nd construon planet, si t\u00eb ishin atje.<\/i><\/p>\n

Prandaj, kur testojm\u00eb ndonj\u00eb migrim, mund t\u00eb mbledhim t\u00eb gjitha planet \u2014 do t\u00eb shohim se si do t\u00eb ndodhte n\u00eb produksion. Sekundat atje do t\u00eb jen\u00eb t\u00eb tjera (m\u00eb t\u00eb ngadalta), por t\u00eb dh\u00ebnat q\u00eb ne lexojm\u00eb n\u00eb t\u00eb v\u00ebrtet\u00eb, dhe vet\u00eb planet (\u00e7far\u00eb JOIN-i dhe t\u00eb tjer\u00eb) dalin krejt si n\u00eb produksion. Dhe paralelisht mund t\u00eb fillojm\u00eb shum\u00eb kontrolle t\u00eb tilla n\u00eb nj\u00eb makin\u00eb.<\/i><\/p>\n

DS<\/b>: A nuk mendon se ka disa probleme k\u00ebtu? E para - \u00ebsht\u00eb nj\u00eb zgjidhje q\u00eb funksionon vet\u00ebm n\u00eb PostgreSQL. Ky qasje \u00ebsht\u00eb shum\u00eb specifike, nuk \u00ebsht\u00eb generic. E dyta - Kubernetes (dhe gjith\u00e7ka q\u00eb tani po shkon n\u00eb teknologjit\u00eb cloud) parashikon shum\u00eb nyje, dhe k\u00ebto nyje jan\u00eb efemere. Nd\u00ebrsa n\u00eb rastin tuaj \u00ebsht\u00eb nj\u00eb nyje stateful, persistente. K\u00ebto gj\u00ebra m\u00eb krijojn\u00eb kontradikta.<\/p>\n

NS<\/b>: E para \u2014 bien dakord, kjo \u00ebsht\u00eb nj\u00eb histori krejt Postgres-i. Mendoj se po t\u00eb kemi ndonj\u00eb IO t\u00eb drejtp\u00ebrdrejt\u00eb dhe nj\u00eb pool buffer pothuajse p\u00ebr gjith\u00eb memorien, nj\u00eb qasje e till\u00eb nuk do t\u00eb funksiononte \u2014 planet do t\u00eb ishin t\u00eb tjera. Por p\u00ebr momentin po punojm\u00eb vet\u00ebm me Postgres-in, nuk po mendojm\u00eb p\u00ebr t\u00eb tjer\u00ebt.<\/i><\/p>\n

P\u00ebr Kubernetes. Ti vet\u00eb e tregon kudo se baza jon\u00eb \u00ebsht\u00eb persistente. N\u00ebse instanca bie, e r\u00ebnd\u00ebsishme \u00ebsht\u00eb t\u00eb ruash diskun. Ja, ne gjithashtu e gjith\u00eb platforma \u00ebsht\u00eb n\u00eb Kubernetes, dhe komponenti me Postgres \u00ebsht\u00eb ve\u00e7mas (edhe pse nj\u00ebher\u00eb do t\u00eb jet\u00eb atje). Prandaj, \u00ebsht\u00eb k\u00ebshtu: instanca bie, por ne kemi ruajtur PV-n\u00eb e saj dhe thjesht e lidhim me nj\u00eb instanc\u00eb tjet\u00ebr (t\u00eb re), si\u00e7 nuk ndodhi asgj\u00eb.<\/i><\/p>\n

DS<\/b>: Nga k\u00ebndv\u00ebshtrimi im, ne krijojm\u00eb pod n\u00eb Kubernetes. K8s \u2014 elastik: nyjet porositen vet\u00eb sipas nevoj\u00ebs. Detyra \u00ebsht\u00eb thjesht t\u00eb krijosh nj\u00eb pod dhe t'i thuash se i nevojiten X burime, e m\u00eb pas K8s do t\u00eb merret me gjith\u00e7ka. Por mb\u00ebshtetja p\u00ebr ruajtjen n\u00eb Kubernetes ende nuk \u00ebsht\u00eb e q\u00ebndrueshme: n\u00eb 1.16<\/a><\/noindex>, n\u00eb 1.17<\/a><\/noindex> (k\u00ebt\u00eb version doli jav\u00eb<\/i> mir\u00ebpas<\/p>\n

Pas do t\u00eb kaloj\u00eb gjasht\u00eb muaj deri nj\u00eb vit \u2014 do t\u00eb b\u00ebhet m\u00eb ndjesh\u00ebm stabil, ose t\u00eb pakt\u00ebn do t\u00eb shpallet si i till\u00eb. At\u00ebher\u00eb mund\u00ebsia p\u00ebr snapshot dhe resize do t\u00eb zgjidh\u00eb plot\u00ebsisht problemin tuaj. Sepse keni nj\u00eb baz\u00eb. Po, ajo mund t\u00eb mos jet\u00eb shum\u00eb e shpejt\u00eb, por shpejt\u00ebsia varet nga ajo q\u00eb \"n\u00eb kapak\", sepse disa zbatime din\u00eb t\u00eb kryejn\u00eb kopjimin dhe copy-on-write n\u00eb nivelin e n\u00ebnstruktur\u00ebs s\u00eb disqet.<\/p>\n

NS<\/b>: K\u00ebtu duhet gjithashtu q\u00eb t\u00eb gjitha motor\u00ebt (Amazon, Google\u2026) t\u00eb fillojn\u00eb t\u00eb mb\u00ebshtesin k\u00ebt\u00eb version - kjo gjithashtu merr ndonj\u00eb koh\u00eb.<\/i><\/p>\n

DS<\/b>: Deri tani nuk i p\u00ebrdorim ato. Ne p\u00ebrdorim tonat.<\/p>\n

Zhvillimi lokal n\u00ebn Kubernetes<\/h2>\n

\nNS<\/b>: A ke hasur ndonj\u00ebher\u00eb n\u00eb nj\u00eb d\u00ebshir\u00eb t\u00eb till\u00eb, kur duhet t\u00eb ngresh t\u00eb gjith\u00eb pod'\u00ebt n\u00eb nj\u00eb makin\u00eb dhe t\u00eb b\u00ebsh nj\u00eb test t\u00eb till\u00eb t\u00eb vog\u00ebl. N\u00eb m\u00ebnyr\u00eb t\u00eb shpejt\u00eb p\u00ebr t\u00eb marr\u00eb nj\u00eb prov\u00eb koncepti, p\u00ebr t\u00eb par\u00eb se si funksionon aplikacioni n\u00eb Kubernetes, pa ndar\u00eb nj\u00eb mori makinash p\u00ebr k\u00ebt\u00eb. Ka Minikube, apo jo?<\/i><\/p>\n

DS<\/b>: M\u00eb duket se ky rast \u2014 t\u00eb vendos\u00ebsh nj\u00eb node \u2014 \u00ebsht\u00eb p\u00ebr zhvillimin lokal ekskluzivisht. Ose disa shprehje t\u00eb k\u00ebtij modeli. Ka Minikube<\/a><\/noindex>, ka k3s<\/a><\/noindex>, KIND<\/a><\/noindex>. Po shkojm\u00eb drejt p\u00ebrdorimit t\u00eb Kubernetes IN Docker. Tani kemi filluar t\u00eb punojm\u00eb me t\u00eb p\u00ebr teste.<\/p>\n

NS<\/b>: M\u00eb par\u00eb mendova se ishte nj\u00eb p\u00ebrpjekje p\u00ebr t\u00eb p\u00ebrfshir\u00eb t\u00eb gjith\u00eb pod'\u00ebt n\u00eb nj\u00eb Docker-image. Por doli se ka t\u00eb b\u00ebj\u00eb me di\u00e7ka krejt\u00ebsisht tjet\u00ebr. S\u00ebrish aty ka kontejner\u00eb t\u00eb ve\u00e7ant\u00eb, pod'\u00eb t\u00eb ve\u00e7ant\u00eb \u2014 thjesht n\u00eb Docker.<\/i><\/p>\n

DS<\/b>: Po. Dhe ka nj\u00eb simulim mjaft t\u00eb qeshur atje, por ideja \u00ebsht\u00eb kjo\u2026 Kemi nj\u00eb utilitar p\u00ebr deploy \u2014 werf<\/a><\/noindex>. Duam t\u00eb b\u00ebjm\u00eb nj\u00eb mod p\u00ebr at\u00eb \u2014 hipotetikisht werf up<\/code>: \u00abM\u00eb ngre Kubernetes lokal\u00bb. Dhe m\u00eb pas t\u00eb nisim nj\u00eb werf follow<\/code>. At\u00ebher\u00eb zhvilluesi do t\u00eb mund t\u00eb b\u00ebj\u00eb ndryshime n\u00eb IDE, nd\u00ebrsa n\u00eb sistem \u00ebsht\u00eb nj\u00eb proces q\u00eb sheh ndryshimet dhe rind\u00ebrton imazhet, duke i ri-ndryshuar ato n\u00eb K8s lokal. K\u00ebshtu duam t\u00eb p\u00ebrpiqemi t\u00eb zgjidhim problemin e zhvillimit lokal.<\/p>\n

Snapshotet dhe klonimi i DB n\u00eb realitetin K8s<\/h2>\n

\nNS<\/b>: N\u00ebse kthehemi te copy-on-write. Kam v\u00ebn\u00eb re se edhe n\u00eb re ka snapshot-e. Ata funksionojn\u00eb ndryshe. P\u00ebr shembull, n\u00eb GCP: ke nj\u00eb instanc\u00eb shum\u00eb-terabajt n\u00eb bregun lindor t\u00eb SHBA. B\u00ebn periodicitet t\u00eb snapshot-eve. Ngre nga nj\u00eb snapshot nj\u00eb kopje disku n\u00eb bregun per\u00ebndimor \u2014 pas disa minutash \u00ebsht\u00eb gjith\u00e7ka gati, punon shum\u00eb shpejt, vet\u00ebm duhet t\u00eb mbush\u00ebsh cache-n\u00eb n\u00eb memorie. Por k\u00ebto klon\u00eb (snapshot-e) jan\u00eb p\u00ebr t\u00eb \"provision\" nj\u00eb volum t\u00eb ri. Kjo \u00ebsht\u00eb fantastike kur duhet t\u00eb krijosh shum\u00eb instanca.<\/i><\/p>\n

Dhe p\u00ebr testet, mendoj se snapshot-et q\u00eb ti flet p\u00ebr n\u00eb Docker ose q\u00eb un\u00eb flas p\u00ebr n\u00eb ZFS, btrfs dhe madje LVM\u2026 \u2014 ata lejojn\u00eb q\u00eb n\u00eb nj\u00eb makin\u00eb t\u00eb mos krijosh t\u00eb dh\u00ebna reale. N\u00eb re, gjithashtu do t\u00eb paguash p\u00ebr to \u00e7do her\u00eb dhe nuk do t\u00eb pres\u00ebsh m\u00eb sekonda, por minuta (dhe n\u00eb rastin e lazy load'ave<\/a><\/noindex>ndoshta edhe or\u00eb).<\/i><\/p>\n

N\u00eb vend t\u00eb k\u00ebsaj, mund t\u00eb marr\u00eb k\u00ebto t\u00eb dh\u00ebna p\u00ebr nj\u00eb sekond\u00eb-dy, t\u00eb zhvillosh testin dhe ta heq\u00ebsh. K\u00ebto snapshotet zgjidhin detyra t\u00eb ndryshme. N\u00eb rastin e par\u00eb \u2014 p\u00ebr t\u00eb u zgjeruar dhe p\u00ebr t\u00eb marr\u00eb replika t\u00eb reja, n\u00eb t\u00eb dyt\u00ebn \u2014 p\u00ebr teste.<\/i><\/p>\n

DS<\/b>: Nuk e pranoj. T\u00eb b\u00ebsh nj\u00eb klonim t\u00eb duhur t\u00eb volumit \u00ebsht\u00eb nj\u00eb detyr\u00eb p\u00ebr cloud. Nuk kam par\u00eb implementimin e tyre, por e di si e b\u00ebjm\u00eb ne n\u00eb hardware. Ne kemi Ceph, ku mund t'i themi \u00e7do volum fizik (RBD<\/a><\/noindex>) dhe t\u00eb marrim p\u00ebr disa milisekonda nj\u00eb volum t\u00eb dyt\u00eb me karakteristika t\u00eb nj\u00ebjta, clone<\/i> IOPS etj. Duhet t\u00eb kuptojm\u00eb se brenda \u00ebsht\u00eb nj\u00eb copy-on-write i nd\u00ebrlikuar. Pse cloud nuk b\u00ebn ashtu? Jam i sigurt se ata p\u00ebrpiqen ta b\u00ebjn\u00eb n\u00eb nj\u00eb m\u00ebnyr\u00eb apo nj\u00eb tjet\u00ebr.<\/a><\/noindex>t\u00eb tjer\u00eb. Duhet t\u00eb kuptosh se atje brenda ka nj\u00eb copy-on-write t\u00eb zgjuar. Pse reja t\u00eb mos e b\u00ebj\u00eb ashtu? Jam i sigurt q\u00eb ata gjithsesi p\u00ebrpiqen ta b\u00ebjn\u00eb k\u00ebt\u00eb.<\/p>\n

NS<\/b>: Pse \u00ebsht\u00eb e nevojshme t\u00eb ngresh nj\u00eb instanc\u00eb t\u00eb t\u00ebr\u00eb? Ne kemi nj\u00eb instanc\u00eb me 32 b\u00ebrtham\u00eb, me 16\u2026 dhe aty mund t\u00eb hyjn\u00eb disa \u2014 p\u00ebr shembull, kat\u00ebr. Kur k\u00ebrkojm\u00eb t\u00eb pestin, do t\u00eb ngrihet nj\u00eb instanc\u00eb, e pastaj do t\u00eb fshihet.<\/i><\/p>\n

DS<\/b>: Po, \u00ebsht\u00eb interesante, n\u00eb Kubernetes del nj\u00eb histori tjet\u00ebr. Ne kemi DB-n\u00eb jo n\u00eb K8s, dhe nj\u00eb instanc\u00eb. Megjithat\u00eb, p\u00ebr klonimin e nj\u00eb baze me shum\u00eb terabajt, nuk merr m\u00eb shum\u00eb se dy sekonda.<\/p>\n

NS<\/b>: Kjo \u00ebsht\u00eb e mrekullueshme. Por mesazhi im fillestar \u00ebsht\u00eb se kjo nuk \u00ebsht\u00eb nj\u00eb zgjidhje generike. Po, \u00ebsht\u00eb e shk\u00eblqyer, por p\u00ebrshtatet vet\u00ebm me Postgres dhe vet\u00ebm n\u00eb nj\u00eb nyje.<\/i><\/p>\n

DS<\/b>: Ajo i p\u00ebrshtatet jo vet\u00ebm p\u00ebr Postgres: k\u00ebto plane, si\u00e7 p\u00ebrshkrova, do t\u00eb funksionojn\u00eb vet\u00ebm n\u00eb t\u00eb. Por n\u00ebse nuk merremi me plane, por na duhen t\u00eb dh\u00ebnat p\u00ebr testimin funksional, at\u00ebher\u00eb kjo do t\u00eb ishte e p\u00ebrshtatshme p\u00ebr \u00e7do SGBD.<\/p>\n

NS<\/b>: Shum\u00eb vite m\u00eb par\u00eb ne b\u00ebm\u00eb di\u00e7ka t\u00eb till\u00eb me snapshotet e LVM. Kjo \u00ebsht\u00eb klasik\u00eb. Ky qasje u p\u00ebrdor shum\u00eb. Thjesht, nyjet stateful jan\u00eb nj\u00eb dhimbje. Sepse duhet t'i kujdesesh, gjithmon\u00eb t\u00eb mbash mend p\u00ebr to...<\/i><\/p>\n

DS<\/b>: A nuk sheh ndonj\u00eb mund\u00ebsi p\u00ebr nj\u00eb hibrid k\u00ebtu? Supozoni se stateful \u00ebsht\u00eb ndonj\u00eb pod, ai punon p\u00ebr disa njer\u00ebz (shum\u00eb testues). Volumi yn\u00eb \u00ebsht\u00eb nj\u00eb, por fal\u00eb sistemit t\u00eb skedar\u00ebve klon\u00ebt \u2014 lokal. N\u00ebse pod-i bie, disku mbetet \u2014 do t\u00eb ngrihet p\u00ebrs\u00ebri pod-i, do t\u00eb lexoj\u00eb informacionin p\u00ebr t\u00eb gjith\u00eb klon\u00ebt, do t\u00eb ngrej\u00eb gjith\u00e7ka mbrapsht dhe do t\u00eb thot\u00eb: \"K\u00ebtu jan\u00eb klon\u00ebt tuaj t\u00eb ngritur n\u00eb k\u00ebto porte, punoni me ta m\u00eb tej\".<\/p>\n

NS<\/b>: Teknikisht, kjo do t\u00eb thot\u00eb se brenda Kubernetes \u00ebsht\u00eb nj\u00eb pod, brenda t\u00eb cilit ne lan\u00e7ohemi shum\u00eb Postgres.<\/i><\/p>\n

DS<\/b>: Teknikisht, kjo do t\u00eb thot\u00eb se n\u00eb kuad\u00ebr t\u00eb Kubernetes, kjo \u00ebsht\u00eb nj\u00eb pod, brenda t\u00eb cilit ne ngrem\u00eb shum\u00eb Postgres'\u00eb.<\/p>\n

NS<\/b>: Po, nuk e shoh as t\u00eb till\u00eb.<\/i><\/p>\n

DS<\/b>: Duhet t\u00eb shtojm\u00eb k\u00ebtu pyetje sigurie. Ky variant organizimi n\u00ebnkupton q\u00eb ky pod ka privilegje t\u00eb larta (capabilities), sepse ai mund t\u00eb kryej\u00eb operacione jo standarde mbi sistemin e skedar\u00ebve\u2026 Por po e p\u00ebrs\u00ebris: mendoj se n\u00eb perspektiv\u00ebn afatmesme n\u00eb Kubernetes do t\u00eb rregullohet storage, n\u00eb cloud do t\u00eb rregullohet e gjith\u00eb historia me volumin \u2014 gjith\u00e7ka do t\u00eb funksionoj\u00eb \u201cthjesht\u201d. Do t\u00eb ket\u00eb resizing, klonim\u2026 Ka nj\u00eb volum \u2014 ne themi: \u201cKrijo nj\u00eb t\u00eb ri mbi baz\u00ebn e atij\u201d, dhe pas nj\u00eb sekonde e gjysm\u00eb fitojm\u00eb at\u00eb q\u00eb na nevojitet.<\/p>\n

NS<\/b>: Nuk e besoj se do t\u00eb b\u00ebhet n\u00eb nj\u00eb sekond\u00eb e gjysm\u00eb p\u00ebr shum\u00eb terabajt\u00eb. N\u00eb Ceph e b\u00ebn vet\u00eb, por po flet p\u00ebr cloud. Shko n\u00eb cloud, n\u00eb EC2 b\u00ebj nj\u00eb klon t\u00eb volumit EBS t\u00eb shum\u00eb terabajt\u00ebve dhe shiko \u00e7far\u00eb performanc\u00eb do t\u00eb ket\u00eb. Nuk do t\u00eb marr\u00eb disa sekonda. M\u00eb intereson shum\u00eb kur do t\u00eb arrijn\u00eb n\u00eb k\u00ebt\u00eb tregues. E kuptoj p\u00ebr \u00e7far\u00eb po flet, por do t\u00eb lejoj veten t\u00eb mos pranoj.<\/i><\/p>\n

DS<\/b>: Ok, por thash\u00eb se n\u00eb perspektiv\u00ebn afatmesme, jo afatshkurt\u00ebr. P\u00ebr disa vite.<\/p>\n

P\u00ebr operatorin p\u00ebr PostgreSQL nga Zalando.<\/h2>\n

\nN\u00eb mes t\u00eb k\u00ebsaj takimi u bashkua gjithashtu Aleksej Kljukin, nj\u00eb ish-programues nga kompania Zalando, i cili tregoi historin\u00eb e operatorit PostgreSQL:<\/p>\n

\u00cbsht\u00eb e shk\u00eblqyer q\u00eb ky tem\u00eb u prek: dhe Postgres, dhe Kubernetes. Kur filluam ta b\u00ebnim k\u00ebt\u00eb n\u00eb Zalando n\u00eb vitin 2017, kjo ishte nj\u00eb tem\u00eb me t\u00eb cil\u00ebn t\u00eb gjith\u00eb d\u00ebshironin t\u00eb merreshin, por askush nuk e b\u00ebnte. T\u00eb gjith\u00eb kishin filluar t\u00eb merrnin Kubernetes, por kur pyeteshin se si t\u00eb veprohej me baza t\u00eb dh\u00ebnash, edhe njer\u00ebz si Kelsey Hightower<\/a><\/noindex>, q\u00eb predikonin K8s, thonin di\u00e7ka si:<\/p>\n

'Shkoni n\u00eb sh\u00ebrbimet e menaxhuara dhe p\u00ebrdorni ato, mos e nisni DB-n\u00eb n\u00eb Kubernetes. P\u00ebrndryshe, K8s juaj do t\u00eb vendos\u00eb, p\u00ebr shembull, t\u00eb b\u00ebj\u00eb nj\u00eb azhurnim, do t\u00eb fik\u00eb t\u00eb gjitha nyjat dhe t\u00eb dh\u00ebnat tuaja do t\u00eb shkojn\u00eb shum\u00eb larg'.<\/i><\/p>\n

Ne vendos\u00ebm t\u00eb b\u00ebjm\u00eb nj\u00eb operator, i cili do t\u00eb funksiononte, pavar\u00ebsisht k\u00ebtij k\u00ebshillimi, p\u00ebr t\u00eb nisur DB-n\u00eb Postgres n\u00eb Kubernetes. Dhe kishim nj\u00eb baz\u00eb t\u00eb mir\u00eb \u2014 Patroni<\/a><\/noindex>. Ky \u00ebsht\u00eb nj\u00eb failover automat p\u00ebr PostgreSQL, i b\u00ebr\u00eb si duhet, pra duke p\u00ebrdorur etcd, consul ose ZooKeeper si nj\u00eb depot p\u00ebr informacionin mbi klastern. Nj\u00eb depot t\u00eb till\u00eb q\u00eb do t'i jepte t\u00eb gjith\u00ebve, q\u00eb pyesnin, p\u00ebr shembull, kush \u00ebsht\u00eb lideri aktual, t\u00eb nj\u00ebjtin informacion \u2014 pavar\u00ebsisht se gjith\u00e7ka \u00ebsht\u00eb e shp\u00ebrndar\u00eb \u2014 p\u00ebr t\u00eb shmangur brain splitin. Plus, ne kishim imazh Docker<\/a><\/noindex> p\u00ebr t\u00eb.<\/p>\n

N\u00eb p\u00ebrgjith\u00ebsi, nevoja p\u00ebr auto failover iu shfaq kompanis\u00eb pas migrazh\u00ebs nga nj\u00eb qend\u00ebr t\u00eb dh\u00ebnash t\u00eb brendshme n\u00eb cloud. Cloud-i ishte i bazuar n\u00eb nj\u00eb zgjidhje PaaS (Platform-as-a-Service) t\u00eb prodhuar vet\u00eb. Ishte Open Source, por p\u00ebr ta ngritur, duhej punuar shum\u00eb. Quhej STUPS<\/a><\/noindex>.<\/p>\n

Fillimisht nuk kishte Kubernetes. M\u00eb sakt\u00ebsisht, kur u zhvillua zgjidhja vetjake, K8s ishte tashm\u00eb, por aq i papjekur saq\u00eb nuk ishte i p\u00ebrshtatsh\u00ebm p\u00ebr prodhim. Ishte, mendoj, viti 2015 ose 2016. Deri n\u00eb vitin 2017, Kubernetes u b\u00eb pak a shum\u00eb i pjekur - u detyruam t\u00eb migronim atje.<\/p>\n

Dhe ne kishim nj\u00eb kontenier Docker. Pat\u00ebm PaaS, e cila p\u00ebrdorte Docker. Pse t\u00eb mos provonim K8s? Pse t\u00eb mos shkruanim operatorin ton\u00eb? Murat Kabilov, i cili erdhi te ne nga Avito, e filloi k\u00ebt\u00eb si nj\u00eb projekt nga iniciativa e tij \u2014 \"t\u00eb luante\" \u2014 dhe projekti \"fluturoi\".<\/p>\n

Por n\u00eb t\u00eb v\u00ebrtet\u00eb, doja t\u00eb flisja p\u00ebr AWS. Pse aty kishte historikisht kod q\u00eb lidhej me AWS...<\/p>\n

Kur nisni ndonj\u00eb gj\u00eb n\u00eb Kubernetes, duhet t\u00eb kuptoni se K8s \u00ebsht\u00eb nj\u00eb pun\u00eb n\u00eb progres. Ai zhvillohet vazhdimisht, p\u00ebrmir\u00ebsohet dhe her\u00eb pas here, madje d\u00ebshtoj\u00eb. Duhet t\u00eb jeni t\u00eb v\u00ebmendsh\u00ebm ndaj t\u00eb gjitha ndryshimeve n\u00eb Kubernetes, duhet t\u00eb jeni t\u00eb gatsh\u00ebm, n\u00eb rast se ndodh ndonj\u00eb gj\u00eb, t\u00eb zhytuni n\u00eb t\u00eb dhe t\u00eb m\u00ebsoni se si funksionon n\u00eb detaje \u2014 ndoshta m\u00eb shum\u00eb se \u00e7far\u00eb do t\u00eb donit. Kjo vlen p\u00ebr \u00e7do platform\u00eb ku nisni bazat e t\u00eb dh\u00ebnave...<\/p>\n

Pra, kur po b\u00ebnim operatorin, ne kishim Postgres q\u00eb punonte me nj\u00eb volum t\u00eb jasht\u00ebm (n\u00eb k\u00ebt\u00eb rast \u2014 EBS, sepse po punonim n\u00eb AWS). Baza e t\u00eb dh\u00ebnave u rrit, n\u00eb nj\u00eb moment u k\u00ebrkua t\u00eb b\u00ebhej resize: p\u00ebr shembull, p\u00ebrmasat fillestare EBS ishin 100 TB, baza arriti n\u00eb at\u00eb madh\u00ebsi, tani duam t\u00eb rrisim EBS n\u00eb 200 TB. Si? Supozoni se mund t\u00eb b\u00ebni dump\/restore n\u00eb nj\u00eb instanc\u00eb t\u00eb re, por kjo merr koh\u00eb dhe ka ndalesa.<\/p>\n

Prandaj do t\u00eb donim nj\u00eb resizing t\u00eb till\u00eb, q\u00eb do t\u00eb rritej seksioni i EBS dhe m\u00eb pas do t\u2019i thoshte sistemit t\u00eb skedar\u00ebve q\u00eb t\u00eb p\u00ebrdor\u00eb hap\u00ebsir\u00ebn e re. Dhe ne e b\u00ebm\u00eb k\u00ebt\u00eb, por n\u00eb at\u00eb koh\u00eb Kubernetes nuk kishte asnj\u00eb API p\u00ebr operacionin e resizing. Pasi punonim n\u00eb AWS, shkruam kodin p\u00ebr API-n\u00eb e tij.<\/p>\n

Askush nuk e pengon t\u00eb b\u00ebj\u00eb t\u00eb nj\u00ebjt\u00ebn gj\u00eb p\u00ebr platforma t\u00eb tjera. N\u00eb operator nuk ka asnj\u00eb lidhje q\u00eb mund t\u00eb niset vet\u00ebm n\u00eb AWS, dhe n\u00eb \u00e7do gj\u00eb tjet\u00ebr nuk do t\u00eb funksionoj\u00eb. N\u00eb p\u00ebrgjith\u00ebsi, ky \u00ebsht\u00eb nj\u00eb projekt Open Source: n\u00ebse dikush d\u00ebshiron t\u00eb p\u00ebrshpejtoj\u00eb shfaqjen e p\u00ebrdorimit t\u00eb API-s\u00eb s\u00eb re \u2014 mir\u00ebseardh\u00ebt. GitHub<\/a><\/noindex>, pull request-\u00ebt \u2014 ekipi i Zalando p\u00ebrpiqet t\u00eb reagoj\u00eb mjaft shpejt ndaj tyre dhe t\u00eb avancoj\u00eb operatorin. Sipas sa di, projekti ka marr\u00eb pjes\u00eb<\/a><\/noindex> n\u00eb Google Summer of Code dhe iniciativa t\u00eb ngjashme. Zalando po punon shum\u00eb aktivisht mbi t\u00eb.\n<\/p><\/blockquote>\n

<\/p>\n

P.S. Bonus!<\/h2>\n

\nN\u00ebse jeni t\u00eb interesuar p\u00ebr tem\u00ebn e PostgreSQL dhe Kubernetes, ju kujtojm\u00eb gjithashtu se jav\u00ebn e kaluar u mbajt Postgres-e mart\u00eb, ku me Nikolain bisedoi Aleksand\u00ebr Kukushkin nga Zalando<\/b>. Video prej tij \u00ebsht\u00eb e disponueshme k\u00ebtu<\/a><\/noindex>.<\/p>\n

P.P.S.<\/h2>\n

\nLexoni gjithashtu n\u00eb blogun ton\u00eb:<\/p>\n