K\u00ebrkohet monitorim t\u00eb dh\u00ebnash t\u00eb m\u00ebtejsh\u00ebm rreth funksionimit t\u00eb failover, dhe ne kemi 12 sharde PostgreSQL, k\u00ebshtu q\u00eb duhet t\u00eb monitorojm\u00eb 12 klaster\u00eb. Me rritjen e numrit t\u00eb shardeve, nuk duhet t\u00eb harrojm\u00eb t\u00eb azhurnojm\u00eb gjithashtu failover.<\/li>\n<\/ul>\nKrijimi i nj\u00eb sistemi t\u00eb shkruar nga vet\u00eb p\u00ebr d\u00ebshtimin duket shum\u00eb i komplikuar dhe k\u00ebrkon mb\u00ebshtetje jo triviale. Duke p\u00ebrdorur nj\u00eb grup PostgreSQL, ky do t\u00eb ishte opsioni m\u00eb i thjesht\u00eb, por nuk \u00ebsht\u00eb i shkall\u00ebzuesh\u00ebm, prandaj nuk na p\u00ebrshtatet.<\/p>\n
Repmgr<\/h3>\n Menaxhuesi i Replikimit p\u00ebr grupet PostgreSQL, i cili din t\u00eb menaxhoj\u00eb funksionimin e grupit PostgreSQL. Megjithat\u00eb, nuk ofron d\u00ebshtim automatik \"nga kutia\", k\u00ebshtu q\u00eb do t\u00eb nevojitet t\u00eb shkruhet nj\u00eb \"mb\u00ebshtjell\u00ebs\" mbi zgjidhjen e gatshme. Prandaj, gjith\u00e7ka mund t\u00eb b\u00ebhet edhe m\u00eb e komplikuar se me skriptet e shkruara nga vet\u00eb, k\u00ebshtu q\u00eb ne nuk e provuam madje edhe Repmgr.<\/p>\n
AWS RDS<\/h3>\n Mb\u00ebshtet gjith\u00e7ka q\u00eb na nevojitet, din t\u00eb b\u00ebj\u00eb kopje rezerv\u00eb dhe mb\u00ebshtet grupimin e lidhjeve. Ka kalim automatik: kur masteri vdes, nj\u00eb replik\u00eb b\u00ebhet masteri i ri, dhe AWS ndryshon regjistrimin DNS p\u00ebr masterin e ri, me replikat q\u00eb mund t\u00eb ndodhen n\u00eb AZ t\u00eb ndryshme.<\/p>\n
Nj\u00eb nga t\u00eb metat \u00ebsht\u00eb mungesa e parametrave t\u00eb holl\u00ebsish\u00ebm t\u00eb konfigurimit. Si shembull t\u00eb parametrave t\u00eb holl\u00ebsish\u00ebm: n\u00eb instancat tona kan\u00eb p\u00ebrllogaritje p\u00ebr lidhjet TCP, gj\u00eb q\u00eb, fatkeq\u00ebsisht, nuk mund t\u00eb realizohet n\u00eb RDS:<\/p>\n
net.ipv4.tcp_keepalive_time=10\nnet.ipv4.tcp_keepalive_intvl=1\nnet.ipv4.tcp_keepalive_probes=5\nnet.ipv4.tcp_retries2=3\n<\/code><\/pre>\nP\u00ebr m\u00eb tep\u00ebr, \u00e7mimi p\u00ebr AWS RDS \u00ebsht\u00eb pothuajse dy her\u00eb m\u00eb i shtrenjt\u00eb se \u00e7mimi i zakonsh\u00ebm i instanc\u00ebs, q\u00eb ka qen\u00eb shkaku kryesor i refuzimit t\u00eb k\u00ebsaj zgjidhjeje.<\/p>\n
Patroni<\/h3>\n Ky \u00ebsht\u00eb nj\u00eb template p\u00ebr python p\u00ebr menaxhimin e PostgreSQL me dokumentacion t\u00eb mir\u00eb, d\u00ebshtim automatik dhe kod burimor n\u00eb github.<\/p>\n
Avantazhet e Patroni:<\/p>\n
\n\u00c7do parametr i konfigurimit \u00ebsht\u00eb i detajuar, kuptohet si funksionon gjith\u00e7ka;<\/li>\n D\u00ebshtimi automatik funksionon nga kutia;<\/li>\n \u00cbsht\u00eb shkruar n\u00eb python, dhe pasi ne vet\u00eb shkruajm\u00eb shum\u00eb n\u00eb python, do t\u00eb na jet\u00eb m\u00eb e leht\u00eb t\u00eb kuptojm\u00eb problemet dhe, ndoshta, madje t\u00eb ndihmojm\u00eb n\u00eb zhvillimin e projektit;<\/li>\n Menaxhon plot\u00ebsisht PostgreSQL-in, lejon ndryshimin e konfiguracionit menj\u00ebher\u00eb n\u00eb t\u00eb gjitha nodet e grupit, dhe n\u00ebse k\u00ebrkohet rinisja e grupit p\u00ebr t\u00eb aplikuar nj\u00eb konfigurim t\u00eb ri, kjo mund t\u00eb b\u00ebhet gjithashtu me ndihm\u00ebn e Patroni.<\/li>\n<\/ul>\nDisavantazhet:<\/p>\n
\nNga dokumentacioni nuk \u00ebsht\u00eb e qart\u00eb se si t\u00eb punoni sakt\u00ebsisht me PgBouncer. Megjithat\u00eb, nuk \u00ebsht\u00eb e leht\u00eb ta quash k\u00ebt\u00eb nj\u00eb t\u00eb met\u00eb, pasi detyra e Patroni \u00ebsht\u00eb t\u00eb menaxhoj\u00eb PostgreSQL-in, dhe se si do t\u00eb kalojn\u00eb lidhjet te Patroni \u00ebsht\u00eb problemi yn\u00eb;<\/li>\n Ka pak shembuj t\u00eb implementimit t\u00eb Patroni n\u00eb volume t\u00eb m\u00ebdha, nd\u00ebrkoh\u00eb q\u00eb ka shum\u00eb shembuj implementimi nga zero.<\/li>\n<\/ul>\nP\u00ebrfundimisht, p\u00ebr t\u00eb krijuar nj\u00eb grup q\u00eb \u00ebsht\u00eb rezistent ndaj d\u00ebshtimeve, ne zgjodh\u00ebm pik\u00ebrisht Patroni.<\/p>\n
Procesi i implementimit t\u00eb Patroni<\/h2>\n Para Patroni kishim 12 shard-e PostgreSQL me nj\u00eb konfigurim nj\u00eb master dhe nj\u00eb replik\u00eb me replikim asinkron. Server\u00ebt e aplikacionit i drejtoheshin bazave t\u00eb t\u00eb dh\u00ebnave p\u00ebrmes Network Load Balancer, p\u00ebr t\u00eb cilin ishin dy instance me PgBouncer, dhe pas tyre ndodheshin t\u00eb gjith\u00eb server\u00ebt PostgreSQL.
P\u00ebr t\u00eb futur Patroni na nevojitej t\u00eb zgjedhim nj\u00eb ruajtje t\u00eb shp\u00ebrndara t\u00eb konfiguracionit t\u00eb klasterit. Patroni funksionon me sisteme t\u00eb shp\u00ebrndara t\u00eb ruajtjes s\u00eb konfiguracioneve si etcd, Zookeeper, Consul. M\u00ebnyra si jemi n\u00eb prodhimin ton\u00eb, kemi nj\u00eb klaster t\u00eb plot\u00eb Consul, i cili funksionon n\u00eb bashk\u00ebpunim me Vault dhe nuk e p\u00ebrdorim m\u00eb tej. Nj\u00eb rast i shk\u00eblqyer p\u00ebr t\u00eb filluar p\u00ebrdorimin e Consul p\u00ebr q\u00ebllimin e tij.<\/p>\n
Si funksionon Patroni me Consul<\/h3>\n Ne kemi nj\u00eb klaster Consul, i cili p\u00ebrb\u00ebhet nga tre nod\u00eb dhe nj\u00eb klaster Patroni, i cili p\u00ebrb\u00ebhet nga nj\u00eb lider dhe nj\u00eb replik\u00eb (n\u00eb Patroni, masteri quhet lider i klasterit dhe skllav\u00ebt - replika). \u00c7do instanc\u00eb e klasterit Patroni d\u00ebrgon vazhdimisht n\u00eb Consul informacion mbi gjendjen e klasterit. Prandaj nga Consul gjithmon\u00eb mund t\u00eb m\u00ebsohet konfiguracioni aktual i klasterit Patroni dhe kush \u00ebsht\u00eb lideri n\u00eb at\u00eb moment.<\/p>\n
P\u00ebr t\u00eb lidhur Patroni me Consul, mjafton t\u00eb studiohet dokumentacioni zyrtar, ku shkruhet se \u00ebsht\u00eb e nevojshme t\u00eb specifikohet host-i n\u00eb formatin http ose https n\u00eb varesi t\u00eb m\u00ebnyr\u00ebs si punojm\u00eb me Consul, dhe skema e lidhjes \u00ebsht\u00eb opsionale:<\/p>\n
host: hosti:port p\u00ebr pik\u00ebn e fundit t\u00eb Consul, n\u00eb formatin: http(s):\/\/host:port\nscheme: (opsionale) http ose https, parazgjedhje \u00ebsht\u00eb http<\/code><\/pre>\nDuket e thjesht\u00eb, por k\u00ebtu fillojn\u00eb problemet. Ne punojm\u00eb me Consul p\u00ebrmes nj\u00eb lidhjeje t\u00eb sigurt p\u00ebrmes https dhe konfigurimi yn\u00eb i lidhjes do t\u00eb duket si m\u00eb posht\u00eb:<\/p>\n
consul:\n host: https:\/\/server.production.consul:8080 \n verify: true\n cacert: {{ consul_cacert }}\n cert: {{ consul_cert }}\n key: {{ consul_key }}<\/code><\/pre>\nPor ashtu nuk funksionon. Gjat\u00eb nisjes s\u00eb Patroni nuk mund t\u00eb lidhet me Consul sepse p\u00ebrpiqet gjithsesi t\u00eb shkoj\u00eb n\u00ebp\u00ebrmjet http.<\/p>\n
T\u00eb kuptojm\u00eb problemin ndihmoi kodi burimor i Patroni. Fatmir\u00ebsisht, ai \u00ebsht\u00eb shkruar n\u00eb python. Duke u zbuluar, parametri host nuk analizohet ashtu si duhet, dhe protokolli duhet t\u00eb specifikohet n\u00eb scheme. Ja si duket blloku i punuesh\u00ebm i konfigurimit p\u00ebr t\u00eb punuar me Consul p\u00ebr ne:<\/p>\n
consul:\n host: server.production.consul:8080\n scheme: https\n verify: true\n cacert: {{ consul_cacert }}\n cert: {{ consul_cert }}\n key: {{ consul_key }}<\/code><\/pre>\nConsul-template<\/h3>\n Pra t\u00eb gjitha, kemi zgjedhur ruajtjen p\u00ebr konfigurimin. Tani duhet t\u00eb kuptojm\u00eb se si PgBouncer do t\u00eb kaloj\u00eb konfigurimin e tij gjat\u00eb ndryshimit t\u00eb liderit n\u00eb klasterin Patroni. Dokumentacioni nuk ofron p\u00ebrgjigje p\u00ebr k\u00ebt\u00eb, pasi nuk e p\u00ebrshkruan pun\u00ebn me PgBouncer n\u00eb t\u00ebr\u00ebsi.<\/p>\n
N\u00eb k\u00ebrkim t\u00eb zgjidhjes, ne gjet\u00ebm nj\u00eb artikull (emrin, fatkeq\u00ebsisht, nuk e mbaj mend) ku ishte shkruar se Sconsul-template ndihmon shum\u00eb n\u00eb lidhjen nd\u00ebrmjet PgBouncer dhe Patroni. Kjo na shtyu n\u00eb k\u00ebrkimin p\u00ebr funksionimin e Consul-template.<\/p>\n
Doli se Sconsul-template monitoron vazhdimisht konfigurimin e klasterit PostgreSQL n\u00eb Consul. Gjat\u00eb ndryshimit t\u00eb liderit, ai p\u00ebrdit\u00ebson konfigurimin e PgBouncer dhe d\u00ebrgon nj\u00eb komand\u00eb p\u00ebr ta rindezur.<\/p>\n
Nj\u00eb p\u00ebrpar\u00ebsi e madhe e template \u00ebsht\u00eb se ajo ruhet si kod, k\u00ebshtu q\u00eb kur shtohet nj\u00eb shard i ri, mjafton t\u00eb b\u00ebhet nj\u00eb commit i ri dhe t\u00eb p\u00ebrdit\u00ebsohet template n\u00eb m\u00ebnyr\u00eb automatike, duke mbajtur parimin e Infrastruktur\u00ebs si Kod.<\/p>\n
Arkitektura e re me Patroni<\/h3>\n Si rezultat, ne mor\u00ebm k\u00ebt\u00eb skem\u00eb pune:
T\u00eb gjitha serverat e aplikacionit iu drejtohen balancuesit \u2192 pas tij q\u00ebndrojn\u00eb dy instance PgBouncer \u2192 n\u00eb \u00e7do instance \u00ebsht\u00eb aktivizuar Sconsul-template, i cili monitoron gjendjen e \u00e7do klasteri Patroni dhe ndjek aktualitetin e konfigurimit PgBouncer, i cili drejton k\u00ebrkesat te lideri aktual i \u00e7do klasteri.<\/p>\n
Testim manual<\/h3>\n K\u00ebt\u00eb skem\u00eb, para se ta nxjerrim n\u00eb prodhim, e kemi ekzekutuar n\u00eb nj\u00eb mjedis testimi t\u00eb vog\u00ebl dhe kontrolluam funksionimin e kalimit automatik. Hapur boardin, l\u00ebviz\u00ebm sticker dhe n\u00eb at\u00eb moment \u201cvram\u00eb\u201d liderin e klasterit. N\u00eb AWS, p\u00ebr k\u00ebt\u00eb mjafton t\u00eb fiket instanca p\u00ebrmes konsol\u00ebs.<\/p>\n
Sticker-i kthehej pas 10-20 sekondash, dhe pastaj fillonte s\u00ebrish t\u00eb l\u00ebvizte normalisht. Kjo tregon se klasteri Patroni funksionoi si\u00e7 duhet: nd\u00ebrroi liderin, d\u00ebrgoi informacionin n\u00eb Sconsul, dhe Sconsul-template menj\u00ebher\u00eb e kapte k\u00ebt\u00eb informacion, z\u00ebvend\u00ebsoi konfigurimin e PgBouncer dhe d\u00ebrgoi komand\u00ebn p\u00ebr reload.<\/p>\n
Si t\u00eb mbijetosh n\u00ebn nj\u00eb ngarkes\u00eb t\u00eb lart\u00eb dhe t\u00eb ruash downtime minimal?<\/h2>\n T\u00eb gjitha funksionon shk\u00eblqyesh\u00ebm! Por po shfaqen pyetje t\u00eb reja: Si do t\u00eb funksionoj\u00eb kjo n\u00ebn nj\u00eb ngarkes\u00eb t\u00eb lart\u00eb? Si t\u00eb shp\u00ebrndajm\u00eb gjith\u00e7ka n\u00eb prodhim shpejt dhe sigurt?<\/p>\n
P\u00ebr t\u00eb p\u00ebrgjigjur pyetjes s\u00eb par\u00eb, na ndihmon nj\u00eb mjedis testimi, ku ne kryejm\u00eb testim ngarkese. Ai \u00ebsht\u00eb plot\u00ebsisht identik me production n\u00eb arkitektur\u00eb dhe ka t\u00eb dh\u00ebna testuese t\u00eb gjeneruara, t\u00eb cilat n\u00eb v\u00ebllim jan\u00eb af\u00ebrsisht t\u00eb barabarta me ato n\u00eb production. Ne vendosim thjesht \"ta vrasim\" nj\u00eb nga masterat e PostgreSQL gjat\u00eb testit dhe t\u00eb shohim se \u00e7far\u00eb do ndodh\u00eb. Por para k\u00ebtij momenti \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb kontrollojm\u00eb automatikisht p\u00ebrhapjen, pasi n\u00eb k\u00ebt\u00eb mjedis kemi disa shard-e t\u00eb PostgreSQL, k\u00ebshtu q\u00eb do t\u00eb kemi nj\u00eb testim t\u00eb shk\u00eblqyer t\u00eb skenar\u00ebve t\u00eb konfigurimit p\u00ebrpara prodhimit.<\/p>\n
T\u00eb dy detyrat duken ambicioze, por ne kemi PostgreSQL 9.6. Po nuk vepruam direkt dhe e p\u00ebrdit\u00ebsuam n\u00eb 11.2?<\/p>\n
Ne vendosim ta b\u00ebjm\u00eb k\u00ebt\u00eb n\u00eb 2 etapa: s\u00eb pari p\u00ebrdit\u00ebsimin n\u00eb versionin 11.2, pastaj t\u00eb drejtojm\u00eb Patroni.<\/p>\n
P\u00ebrdit\u00ebsimi i PostgreSQL<\/h3>\n P\u00ebr t\u00eb p\u00ebrdit\u00ebsuar versionin e PostgreSQL sa m\u00eb shpejt, \u00ebsht\u00eb e nevojshme t\u00eb p\u00ebrdoret opsioni -k<\/b>, n\u00eb t\u00eb cilin krijohen lidhje t\u00eb forta n\u00eb disk dhe nuk ka nevoj\u00eb p\u00ebr kopjimin e t\u00eb dh\u00ebnave tuaja. N\u00eb baza me 300-400 GB, p\u00ebrdit\u00ebsimi zgjat 1 sekond\u00eb.<\/p>\n
Ne kemi shum\u00eb shard-e, k\u00ebshtu q\u00eb p\u00ebrdit\u00ebsimi duhet t\u00eb b\u00ebhet automatikisht. P\u00ebr k\u00ebt\u00eb, ne shkruam nj\u00eb Ansible playbook, i cili kryen t\u00eb gjith\u00eb procesin e p\u00ebrdit\u00ebsimit p\u00ebr ne:<\/p>\n
\/usr\/lib\/postgresql\/11\/bin\/pg_upgrade \n<b>--link <\/b>\n--old-datadir='' --new-datadir='' \n --old-bindir='' --new-bindir='' \n --old-options=' -c config_file=' \n --new-options=' -c config_file='<\/code><\/pre>\nK\u00ebtu \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb theksohet se para se t\u00eb filloni p\u00ebrmir\u00ebsimin, duhet ta kryeni at\u00eb me parametrin \u2014kontrollo<\/b>, p\u00ebr t\u00eb qen\u00eb t\u00eb sigurt n\u00eb mund\u00ebsin\u00eb e p\u00ebrmir\u00ebsimit. Po ashtu, skenari yn\u00eb b\u00ebn z\u00ebvend\u00ebsimin e konfigurimeve gjat\u00eb p\u00ebrmir\u00ebsimit. Skenari yn\u00eb u ekzekutua p\u00ebr 30 sekonda, ky \u00ebsht\u00eb nj\u00eb rezultat i shk\u00eblqyer.<\/p>\nNisja e Patroni<\/h3>\n
P\u00ebr t\u00eb zgjidhur problemin e dyt\u00eb, mjafton t\u00eb shohim konfigurimin e Patroni. N\u00eb depozitat zyrtare ka nj\u00eb shembull konfigurimi me initdb, i cili p\u00ebrgjigjet p\u00ebr inicializimin e nj\u00eb baze t\u00eb re gjat\u00eb nisjes s\u00eb par\u00eb t\u00eb Patroni. Por pasi q\u00eb ne kemi nj\u00eb baz\u00eb t\u00eb gatshme, thjesht e hoq\u00ebm k\u00ebt\u00eb pjes\u00eb nga konfigurimi.<\/p>\n
Kur filluam t\u00eb instalonim Patroni n\u00eb nj\u00eb klaster t\u00eb gatsh\u00ebm PostgreSQL dhe ta nisim at\u00eb, u p\u00ebrball\u00ebm me nj\u00eb problem t\u00eb ri: t\u00eb dy server\u00ebt nisnin si lider. Patroni nuk di asgj\u00eb p\u00ebr gjendjen e m\u00ebparshme t\u00eb klasterit dhe p\u00ebrpiqet t\u00eb nis\u00eb t\u00eb dy server\u00ebt si dy klaster\u00eb t\u00eb ve\u00e7ant\u00eb me t\u00eb nj\u00ebjtin em\u00ebr. P\u00ebr t\u00eb zgjidhur k\u00ebt\u00eb problem, \u00ebsht\u00eb e nevojshme t\u00eb hiqet \u0434\u0438\u0440\u0435\u043a\u0442\u043e\u0440\u0438\u044f e t\u00eb dh\u00ebnave n\u00eb slave:<\/p>\n
rm -rf \/var\/lib\/postgresql\/<\/code><\/pre>\nKjo duhet b\u00ebr\u00eb vet\u00ebm n\u00eb slave!<\/b><\/p>\n
Kur lidhet nj\u00eb replik\u00eb e past\u00ebr, Patroni b\u00ebn backup bazor t\u00eb liderit dhe e rikthen at\u00eb n\u00eb replik\u00eb, pastaj e p\u00ebrdit\u00ebson gjendjen e tanishme p\u00ebrmes wal-log-\u00ebve.<\/p>\n
Nj\u00eb v\u00ebshtir\u00ebsi tjet\u00ebr me t\u00eb cil\u00ebn u p\u00ebrball\u00ebm \u00ebsht\u00eb se t\u00eb gjith\u00eb klasterat PostgreSQL\u9ed8\u8ba4 quhen main. Kur \u00e7do klaster nuk di asgj\u00eb p\u00ebr nj\u00eb tjet\u00ebr, \u00ebsht\u00eb n\u00eb rregull. Por kur d\u00ebshiron t\u00eb p\u00ebrdor\u00ebsh Patroni, t\u00eb gjith\u00eb klasterat duhet t\u00eb ken\u00eb nj\u00eb em\u00ebr unik. Zgjidhja \u00ebsht\u00eb t\u00eb ndryshosh emrin e klasterit n\u00eb konfigurimin e PostgreSQL.<\/p>\n
Testim ngarkese<\/h3>\n Ne filluam nj\u00eb test q\u00eb imiton pun\u00ebn e p\u00ebrdoruesve n\u00eb tabela. Kur ngarkesa arriti niveli ton\u00eb mesatar ditor, ne p\u00ebrs\u00ebrit\u00ebm t\u00eb nj\u00ebjtin test, duke fikur nj\u00eb instanc\u00eb me liderin PostgreSQL. Failover-i automatik funksionoi ashtu si\u00e7 e prisnim: Patroni ndryshoi liderin, Konsul-template p\u00ebrdit\u00ebsoi konfigurimin PgBouncer dhe d\u00ebrgoi nj\u00eb komand\u00eb p\u00ebr rifreskimin. T\u00eb dh\u00ebnat tona n\u00eb Grafana treguan se kishte vonesa prej 20-30 sekondash dhe nj\u00eb sasi t\u00eb vog\u00ebl gabimesh nga server\u00ebt, t\u00eb lidhura me lidhjen n\u00eb baz\u00eb. Kjo \u00ebsht\u00eb nj\u00eb situat\u00eb normale, k\u00ebto vlera jan\u00eb t\u00eb pranueshme p\u00ebr failover-in ton\u00eb dhe sigurisht jan\u00eb m\u00eb t\u00eb mira se ndalesa e sh\u00ebrbimit.<\/p>\n
Dalja e Patroni n\u00eb prodhim<\/h2>\n Si p\u00ebrfundim, plani yn\u00eb ishte si vijon:<\/p>\n
\nDeploy-i i Konsul-template n\u00eb serverat PgBouncer dhe nisja;<\/li>\n P\u00ebrdit\u00ebsimi i PostgreSQL n\u00eb versionin 11.2;<\/li>\n Ndryshimi i emrit t\u00eb klasterit;<\/li>\n Nisja e klasterit Patroni.<\/li>\n<\/ul>\nN\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, skema jon\u00eb lejon q\u00eb t\u00eb realizojm\u00eb pik\u00ebn e par\u00eb pothuajse n\u00eb \u00e7do moment, ne mund t\u00eb heqim nj\u00eb nga nj\u00eb secilin PgBouncer nga funksionimi dhe t\u00eb realizojm\u00eb deploy-n\u00eb dhe nisjen e konsul-template. K\u00ebshtu e b\u00ebm\u00eb.<\/p>\n
P\u00ebr nj\u00eb shp\u00ebrndarje t\u00eb shpejt\u00eb, ne p\u00ebrdor\u00ebm Ansible, pasi t\u00eb gjith\u00eb playbook-\u00ebt i kishim verifikuar tashm\u00eb n\u00eb mjedisin e testimit, dhe koha e realizimit t\u00eb skenarit t\u00eb plot\u00eb ishte nga 1.5 deri n\u00eb 2 minuta p\u00ebr \u00e7do shard. Ne mund\u00ebm ta shp\u00ebrndanim secil\u00ebn shard nj\u00eb nga nj\u00eb pa ndaluar sh\u00ebrbimin ton\u00eb, por do t\u00eb na duhej t\u00eb fiknim \u00e7do PostgreSQL p\u00ebr disa minuta. N\u00eb k\u00ebt\u00eb rast, p\u00ebrdoruesit, t\u00eb cil\u00ebt kishin t\u00eb dh\u00ebna n\u00eb k\u00ebt\u00eb shard, nuk do t\u00eb mund t\u00eb punonin plot\u00ebsisht n\u00eb at\u00eb koh\u00eb, dhe kjo p\u00ebr ne \u00ebsht\u00eb e papranueshme.<\/p>\n
Zgjidhja p\u00ebr k\u00ebt\u00eb situat\u00eb ishte nj\u00eb mir\u00ebmbajtje e planifikuar, q\u00eb zhvillohet \u00e7do 3 muaj. Ky \u00ebsht\u00eb nj\u00eb interval p\u00ebr pun\u00ebt e planifikuara, kur ne e fikim plot\u00ebsisht sh\u00ebrbimin ton\u00eb dhe p\u00ebrdit\u00ebsojm\u00eb instancat e baz\u00ebs s\u00eb t\u00eb dh\u00ebnave. Ishte mbetur nj\u00eb jav\u00eb deri n\u00eb intervalin e radh\u00ebs, dhe ne vendos\u00ebm t\u00eb prisnim dhe t\u00eb p\u00ebrgatiteshim m\u00eb tej. Gjat\u00eb koh\u00ebs s\u00eb pritjes, ne mor\u00ebm masa shtes\u00eb: p\u00ebr \u00e7do shard PostgreSQL ngrit\u00ebm nj\u00eb replik\u00eb rezerv\u00eb p\u00ebr rast t\u00eb d\u00ebshtimit, p\u00ebr t\u00eb ruajtur t\u00eb dh\u00ebnat m\u00eb t\u00eb fundit, dhe shtuam nj\u00eb instanc\u00eb t\u00eb re p\u00ebr \u00e7do shard, e cila do t\u00eb b\u00ebhej nj\u00eb replik\u00eb e re n\u00eb klasterin Patroni, p\u00ebr t\u00eb mos ekzekutuar komand\u00ebn p\u00ebr t\u00eb fshir\u00eb t\u00eb dh\u00ebnat. E gjith\u00eb kjo ndihmoi t\u00eb reduktojm\u00eb sa m\u00eb shum\u00eb rrezikun e gabimeve.
Ne riluajt\u00ebm sh\u00ebrbimin ton\u00eb, gjith\u00e7ka funksionoi si\u00e7 duhej, p\u00ebrdoruesit vazhduan pun\u00ebn, por n\u00eb grafik\u00eb ne v\u00ebrejta nj\u00eb ngarkes\u00eb anomalisht t\u00eb lart\u00eb n\u00eb server\u00ebt e Consul.
Pse nuk e pam\u00eb k\u00ebt\u00eb n\u00eb mjedisin testues? Ky problem ilustron shum\u00eb mir\u00eb nevoj\u00ebn p\u00ebr t\u00eb ndjekur parimin e Infrastruktur\u00ebs si Kod dhe p\u00ebr t\u00eb p\u00ebrmir\u00ebsuar t\u00eb gjith\u00eb infrastruktur\u00ebn, duke filluar nga mjediset testuese deri te prodhimi. Ndryshe, \u00ebsht\u00eb shum\u00eb e leht\u00eb t\u00eb p\u00ebrballesh me nj\u00eb problem t\u00eb till\u00eb si\u00e7 ishte rasti yn\u00eb. \u00c7far\u00eb ndodhi? Consul u shfaq fillimisht n\u00eb prodhim, e m\u00eb pas n\u00eb mjediset testuese, si rezultat, n\u00eb mjediset testuese versioni i Consul ishte m\u00eb i lart\u00eb se n\u00eb prodhim. Pik\u00ebrisht n\u00eb nj\u00eb nga rrelease-t u zgjidh nj\u00eb rrjedhje CPU gjat\u00eb pun\u00ebs me consul-template. K\u00ebshtu q\u00eb thjesht p\u00ebrdit\u00ebsuam Consul, duke zgjidhur k\u00ebshtu problemin.<\/p>\n
Riluaj klasterin Patroni<\/h3>\n Megjithat\u00eb, ne mor\u00ebm nj\u00eb problem t\u00eb ri, p\u00ebr t\u00eb cilin nuk kishim as nj\u00eb ide. Gjat\u00eb p\u00ebrdit\u00ebsimit t\u00eb Consul, thjesht heqim nod\u00ebn e Consul nga klasteri me komand\u00ebn consul leave \u2192 Patroni lidhet me nj\u00eb server tjet\u00ebr t\u00eb Consul \u2192 gjith\u00e7ka funksionon. Por kur arrit\u00ebm n\u00eb instanc\u00ebn e fundit t\u00eb klasterit t\u00eb Consul dhe i d\u00ebrguam asaj komand\u00ebn consul leave, t\u00eb gjith\u00eb klasteret e Patroni thjesht u riluajt\u00ebn, dhe n\u00eb log e shohim gabimin n\u00eb vazhdim:<\/p>\n
Gabim: merrni_kostum\nZgjidhja më e fundit e thirrjes së fundit: \n...\nGabimNëTentativë: 'Kaluar afatin e ri-tre hedo'\nGabim: Gabim në komunikim me DCS\n<b>LOG: sistemi i bazës së të dhënave është mbyllur<\/b><\/code><\/pre>\nKlasteri Patroni nuk arriti t\u00eb marr\u00eb informacion p\u00ebr klasterin e tij dhe u riluajt.<\/p>\n
P\u00ebr t\u00eb gjetur nj\u00eb zgjidhje, ne u drejtuam tek autor\u00ebt e Patroni p\u00ebrmes nj\u00eb problemi n\u00eb github. Ata sugjeruan p\u00ebrmir\u00ebsime t\u00eb skedave tona t\u00eb konfigurimit:<\/p>\n
consul:\n consul.checks: []\nbootstrap:\n dcs:\n retry_timeout: 8<\/code><\/pre>\nNe arrit\u00ebm t\u00eb p\u00ebrs\u00ebrisim problemin n\u00eb mjedisin testues dhe e testuam atje, por, fatkeq\u00ebsisht, ato nuk funksionuan.<\/p>\n
Problemi \u00ebsht\u00eb ende pa zgjidhje. Ne planifikojm\u00eb t\u00eb provojm\u00eb k\u00ebto mund\u00ebsi zgjidhjeje:<\/p>\n
\nT\u00eb p\u00ebrdorim Sconsul-agent n\u00eb \u00e7do instanc\u00eb t\u00eb klasterit Patroni;<\/li>\n T\u00eb rregullojm\u00eb problemin n\u00eb kod.<\/li>\n<\/ul>\nNa \u00ebsht\u00eb e qart\u00eb vendi i shfaqjes s\u00eb gabimit: ndoshta problemi \u00ebsht\u00eb n\u00eb p\u00ebrdorimin e default timeout, i cili nuk ri-shkruhet n\u00ebp\u00ebrmjet dosjes s\u00eb konfigurimit. Kur hiqet serveri i fundit Sconsul nga klasteri, ndodh nj\u00eb ngecje e t\u00ebr\u00eb klasterit Sconsul, q\u00eb zgjas m\u00eb shum\u00eb se nj\u00eb sekond\u00eb, p\u00ebr k\u00ebt\u00eb arsye Patroni nuk mund t\u00eb marr\u00eb gjendjen e klasterit dhe e rinis t\u00ebr\u00eb klasterin.<\/p>\n
Fatmir\u00ebsisht, nuk has\u00ebm m\u00eb asnj\u00eb gabim tjet\u00ebr.<\/p>\n
P\u00ebrmbledhja e p\u00ebrdorimit t\u00eb Patroni<\/h2>\n Pas fillimit t\u00eb suksessh\u00ebm t\u00eb Patroni, kemi shtuar nga nj\u00eb replik\u00eb shtes\u00eb n\u00eb \u00e7do klaster. Tani n\u00eb \u00e7do klaster ka nj\u00eb ngjashm\u00ebri t\u00eb kvorumit: nj\u00eb lider dhe dy replika, p\u00ebr t\u00eb siguruar n\u00eb rastin e split-brain gjat\u00eb kalimit.
N\u00eb prodhim, Patroni ka funksionuar p\u00ebr m\u00eb shum\u00eb se tre muaj. Gjat\u00eb k\u00ebsaj kohe ai ka arritur t\u00eb na ndihmoj\u00eb. S\u00eb fundmi n\u00eb AWS vdiq lideri i nj\u00ebrit prej klaster\u00ebve, dhe failover automatik u aktivizua dhe p\u00ebrdoruesit vazhduan t\u00eb punojn\u00eb. Patroni realizoi detyr\u00ebn e tij kryesore.<\/p>\n
Nj\u00eb p\u00ebrmbledhje e vog\u00ebl e p\u00ebrdorimit t\u00eb Patroni:<\/b><\/p>\n\nLeht\u00ebsia e ndryshimit t\u00eb konfigurimit. Mjafton t\u00eb ndryshoni konfigurimin n\u00eb nj\u00eb instanc\u00eb dhe ai do t\u00eb aplikohet n\u00eb t\u00ebr\u00eb klasterin. N\u00ebse nevojitet rinisja p\u00ebr t\u00eb aplikuar konfigurimin e ri, Patroni do ta njoftoj\u00eb p\u00ebr k\u00ebt\u00eb. Patroni mund t\u00eb rinisi t\u00ebr\u00eb klasterin me nj\u00eb komand\u00eb, q\u00eb gjithashtu \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme.<\/li>\n Failover automatik funksionon dhe na ka ndihmuar tashm\u00eb.<\/li>\n P\u00ebrdit\u00ebsimi i PostgreSQL pa nd\u00ebrprerje t\u00eb aplikacionit. S\u00eb pari, \u00ebsht\u00eb e nevojshme t\u00eb p\u00ebrdit\u00ebsohen replikat n\u00eb versionin e ri, pastaj t\u00eb ndryshohet lideri n\u00eb klasterin Patroni dhe t\u00eb p\u00ebrdit\u00ebsohet lideri i vjet\u00ebr. Gjat\u00eb k\u00ebsaj, b\u00ebhet testimi i nevojsh\u00ebm i failover-it automatik.<\/li>\n<\/ul>\nBurimi: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"\u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u044f \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0443, \u043a\u0430\u043a \u043c\u044b \u043f\u043e\u0434\u043e\u0448\u043b\u0438 \u043a \u0432\u043e\u043f\u0440\u043e\u0441\u0443 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0441\u0442\u0438 PostgreSQL, \u043f\u043e\u0447\u0435\u043c\u0443 \u044d\u0442\u043e \u0441\u0442\u0430\u043b\u043e \u0434\u043b\u044f \u043d\u0430\u0441 \u0432\u0430\u0436\u043d\u043e \u0438 \u0447\u0442\u043e \u0432 \u0438\u0442\u043e\u0433\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c. \u0423 \u043d\u0430\u0441 \u0432\u044b\u0441\u043e\u043a\u043e\u043d\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043d\u044b\u0439 \u0441\u0435\u0440\u0432\u0438\u0441: 2,5 \u043c\u043b\u043d \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u043f\u043e \u0432\u0441\u0435\u043c\u0443 \u043c\u0438\u0440\u0443, 50\u041a+ \u0430\u043a\u0442\u0438\u0432\u043d\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u043a\u0430\u0436\u0434\u044b\u0439 \u0434\u0435\u043d\u044c. \u0421\u0435\u0440\u0432\u0435\u0440\u0430 \u043d\u0430\u0445\u043e\u0434\u044f\u0442\u0441\u044f \u0432 Amazone \u0432 \u043e\u0434\u043d\u043e\u043c \u0440\u0435\u0433\u0438\u043e\u043d\u0435 \u0418\u0440\u043b\u0430\u043d\u0434\u0438\u0438: \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u043f\u043e\u0441\u0442\u043e\u044f\u043d\u043d\u043e 100+ \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432, \u0438\u0437 \u043d\u0438\u0445 \u043f\u043e\u0447\u0442\u0438 50 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":26749,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35706","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t
![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/443f85815b0560fade7db5b639942935.png\")
![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/674dd77de4c8b48a946c9f64c0b2fdd1.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/dabcb7f6006520c4fec85fb925e8975e.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/2c1f1e7d9f7be4fdeb43858520d55e36.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/8e5350d2466311593e11add30d5b1bdc.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/3234594b363904424fdce64c9d77fca5.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/6cf92996a127bb6637ab81dc45fdd60a.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/bf9a9eff675a26039e3f76b16c6cd713.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/d670fe373b774f1b52bfd8a8c3b53a21.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/29e5c5f50dbfebfe5665fa0aeb009728.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/af0eef9ac235706c0aac62d2e1ee179d.png\")
<\/p>\n![\"Kluster](\"\/wp-content\/uploads\/2019\/06\/3bb4c0495fea274b04edd2e99b59eacb.png\")
<\/p>\n