Failover Cluster PostgreSQL + Patroni. Kuita ruzivo

Muchikamu chino ini ndichakuudza nzira yatakasvika nayo nyaya yePostgreSQL kukanganisa kushivirira, nei zvakave zvakakosha kwatiri, uye zvakaitika pakupedzisira.

Tine basa rakaremerwa zvakanyanya: 2,5 miriyoni vashandisi pasi rese, 50K+ vanoshanda zuva rega rega. Masevha ari muAmazone mune imwe dunhu reIreland: 100+ maseva akasiyana anogara achishanda, angangoita makumi mashanu awo ane dhatabhesi.

Iyo yese backend yakakura monolithic stateful Java application iyo inochengetedza inoenderera yewebhu yekubatanidza nemutengi. Kana vashandisi vakati wandei vachishanda panguva imwe chete pabhodhi rimwe chete, vese vanoona shanduko munguva chaiyo, nekuti isu tinorekodha shanduko yega yega mudhatabhesi. Tine zvikumbiro zvinosvika gumi pasekondi kune yedu dhatabhesi. Pamutoro wepamusoro muRedis tinonyora zvikumbiro zve10-80K pasekondi.

Nei takachinja kubva kuRedis kuenda kuPostgreSQL

Pakutanga, sevhisi yedu yakashanda neRedis, kiyi-yakakosha kuchengetedza inochengeta data rese museva RAM.

Zvakanakira Redis:

1. High mhinduro kumhanya, nokuti zvinhu zvose zvakachengetwa mundangariro;
2. Yakanakira backup uye kudzokorora.

Cons yeRedis kwatiri:

1. Iko hakuna kutengeserana chaiko. Takaedza kuvatevedzera padanho redu rekushandisa. Zvinosuruvarisa, izvi hazvina kushanda zvakanaka nguva dzose uye zvaida kunyora kodhi yakaoma kwazvo.
2. Nhamba ye data inogumira nehuwandu hwekuyeuka. Sezvo huwandu hwe data huchiwedzera, chiyeuchidzo chinokura, uye, pakupedzisira, isu tichamhanya mune hunhu hwemuenzaniso wakasarudzwa, uyo muAWS unoda kumisa sevhisi yedu kushandura mhando yemuenzaniso.
3. Izvo zvinodiwa kuti ugare uchichengetedza yakaderera latency level, nekuti Tine nhamba huru kwazvo yezvikumbiro. Iyo yakakwana latency level kwatiri ndeye 17-20 ms. Panhanho ye30-40 ms, tinowana mhinduro refu kune zvikumbiro zvedu zvekushandisa uye kuderedzwa kwebasa. Nehurombo, izvi zvakaitika kwatiri munaGunyana 2018, apo imwe yezviitiko neRedis nekuda kwechimwe chikonzero yakagamuchira latency yaive yakapetwa kaviri kupfuura zvakajairwa. Kuti tigadzirise dambudziko, takamisa sevhisi pakati pezuva rekushanda kuti tigadzirise zvisina kurongwa uye tikatsiva dambudziko reRedis.
4. Zviri nyore kuwana data risingawirirani kunyangwe nezvikanganiso zvidiki mukodhi uye wozopedza nguva yakawanda uchinyora kodhi kugadzirisa iyo data.

Isu takafunga nezvekuipa uye takaona kuti taifanira kutamira kune chimwe chinhu chiri nyore, nekuchinjana kwakajairika uye kushoma kutsamira pane latency. Takaita tsvakiridzo yedu, takaongorora sarudzo dzakawanda uye takasarudza PostgreSQL.

Isu tanga tichienda kune dhatabhesi nyowani kwemakore 1,5 ikozvino uye takangotamisa chikamu chidiki che data, saka ikozvino tiri kushanda panguva imwe chete neRedis uye PostgreSQL. Ruzivo rwakawanda nezvematanho ekufambisa uye kushandura data pakati pedhatabhesi rakanyorwa mukati chinyorwa newandinoshanda naye.

Patakatanga kufamba, application yedu yakashanda zvakananga nedhatabhesi uye yakawana iyo Redis uye PostgreSQL tenzi. Iyo PostgreSQL cluster yaive neyatenzi uye replica ine asynchronous replication. Izvi ndizvo zvakaita dhizaini yekufambiswa kwedatabase:

Kushandisa PgBouncer

Patakanga tichifamba, chigadzirwa chakanga chichikurawo: nhamba yevashandisi uye nhamba yemaseva akashanda nePostgreSQL yakawedzera, uye takatanga kupera kwekubatana. PostgreSQL inogadzira nzira yakaparadzana yekubatanidza yega yega uye inoshandisa zviwanikwa. Iwe unogona kuwedzera nhamba yekubatanidza kusvika kune imwe nzvimbo, kana zvisina kudaro pane mukana wekuti database haishande zvakanaka. Sarudzo yakanaka mumamiriro ezvinhu akadaro ingave yekusarudza maneja wekubatanidza anomira pamberi pe database.

Taive nesarudzo mbiri dzemaneja wekubatanidza: Pgpool uye PgBouncer. Asi yekutanga haitsigire nzira yekushandura yekushanda nedatabase, saka takasarudza PgBouncer.

Isu takagadzirisa chirongwa chebasa chinotevera: yedu application inowana imwe PgBouncer, kuseri uko kune PostgreSQL masters, uye kuseri kwatenzi wega wega kune imwe replica ine asynchronous replication.


Panguva imwecheteyo, isu takanga tisingakwanisi kuchengetedza huwandu hwese data muPostgreSQL uye kukurumidza kwekushanda ne database kwakakosha kwatiri, saka takatanga kugovera PostgreSQL padanho rekushandisa. Chirongwa chakatsanangurwa pamusoro chiri nyore kune izvi: kana uchiwedzera imwe PostgreSQL shard, inokwana kugadzirisa iyo PgBouncer kumisikidzwa uye chishandiso chinogona kushanda nekukurumidza neshadhi nyowani.

PgBouncer Mhosva Kushivirira

Ichi chirongwa chakashanda kudzamara iyo chete PgBouncer muenzaniso yafa. Isu tiri muAWS, uko zviitiko zvese zvinotangwa pane Hardware inofa nguva nenguva. Mumamiriro ezvinhu akadaro, muenzaniso wacho unongotamira kune hardware itsva uye unoshanda zvakare. Izvi zvakaitika nePgBouncer, asi yakazove isingawanikwe. Mhedzisiro yekuparara uku yaive yekuti sevhisi yedu yanga isingawanikwe kwemaminetsi makumi maviri nemashanu. Kune mamiriro akadai, AWS inokurudzira kushandisa redundancy kudivi remushandisi, zvatisina kuita panguva iyoyo.

Mushure meizvozvo, takafunga zvakanyanya nezve kukanganisa kushivirira kwePgBouncer nePostgreSQL masumbu, nekuti mamiriro akafanana anogona kuitika zvakare chero chiitiko muAWS account yedu.

Isu takavaka iyo PgBouncer kukanganisa kushivirira chirongwa sezvizvi: ese maseva ekushandisa anowana iyo Network Load Balancer, kumashure uko kune maviri PgBouncers. Imwe neimwe yePgBouncers inotarisa kune imwecheteyo tenzi PostgreSQL yega yega shard. Kana mamiriro ane AWS chiitiko tsaona achidzokorora, traffic yese inotungamirwa kuburikidza neimwe PgBouncer. Network Load Balancer kukanganisa kushivirira kunopihwa neAWS.

Ichi chirongwa chinokubvumira kuti uwedzere nyore nyore PgBouncer maseva.

Kugadzira iyo PostgreSQL Failover Cluster

Pakugadzirisa dambudziko iri, takafunga nzira dzakasiyana-siyana: kuzvinyora-kutadza, repmgr, AWS RDS, Patroni.

Zvinyorwa zvekuzvinyora

Vanogona kutarisa basa ratenzi uye, kana zvikaramba, kusimudzira replica kuna tenzi uye kugadzirisa PgBouncer configuration.

Zvakanakira nzira iyi ndeyekureruka kwakanyanya, nekuti iwe unonyora zvinyorwa pachako uye unonzwisisa chaizvo kuti zvinoshanda sei.

Cons:

• Tenzi anogona kunge asina kufa; pachinzvimbo, panogona kunge pakatadza network. Failover, asingazivi izvi, achasimudzira replica kuna tenzi, uye tenzi wekare acharamba achishanda. Nekuda kweizvozvo, isu tichawana maviri maseva mune tenzi basa uye hatizoziva kuti ndeupi wavo ane data razvino. Mamiriro ezvinhu aya anonziwo kupatsanurwa-uropi;
• Takasara tisina mhinduro. Mukugadzirisa kwedu kune master uye imwe replica, mushure mekushandura replica inosimudzirwa kune tenzi uye isu hatisisina replicas, saka tinofanira kuwedzera nemaoko replica itsva;
• Tinoda kumwe kutariswa kwekutadza kushanda, uye isu tine gumi nemaviri PostgreSQL shards, zvinoreva kuti tinoda kutarisa masumbu gumi nemaviri. Paunenge uchiwedzera huwandu hwe shards, iwe unofanirwawo kuyeuka kugadzirisa iyo failover.

Kuzvinyora-kukundikana kunoratidzika kwakaoma zvikuru uye kunoda rubatsiro rusiri ruduku. Neimwe PostgreSQL cluster, iyi ichave iri nyore sarudzo, asi haina kuyera, saka haina kukodzera isu.

Repmgr

Replication Maneja wePostgreSQL masumbu, ayo anogona kubata mashandiro ePostgreSQL cluster. Panguva imwecheteyo, haina otomatiki failover kunze kwebhokisi, saka kuti ushande iwe uchafanirwa kunyora yako wega "wrapper" pamusoro peyakagadzirirwa-yakagadziriswa mhinduro. Saka zvese zvinogona kutonyanya kuomarara kupfuura neanozvinyora, ndosaka isu tisina kumboedza Repmgr.

AWS RDS

Inotsigira zvese zvatinoda, inogona kugadzira backups uye inotsigira dziva rekubatanidza. Iyo ine otomatiki switching: kana tenzi afa, replica inova tenzi mutsva, uye AWS inoshandura iyo DNS rekodhi kune tenzi mutsva, nepo replicas inogona kuwanikwa mune dzakasiyana AZ.

Izvo zvisingabatsiri zvinosanganisira kushaikwa kwezvirongwa zvakanaka. Semuenzaniso wekugadzirisa zvakanaka: zviitiko zvedu zvine zvirambidzo zvekubatanidza tcp, izvo, zvinosuruvarisa, hazvigone kuitwa muRSS:

net.ipv4.tcp_keepalive_time=10
net.ipv4.tcp_keepalive_intvl=1
net.ipv4.tcp_keepalive_probes=5
net.ipv4.tcp_retries2=3

Pamusoro pezvo, AWS RDS inodhura zvakapetwa kaviri semutengo wenguva dzose, icho chaive chikonzero chikuru chekusiya mhinduro iyi.

Patroni

Iyi ipython template yekutarisira PostgreSQL ine yakanaka zvinyorwa, otomatiki failover uye source code pa github.

Zvakanakira Patroni:

• Imwe neimwe gadziriso parameter inotsanangurwa, zviri pachena kuti inoshanda sei;
• Automatic failover inoshanda kunze kwebhokisi;
• Yakanyorwa mu python, uye sezvo isu pachedu tichinyora zvakawanda mu python, zvichava nyore kwatiri kutarisana nezvinetso uye, zvichida, kunyange kubatsira kukurudzirwa kweprojekti;
• Inodzora zvizere PostgreSQL, inobvumidza iwe kuti uchinje magadzirirwo pane ese masumbu esumbu kamwechete, uye kana kushandisa iyo nyowani yekumisikidza inoda kutangazve kwesumbu, izvi zvinogona kuitwa zvakare uchishandisa Patroni.

Cons:

• Hazvina kujeka kubva muzvinyorwa kuti ungashanda sei nemazvo nePgBouncer. Kunyangwe zvakaoma kudaidza iyi minus, nekuti basa raPatroni ndere kubata PostgreSQL, uye kuti kubatana kuPatroni kuchashanda sei ratove dambudziko redu;
• Kune mienzaniso mishoma yekushandiswa kwePatroni pazviyero zvakakura, nepo kune mienzaniso yakawanda yekushandiswa kubva pakutanga.

Nekuda kweizvozvo, isu takasarudza Patroni kuti tigadzire failover cluster.

Patroni kuita maitiro

Pamberi paPatroni, takanga tine gumi nemaviri PostgreSQL shards mune imwe tenzi uye imwe replica gadziriso ine asynchronous replication. Masevha ekushandisa akawana dhatabhesi kuburikidza neNetwork Load Balancer, kumashure uko kwaive nezviitiko zviviri nePgBouncer, uye kumashure kwavo kwaive nemaseva ePostgreSQL ese.


Kuti tishandise Patroni, taifanira kusarudza yakagoverwa sumbu gadziriro yekuchengetedza. Patroni anoshanda neakagoverwa masisitimu ekuchengetedza akadai etcd, Zookeeper, Consul. Isu tine yakazara-yakazara Consul cluster mukugadzira, iyo inoshanda pamwe chete neVault uye isu hatichaishandisa zvakare. Chikonzero chikuru chekutanga kushandisa Consul kune chinangwa chayo.

Kuti Patroni anoshanda sei naConsul

Tine Consul cluster, iyo ine nodes nhatu, uye Patroni cluster, iyo ine mutungamiri uye replica (muPatroni, tenzi anonzi mutungamiri weboka, uye varanda vanonzi replicas). Imwe neimwe Patroni cluster muenzaniso inogara ichitumira ruzivo nezve mamiriro esumbu kuConsul. Naizvozvo, kubva kuConsul iwe unogona kugara uchitsvaga ikozvino kumisikidzwa kwePatroni cluster uye ndiani mutungamiri panguva ino.

Kuti ubatanidze Patroni kuConsul, ingo dzidza zvinyorwa zvepamutemo, izvo zvinoti iwe unofanirwa kutsanangura mugadziri mu http kana https fomati, zvichienderana nemashandiro atinoita neConsul, uye dhizaini yekubatanidza, sarudzo:

host: the host:port for the Consul endpoint, in format: http(s)://host:port
scheme: (optional) http or https, defaults to http

Zvinoita sezviri nyore, asi apa ndipo panotangira misungo. NeConsul tinoshanda pamusoro pekubatana kwakachengeteka kuburikidza ne https uye yedu yekubatanidza config ichaita seizvi:

consul:
  host: https://server.production.consul:8080 
  verify: true
  cacert: {{ consul_cacert }}
  cert: {{ consul_cert }}
  key: {{ consul_key }}

Asi hazvishande nenzira iyoyo. Pakutanga, Patroni haakwanise kubatana neConsul nekuti ichiri kuyedza kuenda ne http.

Iyo Patroni source code yakabatsira kugadzirisa dambudziko. Zvakanaka kuti zvakanyorwa mu python. Zvinoitika kuti iyo paramende paramende haina kuparadzaniswa neimwe nzira, uye iyo protocol inofanirwa kutsanangurwa muchirongwa. Izvi ndizvo zvinoita basa rekugadzirisa bhuroka rekushanda neConsul rinotaridzika kwatiri:

consul:
  host: server.production.consul:8080
  scheme: https
  verify: true
  cacert: {{ consul_cacert }}
  cert: {{ consul_cert }}
  key: {{ consul_key }}

Consul-template

Saka, takasarudza gadziriro yekuchengetedza. Iye zvino tinoda kunzwisisa kuti PgBouncer ichachinja sei kugadzirisa kana mutungamiri achinja muboka rePatroni. Iko hakuna mhinduro kumubvunzo uyu mune zvinyorwa, nekuti ... Muchidimbu, kushanda nePgBouncer hakuna kutsanangurwa ipapo.

Mukutsvaga mhinduro, takawana chinyorwa (zvinosuruvarisa, handiyeuki zita racho), apo pakanyorwa kuti Consul-template yakabatsira zvikuru mukubatanidza PgBouncer naPatroni. Izvi zvakaita kuti tidzidze basa reConsul-template.

Zvakazoitika kuti Consul-template inogara ichitarisa kumisikidzwa kwePostgreSQL cluster muConsul. Kana mutungamiri achinge achinja, inogadziridza iyo PgBouncer kumisikidzwa uye inotumira murairo kuti idzore zvakare.

Mukana wakakura wetemplate ndeyekuti inochengetwa sekodhi, saka kana uchiwedzera shard nyowani, zvinokwana kuita kuzvipira patsva uye kugadzirisa template otomatiki, ichitsigira iyo Infrastructure sekodhi kodhi.

Chivakwa chitsva naPatroni

Nekuda kweizvozvo, takagamuchira chirongwa chebasa chinotevera:


Ese maseva ekushandisa anowana iyo balancer → kuseri kwayo kune zviitiko zviviri zvePgBouncer → pane imwe neimwe Consul-template iri kushanda, iyo inotarisisa mamiriro ega ega Patroni cluster uye inotarisisa kukosha kwePgBouncer config, iyo inotungamira zvikumbiro kumutungamiri aripo. yesumbu rimwe nerimwe.

Manual test

Tisati taitanga mukugadzira, takatangisa chirongwa ichi pane diki bvunzo nharaunda uye takatarisa kushanda kwe otomatiki switching. Vakavhura bhodhi, vakafambisa chidhi uye panguva iyoyo "vakauraya" mutungamiri weboka racho. MuAWS, zvese zvaunoda kuti uite kudzima muenzaniso kuburikidza nekoni.

Iyo sticker yakadzoka mukati me10-20 seconds, ndokubva yatanga kufamba zvakanaka zvakare. Izvi zvinoreva kuti boka rePatroni rakashanda nemazvo: rakashandura mutungamiri, rakatumira ruzivo kuna Consul, uye Consul-template yakabva yatora ruzivo urwu, yakatsiva PgBouncer configuration uye yakatumira murairo kuti udzose zvakare.

Nzira yekurarama sei pasi pemutoro wakakwira uye kuchengetedza kushoma kuderera?

Zvose zvinoshanda zvakakwana! Asi mibvunzo mitsva inomuka: Ichashanda sei pasi pemutoro wakakwira? Nzira yekukurumidza uye zvakachengeteka kuburitsa zvese mukugadzira?

Nzvimbo yekuyedza yatinoitisa kuyedzwa kwemutoro inotibatsira kupindura mubvunzo wekutanga. Yakafanana zvachose nekugadzirwa mukuvaka uye yakagadzira data rekuyedza, iro rinenge rakaenzana nehuwandu hwekugadzira. Isu tinosarudza kungo "uraya" imwe yePostgreSQL masters panguva yebvunzo uye toona zvinoitika. Asi izvi zvisati zvaitika, zvakakosha kuti titarise kuburitsa otomatiki, nekuti pane ino nharaunda tine akati wandei PostgreSQL shards, saka isu tinozowana yakanakisa kuyedzwa kwezvigadziriso zvinyorwa zvisati zvagadzirwa.

Mabasa ese ari maviri anotaridzika kuva nechido, asi isu tine PostgreSQL 9.6. Pamwe isu tinogona kuvandudza ku11.2 ipapo ipapo?

Isu tinosarudza kuita izvi mumatanho maviri: tanga wagadziridza vhezheni kusvika 2, wobva watanga Patroni.

PostgreSQL update

Kukurumidza kugadzirisa iyo PostgreSQL vhezheni, unofanirwa kushandisa sarudzo -k, umo iyo yakaoma link inogadzirwa pa diski uye hapana chikonzero chekukopa data yako. Pane dhatabhesi ye300-400 GB, iyo yekuvandudza inotora 1 sekondi.

Isu tine shards yakawanda, saka iyo yekuvandudza inoda kuitwa yega. Kuti tiite izvi, takanyora Ansible playbook iyo inotiitira iyo yese yekuvandudza maitiro:

/usr/lib/postgresql/11/bin/pg_upgrade 
<b>--link </b>
--old-datadir='' --new-datadir='' 
 --old-bindir=''  --new-bindir='' 
 --old-options=' -c config_file=' 
 --new-options=' -c config_file='

Zvakakosha kucherechedza pano kuti usati watanga kukwidziridzwa, unofanira kuzviita neparameter --checkkuve nechokwadi chekuti kukwidziridzwa kunogoneka. Yedu script zvakare inotsiva configs panguva yekusimudzira. Runyoro rwedu rwakapera mumasekonzi makumi matatu, inova mhedzisiro yakanaka.

Kutanga kwePatroni

Kuti ugadzirise dambudziko rechipiri, ingotarisa iyo Patroni kumisikidzwa. Iyo yepamutemo repository ine muenzaniso wekumisikidzwa neinitdb, iyo ine basa rekutanga dhatabhesi nyowani kana Patroni atanga kutangwa. Asi sezvo isu tatova nedatabase yakagadzirira, isu takangobvisa chikamu ichi kubva pakugadziriswa.

Patakatanga kuisa Patroni pane yakagadzirira-yakagadzirwa PostgreSQL cluster nekuivhura, takasangana nedambudziko idzva: ese ari maviri maseva akatangwa semutungamiri. Patroni hapana chaanoziva nezvekutanga mamiriro esumbu uye anoedza kumhanyisa maseva ese ari maviri akaparadzana masumbu ane zita rimwe chete. Kuti ugadzirise dambudziko iri, unofanirwa kudzima dhairekitori re data pamuranda:

rm -rf /var/lib/postgresql/

Izvi zvinofanira kuitwa chete pamuranda!

Kana uchibatanidza replica yakachena, Patroni anoita mutungamiri webhesikepu uye anoidzorera kune replica, uye obva abata nemamiriro azvino achishandisa matanda emadziro.

Rimwe dambudziko ratakasangana naro nderekuti masumbu ese ePostgreSQL anonzi main by default. Kana sumbu rimwe nerimwe risingazivi nezverimwe, izvi zvakajairika. Asi kana iwe uchida kushandisa Patroni, masumbu ese anofanira kunge aine zita rakasiyana. Mhinduro ndeyekushandura zita resumbu muPostgreSQL kumisikidza.

Mutoro bvunzo

Takatanga bvunzo inotevedzera mashandisiro anoita vashandisi pamabhodhi. Mutoro pawakasvika paavhareji yedu yemazuva ese, takadzokorora bvunzo dzakafanana, takadzima chiitiko chimwe chete nemutungamiri PostgreSQL. Iyo otomatiki failover yakashanda sezvataitarisira: Patroni akachinja mutungamiri, Consul-template yakagadziridza iyo PgBouncer kumisikidzwa uye yakatumira murairo wekurodha zvakare. Maererano nemagirafu edu muGrafana, zvaive pachena kuti pane kunonoka kwe20-30 seconds uye zvishoma zvikanganiso kubva kumaseva ane chokuita nekubatanidza kune database. Aya ndiwo mamiriro akajairwa, hunhu hwakadai hunogamuchirwa kune yedu failover uye zviri nani pane sevhisi yekudzikira.

Kuvhurwa kwePatroni mukugadzirwa

Nekuda kweizvozvo, takauya nechirongwa chinotevera:

• Shandisa Consul-template kuPgBouncer server uye utange;
• PostgreSQL inogadziridza kune vhezheni 11.2;
• Kuchinja zita remasumbu;
• Kutangwa kwePatroni cluster.

Panguva imwecheteyo, chirongwa chedu chinotibvumira kuita poindi yekutanga chero nguva; isu tinogona kubvisa imwe neimwe PgBouncer kubva kubasa imwe neimwe uye kuita deployment uye kuvhurwa kweconsul-template pairi. Ndozvatakaita.

Kuti tiedze nekukurumidza, takashandisa Ansible, sezvo takanga tatoedza bhuku rese rekutamba pane imwe nzvimbo yekuyedza, uye nguva yekuuraya yeiyo yakazara script yaive kubva ku1,5 kusvika ku2 maminetsi kune yega yega shard. Tinogona kuburitsa zvese chimwe nechimwe kune imwe neimwe shard pasina kumisa sevhisi yedu, asi isu taizofanira kudzima imwe neimwe PostgreSQL kwemaminitsi akati wandei. Muchiitiko ichi, vashandisi vane data iri pane shard iyi havazokwanisi kushanda zvakakwana panguva ino, uye izvi hazvigamuchirwi kwatiri.

Nzira yekubuda mumamiriro ezvinhu aya yakarongwa kuchengetedza, izvo zvatinoita mwedzi mitatu yega yega. Iri ihwindo rebasa rakarongwa, patinodzima zvachose sevhisi yedu uye kugadzirisa dhatabhesi zviitiko. Kwakanga kwasara vhiki imwe chete kuti tisvike pahwindo raitevera, uye takasarudza kungomirira uye kugadzirira zvimwe. Munguva yekumirira, isu takawedzera mabheti edu: kune yega yega PostgreSQL shard isu takasimudza replica yakasara kana ikatadza, kuitira kuchengetedza data razvino, uye tikawedzera muenzaniso mutsva kune yega yega shard, iyo inofanirwa kuve replica nyowani muPatroni. cluster, kuitira kuti usaburitse murairo wekudzima data. Zvese izvi zvakabatsira kuderedza njodzi yekukanganisa.


Takatangazve sevhisi yedu, zvese zvakashanda sezvazvaifanira, vashandisi vakaramba vachishanda, asi pamagirafu takaona mutoro wakakwira zvakanyanya pamaseva eConsul.


Sei tisina kuona izvi mumamiriro ekuedzwa? Iri dambudziko rinonyatso ratidza kuti zvinodikanwa kutevedzera Infrastructure semusimboti wekodhi uye kuvandudza hurongwa hwese, kubva munzvimbo dzekuyedza kusvika pakugadzira. Zvikasadaro, zviri nyore kwazvo kuwana dambudziko ratakawana. Chii chaitika? Consul yakatanga kuoneka mukugadzira, uyezve munzvimbo dzekuyedzwa; semhedzisiro, munzvimbo dzekuyedzwa shanduro yeConsul yaive yakakwira kupfuura mukugadzirwa. Ingori mune imwe yekuburitswa, CPU inodonha kana uchishanda neconsul-template yakagadziriswa. Saka isu takangogadzirisa Consul, nekudaro kugadzirisa dambudziko.

Tangazve Patroni cluster

Zvisinei, takava nedambudziko idzva ratakanga tisina kumbofungira nezvaro. Kana tichivandudza Consul, tinongobvisa Consul node kubva kuboka tichishandisa consul siya murairo → Patroni inobatanidza kune imwe Consul server → zvose zvinoshanda. Asi patakasvika pachiitiko chekupedzisira cheConsul cluster ndokuitumira iyo consul leave command, ese Patroni masumbu akatangazve, uye mumatanda takaona chikanganiso chinotevera:

ERROR: get_cluster
Traceback (most recent call last):
...
RetryFailedError: 'Exceeded retry deadline'
ERROR: Error communicating with DCS
<b>LOG: database system is shut down</b>

Iyo Patroni cluster haina kukwanisa kuwana ruzivo nezve cluster yayo uye yakatangazve.

Kuti tiwane mhinduro, takabata vanyori vePatroni kuburikidza nenyaya pa github. Vakakurudzira kuvandudzwa kwemafaira edu ekugadzirisa:

consul:
 consul.checks: []
bootstrap:
 dcs:
   retry_timeout: 8

Isu takakwanisa kudzokorora nyaya munzvimbo yekuyedza uye tikaedza marongero aya ipapo, asi zvinosuwisa haana kushanda.

Dambudziko richiri risati ragadziriswa. Isu tinoronga kuedza zvinotevera mhinduro:

• Shandisa Consul-agent pane imwe neimwe chiitiko chePatroni cluster;
• Gadzirisa dambudziko mukodhi.

Isu tinonzwisisa pakaitika chikanganiso: pamwe dambudziko riri pakushandisa iyo default timeout, iyo isina kupfuudzwa kuburikidza nefaira rekugadzirisa. Kana iyo yekupedzisira Consul server yabviswa kubva musumbu, iyo yese Consul cluster inotonhora, iyo inotora nguva yakareba kupfuura sekondi; nekuda kweizvi, Patroni haakwanise kuwana iyo mamiriro esumbu uye anotangazve sumbu rese.

Sezvineiwo, hatina kuzosangana zvakare nekukanganisa.

Migumisiro yekushandisa Patroni

Mushure mekubudirira kuvhurwa kwePatroni, isu takawedzera imwe replica musumbu rega rega. Ikozvino cluster yega yega ine ssemblance yequorum: mutungamiri mumwechete uye maviri replicas, kudzivirira kubva pakupatsanurwa-uropi paunenge uchichinja.


Patroni anga achishanda mukugadzira kweanopfuura mwedzi mitatu. Panguva iyi, akatokwanisa kutibatsira. Munguva ichangopfuura, mutungamiri weimwe yemasumbu akafa muAWS, iyo automatic failover yakashanda uye vashandisi vakaramba vachishanda. Patroni akazadzisa basa rayo guru.

Pfupiso pfupi yekushandisa Patroni:

• Kureruka kwekuchinja kwekugadzirisa. Zvakakwana kushandura gadziriro pane imwe nguva uye ichashanda kune yese cluster. Kana reboot ichidikanwa kuti uise iyo nyowani gadziriso, Patroni achakuzivisa iwe nezve izvi. Patroni anogona kutangazve sumbu rese nekuraira mumwechete, izvo zvakare zviri nyore kwazvo.
• Iyo otomatiki failover iri kushanda uye yakatotibatsira kunze.
• Kugadziridza PostgreSQL pasina application downtime. Iwe unofanirwa kutanga wagadziridza replicas kune iyo nyowani vhezheni, wozoshandura mutungamiriri muPatroni cluster uye kugadzirisa mutungamiri wekare. Muchiitiko ichi, kuyedzwa kwakakodzera kweotomatiki failover kunoitika.

Source: www.habr.com