M'nkhaniyi ndikuuzani momwe tinayendera nkhani ya PostgreSQL zolakwika kulolerana, chifukwa chake zidakhala zofunika kwa ife, ndi zomwe zinachitika pamapeto pake.
Tili ndi ntchito yodzaza kwambiri: Ogwiritsa ntchito 2,5 miliyoni padziko lonse lapansi, ogwiritsa ntchito 50K+ tsiku lililonse. Ma seva ali ku Amazone m'chigawo chimodzi cha Ireland: 100+ ma seva osiyanasiyana akugwira ntchito nthawi zonse, pafupifupi 50 mwa iwo omwe ali ndi database.
Kumbuyo konseko ndi pulogalamu yayikulu ya Java ya monolithic yomwe imasunga kulumikizana kosalekeza kwa websocket ndi kasitomala. Pamene ogwiritsa ntchito angapo amagwira ntchito nthawi imodzi pa bolodi lomwelo, onse amawona kusintha mu nthawi yeniyeni, chifukwa timalemba kusintha kulikonse mu database. Tili ndi zopempha pafupifupi 10K pa sekondi imodzi pazosungidwa zathu. Pachimake ku Redis timalemba zopempha 80-100K pamphindikati.
Chifukwa chomwe tidasinthira kuchoka ku Redis kupita ku PostgreSQL
Poyambirira, ntchito yathu inkagwira ntchito ndi Redis, chosungira chamtengo wapatali chomwe chimasunga zonse mu RAM ya seva.
Ubwino wa Redis:
- Kuthamanga kwakukulu, chifukwa zonse zimasungidwa mu kukumbukira;
- Kusunga kosavuta ndi kubwereza.
Zoyipa za Redis kwa ife:
- Palibe zochitika zenizeni. Tinayesetsa kuwatsanzira pamlingo wofunsira. Tsoka ilo, izi sizinagwire ntchito bwino nthawi zonse ndipo zimafunikira kulemba ma code ovuta kwambiri.
- Kuchuluka kwa deta kumachepetsedwa ndi kuchuluka kwa kukumbukira. Pamene kuchuluka kwa deta kumawonjezeka, kukumbukira kumakula, ndipo, pamapeto pake, tidzathamangira muzochita zosankhidwa, zomwe mu AWS zimafuna kuyimitsa ntchito yathu kuti tisinthe mtundu wa chitsanzo.
- Ndikofunika kuti nthawi zonse mukhale ndi latency level, chifukwa Tili ndi zopempha zambiri. Mulingo woyenera kwambiri wa latency kwa ife ndi 17-20 ms. Pamlingo wa 30-40 ms, timapeza mayankho ataliatali ku zopempha zathu ndikuwonongeka kwa ntchito. Tsoka ilo, izi zidatichitikira mu Seputembara 2018, pomwe imodzi mwazochitika za Redis pazifukwa zina idalandira latency yomwe inali yokwera 2 kuposa nthawi zonse. Kuti tithane ndi vutoli, tidayimitsa ntchitoyo mkati mwa tsiku logwira ntchito kuti tikonze zomwe sizinachitike ndikulowa m'malo mwa zovuta za Redis.
- Ndikosavuta kupeza deta yosagwirizana ngakhale ndi zolakwika zazing'ono mu code ndikuthera nthawi yambiri ndikulemba code kuti mukonze detayo.
Tinaganizira za zovutazo ndipo tinazindikira kuti tifunika kusamukira ku chinthu china chosavuta, ndi zochitika zachilendo komanso osadalira kwambiri latency. Tinachita kafukufuku wathu, kusanthula njira zambiri ndikusankha PostgreSQL.
Takhala tikusamukira ku database yatsopano kwa zaka 1,5 tsopano ndipo tangosamutsa gawo laling'ono la deta, kotero tsopano tikugwira ntchito imodzi ndi Redis ndi PostgreSQL. Zambiri zokhudzana ndi magawo osuntha ndikusintha deta pakati pa nkhokwe zalembedwamo .
Titayamba kusuntha, pulogalamu yathu idagwira ntchito mwachindunji ndi nkhokwe ndikufikira Redis ndi PostgreSQL master. Gulu la PostgreSQL linali ndi mbuye ndi chofananira chokhala ndi kubwereza kosasinthika. Izi ndi zomwe mayendedwe a database amawoneka ngati:
Kukhazikitsa PgBouncer
Pamene tinali kusuntha, mankhwalawa anali akukulanso: chiwerengero cha ogwiritsa ntchito ndi chiwerengero cha ma seva omwe ankagwira ntchito ndi PostgreSQL chinawonjezeka, ndipo tinayamba kutha kugwirizana. PostgreSQL imapanga njira yosiyana yolumikizirana iliyonse ndikugwiritsa ntchito zinthu. Mutha kuwonjezera kuchuluka kwa maulumikizidwe mpaka pamlingo wina, apo ayi pali mwayi woti database siyigwira ntchito bwino. Njira yabwino muzochitika zotere ingakhale kusankha woyang'anira kugwirizana yemwe adzayime kutsogolo kwa database.
Tidali ndi zosankha ziwiri za manejala wolumikizira: Pgpool ndi PgBouncer. Koma yoyamba sichigwirizana ndi njira yogwirira ntchito ndi nkhokwe, choncho tinasankha PgBouncer.
Takonza chiwembu chotsatirachi: ntchito yathu imapeza PgBouncer imodzi, kumbuyo komwe kuli masters a PostgreSQL, ndipo kumbuyo kwa mbuye aliyense pali chofanana chimodzi chokhala ndi kubwereza kosagwirizana.
Panthawi imodzimodziyo, sitinathe kusunga chiwerengero chonse cha deta mu PostgreSQL ndipo liwiro logwira ntchito ndi database linali lofunika kwa ife, choncho tinayamba kugawana PostgreSQL pa mlingo wa ntchito. Chiwembu chomwe chafotokozedwa pamwambapa ndichosavuta kwa izi: powonjezera shard yatsopano ya PostgreSQL, ndikokwanira kusinthira kasinthidwe ka PgBouncer ndipo pulogalamuyo imatha kugwira ntchito ndi shard yatsopanoyo.
PgBouncer Fault Tolerance
Chiwembuchi chinagwira ntchito mpaka chitsanzo chokha cha PgBouncer chinamwalira. Tili mu AWS, pomwe zochitika zonse zimayambitsidwa pa hardware yomwe imafa nthawi ndi nthawi. Zikatero, chitsanzocho chimangosunthira ku hardware yatsopano ndikugwiranso ntchito. Izi zidachitika ndi PgBouncer, koma sizinapezeke. Chotsatira cha ngoziyi chinali chakuti ntchito yathu sinapezeke kwa mphindi 25. Pazifukwa zotere, AWS imalimbikitsa kugwiritsa ntchito redundancy kumbali ya ogwiritsa ntchito, zomwe sitinagwiritse ntchito panthawiyo.
Pambuyo pake, tidaganizira mozama za kulolerana kolakwika kwa magulu a PgBouncer ndi PostgreSQL, chifukwa zomwezi zitha kuchitikanso ndi zochitika zilizonse mu akaunti yathu ya AWS.
Tinapanga dongosolo lololera zolakwa za PgBouncer motere: ma seva onse ogwiritsira ntchito amapeza Network Load Balancer, kumbuyo komwe kuli ma PgBouncers awiri. Iliyonse ya PgBouncers imayang'ana mbuye yemweyo wa PostgreSQL wa shard iliyonse. Ngati vuto la ngozi ya AWS likubwereza, magalimoto onse amatumizidwa kudzera pa PgBouncer ina. Kulekerera zolakwika za Network Load Balancer kumaperekedwa ndi AWS.
Chiwembuchi chimakupatsani mwayi wowonjezera ma seva atsopano a PgBouncer mosavuta.
Kupanga PostgreSQL Failover Cluster
Pothetsa vutoli, tidaganizira njira zosiyanasiyana: zolembera zokha, repmgr, AWS RDS, Patroni.
Zolemba zodzilemba zokha
Atha kuyang'anira ntchito ya mbuyeyo ndipo, ngati italephera, limbikitsani chithunzicho kwa mbuye ndikusintha kasinthidwe ka PgBouncer.
Ubwino wa njirayi ndi kuphweka kwakukulu, chifukwa mumalemba zolemba nokha ndikumvetsetsa momwe zimagwirira ntchito.
Wotsatsa:
- Mbuyeyo mwina sanafe, m'malo mwake, pakhoza kukhala vuto la intaneti. Failover, osadziwa izi, adzalimbikitsa chofanizira kwa mbuye, ndipo mbuye wakale adzapitiriza kugwira ntchito. Zotsatira zake, tipeza ma seva awiri paudindo waukulu ndipo sitidziwa kuti ndi ndani mwa iwo omwe ali ndi deta yaposachedwa. Izi zimatchedwanso kugawanika-ubongo;
- Tinatsala opanda choyankha. Mu kasinthidwe kwathu pali master ndi chofananira chimodzi, mutasintha chofananiracho chimakwezedwa kukhala mbuye ndipo tilibenso zofananira, chifukwa chake tiyenera kuwonjezera pamanja chofananira chatsopano;
- Tikufuna kuwunika kowonjezereka kwa ntchito ya failover, ndipo tili ndi 12 PostgreSQL shards, zomwe zikutanthauza kuti tiyenera kuyang'anira masango 12. Mukamawonjezera kuchuluka kwa shards, muyenera kukumbukiranso kusintha fayilo ya failover.
Wolephera wodzilemba yekha akuwoneka wovuta kwambiri ndipo amafuna chithandizo chopanda pake. Ndi gulu limodzi la PostgreSQL, iyi ikhala njira yosavuta kwambiri, koma siyikula, chifukwa chake siyoyenera kwa ife.
Repmgr
Replication Manager wamagulu a PostgreSQL, omwe amatha kuyang'anira magwiridwe antchito a gulu la PostgreSQL. Nthawi yomweyo, ilibe kulephera kwadzidzidzi m'bokosi, kotero kuti mugwire ntchito muyenera kulemba "wrapper" yanu pamwamba pa yankho lomwe lapangidwa kale. Chifukwa chake zonse zitha kukhala zovuta kwambiri kuposa zolemba zodzilemba zokha, ndichifukwa chake sitinayeserenso Repmgr.
AWS RDS
Imathandizira chilichonse chomwe timafunikira, imatha kupanga zosunga zobwezeretsera ndikuthandizira maulumikizidwe ambiri. Imakhala ndi kusintha kwadzidzidzi: mbuyeyo akamwalira, chofananacho chimakhala mbuye watsopano, ndipo AWS imasintha mbiri ya DNS kukhala mbuye watsopano, pamene zojambulazo zikhoza kukhala mu AZs zosiyanasiyana.
Zoyipa zake ndi kusowa kwa zoikamo zabwino. Monga chitsanzo cha kukonza bwino: zochitika zathu zili ndi zoletsa zolumikizira za tcp, zomwe, mwatsoka, sizingachitike mu RDS:
net.ipv4.tcp_keepalive_time=10
net.ipv4.tcp_keepalive_intvl=1
net.ipv4.tcp_keepalive_probes=5
net.ipv4.tcp_retries2=3
Kuphatikiza apo, AWS RDS ndiyokwera mtengo pafupifupi kawiri kuposa mtengo wanthawi zonse, chomwe chinali chifukwa chachikulu chosiyira yankholi.
Mtsogoleri
Ichi ndi template ya python yoyang'anira PostgreSQL yokhala ndi zolembedwa zabwino, zolephera zokha komanso ma code source pa github.
Ubwino wa Patroni:
- Chigawo chilichonse cha kasinthidwe chikufotokozedwa, zikuwonekeratu momwe zimagwirira ntchito;
- Automatic failover imagwira ntchito kunja kwa bokosi;
- Zolembedwa mu python, ndipo popeza ife tokha timalemba zambiri mu python, zidzakhala zosavuta kuti tithane ndi mavuto ndipo, mwinamwake, ngakhale kuthandizira chitukuko cha polojekitiyi;
- Kuwongolera kwathunthu PostgreSQL, kukulolani kuti musinthe kasinthidwe pamagulu onse a masango nthawi imodzi, ndipo ngati kugwiritsa ntchito kusintha kwatsopano kumafuna kuyambiranso kwa gululo, izi zikhoza kuchitika kachiwiri pogwiritsa ntchito Patroni.
Wotsatsa:
- Sizikudziwika bwino pazolembedwa momwe mungagwirire ntchito moyenera ndi PgBouncer. Ngakhale kuti ndizovuta kutchula izi kuti ndizochepa, chifukwa ntchito ya Patroni ndiyo kuyang'anira PostgreSQL, ndi momwe kugwirizana kwa Patroni kudzagwirira ntchito kale vuto lathu;
- Pali zitsanzo zochepa za kukhazikitsidwa kwa Patroni pamiyeso yayikulu, pomwe pali zitsanzo zambiri zoyendetsera kuyambira pachiyambi.
Zotsatira zake, tinasankha Patroni kuti apange gulu la failover.
Njira yoyendetsera Patroni
Pamaso pa Patroni, tinali ndi ma shards 12 a PostgreSQL mumbuye m'modzi komanso kasinthidwe kamodzi kofanana ndi kubwereza kosagwirizana. Ma seva ogwiritsira ntchito adapeza ma database kudzera pa Network Load Balancer, kumbuyo komwe kunali zochitika ziwiri ndi PgBouncer, ndipo kumbuyo kwawo kunali ma seva onse a PostgreSQL.
Kuti tigwiritse ntchito Patroni, tinkafunika kusankha zosungirako zogawa zamagulu. Patroni amagwira ntchito ndi makina osungirako ogawa monga etcd, Zookeeper, Consul. Tili ndi gulu lathunthu la Consul pakupanga, lomwe limagwira ntchito limodzi ndi Vault ndipo sitiligwiritsanso ntchito. Chifukwa chachikulu choyambira kugwiritsa ntchito Consul pazolinga zake.
Momwe Patroni amagwirira ntchito ndi Consul
Tili ndi gulu la Consul, lomwe lili ndi mfundo zitatu, ndi gulu la Patroni, lomwe lili ndi mtsogoleri ndi chojambula (mu Patroni, mbuyeyo amatchedwa mtsogoleri wamagulu, ndipo akapolo amatchedwa replicas). Chitsanzo chilichonse chamagulu a Patroni nthawi zonse chimatumiza zambiri za momwe gululi lilili kwa Consul. Chifukwa chake, kuchokera kwa Consul mutha kudziwa nthawi zonse masanjidwe apano a gulu la Patroni komanso yemwe ali mtsogoleri pakadali pano.
Kuti mulumikizane ndi Patroni ku Consul, ingophunzirani zolembedwa zovomerezeka, zomwe zimati muyenera kufotokozera wolandirayo mumtundu wa http kapena https, kutengera momwe timagwirira ntchito ndi Consul, ndi chithunzi cholumikizira, mwakufuna:
host: the host:port for the Consul endpoint, in format: http(s)://host:port
scheme: (optional) http or https, defaults to httpZikuwoneka zosavuta, koma apa ndipamene misampha imayambira. Ndi Consul timagwira ntchito yolumikizana motetezeka kudzera pa https ndipo kasinthidwe kathu kakuwoneka motere:
consul:
host: https://server.production.consul:8080
verify: true
cacert: {{ consul_cacert }}
cert: {{ consul_cert }}
key: {{ consul_key }}Koma sizimagwira ntchito mwanjira imeneyo. Poyambira, Patroni sangathe kulumikiza ku Consul chifukwa amayesabe kudzera pa http.
Khodi yochokera ku Patroni idathandizira kuthetsa vutoli. Ndibwino kuti zalembedwa mu python. Zikuoneka kuti parameter ya wolandirayo siinasinthidwe mwanjira iliyonse, ndipo protocol iyenera kufotokozedwa mu chiwembu. Umu ndi momwe chipilala chogwirira ntchito chogwirira ntchito ndi Consul chimawonekera kwa ife:
consul:
host: server.production.consul:8080
scheme: https
verify: true
cacert: {{ consul_cacert }}
cert: {{ consul_cert }}
key: {{ consul_key }}Consul-template
Kotero, tasankha kusungirako kasinthidwe. Tsopano tikuyenera kumvetsetsa momwe PgBouncer angasinthire masinthidwe ake pamene mtsogoleri asintha mu gulu la Patroni. Palibe yankho ku funso ili muzolemba, chifukwa ... Kwenikweni, kugwira ntchito ndi PgBouncer sikunafotokozedwe pamenepo.
Pofufuza yankho, tinapeza nkhani (mwatsoka, sindikukumbukira dzina), kumene kunalembedwa kuti Consul-template inali yothandiza kwambiri pophatikiza PgBouncer ndi Patroni. Izi zidatipangitsa kuphunzira ntchito ya Consul-template.
Zinapezeka kuti Consul-template nthawi zonse imayang'anira kasinthidwe kwa gulu la PostgreSQL mu Consul. Mtsogoleri akasintha, amasintha kasinthidwe ka PgBouncer ndikutumiza lamulo kuti muyikenso.
Ubwino waukulu wa template ndikuti umasungidwa ngati kachidindo, kotero powonjezera shard yatsopano, ndikokwanira kupanga chopereka chatsopano ndikusinthira template yokha, ndikuchirikiza Infrastructure monga code code.
Zomangamanga zatsopano ndi Patroni
Chifukwa chake, tidalandira dongosolo lotsatirali la ntchito:
Ma seva onse ogwiritsira ntchito amapeza balancer β kuseri kwake pali zochitika ziwiri za PgBouncer β nthawi iliyonse Consul-template ikugwira ntchito, yomwe imayang'anira chikhalidwe cha gulu lililonse la Patroni ndikuyang'anira kufunikira kwa PgBouncer config, yomwe imatsogolera zopempha kwa mtsogoleri wamakono. wa gulu lililonse.
Kuyesa pamanja
Tisanayambitse kupanga, tidayambitsa chiwembuchi pamayeso ang'onoang'ono ndikuwunika momwe zimasinthira zokha. Iwo anatsegula bolodi, anasuntha zomata ndipo panthawiyo "anapha" mtsogoleri wa gululo. Mu AWS, zomwe muyenera kuchita ndikuzimitsa chitsanzocho kudzera pa console.
Chomata chinabweranso mkati mwa masekondi 10-20, kenako chinayambanso kuyenda bwino. Izi zikutanthauza kuti gulu la Patroni linagwira ntchito bwino: linasintha mtsogoleri, linatumiza chidziwitso kwa Consul, ndipo Consul-template nthawi yomweyo inatenga chidziwitsochi, m'malo mwa PgBouncer kasinthidwe ndikutumiza lamulo loti liyikenso.
Kodi mungapulumuke bwanji ndikulemedwa kwambiri ndikukhalabe ndi nthawi yochepa?
Zonse zimagwira ntchito mwangwiro! Koma mafunso atsopano amabuka: Zidzagwira ntchito bwanji pansi pa katundu wambiri? Momwe mungatulutsire mwachangu komanso mosamala zonse zomwe zikupanga?
Malo oyesera omwe timayesapo katundu amatithandiza kuyankha funso loyamba. Ndizofanana kwathunthu ndi kapangidwe kazomangamanga ndipo zapanga data yoyeserera, yomwe ili pafupifupi yofanana ndi kuchuluka kwa kupanga. Tasankha "kupha" m'modzi mwa ambuye a PostgreSQL panthawi ya mayeso ndikuwona zomwe zimachitika. Koma izi zisanachitike, ndikofunikira kuyang'ana kutulutsa kodziwikiratu, chifukwa pamalo ano tili ndi ma shards angapo a PostgreSQL, chifukwa chake tipeza kuyesedwa kwabwino kwa zolembedwa zosinthidwa tisanapangidwe.
Ntchito zonsezi zimawoneka zolakalaka, koma tili ndi PostgreSQL 9.6. Mwina titha kusintha mpaka 11.2 nthawi yomweyo?
Tasankha kuchita izi mu magawo awiri: choyamba sinthani mtunduwo kukhala 2, kenako ndikuyambitsa Patroni.
Kusintha kwa PostgreSQL
Kuti musinthe mwachangu mtundu wa PostgreSQL, muyenera kugwiritsa ntchito njirayi -k, momwe cholumikizira cholimba chimapangidwa pa diski ndipo palibe chifukwa chokopera deta yanu. Pama database a 300-400 GB, zosinthazi zimatenga sekondi imodzi.
Tili ndi shards zambiri, kotero zosinthazo ziyenera kuchitidwa zokha. Kuti tichite izi, tidalemba buku lamasewera la Ansible lomwe limatipangira zonse zosinthira:
/usr/lib/postgresql/11/bin/pg_upgrade
<b>--link </b>
--old-datadir='' --new-datadir=''
--old-bindir='' --new-bindir=''
--old-options=' -c config_file='
--new-options=' -c config_file='Ndikofunika kuzindikira apa kuti musanayambe kukweza, muyenera kuchita ndi parameter -- onanikuonetsetsa kuti kukweza ndi kotheka. Script yathu imalowetsanso ma configs panthawi yokweza. Zolemba zathu zidatha mumasekondi 30, zomwe ndi zotsatira zabwino kwambiri.
Kukhazikitsidwa kwa Patroni
Kuti muthetse vuto lachiwiri, ingoyang'anani kasinthidwe ka Patroni. Malo ovomerezeka ali ndi chitsanzo cha kasinthidwe ndi initdb, yomwe ili ndi udindo woyambitsa nkhokwe yatsopano pamene Patroni akhazikitsidwa koyamba. Koma popeza tili ndi database yopangidwa kale, tidangochotsa gawo ili pakukonzekera.
Pamene tidayamba kukhazikitsa Patroni pamagulu okonzeka a PostgreSQL ndikuyambitsa, tidakumana ndi vuto latsopano: ma seva onsewa adakhazikitsidwa ngati mtsogoleri. Patroni sadziwa kalikonse za momwe gululi linayambira ndipo amayesa kuyendetsa ma seva onse ngati magulu awiri osiyana omwe ali ndi dzina lomwelo. Kuti muthetse vutoli, muyenera kuchotsa chikwatu cha data pa kapolo:
rm -rf /var/lib/postgresql/Izi ziyenera kuchitidwa pa kapolo yekha!
Pogwirizanitsa chojambula choyera, Patroni amapanga mtsogoleri wotsalira ndikubwezeretsanso ku chithunzicho, kenako akugwirana ndi zomwe zikuchitika pogwiritsa ntchito zipika za khoma.
Vuto lina lomwe tidakumana nalo ndikuti magulu onse a PostgreSQL amatchedwa main by default. Pamene gulu lirilonse silidziwa kanthu za linzake, izi ndi zachilendo. Koma mukafuna kugwiritsa ntchito Patroni, masango onse ayenera kukhala ndi dzina lapadera. Yankho lake ndikusintha dzina lamagulu mu kasinthidwe ka PostgreSQL.
Katundu mayeso
Tidayambitsa mayeso omwe amatengera momwe ogwiritsa ntchito amagwirira ntchito pama board. Pamene katunduyo adafika pafupifupi tsiku lililonse, tidabwereza mayeso omwewo, tidazimitsa nthawi imodzi ndi mtsogoleri PostgreSQL. Kulephera kodziwikiratu kunagwira ntchito monga momwe timayembekezera: Patroni adasintha mtsogoleri, Consul-template adasintha kasinthidwe ka PgBouncer ndikutumiza lamulo kuti mutsegulenso. Malingana ndi ma graph athu ku Grafana, zinali zoonekeratu kuti panali kuchedwa kwa masekondi 20-30 ndi zolakwika zochepa kuchokera ku maseva okhudzana ndi kugwirizana kwa deta. Izi ndizabwinobwino, zikhalidwe zotere ndizovomerezeka kulephera kwathu ndipo ndizabwinoko kuposa nthawi yopumira.
Kukhazikitsidwa kwa Patroni mukupanga
Chifukwa chake, tinapanga dongosolo ili:
- Ikani Consul-template ku seva ya PgBouncer ndikuyambitsa;
- Zosintha za PostgreSQL ku mtundu wa 11.2;
- Kusintha dzina lamagulu;
- Kukhazikitsidwa kwa gulu la Patroni.
Nthawi yomweyo, chiwembu chathu chimatilola kuchita mfundo yoyamba pafupifupi nthawi iliyonse; titha kuchotsa PgBouncer iliyonse kuntchito imodzi ndi imodzi ndikutumiza ndikukhazikitsa template ya consul. Ndi zomwe tinachita.
Kuti tiyese mwamsanga, tinagwiritsa ntchito Ansible, popeza tinali titayesa kale buku lonse lamasewera pa malo oyesera, ndipo nthawi yopangira zolemba zonse inali kuchokera ku 1,5 mpaka maminiti a 2 pa shard iliyonse. Titha kutulutsa chilichonse chimodzi ndi chimodzi ku shard iliyonse osayimitsa ntchito yathu, koma tiyenera kuzimitsa PostgreSQL iliyonse kwa mphindi zingapo. Pankhaniyi, ogwiritsa ntchito omwe deta yawo ili pa shard iyi sangathe kugwira ntchito mokwanira panthawiyi, ndipo izi ndizosavomerezeka kwa ife.
Njira yothetsera vutoli idakonzedwa kukonza, zomwe timachita miyezi itatu iliyonse. Ili ndi zenera la ntchito zomwe zakonzedwa, tikathimitsa ntchito yathu ndikusintha ma database. Panatsala sabata imodzi kuti ifike zenera lotsatira, ndipo tinaganiza zongodikira ndikukonzekera zina. Pakudikirira, tidawonjezeranso kubetcha kwathu: pa shard iliyonse ya PostgreSQL tidakweza choyimira chotsalira ngati chalephera, kuti tisunge zomwe zachitika posachedwa, ndikuwonjezera chitsanzo chatsopano pa shard iliyonse, yomwe iyenera kukhala chithunzi chatsopano mu Patroni. cluster, kuti musapereke lamulo lochotsa deta. Zonsezi zinathandiza kuchepetsa chiopsezo cha zolakwika.
Tinayambiranso ntchito yathu, zonse zinagwira ntchito monga momwe ziyenera kukhalira, ogwiritsa ntchito anapitirizabe kugwira ntchito, koma pazithunzi tidawona katundu wochuluka kwambiri pa ma seva a Consul.
Chifukwa chiyani sitinawone izi m'malo oyeserera? Vutoli likuwonetsa bwino kwambiri kuti ndikofunikira kutsata Infrastructure ngati mfundo ndikuwongolera zida zonse, kuyambira malo oyesera mpaka kupanga. Kupanda kutero, ndikosavuta kupeza vuto lomwe tili nalo. Chinachitika ndi chiyani? Consul adawonekera koyamba pakupanga, kenako m'malo oyesera; zotsatira zake, m'malo oyesera mtundu wa Consul unali wapamwamba kuposa momwe amapangira. Mu imodzi mwazotulutsa, kutayikira kwa CPU pogwira ntchito ndi consul-template kunathetsedwa. Chifukwa chake tidangosintha a Consul, ndikuthetsa vutoli.
Yambitsaninso gulu la Patroni
Komabe, tinapeza vuto latsopano limene sitinali kulikayikira nβkomwe. Pamene tikukonzekera Consul, timangochotsa mfundo ya Consul kuchokera ku tsango pogwiritsa ntchito lamulo la kuchoka kwa consul β Patroni akugwirizanitsa ndi seva ina ya Consul β chirichonse chimagwira ntchito. Koma titafika kumapeto kwa gulu la Consul ndikutumiza lamulo la consul leave, magulu onse a Patroni adangoyambiranso, ndipo m'zipika tidawona cholakwika chotsatirachi:
ERROR: get_cluster
Traceback (most recent call last):
...
RetryFailedError: 'Exceeded retry deadline'
ERROR: Error communicating with DCS
<b>LOG: database system is shut down</b>Gulu la Patroni silinathe kupeza zambiri za gulu lake ndikuyambiranso.
Kuti tipeze yankho, tinalumikizana ndi olemba a Patroni kudzera pa nkhani pa github. Adapereka malingaliro owongolera mafayilo athu osinthira:
consul:
consul.checks: []
bootstrap:
dcs:
retry_timeout: 8Tinatha kubwerezanso nkhaniyi m'malo oyesera ndikuyesa zosinthazi pamenepo, koma mwatsoka sizinagwire ntchito.
Vutoli silinathe. Tikukonzekera kuyesa njira zotsatirazi:
- Gwiritsani ntchito Consul-agent pazochitika zilizonse za gulu la Patroni;
- Konzani vuto mu code.
Timamvetsetsa pomwe cholakwikacho chidachitika: mwina vuto liri pakugwiritsa ntchito nthawi yokhazikika, yomwe siyikupitilira mufayilo yosinthira. Seva yomaliza ya Consul ikachotsedwa pagululo, gulu lonse la Consul limaundana, lomwe limatenga nthawi yayitali kuposa sekondi imodzi; chifukwa cha izi, Patroni sangathe kupeza chikhalidwe cha gululo ndikuyambitsanso gulu lonselo.
Mwamwayi, sitinakumanenso ndi zolakwika.
Zotsatira zogwiritsa ntchito Patroni
Pambuyo pa kukhazikitsidwa bwino kwa Patroni, tidawonjezeranso chofananira mgulu lililonse. Tsopano gulu lirilonse limakhala ndi mawonekedwe a quorum: mtsogoleri m'modzi ndi zofananira ziwiri, kuti ateteze ku ubongo wogawanika mukasintha.
Patroni wakhala akugwira ntchito yopanga kwa miyezi yoposa itatu. Panthawi imeneyi, iye anakwanitsa kale kutithandiza. Posachedwapa, mtsogoleri wa gulu limodzi adamwalira mu AWS, kulephera kwadzidzidzi kunagwira ntchito ndipo ogwiritsa ntchito anapitirizabe kugwira ntchito. Patroni wakwaniritsa ntchito yake yaikulu.
Chidule chachidule chogwiritsa ntchito Patroni:
- Kusavuta kusintha kasinthidwe. Ndikokwanira kusintha kasinthidwe pa nthawi imodzi ndipo idzagwira ntchito ku gulu lonse. Ngati kuyambiransoko kukufunika kuti mugwiritse ntchito kasinthidwe kwatsopano, Patroni adzakudziwitsani za izi. Patroni akhoza kuyambitsanso gulu lonse ndi lamulo limodzi, lomwe limakhalanso lothandiza kwambiri.
- The automatic failover ikugwira ntchito ndipo yatithandiza kale.
- Kusintha PostgreSQL popanda kugwiritsa ntchito nthawi. Choyamba muyenera kusintha zofananira ku mtundu watsopano, kenako sinthani mtsogoleri mu gulu la Patroni ndikusintha mtsogoleri wakale. Pankhaniyi, kuyezetsa koyenera kwa failover yodziwikiratu kumachitika.
Source: www.habr.com