Ukwandisa, abathengi bafumana ezi zicelo zilandelayo: "Sifuna njenge-Amazon RDS, kodwa ingabizi"; "Siyifuna njenge-RDS, kodwa kuyo yonke indawo, kuyo nayiphi na isiseko." Ukuphumeza isisombululo esinjalo esilawulwayo kwi-Kubernetes, sijonge imeko yangoku yabaqhubi abadumileyo be-PostgreSQL (i-Stolon, abaqhubi abavela kwi-Crunchy Data kunye ne-Zalando) kwaye senze ukhetho lwethu.
Eli nqaku ngamava esiwafumene kokubini kwimbono yethiyori (uphononongo lwezisombululo) kwaye ukusuka kwicala elisebenzayo (yintoni ekhethiweyo kunye nento eyafika kuyo). Kodwa kuqala, makhe sijonge ukuba zeziphi iimfuno jikelele zokutshintshwa okunokubakho kwe-RDS...
Yintoni i-RDS
Xa abantu bethetha nge-RDS, kumava ethu, bathetha inkonzo elawulwayo ye-DBMS ethi:
- kulula ukuyiqwalasela;
- inamandla okusebenza ngeesnapshots kwaye iphinde ibuyiselwe kuzo (ngokukhethekileyo ngenkxaso );
- ikuvumela ukuba wenze i-topology ye-master-slave;
- inoluhlu olutyebileyo lwezandiso;
- ibonelela ngophicotho-zincwadi kunye nolawulo lomsebenzisi/ufikelelo.
Ngokuqhelekileyo, iindlela zokuphumeza umsebenzi okhoyo zinokwahluka kakhulu, kodwa indlela enemeko enokuthi i-Ansible ayisondeli kuthi. (Abalingane abavela kwi-2GIS bafikelela kwisigqibo esifanayo njengesiphumo yenza "isixhobo sokuthumela ngokukhawuleza i-postgres-based failover cluster.")
Abaqhubi bayindlela eqhelekileyo yokusombulula iingxaki ezifanayo kwi-ecosystem ye-Kubernetes. Umlawuli wezobugcisa we "Flanta" sele ethethe ngokubanzi malunga nabo ngokumalunga nedatha eqaliswe ngaphakathi kweKubernetes. ngaphakathi .
NB: Ukudala ngokukhawuleza abaqhubi abalula, sicebisa ukuba sinikele ingqalelo kuMthombo ovulekileyo oluncedo . Ukuyisebenzisa, unokwenza oku ngaphandle kolwazi lwe-Go, kodwa ngeendlela eziqhelekileyo kubalawuli benkqubo: eBash, Python, njl.
Kukho uninzi lwabasebenzisi be-K8s abadumileyo bePostgreSQL:
- Stolon;
- I-Crunchy Data PostgreSQL Operator;
- Zalando Postgres Operator.
Masizijonge ngakumbi.
Ukhetho lomsebenzisi
Ukongeza kwizinto ezibalulekileyo esele zikhankanyiwe ngasentla, thina - njengeenjineli zokusebenza kweziseko ezingundoqo zeKubernetes - sikwalindele oku kulandelayo kubasebenzi:
- ukuthunyelwa ukusuka kwiGit kunye ne ;
- inkxaso ye-pod anti-affinity;
- ukufaka i-node affinity okanye umkhethi we-node;
- ukufakwa kokunyamezela;
- ukufumaneka kobuchule bokushuna;
- itekhnoloji eqondakalayo kunye nemiyalelo.
Ngaphandle kokungena kwiinkcukacha kwinqaku ngalinye (buza kumagqabantshintshi ukuba usenemibuzo malunga nabo emva kokufunda lonke inqaku), ndiya kuqaphela ngokubanzi ukuba ezi parameters ziyafuneka ukuchaza ngokuchanekileyo ngakumbi ubungcali beendawo zeqela ukuze ziodole kwizicelo ezithile. Ngale ndlela sinokufikelela kwibhalansi efanelekileyo malunga nokusebenza kunye neendleko.
Ngoku masiqhubele phambili kubasebenzisi bePostgreSQL ngokwabo.
1. Stolon
evela kwinkampani yaseItali iSorint.lab kwi yathathwa njengomgangatho womgangatho phakathi kwabaqhubi be-DBMS. Le yiprojekthi endala: ukukhululwa kwayo okokuqala kuluntu kwenzeka ngoNovemba ka-2015 (!), kwaye indawo yokugcina i-GitHub iqhayisa malunga neenkwenkwezi ze-3000 kunye nabaxhasi be-40+.
Ewe, uStolon ngumzekelo obalaseleyo woyilo olucingisisiweyo:
Isixhobo salo mqhubi sinokufumaneka ngokweenkcukacha kwingxelo okanye . Ngokubanzi, kwanele ukuthetha ukuba inokwenza yonke into echazwe: i-failover, i-proxies yokufikelela kumthengi ocacileyo, i-backups ... Ngaphezu koko, iiproksi zibonelela ngokufikelela ngenkonzo enye yokuphela - ngokungafaniyo nezinye izisombululo ezibini ezixutyushwa ngezantsi (ngamnye uneenkonzo ezimbini ze isiseko sokufikelela).
Nangona kunjalo, uStolon , Kungenxa yoko le nto ayikwazi ukuthunyelwa ngendlela elula kwaye ikhawuleza - "njengamaqebengwane ashushu" - ukudala iimeko ze-DBMS kwi-Kubernetes. Ulawulo lwenziwa nge-utility stolonctl, unikezelo lwenziwa ngetshathi yeHelm, kwaye ezo siko zichaziwe kwaye zicaciswe kwiConfigMap.
Ngakolunye uhlangothi, kubonakala ukuba umqhubi akayena ngokwenene umqhubi (emva koko, akasebenzisi i-CRD). Kodwa kwelinye icala, yinkqubo eguquguqukayo ekuvumela ukuba uqwalasele izixhobo kwii-K8s njengoko ubona kufanelekile.
Ukushwankathela, kuthi ngokobuqu akukhange kubonakale kufanelekile ukwenza itshathi eyahlukileyo yesiseko sedatha ngasinye. Ngoko ke, saqalisa ukukhangela ezinye iindlela.
2. I-Crunchy Data PostgreSQL Operator
, inkwenkwe yaseMerika eselula, yabonakala iyindlela esengqiqweni. Imbali yayo yoluntu iqala ngokukhululwa kokuqala ngo-Matshi 2017, ukususela ngoko indawo yokugcina i-GitHub ifumene ngaphantsi kwe-1300 yeenkwenkwezi kunye nabaxhasi be-50 +. Ukukhutshwa kwamva nje ngoSeptemba kuhlolwe ukusebenza kunye ne-Kubernetes 1.15-1.18, i-OpenShift 3.11+ kunye ne-4.4+, i-GKE kunye ne-VMware Enterprise PKS 1.3+.
Uyilo lweCrunchy Data PostgreSQL Operator ikwahlangabezana neemfuno ezichaziweyo:
Ulawulo lwenzeka ngokusetyenziswa pgo, nangona kunjalo, ivelisa iZibonelelo zeSiko leKubernetes. Ke ngoko, umsebenzisi ukholisiwe kuthi njengabasebenzisi abanokubakho:
- kukho ulawulo ngeCRD;
- ulawulo lomsebenzisi olufanelekileyo (ngokusebenzisa iCRD);
- ukudibanisa namanye amacandelo - ingqokelela ekhethekileyo yemifanekiso yesikhongozeli sePostgreSQL kunye nezixhobo zokusebenza kunye nayo (kubandakanywa ne-pgBackRest, pgAudit, izandiso ezivela kwi-contrib, njl.).
Nangona kunjalo, iinzame zokuqalisa ukusebenzisa umsebenzisi ovela kwiDatha yeCrunchy zibonakalise iingxaki ezininzi:
- Kwakungekho nto yokunyamezela - kuphela i-nodeSelector inikezelwa.
- Iipods ezenziweyo beziyinxalenye yokusasazwa, ngaphandle kwento yokuba siye safaka isicelo esisemthethweni. Ngokungafaniyo ne-StatefulSets, ukuhanjiswa akukwazi ukwenza iidiski.
I-drawback yokugqibela ikhokelela kwixesha elihlekisayo: kwindawo yokuvavanya sikwazi ukuqhuba ii-replicas ezi-3 ngediski enye indawo yokugcina, ebangela ukuba umqhubi anike ingxelo yokuba ii-replicas ezi-3 zazisebenza (nangona zazingekho).
Olunye uphawu lwalo mqhubi ludibaniso olulungeleyo kunye neenkqubo ezahlukeneyo ezincedisayo. Umzekelo, kulula ukuyifaka i-pgAdmin kunye ne-pgBounce, kwaye kwi iGrafana kunye nePrometheus esele ziqwalaselwe ziyaqwalaselwa. Kutshanje Ukudityaniswa okuphuculweyo kunye neprojekthi kuqatshelwa ngokwahlukeneyo , enkosi apho umqhubi unikezela ngokubonwayo okucacileyo kwePgSQL metrics ngaphandle kwebhokisi.
Nangona kunjalo, ukhetho olungaqhelekanga lwezixhobo eziveliswe nguKubernetes zasikhokelela kwisidingo sokufumana isisombululo esahlukileyo.
3. Zalando Postgres Operator
Siyazi iimveliso zeZalando ixesha elide: sinamava ngokusebenzisa iZalenium kwaye, ngokuqinisekileyo, sizamile sisisombululo sabo se-HA esidumileyo sePostgreSQL. Malunga nendlela yenkampani yokudala omnye wababhali bayo, u-Alexey Klyukin, uthe emoyeni , kwaye sayithanda.
Esi sesona sisombululo sincinci sixutyushwa kwinqaku: ukukhutshwa kokuqala kwenzeka ngo-Agasti 2018. Nangona kunjalo, nangona inani elincinci lokukhutshwa ngokusemthethweni, le projekthi ifikile indlela ende, sele idlula ekuthandeni isisombululo esivela kwi-Crunchy Data kunye neenkwenkwezi ze-1300 + kwi-GitHub kunye nenani eliphezulu labaxhasi (70+).
"Ngaphantsi kwe-hood" lo mqhubi usebenzisa izisombululo ezivavanywa ixesha:
- Patroni kunye Ukuqhuba,
- - kwii-backups,
- - njengedama lokudibanisa.
Yile ndlela uyilo lwabasebenzi oluvela eZalando luboniswa:
Umsebenzisi ulawulwa ngokupheleleyo ngeZibonelelo zeSiko, uzenzele ngokuzenzekelayo iStatefulSet kwizikhongozeli, ezinokuthi emva koko zenziwe ngokwezifiso ngokongeza ii-sidecars ezahlukeneyo kwipod. Konke oku kuluncedo olukhulu xa kuthelekiswa nomsebenzisi ovela kwiCrunchy Data.
Ekubeni sikhethe isisombululo esivela ku-Zalando phakathi kweenketho ze-3 ezicatshangelwayo, inkcazo eyongezelelweyo yezakhono zayo iya kuboniswa ngezantsi, ngokukhawuleza kunye nokusebenza kwesicelo.
Ziqhelanise nePostgres Operator evela eZalando
Ubeko lomsebenzisi lulula kakhulu: khuphela nje ukhupho lwangoku kwiGitHub kwaye usebenzise iifayile zeYAML kulawulo. . Kungenjalo, ungasebenzisa kwakhona .
Emva kofakelo, kufuneka ukhathazeke ngokuseta . Oku kwenziwa ngeConfigMap postgres-operator kwindawo yegama apho ufake khona umsebenzisi. Nje ukuba iindawo zokugcina ziqwalaselwe, ungathumela iqela lakho lokuqala lePostgreSQL.
Umzekelo, ukuhanjiswa kwethu okusemgangathweni kujongeka ngolu hlobo:
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
name: staging-db
spec:
numberOfInstances: 3
patroni:
synchronous_mode: true
postgresql:
version: "12"
resources:
limits:
cpu: 100m
memory: 1Gi
requests:
cpu: 100m
memory: 1Gi
sidecars:
- env:
- name: DATA_SOURCE_URI
value: 127.0.0.1:5432
- name: DATA_SOURCE_PASS
valueFrom:
secretKeyRef:
key: password
name: postgres.staging-db.credentials
- name: DATA_SOURCE_USER
value: postgres
image: wrouesnel/postgres_exporter
name: prometheus-exporter
resources:
limits:
cpu: 500m
memory: 100Mi
requests:
cpu: 100m
memory: 100Mi
teamId: staging
volume:
size: 2Gi
Lo mboniso usasaza iqela leemeko ezi-3 ezinemoto esecaleni ngendlela , apho sithatha iimetriki zesicelo. Njengoko ubona, yonke into ilula kakhulu, kwaye ukuba unqwenela, unokwenza inani elingenamkhawulo lamaqela.
Kufanelekile ukunikela ingqalelo iphaneli yolawulo lwewebhu - . Iza nomsebenzisi kwaye ikuvumela ukuba udale kwaye ucime amaqela, kunye nokusebenza ngee-backups ezenziwe ngumqhubi.
Uluhlu lwamaqela e-PostgreSQL
Ulawulo logcino
Enye into enomdla yinkxaso . Lo matshini wenza ngokuzenzekelayo iindima kwiPostgreSQL, ngokusekelwe kuluhlu lweziphumo lwamagama abasebenzisi. I-API ke ikuvumela ukuba ubuyisele uluhlu lwabasebenzisi abo iindima zidalwe ngokuzenzekelayo.
Iingxaki kunye nezisombululo
Nangona kunjalo, ukusetyenziswa komqhubi ngokukhawuleza kubonise iintsilelo ezininzi ezibalulekileyo:
- ukungabikho kwe-nodeSelector inkxaso;
- ukungakwazi ukukhubaza i-backups;
- xa usebenzisa umsebenzi wokudala wesiseko sedata, amalungelo angagqibekanga awabonakali;
- Ngamanye amaxesha amaxwebhu alahlekile okanye aphelelwe lixesha.
Ngethamsanqa, ezininzi zazo zinokusonjululwa. Masiqale ukusuka ekugqibeleni - iingxaki nge uxwebhu.
Okunokwenzeka, uya kudibana nenyaniso yokuba ayisoloko icacile indlela yokubhalisa i-backup kunye nendlela yokudibanisa ibhakethi yokugcina kwi-UI yoMsebenzisi. Amaxwebhu athetha ngale nto ngokudlulayo, kodwa inkcazo yokwenyani ingaphakathi :
- kufuneka wenze imfihlo;
- yigqithisele kumsebenzisi njenge parameter
pod_environment_secret_namekwi-CRD ngoseto lomsebenzisi okanye kwi-ConfigMap (kuxhomekeke kwindlela ogqiba ngayo ukufaka umsebenzisi).
Nangona kunjalo, njengoko kuvela, oku akunakwenzeka okwangoku. Yiyo loo nto siqokelele kunye nophuhliso olongezelelweyo lomntu wesithathu. Ukuze ufumane inkcazelo engakumbi ngayo, bona ngezantsi.
Ukuba ugqithisa iiparameters zogcino kumsebenzisi, oko kukuthi - wal_s3_bucket kunye nezitshixo zokufikelela kwi-AWS S3, emva koko izogcina yonke into: kungekhona kuphela iziseko kwimveliso, kodwa kunye nesiteji. Oku bekungasifanelanga.
Kwinkcazo yeeparamitha zeSpilo, esisiseko seDocker esongayo sePgSQL xa usebenzisa umsebenzisi, kwavela: ungadlula iparameter. WAL_S3_BUCKET engenanto, ngokwenza oko ikhubaza ugcino. Ngaphezu koko, ndafumana uvuyo olukhulu , esayamkela ngoko nangoko kwifolokhwe yethu. Ngoku ufuna nje ukongeza enableWALArchiving: false kumthombo weqela lePostgreSQL.
Ewe, bekukho ithuba lokuyenza ngokuhlukileyo ngokuqhuba abaqhubi be-2: enye ye-staging (ngaphandle kwe-backups), kunye neyesibini kwimveliso. Kodwa sakwazi ukwenza enye.
Kulungile, sifunde indlela yokudlulisa ukufikelela kwi-database ye-S3 kunye nee-backups zaqala ukungena kwindawo yokugcina. Uwenza njani amaphepha agcinwayo asebenze kwi-UI yoMsebenzisi?
Kuya kufuneka udibanise izinto ezi-3 kwi-UI yoMsebenzisi:
-
SPILO_S3_BACKUP_BUCKET -
AWS_ACCESS_KEY_ID -
AWS_SECRET_ACCESS_KEY
Emva koku, ulawulo lwee-backups luya kufumaneka, oluya kuthi kwimeko yethu lube lula umsebenzi kunye nesiteji, ukusivumela ukuba sihambise iziqwenga ezivela kwimveliso apho ngaphandle kwezikripthi ezongezelelweyo.
Enye inzuzo yayingumsebenzi kunye ne-API yeQela kunye namathuba amaninzi okudala i-database kunye neendima usebenzisa izixhobo zokusebenza. Nangona kunjalo, yenziwe iindima zazingenamalungelo ngokungagqibekanga. Ngokufanelekileyo, umsebenzisi onamalungelo okufunda akakwazanga ukufunda iitafile ezintsha.
Kutheni kunjalo? Nangona kunjalo kwikhowudi kunyanzelekile GRANT, azisoloko zisetyenziswa. Kukho iindlela ezi-2: syncPreparedDatabases ΠΈ syncDatabases. Ku syncPreparedDatabases - nangona kunjalo kwicandelo preparedDatabases kukho imeko defaultRoles ΠΈ defaultUsers ukwenza iindima, amalungelo angagqibekanga awasetyenziswa. Sikwinkqubo yokulungisa isiqwenga ukuze la malungelo asetyenziswe ngokuzenzekelayo.
Kwaye inqaku lokugqibela kuphuculo olufanelekileyo kuthi - , eyongeza i-Node Affinity kwi-StatefulSet eyenziweyo. Abathengi bethu bahlala bekhetha ukunciphisa iindleko ngokusebenzisa iimeko zeendawo, kwaye ngokucacileyo abafanelekanga ukubamba iinkonzo zesiseko sedatha. Lo mbandela unokusonjululwa ngokunyamezela, kodwa ubukho beNode Affinity bunika ukuzithemba okukhulu.
Kweneke ntoni?
Ngokusekwe kwiziphumo zokusombulula ezi ngxaki zingentla, safoxa iPostgres Operator ukusuka eZalando ukuya , apho iqokelelwa kunye neepatches eziluncedo ezinjalo. Kwaye ukuze kube lula ngakumbi, nathi siqokelele .
Uluhlu lweePRs zamkelwe kwifolokhwe:
- ;
- ;
- ;
- .
Kuya kuba kuhle ukuba uluntu luxhasa ezi PRs ukwenzela ukuba zinyuke ngenguqu elandelayo yomsebenzisi (1.6).
Ibhonasi! Ibali lempumelelo yokufuduka kwemveliso
Ukuba usebenzisa iPatroni, imveliso ephilayo inokufuduselwa kumqhubi ngexesha elincinci lokuphumla.
I-Spilo ikuvumela ukuba wenze amaqela alindileyo nge-S3 yokugcina nge , xa i-log yokubini ye-PgSQL igcinwa kuqala kwi-S3 kwaye emva koko imponthwe yi-replica. Kodwa yintoni onokuyenza ukuba unayo hayi isetyenziswe nguWal-E kwiziseko zophuhliso ezindala? Isisombululo sale ngxaki sele sikhona kwi hub.
Ukuphindaphinda okunengqiqo kwePostgreSQL kuza kuhlangula. Nangona kunjalo, asiyi kungena kwiinkcukacha malunga nendlela yokwenza iimpapasho kunye nokubhaliselwa, kuba ... isicwangciso sethu sasiyi-fiasco.
Inyani kukuba i-database yayineetafile ezininzi ezilayishiweyo kunye nezigidi zemigca, ezo, ngaphezu koko, zazihlala zizaliswa kwaye zicinywa. Ρ copy_data, xa i-replica entsha ikopisha zonke iziqulatho ezivela kwi-master, ayinakukwazi ukuhambelana nenkosi. Ukukopa umxholo usebenze iveki, kodwa awuzange ubanjwe nenkosi. Ekugqibeleni, yandinceda ukusombulula ingxaki oogxa abavela kwi-Avito: unokudlulisa idatha usebenzisa pg_dump. Ndiza kuchaza inguqulelo yethu (eguqulwe kancinane) yale algorithm.
Umbono kukuba ungenza umrhumo okhubazekileyo ubotshelelwe kwindawo ethile yokuphindaphinda, kwaye emva koko ulungise inombolo yentengiselwano. Kwakukho iikopi ezifumanekayo zomsebenzi wokuvelisa. Oku kubalulekile kuba i-replica iya kunceda ukudala ukulahla okuhambelanayo kwaye uqhubeke nokufumana utshintsho oluvela kwinkosi.
Imiyalelo elandelayo echaza inkqubo yokufuduka iya kusebenzisa olu lwazi lulandelayo.
- inkosi - iseva yomthombo;
- ikopi1 - i-replica yokusakaza kwimveliso yakudala;
- ikopi2 -Ireplica entsha enengqiqo.
Isicwangciso sokufuduka
1. Yenza umrhumo kwinkosi yazo zonke iitafile ezikwi-schema public isiseko dbname:
psql -h master -d dbname -c "CREATE PUBLICATION dbname FOR ALL TABLES;"
2. Yenza indawo yokuphindaphinda kwinkosi:
psql -h master -c "select pg_create_logical_replication_slot('repl', 'pgoutput');"
3. Yeka ukuphindaphinda kwikopi endala:
psql -h replica1 -c "select pg_wal_replay_pause();"
4. Fumana inombolo yetransekshini kumphathi:
psql -h master -c "select replay_lsn from pg_stat_replication where client_addr = 'replica1';"
5. Susa inkunkuma kwikopi endala. Siza kwenza oku kwimisonto emininzi, eya kunceda ukukhawulezisa inkqubo:
pg_dump -h replica1 --no-publications --no-subscriptions -O -C -F d -j 8 -f dump/ dbname
6. Layisha ukulahla kwiseva entsha:
pg_restore -h replica2 -F d -j 8 -d dbname dump/
7. Emva kokukhuphela ukulahla, ungaqala ukuphindaphinda kwi-replica yostrimisho:
psql -h replica1 -c "select pg_wal_replay_resume();"
7. Masenze umrhumo kwikopi entsha enengqiqo:
psql -h replica2 -c "create subscription oldprod connection 'host=replica1 port=5432 user=postgres password=secret dbname=dbname' publication dbname with (enabled = false, create_slot = false, copy_data = false, slot_name='repl');"
8. Makhe sifumane oid imirhumo:
psql -h replica2 -d dbname -c "select oid, * from pg_subscription;"
9. Masithi yamkelwe oid=1000. Masisebenzise inombolo yetransekshini kumrhumo:
psql -h replica2 -d dbname -c "select pg_replication_origin_advance('pg_1000', 'AA/AAAAAAAA');"
10. Masiqale ukuphindaphinda:
psql -h replica2 -d dbname -c "alter subscription oldprod enable;"
11. Jonga ubume bomrhumo, ukuphindaphinda kufuneka kusebenze:
psql -h replica2 -d dbname -c "select * from pg_replication_origin_status;"
psql -h master -d dbname -c "select slot_name, restart_lsn, confirmed_flush_lsn from pg_replication_slots;"
12. Emva kokuba uphindaphindo luqalisiwe kwaye oovimba beenkcukacha bangqamaniswe, ungatshintsha.
13. Emva kokuvala ukuphindaphinda, kufuneka ulungise ulandelelwano. Oku kuchazwe kakuhle .
Ngombulelo kwesi sicwangciso, utshintsho lwenzeka ngokulibaziseka okuncinci.
isiphelo
Abaqhubi be-Kubernetes bakuvumela ukuba wenze lula izenzo ezahlukeneyo ngokuzinciphisa ekudalweni kwezixhobo ze-K8s. Nangona kunjalo, ukuba ufezekise i-automation ephawulekayo ngoncedo lwabo, kuyafaneleka ukukhumbula ukuba kunokuzisa inani lee-nuances ezingalindelekanga, ngoko khetha abaqhubi bakho ngobulumko.
Emva kokuqwalasela abaqhubi abathathu abadumileyo be-Kubernetes kwi-PostgreSQL, sikhethe iprojekthi kwi-Zalando. Kwaye kuye kwafuneka soyise ubunzima obuthile ngayo, kodwa isiphumo besikholisa ngokwenene, ke siceba ukwandisa la mava kolunye ufakelo lwePgSQL. Ukuba unamava usebenzisa izisombululo ezifanayo, siya kuvuya ukubona iinkcukacha kwizimvo!
PS
Funda nakwibhlog yethu:
- Β«";
- Β«";
- «».
umthombo: www.habr.com