Kudzokorora hakusi kuchengetedza. Kana kuti kwete? Heano mashandisiro atakaita deferred replication kupora kubva pakudzima mapfupi netsaona.
GitLab inotarisira basa racho - iyo yakakura GitLab muenzaniso mune zvakasikwa. Iine mamirioni matatu vashandisi uye angangoita mamirioni manomwe mapurojekiti, ndiyo imwe yakakura yakavhurika sosi SaaS saiti ine yakazvitsaurira dhizaini. Pasina iyo PostgreSQL dhatabhesi system, iyo GitLab.com zvivakwa hazviende kure, uye chii chatiri kuita kuti tive nechokwadi chekushivirira kana paine kutadza kana data rikarasika. Hazvigoneki kuti njodzi yakadai iitike, asi isu takagadzirira uye takaiswa neakasiyana siyana ekuchengetedza uye nzira dzekudzokorora.
Kudzokorora haisi nzira yekutsigira dhatabhesi () Asi ikozvino tichaona nzira yekukurumidza kudzoreredza netsaona yakadzimwa data uchishandisa usimbe kudzokorora: on mushandisi zvepurojekiti uye yakarasika kubatana nekubatanidza zvikumbiro uye mabasa.
Iine replica yakamisikidzwa, takatora data mumaawa 1,5 chete. Tarirai kuti zvakaitika sei.
Pota munguva yekupora nePostgreSQL
PostgreSQL ine yakavakirwa-mukati basa inodzoreredza mamiriro edhatabhesi kune yakatarwa nguva nenguva. Inonzi (PITR) uye inoshandisa nzira dzakafanana dzinoita kuti replica ienderere mberi: kutanga neakavimbika snapshot yesese database cluster (base backup), tinoshandisa nhevedzano yekuchinja kwenyika kusvika pane imwe nguva nenguva.
Kuti tishandise ichi chimiro chekuchengetedza inotonhora, isu tinogara tichigadzira dhatabhesi rekutanga uye nekuichengeta mudura (GitLab archives inogara mukati. ) Isu tinotarisisawo shanduko yemamiriro edhatabhesi nekuchengetedza rondedzero yekunyora-mberi (, WAL). Uye nezvose izvi panzvimbo, tinogona kuita PITR yekudzorera njodzi: kutanga nemufananidzo wakatorwa kusati kwakundikana, uye kushandisa shanduko kubva kuWAL archive kusvika pakukundikana.
Chii chinonzi deferred replication?
Simbe kudzokorodza iko kushandisa shanduko kubva kuWAL nekunonoka. Kureva kuti, kutengeserana kwakaitika muawa X, asi ichaonekwa mune replica nekunonoka d muawa X + d.
PostgreSQL ine nzira mbiri dzekumisikidza dhatabhesi replica: kudzoreredza kudzoreredza uye kutenderera kudzokorora. , inoshanda sePITR, asi ichienderera: isu tinogara tichitora shanduko kubva kuWAL archive todziisa kune replica. A inotora zvakananga iyo WAL rukova kubva kune inokwira dhatabhesi host. Isu tinosarudza kudzoreredza kuchengetedza - zviri nyore kubata uye zvine zvakajairika kuita zvinofambirana neboka rekugadzira.
Maitiro ekuseta kunonoka kudzoreredza kubva kune archive
inotsanangurwa mufaira recovery.conf. Muenzaniso:
standby_mode = 'on'
restore_command = '/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-fetch -p 4 "%f" "%p"'
recovery_min_apply_delay = '8h'
recovery_target_timeline = 'latest'Nema paramita aya, isu takagadzira replica yakadzokororwa ine backup yekudzoreredza. Pano rinoshandiswa kubvisa WAL zvikamu (restore_command) kubva mudura, uye shanduko dzichashandiswa mushure memaawa masere (recovery_min_apply_delay) Iyo replica ichaona shanduko yenguva mudura, semuenzaniso nekuda kwekutadza kweboka (recovery_target_timeline).
Π‘ recovery_min_apply_delay Iwe unogona kuseta kudzokorora kutenderera nekunonoka, asi pane akati wandei makomba pano ane chekuita nekudzokorora slots, inopisa yekumira mhinduro, zvichingodaro. Iyo WAL dura inobvumidza iwe kuti udzivise.
Parameter recovery_min_apply_delay yakaonekwa chete muPostgreSQL 9.3. Mune shanduro dzakapfuura, kudzokorodza yakadzokororwa unofanirwa kugadzirisa musanganiswa (pg_xlog_replay_pause(), pg_xlog_replay_resume()) kana kubata WAL zvikamu mudura kwenguva yekunonoka.
PostgreSQL inoita sei izvi?
Zvinonakidza kuona kuti PostgreSQL inoshandisa sei usimbe kupora. Ngatitarisei . Inonzi kubva kune yega yega yekupinda kubva kuWAL.
static bool
recoveryApplyDelay(XLogReaderState *record)
{
uint8 xact_info;
TimestampTz xtime;
long secs;
int microsecs;
/* nothing to do if no delay configured */
if (recovery_min_apply_delay <= 0)
return false;
/* no delay is applied on a database not yet consistent */
if (!reachedConsistency)
return false;
/*
* Is it a COMMIT record?
*
* We deliberately choose not to delay aborts since they have no effect on
* MVCC. We already allow replay of records that don't have a timestamp,
* so there is already opportunity for issues caused by early conflicts on
* standbys.
*/
if (XLogRecGetRmid(record) != RM_XACT_ID)
return false;
xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK;
if (xact_info != XLOG_XACT_COMMIT &&
xact_info != XLOG_XACT_COMMIT_PREPARED)
return false;
if (!getRecordTimestamp(record, &xtime))
return false;
recoveryDelayUntilTime =
TimestampTzPlusMilliseconds(xtime, recovery_min_apply_delay);
/*
* Exit without arming the latch if it's already past time to apply this
* record
*/
TimestampDifference(GetCurrentTimestamp(), recoveryDelayUntilTime,
&secs, µsecs);
if (secs <= 0 && microsecs <= 0)
return false;
while (true)
{
// Shortened:
// Use WaitLatch until we reached recoveryDelayUntilTime
// and then
break;
}
return true;
}Chinokosha ndechekuti kunonoka kunoenderana nenguva yemuviri yakanyorwa mukutengeserana kuita timestamp (xtime) Sezvauri kuona, kunonoka kunoshanda chete kune kuzvipira uye hakukanganise zvimwe zvinyorwa - shanduko dzese dzinoiswa zvakananga, uye kuzvipira kunonokerwa, saka isu tichangoona shanduko mushure mekugadzirisa kunonoka.
Maitiro ekushandisa yakanonoka replica kudzoreredza data
Ngatitii isu tine dhatabhesi cluster uye replica ine kunonoka kwemaawa masere mukugadzira. Ngationei nzira yekudzoreredza data uchishandisa muenzaniso .
Patakadzidza nezvedambudziko, isu kune replica yakamisikidzwa:
SELECT pg_xlog_replay_pause();Nekumbomira, takanga tisina ngozi yokuti mufananidzo wacho waizodzokorora chikumbiro chacho DELETE. Chinhu chinobatsira kana iwe uchida nguva yekufunga zvese.
Icho chiripo ndechekuti replica yakamisikidzwa inofanira kusvika nguva isati yakumbira DELETE. Isu taiziva nguva yekubviswa. Tabvisa recovery_min_apply_delay uye akawedzera recovery_target_time Π² recovery.conf. Iyi ndiyo nzira iyo replica inosvika panguva chaiyo pasina kunonoka:
recovery_target_time = '2018-10-12 09:25:00+00'Nematampu enguva, zviri nani kuderedza kuwanda kuitira kuti usapotsa. Ichokwadi, iyo yakakura kuderera, iyo yakawanda data yatinorasikirwa nayo. Zvakare, kana tikapotsa chikumbiro DELETE, zvese zvichadzimwa zvakare uye uchafanirwa kutanga patsva (kana kutotora inotonhora backup yePITR).
Takatangazve iyo yakadzoserwa Postgres muenzaniso uye zvikamu zveWAL zvakadzokororwa kusvika panguva yakatarwa. Iwe unogona kutarisa kufambira mberi panguva ino nekubvunza:
SELECT
-- current location in WAL
pg_last_xlog_replay_location(),
-- current transaction timestamp (state of the replica)
pg_last_xact_replay_timestamp(),
-- current physical time
now(),
-- the amount of time still to be applied until recovery_target_time has been reached
'2018-10-12 09:25:00+00'::timestamptz - pg_last_xact_replay_timestamp() as delay;Kana iyo timestamp isingachachinji, kudzoreredza kwapera. Chiito chinogona kugadzirwa kuvhara, kusimudzira, kana kumbomira chiitiko mushure mekuedza zvakare (inomiswa nekusarudzika).
Dhatabhesi yakadzokera kumamiriro ayo pamberi pechikumbiro chinosiririsa. Iye zvino unogona, semuenzaniso, kutumira kunze data. Isu takatumira kunze iyo yakadzimwa label data uye zvese zvinongedzo kune nyaya uye nekubatanidza zvikumbiro uye tozviendesa mudura rekugadzira. Kana kurasikirwa kuri kukuru, unogona kungosimudzira replica uye woishandisa seyo huru. Asi zvino shanduko dzese mushure meiyo pfungwa yatakawana icharasika.
Panzvimbo pezvitambi zvenguva, zviri nani kushandisa transaction IDs. Izvo zvinobatsira kurekodha ma ID aya, semuenzaniso, kune DDL zvirevo (senge DROP TABLE), nekushandisa log_statements = 'ddl'. Dai taiva netransaction ID, taitora recovery_target_xid uye akamhanyisa zvese pasi kune kutengeserana pamberi pekukumbira DELETE.
Kudzokera kubasa kuri nyore: bvisa shanduko dzese kubva recovery.conf uye tangazve Postgres. Replica ichakurumidza kunonoka kwemaawa masere zvakare, uye takagadzirira matambudziko anotevera.
Recovery Benefits
Iine replica yakamisikidzwa pachinzvimbo chekuchengetedza inotonhora, haufanirwe kupedza maawa uchidzoreredza mufananidzo wese kubva mudura. Semuyenzaniso, zvinotora isu maawa mashanu kuti tiwane ese ekutanga 2 TB backup. Uye ipapo iwe uchiri kushandisa iyo yemazuva ese WAL kuti udzoke kune yaunoda mamiriro (mune yakaipisisa kesi).
Replica yakamisikidzwa iri nani pane inotonhora backup munzira mbiri:
- Iko hakuna chikonzero chekubvisa yese yekutanga backup kubva mudura.
- Pane hwindo rakagadziriswa maawa masere ezvikamu zveWAL zvinofanirwa kudzokororwa.
Isu tinogara tichitarisa kuti tione kana zvichikwanisika kugadzira PITR kubva kuWAL, uye isu taizokurumidza kuona huwori kana mamwe matambudziko neWAL archive nekutarisa kusakara kweiyo deferred replica.
Mumuenzaniso uyu, zvakatitorera maminetsi makumi mashanu kudzoreredza, zvichireva kuti kumhanya kwaive 50 GB yeWAL data paawa (iyo archive yaive ichiri kushanda. ) Pakazara, takagadzirisa dambudziko uye takadzora iyo data mumaawa 1,5.
Mhedzisiro: uko replica yakamisikidzwa inobatsira (uye paisiri)
Shandisa kunonoka kudzokorodza seyamuro yekutanga kana iwe wakarasa data netsaona uye waona dambudziko iri mukati mekunonoka kwakarongwa.
Asi ramba uchifunga: kudzokorora hakusi kuchengetedza.
Backup uye kudzokorora kune zvinangwa zvakasiyana. Iyo inotonhora backup ichauya inobatsira kana iwe wakaita netsaona DELETE kana DROP TABLE. Isu tinoita backup kubva kunotonhora kuchengetedza uye kudzoreredza iyo yapfuura mamiriro etafura kana iyo database yese. Asi panguva imwe chete chikumbiro DROP TABLE inenge yangogadzirwazve ipapo mune zvese replicas pane iri kushanda cluster, saka kudzokorora kwakajairwa hakuzobatsiri pano. Replication pachayo inochengeta dhatabhesi iripo kana maseva ega ega achirendwa uye achigovera mutoro.
Kunyangwe iine replica yakamisikidzwa, isu dzimwe nguva tinoda chaizvo kuchengetedza inotonhora munzvimbo yakachengeteka kana data data kutadza, kukuvara kwakavanzika, kana zvimwe zviitiko zvisingaonekwe nekukurumidza zvikaitika. Kudzokorora chete hakuna basa pano.
taura pfungwa... Vhura Isu parizvino tinodzivirira kubva pakurasikirwa kwedata padanho rehurongwa uye hatidzore data padanho remushandisi.
Source: www.habr.com