ProHoster > Blog > Utongi > Yakagoverwa kukiya uchishandisa Redis

Yakagoverwa kukiya uchishandisa Redis

Hei Habr!

Nhasi tinokuunzira iwe kududzirwa kwechinyorwa chakaoma pamusoro pekushandiswa kwekuvharidzirwa kwekuvhara uchishandisa Redis uye kukukoka iwe kuti utaure nezve tarisiro yeRedis sechinyorwa. Ongororo yeRedlock algorithm iri mubvunzo kubva kuna Martin Kleppmann, munyori webhuku "High Load Applications", akapiwa pano.

Kuvharirwa kukiya chinhu chekare chinobatsira kwazvo chinoshandiswa munzvimbo dzakawanda umo maitiro akasiyana anofanirwa kushanda pane zvakagovaniswa zviwanikwa nenzira yakasarudzika.

Kune akati wandei emaraibhurari uye mameseji kunze uko anotsanangura maitiro ekuita DLM (Distributed Lock Manager) uchishandisa Redis, asi raibhurari yega yega inotora nzira yakasiyana uye vimbiso yavanopa haina kusimba kana ichienzaniswa neinogoneka neyakaomesesa dhizaini.

Muchikamu chino, isu tichaedza kutsanangura inomisikidzwa canonical algorithm inoratidza maitiro ekuita yakagoverwa kukiya uchishandisa Redis. Tichataura nezve algorithm inonzi Redlock, inoshandisa yakagoverwa yekiyi maneja uye, semaonero edu, iyi algorithm yakachengeteka kupfuura yakajairwa imwe-chiitiko maitiro. Tinovimba kuti nharaunda ichaiongorora, kupa mhinduro, uye kuishandisa sepokutangira kune mamwe mapurojekiti akaomarara kana mamwe.

Implementation

Tisati taenderera kune tsananguro yegorgorithm, tinopa akati wandei malink kune akagadzirira-akaitwa mashandisirwo. Vanogona kushandiswa sereferensi.

Chengetedzo uye Kuwanikwa Guarantees

Tichazomodhera dhizaini yedu nezvivakwa zvitatu chete zvatinofunga kuti zvinopa vimbiso shoma inodiwa kuti tishandise nemazvo kukiya kukiya.

  1. Chengetedzo pfuma: Kusabatanidzwa. Panguva ipi zvayo, mutengi mumwe chete ndiye anogona kubata kiyi.
  2. Kuwanika Property A: Hapana madhiri. Zvinogara zvichigoneka kuti pakupedzisira uwane kukiya, kunyangwe mutengi akakiya sosi akatadza kana kugara pane imwe dhisiki segment.
  3. Kuwanikwa Property B: Mhosva Kushivirira. Chero bedzi mazhinji eRedis node ari kushanda, vatengi vanokwanisa kuwana nekusunungura makiyi.

Sei kushandiswa kunobva pakukundikana kupora hakuna kukwana munyaya iyi
Kuti tinzwisise zvatiri kuzovandudza, ngationgororei mamiriro ezvinhu aripo nemaraibhurari akawanda akavharirwa ekuvhara akavakirwa paRedis.

Iyo yakapusa nzira yekukiya sosi uchishandisa Redis kugadzira kiyi mumuenzaniso. Kazhinji, kiyi inogadzirwa nehupenyu hushoma, izvi zvinowanikwa uchishandisa iyo inopera nguva yakapihwa muRedis, saka nekukurumidza kana gare gare kiyi iyi inoburitswa (property 2 mune yedu runyorwa). Kana mutengi achida kuburitsa sosi, inodzima kiyi.

Pakutanga kuona, mhinduro iyi inoshanda chaizvo, asi pane dambudziko: dhizaini yedu inogadzira imwe pfungwa yekutadza. Chii chinoitika kana muenzi Redis muenzaniso akakundikana? Ngatiwedzere muranda ipapo! Uye isu tichaishandisa kana muratidziri asipo. Zvinosuruvarisa, iyi sarudzo haigoneki. Nekuita izvi, isu hatizokwanisa kuita nemazvo iyo mutual exclusion pfuma yatinoda kuti tive nechokwadi chekuchengetedza, nekuti kudzokorora muRedis kune asynchronous.

Zviripachena, mumuenzaniso wakadaro mamiriro emujaho anoitika:

  1. Mutengi A anowana kiyi pane tenzi.
  2. Tenzi anokundikana kiyi yekupinda isati yaendeswa kumuranda.
  3. Muteveri anosimudzirwa kuva mutungamiriri.
  4. Mutengi B anotora kiyi pane imwechete sosi iyo A yakatokiya. KUNYORWA KWEZVINHU!

Dzimwe nguva zvakajairika kuti mumamiriro ezvinhu akasarudzika, sekutadza, vatengi vazhinji vanogona kubata kukiya panguva imwe chete. Muzviitiko zvakadaro, replication-based solution inogona kushandiswa. Kana zvisina kudaro, tinokurudzira mhinduro inotsanangurwa munyaya ino.

Kugadzirisa kuita nemuenzaniso mumwe chete

Usati waedza kukurira kukanganisa kweiyo imwe-chiitiko kumisikidzwa yakatsanangurwa pamusoro, ngatinzwisise kuti tingaite sei nemazvo nyaya iyi iri nyore, sezvo mhinduro iyi inoshanda mumashandisirwo ayo mamiriro erudzi anogamuchirwa nguva nenguva, uye zvakare nekuti kuvharira pane imwe muenzaniso inoshanda sehwaro hunoshandiswa mune yakagoverwa algorithm inotsanangurwa pano.

Kuti uwane lock, ita izvi:

SET resource_name my_random_value NX PX 30000

Uyu murairo unoisa kiyi chete kana isati yavepo (NX sarudzo), ine nguva yechokwadi ye30000 milliseconds (PX sarudzo). Makiyi akaiswa ku"myrandomvalue" Kukosha uku kunofanirwa kuve kwakasiyana kune vese vatengi uye zvese zvikumbiro zvekiyi.
Chaizvoizvo, kukosha kwakasarudzika kunoshandiswa kusunungura kiyi zvakachengeteka, ine script inotaurira Redis: bvisa chete kiyi kana iripo uye kukosha kwakachengetwa mairi ndiko chaiko kwaitarisirwa. Izvi zvinowanikwa uchishandisa inotevera Lua script:

if redis.call("get",KEYS[1]) == ARGV[1] then
    return redis.call("del",KEYS[1])
else
    return 0
end

Izvi zvakakosha kudzivirira kiyi inobatwa nemumwe mutengi kubva pakubviswa. Semuyenzaniso, mutengi anogona kutora kiyi, obva avharirwa mune kumwe kushanda kunotora nguva yakareba kupfuura kukiya kwekutanga (kuitira kuti kiyi ive nenguva yekupera), uye wozobvisa kiyi iyo yaiswa nemumwe mutengi.
Kushandisa DEL iri nyore hakuna kuchengetedzeka nekuti mutengi anogona kubvisa kiyi yakabatwa nemumwe mutengi. Mukupesana, kana uchishandisa script iri pamusoro, kiyi yega yega "yakasainwa" netambo isina kujairika, saka mutengi chete akamboiisa anogona kuibvisa.

Chii chinofanira kunge chiri tambo isina kurongeka iyi? Ndiri kufungidzira kuti inofanira kunge iri 20 bytes kubva /dev/urandom, asi unogona kuwana nzira dzisingadhure dzekuita kuti tambo ive yakasarudzika zvakakwana kune zvinangwa zvako. Semuenzaniso, zvingave zvakanaka kudyara RC4 ne /dev/urandom wobva wagadzira pseudo-random rwizi kubva mairi. Mhinduro yakapusa inosanganisira musanganiswa wenguva unix mukugadziriswa kwemicrosecond pamwe neiyo ID yemutengi; haina kuchengetedzeka, asi pamwe iri pamusoro pebasa racho mumamiriro mazhinji.

Nguva yatinoshandisa sechiyero chehupenyu hwekiyi inonzi "lock lifetime". Ukoshi uhwu huviri huwandu hwenguva kiyi isati yaburitswa otomatiki uye huwandu hwenguva yemutengi yekupedza oparesheni mumwe mutengi asati akwanisa kukiya iyo sosi, pasina kutyora vimbiso yekusabatanidzwa. Iyi vimbiso inongogumira kune imwe hwindo yenguva, iyo inotanga kubva panguva iyo kukiya kunotengwa.

Saka takakurukura nzira yakanaka yekuwana nekusunungura kiyi. Iyo sisitimu (kana isu tiri kutaura nezve isina-kugovaniswa system ine imwechete uye inogara iripo muenzaniso) yakachengeteka. Ngatiwedzerei pfungwa iyi kune yakagoverwa system, uko isu tisina vimbiso dzakadaro.

Redlock algorithm

Iyo yakagoverwa vhezheni yealgorithm inofunga kuti isu tine N Redis masters. Aya ma node akazvimirira zvachose kubva kune mumwe nemumwe, saka isu hatishandise kudzokorodza kana chero imwe yakasarudzika yekubatanidza system. Isu takatovhara nzira yekuwana zvakachengeteka uye kusunungura kiyi pane imwechete muenzaniso. Isu tinozvitora sezviri pachena kuti iyo algorithm, kana ichishanda nemuenzaniso mumwe chete, ichashandisa chaiyo nzira iyi. Mumienzaniso yedu tinoisa N kusvika 5, inova yakakosha. Nekudaro, isu tichada kushandisa mashanu Redis masters pamakomputa akasiyana kana chaiwo muchina kuona kuti vanoita zvakanyanya vakazvimiririra.

Kuti awane kukiya, mutengi anoita zvinotevera:

  1. Inowana nguva iripo mumamilliseconds.
  2. Sequentially kuyedza kuwana kukiya pane ese N zviitiko, uchishandisa iro rimwechete kiyi zita uye zvisina kujairika kukosha mune ese kesi. MuChikamu chechipiri, kana mutengi achiisa kiyi pane imwe-chiitiko, mutengi anoshandisa kunonoka kuitora iyo ipfupi zvakakwana kana ichienzaniswa nenguva iyo yekukiya inoburitswa otomatiki. Semuenzaniso, kana nguva yekuvharira iri masekonzi gumi, saka kunonoka kunogona kunge kuri pakati pe ~ 2-10 milliseconds. Izvi zvinobvisa mamiriro ezvinhu umo mutengi anogona kuramba akavharirwa kwenguva yakareba achiedza kusvika yakakundikana Redis node: kana muenzaniso usipo, saka tinoedza kubatana kune imwe chiitiko nekukurumidza.
  3. Kutora kukiya, mutengi anoverenga kuti inguva yakadii yapfuura; Kuti uite izvi, inobvisa kubva kunguva chaiyo kukosha iyo timestamp yakawanikwa mudanho 1. Kana uye chete kana mutengi akakwanisa kuwana kukiya pazvizhinji zvezviitiko (inenge 3), uye iyo yakazara nguva yaitora kuwana kukiyiwa, zvishoma kudarika nguva yekuvhara, kuvhara kunofungidzirwa kuti kwakawanikwa.
  4. Kana kukiyiwa kwave kuwanikwa, nguva yekuvhara inotorwa kuve yepakutanga kukiya nguva kubvisa nguva yakapfuura yakaverengerwa mudanho rechitatu.
  5. Kana mutengi akatadza kuwana kiyi nekuda kwechimwe chikonzero (zvichida haina kukwanisa kukiya N/2+1 kesi, kana kukiya nguva yanga isina kunaka), inozoedza kuvhura makiyi ese (kunyangwe ayo aaifunga kuti haakwanise kuvharisa. )

Iyo algorithm ndeye asynchronous?

Iyi algorithm yakavakirwa pafungidziro yekuti, kunyangwe pasina wachi yakawiriraniswa paizoshanda maitiro ese, nguva yemuno muhurongwa hwega hwega ichiri kuyerera nekufamba kwakafanana, uye kukanganisa kudiki kana zvichienzaniswa nenguva yakazara mushure mekukiya. yakabudiswa otomatiki. Kufungidzira uku kwakafanana nemamiriro ezvinhu akajairika kumakomputa akajairwa: komputa imwe neimwe ine wachi yemuno, uye isu tinogona kazhinji kuvimba nenyaya yekuti musiyano wenguva pakati pemakomputa akasiyana idiki.

Panguva ino, isu tinofanira kunyatso gadzira mutemo wedu wekuti tisiyane: kusabatanidzwa kunovimbiswa chete kana mutengi akabata kiyi achibuda panguva iyo kukiya kunenge kuri kushanda (ukoshi uhu hunowanikwa mudanho rechitatu), kubvisa imwe nguva (yakazara mashoma. milliseconds kutsiva musiyano wenguva pakati pemaitiro).

Chinyorwa chinotevera chinonakidza chinotaurira zvakawanda nezve masisitimu akadaro anoda kurongeka kwenguva dzenguva: Leases: inoshanda yekutadza-kushivirira nzira yekugovera faira cache kuenderana.

Edzazve pakukundikana

Kana mutengi atadza kuwana kiyi, inofanirwa kuyedza zvakare mushure mekunonoka kunonoka; izvi zvinoitirwa kumisa-synchronize vatengi vakawanda vachiedza kuwana kiyi pane imwechete sosi panguva imwe chete (izvo zvinogona kutungamira kune "kutsemuka-uropi" mamiriro umo pasina vanokunda). Pamusoro pezvo, iyo inokurumidza mutengi anoedza kuwana kiyi pane yakawanda yeRedis zviitiko, iyo yakatetepa hwindo umo kupatsanurwa-uropi mamiriro anogona kuitika (uye kushoma kudiwa kwekuyedzazve). Naizvozvo, zvine musoro, mutengi anofanira kuedza kutumira SET mirairo kuN zviitiko panguva imwe chete uchishandisa multiplexing.

Zvakakosha kusimbisa pano kuti zvakakosha sei kune vatengi vanotadza kuwana ruzhinji rwemakiyi kuti vasunungure (zvishoma) makiyi akawanikwa, kuti vasazomirira kuti kiyi ipere kukiyiwa pachishandiswa kunogona kuwanikwa zvakare. (kunyangwe kana kupatsanurwa kwenetiweki kukaitika, uye mutengi akarasikirwa nekubatana neRedis zviitiko, saka unofanirwa kubhadhara chirango chekuwanikwa uchimirira kiyi kupera).

Sunungura kukiya

Kusunungura kiyi ibasa rakareruka rinongoda kuvhura zviitiko zvese, zvisinei nekuti mutengi anoita seakabudirira kukiya imwe muenzaniso.

Chengetedzo Kufunga

Iyo algorithm yakachengeteka here? Ngatiedzei kufungidzira zvinoitika muzviitiko zvakasiyana.

Kutanga, ngatifungei kuti mutengi akakwanisa kuwana kiyi pazvizhinji zvezviitiko. Muenzaniso wega wega uchange uine kiyi ine hupenyu hwakafanana kune vese. Nekudaro, imwe neimwe yemakiyi aya akaiswa pane imwe nguva, saka anopera nenguva dzakasiyana. Asi, kana kiyi yekutanga yakaiswa panguva isina kuipa kupfuura T1 (nguva yatinosarudza tisati tasangana nesevha yekutanga), uye kiyi yekupedzisira yakaiswa panguva isina kuipa kupfuura T2 (nguva iyo mhinduro yakagamuchirwa. kubva kune yekupedzisira server), saka isu tine chivimbo chekuti kiyi yekutanga museti inopera ichararama zvishoma MIN_VALIDITY=TTL-(T2-T1)-CLOCK_DRIFT. Mamwe makiyi ese achapera gare gare, saka tinogona kuva nechokwadi chekuti makiyi ese achashanda panguva imwe chete kwenguva ino.

Munguva iyo makiyi mazhinji anoramba achishanda, mumwe mutengi haazokwanisi kuwana kiyi, sezvo N/2+1 SET NX mashandiro asingagone kubudirira kana N/2+1 makiyi atovapo. Naizvozvo, kana kukiya kwave kuwanikwa, hazvibviri kuiwana zvakare panguva imwe chete (izvi zvinotyora mutual exclusion property).
Nekudaro, isu tinoda kuve nechokwadi chekuti vatengi vakawanda vari kuyedza kuwana kiyi panguva imwe chete havagone kubudirira panguva imwe chete.

Kana mutengi akakiya mazhinji emakesi kweinoda kana kupfuura iyo yakanyanya kukiya nguva, inozofunga kukiya iyo isiriyo uye kuvhura zviitiko. Naizvozvo, isu tinongofanirwa kufunga nezve nyaya iyo mutengi akakwanisa kuvharira mazhinji ezviitiko munguva isingasviki zuva rekupera. Muchiitiko ichi, maererano nenharo iri pamusoro, panguva MIN_VALIDITY hapana mutengi anofanira kukwanisa kutorazve kukiya. Naizvozvo, vatengi vazhinji vanozokwanisa kukiya N/2+1 zviitiko panguva imwe chete (inopera pakupera kwechikamu chechipiri) chete kana nguva yekuvhara iyo yakawanda yaive yakakura kupfuura nguva yeTTL, iyo inoita kuti kukiya kuve kusashanda.

Unogona here kupa humbowo hwakarongeka hwekuchengetedza, kuratidza maalgorithms aripo akafanana, kana kutsvaga bug mune zviri pamusoro?

Kuwanika Kufunga

Kuwanikwa kweSystem kunoenderana nemhando nhatu huru:

  1. Kuburitsa otomatiki makiyi (sezvo makiyi apera): Makiyi anozopedzisira ave kuwanikwa zvakare kuti ashandiswe kukiya.
  2. Icho chokwadi chekuti vatengi vanowanzo batsirana nekubvisa makiyi kana kiyi yaidiwa isati yawanikwa, kana yawanikwa uye basa rapera; saka zvingangoita kuti hatizomirira kuti kiyi dzipere kuti titorezve kukiya.
  3. Icho chokwadi chekuti kana mutengi achifanira kuedza zvakare kutora kiyi, inomirira kwenguva yakati rebei kupfuura iyo nguva inodiwa kuwana makiyi mazhinji. Izvi zvinoderedza mukana wekuti kukamukana-brain mamiriro anomuka pakukwikwidza zviwanikwa.

Zvisinei, pane chirango chekuwanika chakaenzana neTTL yezvikamu zvetiweki, saka kana pane zvikamu zvinosangana, chirango chinogona kunge chisingagumi. Izvi zvinoitika pese mutengi paanotora kiyi obva abvarura kuenda kune chimwe chikamu asati aisunungura.

Mumusimboti, kupihwa kusingaperi contiguous network zvikamu, sisitimu inogona kuramba isingawanikwe kwenguva isingaperi yenguva.

Performance, failover uye fsync

Vanhu vazhinji vanoshandisa Redis nekuti vanoda yakakwira yekuvhara server performance maererano neiyo latency inodikanwa kuwana nekusunungura makiyi, uye nhamba yekutora / kuburitswa kunogona kupedzwa pasekondi. Kuzadzisa ichi chinodiwa, pane zano rekutaurirana neN Redis maseva kudzikisa latency. Iri izano rekuwandisa (kana "kuwandisa kwemurume murombo", uko soketi inoiswa mune isiri-yekuvharisa modhi, inotumira mirairo yese, uye inoverenga mirairo gare gare, uchifungidzira kuti nguva yekufamba-famba pakati pemutengi uye chiitiko chimwe nechimwe chakafanana) .

Nekudaro, isu tinofanirwawo kufunga nezve kutariswa kwakabatana nekureba data kuchengetedza kana isu tichiedza kugadzira modhi ine yakavimbika kupora kubva mukukundikana.

Chaizvoizvo, kuti tijekese nyaya, ngatifungei kuti isu tinogadzirisa Redis isina-yenguva refu yekuchengetedza data zvachose. Mutengi anokwanisa kuvharira 3 kubva pa5 zviitiko. Imwe yezviitiko izvo mutengi akakwanisa kuvharira inotangwazve, uye panguva ino pane 3 zviitiko zvakare kune imwecheteyo sosi, iyo yatinogona kuvharira, uye mumwe mutengi anogona, zvakare, kuvharidzira chiitiko chatangwazve, kutyora iyo yekuchengetedza pfuma iyo. inotora chete kukiya.

Kana iwe ukagonesa data kumberi (AOF), mamiriro acho anovandudza zvishoma. Semuenzaniso, unogona kusimudzira sevha nekutumira iyo SHUTDOWN kuraira uye kuitangazve. Sezvo mashandiro ekupera muRedis achiitwa semantic nenzira yekuti nguva irambe ichiyerera kunyangwe sevha yakadzimwa, zvese zvatinoda zvakanaka. Izvi zvakajairika chero bedzi kuvharwa kwakajairwa kuchiitwa. Chii chekuita kana magetsi akadzimwa? Kana Redis yakagadziriswa nekusarudzika, ine fsync synchronizing pa diski sekondi yega yega, zvino zvinogoneka kuti mushure mekutangazve isu hatizove nekiyi yedu. Nechepfungwa, kana tichida kuvimbisa kukiya chengetedzo panguva ipi neipi yekutangisazve, isu tinofanirwa kugonesa fsync=always muzvirongwa zvekuchengetedza data kwenguva refu. Izvi zvichauraya zvachose kuita, kusvika padanho reCP masisitimu ayo anowanzo shandiswa kuchengetedza makiyi akagoverwa.

Asi mamiriro acho ezvinhu ari nani pane zvaanoita pakutanga. Muchidimbu, chengetedzo yegorgorithm inochengetedzwa nekuti kana iyo muenzaniso yatangwazve mushure mekutadza, haisisiri kutora chikamu mune chero kukiya kuri kushanda izvozvi.

Kuti tive nechokwadi cheizvi, isu tinongoda kuve nechokwadi chekuti mushure mekutadza muenzaniso unoramba usipo kwenguva yakati rebei kupfuura iyo yakanyanya TTL yatinoshandisa. Nenzira iyi isu tichamirira kusvika zuva rekupera uye otomatiki kuburitswa kwese makiyi aive achishanda panguva yekutadza.

Uchishandisa kunonoka kudzoreredza, zviri musimboti zvinogoneka kuwana chengetedzo kunyangwe pasina kutsungirira kwenguva refu muRedis. Ziva, zvisinei, kuti izvi zvinogona kuguma nefaindi yekutyora kuwanikwa. Semuenzaniso, kana ruzhinji rwezviitiko dzikatadza, sisitimu inozove isingawanikwe pasi rose kuTTL (uye hapana sosi inogona kuvharika panguva ino).

Isu tinowedzera kuwanikwa kweiyo algorithm: isu tinowedzera kuvharika

Kana basa rinoitwa nevatengi riine nhanho diki, zvinokwanisika kudzikisira nguva yekukiya yekuvhara uye kushandisa nzira yekuwedzera makiyi. Muchidimbu, kana mutengi ari mubishi rekushandisa komputa uye kukosha kwekupera kwekiyi kwakadzikira zvine njodzi, unogona kutumira zviitiko zvese runyoro rweLua rwunowedzera TTL yekiyi kana kiyi ichiripo uye kukosha kwayo kuchiri kukosha kwakawanikwa pakawanikwa kiyi. .

Mutengi anofanirwa kungofunga kiyi kuti iwanikwezve kana yakakwanisa kukiya mazhinji ezviitiko mukati menguva yechokwadi.

Chokwadi, tekinoroji iyo algorithm haishanduke, saka huwandu hwepamusoro hwekuyedza kuwana makiyi hunofanirwa kuve hushoma, zvikasadaro zvivakwa zvekuwanikwa zvichatyorwa.

Source: www.habr.com

Voeg