🥇Tarantool Cartridge: kutsemura rua backend mumitsara mitatu

PaMail.ru Group tine Tarantool - iyi sevha yekushandisa muLua, iyo inopetwa kaviri sedhatabhesi (kana zvinopesana?). Inokurumidza uye inotonhorera, asi kugona kweimwe sevha haisati isina magumo. Vertical scaling zvakare haisi panacea, saka Tarantool ine maturusi ekuyera kuyera - iyo vshard module. [1]. Iyo inokutendera iwe kuti utore data kune akati wandei maseva, asi iwe unofanirwa kuverengera nayo kuti uimise uye ubatanidze iyo bhizinesi pfungwa.

Nhau dzakanaka: takaunganidza mapfuti makuru (semuenzaniso [2], [3]) uye yakagadzira imwe dhizaini inozorerutsa zvakanyanya mhinduro yedambudziko iri.

Tarantool Cartridge igadziriro itsva yekugadzira masisitimu akaomarara akaparadzirwa. Iyo inokutendera kuti utarise pakunyora bhizinesi logic pane kugadzirisa matambudziko ezvivakwa. Pazasi pekuchekwa ini ndichakuudza kuti iyi dhizaini inoshanda sei uye maitiro ekunyora akagoverwa masevhisi uchiishandisa.

Dambudziko nderei?

Tine tarantula, isu tine vshard - chii chimwe chaungade?

Chekutanga, inyaya yekureruka. Iyo vshard gadziriso inogadziriswa kuburikidza neLua matafura. Kuti iyo yakagoverwa sisitimu yeakawanda maTarantool maitiro ashande nemazvo, iyo gadziriso inofanirwa kunge yakafanana kwese kwese. Hapana anoda kuita izvi nemaoko. Naizvozvo, marudzi ese ezvinyorwa, Ansible, uye deployment systems anoshandiswa.

Cartridge pachayo inogadzirisa iyo vshard kumisikidzwa, inoita izvi zvichibva pane yayo yega yakagovaniswa gadziriso. Iri rakareruka YAML faira, kopi inochengetwa mune yega yega Tarantool muenzaniso. Iyo yekurerutsa ndeyekuti iyo sisitimu pachayo inotarisisa kumisikidzwa kwayo uye inova nechokwadi chekuti yakafanana kwese kwese.

Chechipiri, zvakare inyaya yekureruka. Iyo vshard configuration haina chekuita nekuvandudzwa kwebhizinesi logic uye inongovhiringidza mugadziri kubva kubasa rake. Patinokurukura mavakirwo epurojekiti, isu tinowanzo taura nezve ega zvikamu uye kudyidzana kwavo. Kwakasira kufunga nezve kuburitsa sumbu kune 3 data nzvimbo.

Isu takagadzirisa matambudziko aya kakawanda, uye pane imwe nguva takakwanisa kugadzira nzira yakareruka kushanda nechishandiso mukati mehupenyu hwayo hwese kutenderera: kusikwa, kusimudzira, kuyedzwa, CI/CD, kugadzirisa.

Cartridge inosuma iyo pfungwa yebasa kune yega yega Tarantool maitiro. Mabasa ipfungwa inobvumira mugadziri kuti atarise pakunyora kodhi. Ese mabasa anowanikwa mupurojekiti anogona kuitiswa pane imwe Tarantool muenzaniso, uye izvi zvichakwana kumiedzo.

Zvinhu zvakakosha zveTarantool Cartridge:

otomatiki cluster orchestration;
kuwedzera kushanda kwechishandiso uchishandisa mabasa matsva;
application template yekusimudzira uye kutumira;
yakavakirwa-mukati otomatiki sharding;
kubatanidzwa neLuatest yekuyedza chimiro;
cluster management uchishandisa WebUI uye API;
kurongedza uye kutumira zvishandiso.

Mhoroi, Nyika!

Ini handigone kumirira kuratidza iyo chimiro pachayo, saka isu tichasiya iyo nyaya yezvivakwa gare gare totanga nechimwe chinhu chiri nyore. Kana isu tikafungidzira kuti Tarantool pachayo yakatoiswa, saka chasara kuita

$ tarantoolctl rocks install cartridge-cli
$ export PATH=$PWD/.rocks/bin/:$PATH

Iyi mirairo miviri ichaisa iyo yekuraira mutsara zvishandiso uye inokubvumidza iwe kugadzira yako yekutanga application kubva kutemplate:

$ cartridge create --name myapp

Uye izvi ndizvo zvatinowana:

myapp/
├── .git/
├── .gitignore
├── app/roles/custom.lua
├── deps.sh
├── init.lua
├── myapp-scm-1.rockspec
├── test
│   ├── helper
│   │   ├── integration.lua
│   │   └── unit.lua
│   ├── helper.lua
│   ├── integration/api_test.lua
│   └── unit/sample_test.lua
└── tmp/

Iyi igit repository ine yakagadzirira-yakagadzirwa "Mhoro, Nyika!" application. Ngatiedzei kuimhanyisa ipapo, takamboisa zvinotsamira (kusanganisira iyo chimiro pachayo):

$ tarantoolctl rocks make
$ ./init.lua --http-port 8080

Saka, isu tine imwe node inomhanya yeramangwana sharded application. Munhu anobvunzisisa anogona kuvhura iyo webhu interface, gadzirisa sumbu reimwe node negonzo uye anakirwe nemhedzisiro, asi kuchiri kutanga kufara. Parizvino, iyo application haigone kuita chero chinhu chinobatsira, saka ini ndichakuudza nezve kutumirwa gare gare, asi ikozvino yave nguva yekunyora kodhi.

Kuvandudza Application

Chimbofungidzira, tiri kugadzira purojekiti inofanirwa kugamuchira data, kuichengeta uye kuvaka rondedzero kamwe chete pazuva.

Tinotanga kudhirowa dhayagiramu uye toisa zvinhu zvitatu pairi: gedhi, kuchengetedza uye kuronga. Isu tiri kushanda pane architecture mberi. Sezvo isu tinoshandisa vshard sekuchengetedza, isu tinowedzera vshard-router uye vshard-kuchengetedza kune chirongwa. Hapana gedhi kana mugadziri anowana zvakananga kuchengetera; ndizvo zvakaitirwa router, ndizvo zvayakasikirwa.

Dhiagiramu iyi haisati yanyatso miririra izvo zvatichange tichivaka mupurojekiti nekuti zvikamu zvinotaridzika. Tichiri kuda kuona kuti izvi zvinozorongwa sei paTarantool chaiyo - ngatiunganidzei zvikamu zvedu nemaitiro.

Pane chidiki chekuchengeta vshard-router uye gedhi pane dzakasiyana zviitiko. Nei tichida kusefa network zvakare kana iri ratova basa reiyo router? Dzinofanirwa kuitwa mukati memaitiro akafanana. Kureva kuti, zvese gedhi uye vshard.router.cfg zvinotangwa mune imwe nzira, uye ita kuti zvidyidzane munharaunda.

Padanho rekugadzira, zvaive nyore kushanda nezvikamu zvitatu, asi ini, semugadziri, ndichinyora kodhi, handidi kufunga nezve kutangisa matatu eTarnatool. Ini ndoda kumhanyisa bvunzo uye tarisa kuti ndanyora gateway nemazvo. Kana kuti zvimwe ndinoda kuratidza chimiro kune vandinoshanda navo. Sei ndichifanira kuenda nekunetseka kwekuendesa makopi matatu? Aya ndiwo matangiro akaita pfungwa yemabasa. Basa inguva yakajairwa luash module ine hupenyu kutenderera inotungamirwa neCartridge. Mumuenzaniso uyu pane zvina - gedhi, router, kuchengetedza, kuronga. Panogona kunge paine zvakawanda mune chimwe chirongwa. Mabasa ese anogona kuitwa mune imwe nzira, uye izvi zvinokwana.

Uye kana zvasvika pakuendeswa kune staging kana kugadzirwa, isu tinozopa yega yega Tarantool process yayo yega seti yemabasa zvichienderana nekugona kwehardware:

Topology manejimendi

Ruzivo rwekuti ndeapi mabasa ari kuitwa anofanira kuchengetwa pane imwe nzvimbo. Uye iyi "kune imwe nzvimbo" ndiyo yakagoverwa gadziriro, yandatotaura pamusoro apa. Chinhu chinonyanya kukosha pazviri ndeye cluster topology. Heano mapoka matatu ekudzokorora e3 Tarantool maitiro:

Hatidi kurasikirwa nedata, saka tinobata ruzivo rwekumhanyisa maitiro nekungwarira. Cartridge inochengeta iyo gadziriso uchishandisa maviri-chikamu kuzvipira. Kana tangoda kugadzirisa zvigadziriso, inotanga kutarisa kuti zviitiko zvese zviripo uye zvakagadzirira kugamuchira kugadziridzwa kutsva. Mushure meizvi, chikamu chechipiri chinoshandisa config. Nokudaro, kunyange kana kopi imwe chete ikazova isingawanikwi kwenguva pfupi, hapana chakaipa chichaitika. Iyo gadziriso haingoshandiswe uye iwe uchaona kukanganisa pamberi.

Zvakare muchikamu chetopology, parameter yakakosha semutungamiri weboka rega rega rekudzokorora inoratidzwa. Kazhinji iyi ndiyo kopi iri kurekodhwa. Zvimwe zvacho zvinowanzoverengwa-chete, kunyange zvazvo panogona kunge paine zvisizvo. Dzimwe nguva vanogadzira vakashinga havatyi kupokana uye vanogona kunyora dhata kune akati wandei akateedzana, asi kune mamwe maoparesheni ayo, zvisinei nekuti chii, haafanirwe kuitwa kaviri. Nokuda kweizvi pane chiratidzo chemutungamiri.

Hupenyu hwemabasa

Kuti basa risinganzwisisike rivepo mukuvaka kwakadai, chimiro chinofanira kuzvibata neimwe nzira. Nomuzvarirwo, kutonga kunoitika pasina kutangazve Tarantool maitiro. Pane 4 callbacks yekugadzirisa mabasa. Cartridge pachayo inovadaidza zvichienderana nezvakanyorwa mukumisikidzwa kwayo kwakagoverwa, nekudaro ichishandisa gadziriso kune mamwe mabasa.

function init()
function validate_config()
function apply_config()
function stop()

Basa rimwe nerimwe rine basa init. Inodaidzwa kamwe chete kana basa raitwa kana Tarantool yatangwazve. Zviri nyore ipapo, semuenzaniso, kutanga box.space.create, kana mugadziri anogona kuvhura imwe kumashure fiber ichaita basa pane imwe nguva nguva.

Imwe basa init zvingasakwana. Cartridge inobvumira mabasa kutora mukana weiyo yakagoverwa kumisikidzwa yaanoshandisa kuchengetedza iyo topology. Tinogona kuzivisa chikamu chitsva mukugadzirisa kwakafanana uye kuchengetedza chidimbu chekugadzirisa bhizinesi mairi. Mumuenzaniso wangu, iyi inogona kunge iri data schema kana kuronga marongero ebasa rekuronga.

Cluster mafoni validate_config и apply_config nguva dzose iyo yakagoverwa configuration inoshanduka. Kana gadziriso ikashandiswa neaviri-chikamu kuzvipira, sumbu rinotarisa kuti chikamu chega chega chagadzirira kugamuchira iyi nyowani uye, kana zvichidikanwa, inoshuma kukanganisa kumushandisi. Kana munhu wese achibvumirana kuti iyo gadziriso yakajairika, ipapo iyo apply_config.

Uyewo mabasa ane nzira stop, iyo inodiwa kuchenesa zvinobuda mubasa racho. Kana isu tikati scheduler haichadiwe pane iyi server, inogona kumisa iwo mafiber ayo akatanga nawo init.

Mabasa anogona kudyidzana. Isu takajaira kunyora mabasa ekufona muLua, asi zvinogona kuitika kuti yakapihwa maitiro haina basa ratinoda. Kufambisa mafoni pamusoro petiweki, tinoshandisa iyo rpc (remote process call) yekubatsira module, inovakwa pahwaro hweiyo yakajairwa netbox yakavakwa muTarantool. Izvi zvinogona kubatsira kana, semuenzaniso, gedhi rako richida kubvunza zvakananga iye anoronga kuita basa izvozvi, pane kumirira zuva.

Imwe pfungwa yakakosha kuve nechokwadi chekushivirira mhosva. Cartridge inoshandisa iyo SWIM protocol kutarisa hutano [4]. Muchidimbu, maitiro ekuchinjana "runyerekupe" kune mumwe nemumwe pamusoro peUDP - yega yega inoudza vavakidzani vayo nhau dzichangoburwa, uye vanopindura. Kana kamwe kamwe mhinduro ikasauya, Tarantool inotanga kufungidzira kuti chimwe chinhu chakashata, uye mushure mechinguva inodzokorora rufu uye inotanga kuudza munhu wose akakomberedza nhau idzi.

Kubva pane iyi protocol, Cartridge inoronga otomatiki kutadza kugadzirisa. Maitiro ega ega anotarisisa zvakatipoteredza, uye kana mutungamiri akangoerekana amira kupindura, replica inogona kutora chinzvimbo chayo, uye Cartridge inogadzirisa mabasa ekumhanya zvinoenderana.

Iwe unofanirwa kungwarira pano, nekuti kugara uchichinja nekudzoka kumashure kunogona kutungamira kune kusawirirana kwedata panguva yekudzokorora. Ehezve, haufanirwe kugonesa otomatiki kukundikana chero. Tinofanira kunyatsonzwisisa zviri kuitika uye kuva nechokwadi chokuti kudzokorora hakuzoputsika mushure mokunge mutungamiri adzorerwa uye korona yadzorerwa kwaari.

Kubva pane zvese izvi, unogona kuwana manzwiro ekuti mabasa akafanana ne microservices. Mune imwe pfungwa, ivo ndizvozvo, chete semamodule mukati meTarantool maitiro. Asi kunewo misiyano yakati wandei inokosha. Chekutanga, ese mabasa eprojekiti anofanirwa kugara mune imwechete kodhi base. Uye maitiro ese eTarantool anofanirwa kutangwa kubva kune imwechete kodhi base, kuitira kuti pasave nekushamisika seizvo kana isu tichiedza kutanga iyo scheduler, asi haipo. Uyewo, haufaniri kubvumira kusiyana kwekodhi shanduro, nokuti maitiro ehurongwa mumamiriro ezvinhu akadaro akaoma zvikuru kufanotaura uye kugadzirisa.

Kusiyana neDocker, isu hatigone kungotora chikamu "mufananidzo", tora kune mumwe muchina woumhanyisa ipapo. Basa redu harina kuparadzaniswa semidziyo yeDocker. Zvakare, isu hatigone kuita mabasa maviri akafanana pane imwe nguva. Basa rinogona kunge riripo kana kuti harisi; neimwe nzira, ingori imwe chete. Uye chechitatu, mabasa anofanirwa kunge akafanana mukati meboka rese rekudzokorora, nekuti zvikasadaro zvingave zvisina musoro - data rakafanana, asi kurongeka kwakasiyana.

Deployment tools

Ndakavimbisa kuratidza kuti Cartridge inobatsira sei kutumira zvikumbiro. Kuita kuti hupenyu huve nyore kune vamwe, iyo sisitimu mapakeji RPM mapakeji:

$ cartridge pack rpm myapp -- упакует для нас ./myapp-0.1.0-1.rpm
$ sudo yum install ./myapp-0.1.0-1.rpm

Iyo yakaiswa package ine zvinenge zvese zvaunoda: zvese zviri zviviri application uye zvakaiswa zvinotsamira. Tarantool ichasvikawo pane sevha sekutsamira kweRPM package, uye sevhisi yedu yakagadzirira kutangwa. Izvi zvinoitwa kuburikidza ne systemd, asi chekutanga iwe unofanirwa kunyora zvishoma zvigadziriso. Pashoma, tsanangura iyo URI yega yega maitiro. Zvitatu zvakakwana semuenzaniso.

$ sudo tee /etc/tarantool/conf.d/demo.yml <<CONFIG
myapp.router: {"advertise_uri": "localhost:3301", "http_port": 8080}
myapp.storage_A: {"advertise_uri": "localhost:3302", "http_enabled": False}
myapp.storage_B: {"advertise_uri": "localhost:3303", "http_enabled": False}
CONFIG

Pane inonakidza nuance pano. Panzvimbo pekutsanangura iyo bhinary protocol port, isu tinotsanangura iyo yese kero yeruzhinji yemaitiro kusanganisira zita remugamuchiri. Izvi zvinodikanwa kuitira kuti masumbu emasumbu azive nzira yekubatanidza kune mumwe nemumwe. Ipfungwa yakaipa kushandisa 0.0.0.0 sead_uri kero; inofanirwa kunge iri yekunze IP kero, kwete socket bind. Pasina iyo, hapana chinozoshanda, saka Cartridge haingokuregedze iwe kuvhura node ine isiriyo advertise_uri.

Iye zvino kuti iyo gadziriso yakagadzirira, unogona kutanga maitiro. Sezvo yakajairwa systemd unit isingatenderi inopfuura nzira imwe kutanga, maapplication ari paCartridge anoiswa nevanonzi. instanted units inoshanda seizvi:

$ sudo systemctl start myapp@router
$ sudo systemctl start myapp@storage_A
$ sudo systemctl start myapp@storage_B

Mukugadzirisa, takatsanangura HTTP port iyo Cartridge inoshandisa web interface - 8080. Ngatiendei kwairi uye titarise:

Isu tinoona kuti kunyangwe maitiro ari kushanda, haasati agadziriswa. Iyo cartridge haisati yaziva kuti ndiani anofanira kudzokorora naye uye haakwanise kuita sarudzo ari ega, saka iri kumirira zviito zvedu. Asi isu hatina sarudzo yakawanda: hupenyu hweboka idzva hunotanga nekugadziriswa kwekutanga node. Zvadaro tichawedzera vamwe kuboka, tovapa mabasa, uye panguva ino kutumirwa kunogona kunzi kwakapedzwa kubudirira.

Ngatidire girazi rechinwiwa chaunofarira uye uzorore mushure mevhiki refu rekushanda. Chikumbiro chinogona kushandiswa.