Architecture uye kugona kweTarantool Data Grid

Muna 2017, takahwina mukwikwidzi wekuvandudza musimboti webhizinesi rekudyara reAlfa-Bank uye takatanga basa (paHighLoad ++ 2018 neshumo pamusoro pepakati pebhizimusi rekudyara. kuitwa Vladimir Drynkin, mukuru weiyo transactional musimboti webhizinesi rekudyara reAlfa Bank). Iyi sisitimu yaifanirwa kuunganidza data rekutengesa kubva kwakasiyana mafomati akasiyana, kuunza iyo data mufomu yakabatana, kuichengeta uye kupa mukana kwairi.

Munguva yekuvandudza, sisitimu yakashanduka uye yakawana mashandiro, uye pane imwe nguva takaona kuti isu takanga tichiratidzira chimwe chinhu chinopfuura kungoita software yesoftware yakagadzirwa kugadzirisa huwandu hwakanyatsotsanangurwa hwemabasa: takabudirira. sisitimu yekuvaka yakagovaniswa zvikumbiro neinopfuurira kuchengetedza. Ruzivo rwatakawana rwakagadzira hwaro hwechigadzirwa chitsva - Tarantool Data Grid (TDG).

Ini ndinoda kutaura nezve TDG yekuvakisa uye mhinduro dzatakauya kwadziri panguva yekuvandudza, kukuzivisa iwe kune chikuru mashandiro uye kuratidza kuti chigadzirwa chedu chingave sei hwaro hwekuvaka yakakwana mhinduro.

Architecturally, isu takakamura sisitimu kuita zvakasiyana mabasa, imwe neimwe ine basa rekugadzirisa mamwe matambudziko. Imwe chete inomhanya application muenzaniso inoshandisa imwechete kana akawanda emhando yemhando. Panogona kuve nemabasa akati wandei emhando imwechete musumbu:

Connector

Connector ine basa rekutaurirana nenyika yekunze; basa rayo nderekugamuchira chikumbiro, chekuenzanisa, uye kana izvi zvikabudirira, wobva watumira iyo data kuti igadziriswe kune yekupinza processor. Isu tinotsigira HTTP, SIPO, Kafka, FIX mafomati. Iyo dhizaini inobvumidza iwe kuti uwedzere rutsigiro rwemafomu matsva, nerutsigiro rweBMM MQ iri kuuya munguva pfupi. Kana kupatsanura chikumbiro kwatadza, chinongedzo chinodzosa chikanganiso; kana zvisina kudaro, ichapindura kuti chikumbiro chakagadziriswa zvinobudirira, kunyange kana kukanganisa kwakaitika panguva yekuenderera mberi kwayo. Izvi zvakaitwa zvakananga kuitira kushanda nemasisitimu asingazivi nzira yekudzokorora zvikumbiro - kana, pane zvinopesana, zviitewo uchishingirira. Kuti urege kurasikirwa nedheta, mutsara wekugadzirisa unoshandiswa: chinhu chinotanga kupinda mairi uye chete mushure mekubudirira kwekugadzirisa kubviswa kubva pairi. Mutungamiri anogona kugamuchira chenjedzo nezvezvinhu zvakasara mumutsara wekugadzirisa, uye mushure mekubvisa kukanganisa kwesoftware kana kutadza kwehardware, edza zvakare.

Input processor

Iyo Input processor inoisa iyo yakagamuchirwa data zvinoenderana nehunhu uye inodaidza akakodzera processor. MaHandler iLua kodhi inomhanya mubhokisi rejecha, saka haigone kukanganisa kushanda kwehurongwa. Panguva ino, iyo data inogona kuderedzwa kusvika kune inodiwa fomu, uye, kana zvichidikanwa, nhamba yekupokana yemabasa inogona kutangwa iyo inogona kuita inodiwa logic. Semuenzaniso, muMDM (Master Data Management) chigadzirwa chakavakirwa paTarantool Data Grid, pakuwedzera mushandisi mutsva, kuitira kuti tisanonoke kugadzirisa chikumbiro, tinotangisa kusikwa kwerekodhi regoridhe sebasa rakasiyana. Bhokisi rejecha rinotsigira zvikumbiro zvekuverenga, kushandura uye kuwedzera data, rinokutendera kuti uite rimwe basa pane ese emhando yekuchengetedza uye kuunganidzwa kwemhedzisiro (mepu / kuderedza).

Handlers inogona kutsanangurwa mumafaira:

sum.lua

local x, y = unpack(...)
return x + y

Uye zvino, yakaziviswa mukugadziriswa:

functions:
  sum: { __file: sum.lua }

Sei Lua? Lua mutauro wakapfava zvikuru. Kubva pane zvakaitika kwatiri, maawa mashoma mushure mekuzviziva, vanhu vanotanga kunyora kodhi inogadzirisa dambudziko ravo. Uye izvi hazvisi chete vanogadzira nyanzvi, asi, semuenzaniso, vaongorori. Mukuwedzera, nekuda kwejit compiler, Lua inomhanya nekukurumidza.

chengetedzo

Chengetedzo inochengetedza data inoramba iripo. Usati wachengeta, data inosimbiswa inopesana ne data schema. Kutsanangura dunhu tinoshandisa chimiro chakawedzerwa Apache Avro. Muenzaniso:

{
    "name": "User",
    "type": "record",
    "logicalType": "Aggregate",
    "fields": [ 
        { "name": "id", "type": "string"}, 
        {"name": "first_name", "type": "string"}, 
        {"name": "last_name", "type": "string"} 
    ], 
    "indexes": ["id"] 
}

Kubva pane tsananguro iyi, DDL (Data Tsanangudzo Mutauro) inogadzirwa otomatiki yeTarantula DBMS uye. GraphQL schema yekuwana data.

Asynchronous data replication inotsigirwa (kune zvirongwa zvekuwedzera synchronous imwe).

Output processor

Dzimwe nguva zvinodikanwa kuzivisa vatengi vekunze nezvekuuya kwedata nyowani; nechikonzero ichi, pane iyo Output processor basa. Mushure mekuchengetedza iyo data, inogona kupfuudzwa kune inoenderana inobata (semuenzaniso, kuiunza kune iyo fomu inodiwa nemutengi) - uye yozopfuudzwa kune chinongedzo chekutumira. Mutsara wekugadzirisa unoshandiswawo pano: kana pasina akagamuchira chinhu, mutungamiri anogona kuedza zvakare gare gare.

Kuyera

Iyo yekubatanidza, yekupinza processor, uye inobuda processor mabasa haana chirevo, zvichitibvumira kuyera sisitimu yakatwasuka nekungowedzera zviitiko zvitsva zvekushandisa nemhando yemhando yaunoda inogoneswa. Kuchengetera kunoshandiswa kune horizontal scaling maitiro kuronga sumbu uchishandisa mabhakiti chaiwo. Mushure mekuwedzera sevha itsva, mamwe mabhakiti kubva kumaseva ekare anoendeswa kune itsva server kumashure; izvi zvinoitika pachena kune vashandisi uye hazvikanganisi kushanda kwehurongwa hwose.

Data Properties

Zvinhu zvinogona kunge zvakakura uye zvine zvimwe zvinhu. Isu tinova nechokwadi cheatomicity yekuwedzera nekuvandudza data nekuchengetedza chinhu chine zvese zvinotsamira mubhakiti rimwe chete. Izvi zvinodzivirira chinhu kubva "kupararira" kune akati wandei maseva emuviri.

Kushandura kunotsigirwa: imwe neimwe update yechinhu inogadzira imwe vhezheni, uye isu tinokwanisa kugara tichitora chidimbu chenguva uye kuona kuti nyika yakataridzika sei ipapo. Kune data risingade nhoroondo refu, tinogona kudzikamisa huwandu hweshanduro kana kutochengeta imwe chete - yazvino - kureva, kudzima shanduro yeimwe mhando. Iwe unogona zvakare kudzikisira nhoroondo nenguva: semuenzaniso, bvisa zvinhu zvese zveimwe mhando yakakura pane 1 gore. Kuchengetedza kunotsigirwawo: tinogona kuburitsa zvinhu zvekare kupfuura nguva yakatarwa, tichisunungura nzvimbo musumbu.

mabasa

Pakati pezvinhu zvinonakidza, zvakakosha kucherechedza kugona kuvhura mabasa pane hurongwa, pakukumbira kwemushandisi, kana zvakarongwa kubva kubhokisi rejecha:

Pano tinoona rimwe basa - mumhanyi. Iri basa harina chimiro, uye mamwe ekuwedzera ekushandisa ane basa iri anogona kuwedzerwa kune sumbu sezvinodiwa. Basa remumhanyi kupedzisa mabasa. Sezvataurwa, zvinokwanisika kugadzira mabasa matsva kubva kubhokisi rejecha; ivo vanoponeswa mumutsara pakuchengetedza uye vozourayiwa pamumhanyi. Basa remhando iyi rinonzi Jobho. Isu tinewo rudzi rwebasa rwunodaidzwa kuti Task - aya mabasa anotsanangurwa nemushandisi anomhanya pane hurongwa (uchishandisa cron syntax) kana pakuda. Kuvhura uye kuronda mabasa akadai, isu tine ari nyore basa maneja. Kuti basa iri rivepo, unofanirwa kugonesa basa rehurongwa; basa iri rine nyika, saka harina kuyera, izvo, zvisinei, hazvidiwi; panguva imwe chete, semamwe mabasa ose, inogona kuva nereplica inotanga kushanda kana tenzi akangoerekana aramba.

Logger

Rimwe basa rinonzi logger. Inounganidza matanda kubva kunhengo dzese dzeboka uye inopa interface yekuisa uye kuaona kuburikidza newebhu interface.

Services

Zvakakodzera kutaura kuti sisitimu inoita kuti zvive nyore kugadzira masevhisi. Mune faira yekumisikidza, unogona kutsanangura kuti ndezvipi zvikumbiro zvinotumirwa kumushandisi-akanyorwa mushandisi anomhanya mubhokisi rejecha. Mumubati uyu, unogona, semuenzaniso, kumhanyisa imwe mhando yemubvunzo wekuongorora uye wodzosera mhedzisiro.

Iyo sevhisi inotsanangurwa mufaira rekugadzirisa:

services:
   sum:
      doc: "adds two numbers"
      function: sum
      return_type: int
      args:
         x: int
         y: int

Iyo GraphQL API inogadzirwa otomatiki uye sevhisi inowanikwa pakufona:

query {
   sum(x: 1, y: 2) 
}

Izvi zvinodaidza mubati sumizvo zvichadzosera mhedzisiro:

3

Query Profileling uye Metrics

Kuti tinzwisise mashandiro ehurongwa uye zvikumbiro zveprofile, takaisa rutsigiro rweOpenTracing protocol. Iyo sisitimu inogona kutumira ruzivo nezve kudiwa kune zvishandiso zvinotsigira iyi protocol, seZipkin, izvo zvinokutendera iwe kuti unzwisise kuti chikumbiro chakaitwa sei:

Nomuzvarirwo, iyo sisitimu inopa yemukati metrics inogona kuunganidzwa uchishandisa Prometheus uye inooneswa uchishandisa Grafana.

Deploy

Tarantool Data Grid inogona kuendeswa kubva kuRPM mapakeji kana archive, uchishandisa utility kubva pakugovera kana Ansible, kune zvakare rutsigiro rweKubernetes (Tarantool Kubernetes Operator).

Chishandiso chinoshandisa iyo bhizinesi logic (kugadziridza, zvibatiso) chinotakurwa muakaiswa Tarantool Data Grid cluster muchimiro chekuchengetedza kuburikidza neUI kana kushandisa script kuburikidza neAPI yakapihwa nesu.

Sample Applications

Ndeapi maapplication anogona kugadzirwa uchishandisa Tarantool Data Grid? Muchokwadi, mazhinji mabasa ebhizinesi ane hukama nekugadzirisa, kuchengetedza uye kuwana kuyerera kwedata. Naizvozvo, kana uine hova hombe dzedata dzinoda kuchengetedzwa zvakachengeteka uye kuwanikwa, saka chigadzirwa chedu chinogona kukuchengetedza yakawanda yenguva yekusimudzira uye kutarisa pane rako bhizinesi logic.

Semuenzaniso, tinoda kuunganidza ruzivo pamusoro pemusika wekutengesa, kuitira kuti mune ramangwana, semuenzaniso, tive neruzivo pamusoro pezvakanakisa zvinopa. Muchiitiko ichi, tichasimbisa zvinotevera mabasa:

1. Marobhoti anounganidza ruzivo kubva kune akavhurika masosi achave edu masosi data. Iwe unogona kugadzirisa dambudziko iri uchishandisa yakagadzirira-yakagadzirwa mhinduro kana kunyora kodhi mune chero mutauro.
2. Tevere, Tarantool Data Grid inobvuma uye chengetedza iyo data. Kana iyo data data kubva kune akasiyana masosi akasiyana, saka unogona kunyora kodhi muLua iyo ichaita shanduko kune imwechete fomati. Padanho repamberi rekugadzirisa, iwe zvakare uchakwanisa, semuenzaniso, kusefa zvakapetwa zvinopihwa kana kuwedzera ruzivo nezve vamiririri vanoshanda mumusika mudhatabhesi.
3. Iye zvino iwe watove uine scalable mhinduro musumbu rinogona kuzadzwa nedata uye kuita sarudzo dzedata. Ipapo iwe unokwanisa kuita hutsva hutsva, semuenzaniso, nyora sevhisi iyo ichaita chikumbiro che data uye kupa iyo inobatsira zvakanyanya zuva nezuva - izvi zvinoda mitsetse mishoma mufaira rekugadzirisa uye diki Lua kodhi.

Chii chinotevera?

Chekutanga chedu ndechekuvandudza kureruka kwekusimudzira kushandisa Tarantool Data Grid. Semuyenzaniso, iyi IDE ine rutsigiro rwekuprofiling nekugadzirisa zvibatiso zvinomhanya mubhokisi rejecha.

Isu tinoteererawo zvakanyanya kune nyaya dzekuchengetedza. Parizvino tiri kuita certification neFSTEC yeRussia yekusimbisa mwero wepamusoro wekuchengetedza uye kusangana nezvinodiwa zvekupihwa zvitupa zvesoftware zvinoshandiswa mumasisitimu eruzivo rwedata uye masisitimu ehurumende.

Source: www.habr.com