Zomangamanga ndi kuthekera kwa Tarantool Data Grid

Mu 2017, tidapambana mpikisano wokhazikitsa maziko abizinesi ya Alfa-Bank ndipo tidayamba ntchito (pa HighLoad ++ 2018 ndi lipoti lachiyambi cha bizinesi yogulitsa ndalama. analankhula Vladimir Drynkin, wamkulu wa bizinesi yamalonda ya Alfa Bank). Dongosololi limayenera kuphatikizira deta yochokera kuzinthu zosiyanasiyana m'mitundu yosiyanasiyana, kubweretsa deta mu mawonekedwe ogwirizana, kuisunga ndikupereka mwayi wopeza.

Panthawi yachitukuko, dongosololi lidasinthika ndikupeza magwiridwe antchito, ndipo nthawi ina tidazindikira kuti tikuwunikira china chake kuposa kungogwiritsa ntchito mapulogalamu omwe adapangidwa kuti athetse ntchito zosiyanasiyana: tapambana. dongosolo kumanga anagawira ntchito ndi kusunga mosalekeza. Zomwe tapeza zidapanga maziko a chinthu chatsopano - Tarantool Data Grid (TDG).

Ndikufuna kulankhula za zomangamanga za TDG ndi mayankho omwe tidabwera nawo panthawi yachitukuko, ndikudziwitseni magwiridwe antchito ndikuwonetsa momwe malonda athu angakhalire maziko opangira mayankho athunthu.

Zomangamanga, tinagawaniza dongosolo kukhala losiyana maudindo, iliyonse yomwe ili ndi udindo wothetsa mavuto osiyanasiyana. Ntchito imodzi yogwiritsira ntchito imagwiritsa ntchito mtundu umodzi kapena zingapo. Pakhoza kukhala maudindo angapo amtundu womwewo mumagulu:

cholumikizira

Cholumikizira ali ndi udindo wolumikizana ndi dziko lakunja; ntchito yake ndikuvomera pempho, kusanthula, ndipo ngati izi zikuyenda bwino, tumizani deta kuti ikonzedwe kwa purosesa yolowera. Timathandizira mawonekedwe a HTTP, SOAP, Kafka, FIX. Zomangamangazi zimakupatsani mwayi wongowonjezera chithandizo chamitundu yatsopano, mothandizidwa ndi IBM MQ ikubwera posachedwa. Ngati kusanja pempho kwalephera, cholumikizira chidzabwezera cholakwika; apo ayi, idzayankha kuti pempholo lidakonzedwa bwino, ngakhale cholakwika chikachitika pakukonzanso kwake. Izi zidachitidwa makamaka kuti agwire ntchito ndi machitidwe omwe sadziwa kubwereza zopempha - kapena, m'malo mwake, chitani mosalekeza. Kuti musataye deta, mzere wokonza umagwiritsidwa ntchito: chinthu choyamba chimalowa m'menemo ndipo pokhapokha atachita bwino atachotsedwa. Woyang'anira atha kulandira zidziwitso za zinthu zomwe zatsala pamzere wokonza, ndipo mutatha kuchotsa cholakwika cha pulogalamu kapena kulephera kwa hardware, yesaninso.

Purosesa yolowetsa

Purosesa ya Input imayika data yomwe yalandilidwa molingana ndi mawonekedwe ake ndikuyitanitsa mapurosesa oyenera. Handlers ndi Lua code yomwe imayenda mu sandbox, kotero izo sizingakhudze magwiridwe antchito a dongosolo. Panthawiyi, deta ikhoza kuchepetsedwa kukhala mawonekedwe ofunikira, ndipo, ngati n'koyenera, chiwerengero cha ntchito chikhoza kukhazikitsidwa chomwe chingathe kukhazikitsa mfundo zofunika. Mwachitsanzo, mu mankhwala a MDM (Master Data Management) omwe amamangidwa pa Tarantool Data Grid, powonjezera wogwiritsa ntchito watsopano, kuti asachepetse kukonzanso pempho, timayambitsa kupanga mbiri ya golide ngati ntchito yosiyana. Bokosi la mchenga limathandizira zopempha zowerengera, kusintha ndi kuwonjezera deta, zimakulolani kuchita ntchito zina pamagulu onse amtundu wosungirako ndikuphatikiza zotsatira (mapu / kuchepetsa).

Ma Handlers amatha kufotokozedwa m'mafayilo:

sum.lua

local x, y = unpack(...)
return x + y

Kenako, adalengezedwa mu kasinthidwe:

functions:
  sum: { __file: sum.lua }

Chifukwa Lua? Lua ndi chilankhulo chosavuta. Kutengera zomwe takumana nazo, patadutsa maola angapo titadziwa, anthu amayamba kulemba ma code omwe amathetsa vuto lawo. Ndipo awa si akatswiri okhawo opanga, koma, mwachitsanzo, akatswiri. Kuphatikiza apo, chifukwa cha jit compiler, Lua imathamanga mwachangu kwambiri.

yosungirako

Kusungirako kumasunga zidziwitso zokhazikika. Musanasunge, deta imatsimikiziridwa motsutsana ndi schema ya data. Kufotokozera dera timagwiritsa ntchito mawonekedwe otalikirapo Apache Avro. Chitsanzo:

{
    "name": "User",
    "type": "record",
    "logicalType": "Aggregate",
    "fields": [ 
        { "name": "id", "type": "string"}, 
        {"name": "first_name", "type": "string"}, 
        {"name": "last_name", "type": "string"} 
    ], 
    "indexes": ["id"] 
}

Kutengera malongosoledwe awa, DDL (Data Definition Language) imangopangidwira kwa Tarantula DBMS ndi Chithunzi cha QL schema kuti mupeze deta.

Kubwereza kwa data kwa Asynchronous kumathandizidwa (pali mapulani owonjezera amodzi).

Purosesa yotulutsa

Nthawi zina ndikofunikira kudziwitsa ogula akunja zakubwera kwa data yatsopano; pachifukwa ichi, pali gawo la purosesa ya Output. Pambuyo posunga deta, ikhoza kuperekedwa kwa wothandizira wofanana (mwachitsanzo, kuti abweretse ku mawonekedwe omwe ogula amafunikira) - kenako amaperekedwa kwa cholumikizira kuti atumize. Mzere wokonza umagwiritsidwanso ntchito pano: ngati palibe amene adavomereza chinthucho, woyang'anira akhoza kuyesanso pambuyo pake.

Kukulitsa

Cholumikizira, purosesa yolowetsa, ndi ntchito zotulutsa zotulutsa ndizosawerengeka, zomwe zimatilola kuti tiwongolere dongosolo mopingasa ndikungowonjezera zochitika zatsopano ndi gawo lomwe tikufuna. Kusungirako kumagwiritsidwa ntchito kukweza mopingasa njira kupanga gulu pogwiritsa ntchito ndowa zenizeni. Pambuyo powonjezera seva yatsopano, zidebe zina kuchokera kumaseva akale zimasunthidwa ku seva yatsopano kumbuyo; izi zimachitika mowonekera kwa ogwiritsa ntchito ndipo sizikhudza magwiridwe antchito a dongosolo lonse.

Data Properties

Zinthu zimatha kukhala zazikulu komanso kukhala ndi zinthu zina. Timaonetsetsa kuti atomiki yowonjezera ndikusintha deta posunga chinthu ndi zodalira zonse mu chidebe chimodzi. Izi zimalepheretsa chinthucho "kufalikira" pamaseva angapo akuthupi.

Kusintha kumathandizidwa: kusintha kulikonse kwa chinthu kumapanga mtundu watsopano, ndipo nthawi zonse titha kutenga kagawo kakang'ono ndikuwona momwe dziko lidawonekera panthawiyo. Pazidziwitso zomwe sizifunikira mbiri yakale, titha kuchepetsa kuchuluka kwa mitundu kapena kusunga imodzi yokha - yatsopano - ndiko kuti, kuletsa kumasulira kwamtundu wina. Muthanso kuchepetsa mbiriyo ndi nthawi: mwachitsanzo, chotsani zinthu zonse zamtundu wina wakale kuposa chaka chimodzi. Kusunga zakale kumathandizidwanso: titha kutsitsa zinthu zakale kuposa nthawi yomwe tafotokoza, ndikumasula malo mgulu.

ntchito

Zina mwazinthu zosangalatsa, ndikofunika kuzindikira kuthekera koyambitsa ntchito pa ndandanda, pa pempho la wogwiritsa ntchito, kapena mwadongosolo kuchokera pa sandbox:

Apa tikuwona udindo wina - wothamanga. Ntchitoyi ndi yopanda malire, ndipo zowonjezera zowonjezera zomwe zili ndi gawoli zitha kuwonjezeredwa kumagulu ngati pakufunika. Udindo wa wothamanga ndi kumaliza ntchito. Monga tafotokozera, ndizotheka kupanga ntchito zatsopano kuchokera ku sandbox; amasungidwa pamzere wosungirako ndikuphedwa pa wothamanga. Ntchito imeneyi imatchedwa Yobu. Tilinso ndi mtundu wa ntchito wotchedwa Task - izi ndi ntchito zofotokozedwa ndi ogwiritsa ntchito zomwe zimayenda pa ndandanda (pogwiritsa ntchito cron syntax) kapena pakufunika. Kuti tiyambitse ndikutsata ntchito zotere, tili ndi woyang'anira ntchito wosavuta. Kuti ntchitoyi ipezeke, muyenera kuloleza gawo la scheduler; udindo uwu uli ndi boma, kotero sichimakula, chomwe, komabe, sichifunika; panthawi imodzimodziyo, monga maudindo ena onse, ikhoza kukhala ndi chofanana chomwe chimayamba kugwira ntchito ngati mbuye akukana mwadzidzidzi.

Logger

Ntchito ina imatchedwa logger. Imasonkhanitsa zipika kuchokera kwa mamembala onse a gululo ndipo imapereka mawonekedwe oti muyike ndikuyiwona kudzera pa intaneti.

Ntchito

Ndikoyenera kutchula kuti dongosolo limapangitsa kukhala kosavuta kupanga mautumiki. Mufayilo yosinthira, mutha kufotokoza zomwe zimatumizidwa kwa wogwiritsa ntchito yemwe amayenda mu sandbox. Mwa chogwirizira ichi, mutha, mwachitsanzo, kuyendetsa mtundu wina wafunso ndikubweza zotsatira.

Ntchitoyi ikufotokozedwa mufayilo yosinthira:

services:
   sum:
      doc: "adds two numbers"
      function: sum
      return_type: int
      args:
         x: int
         y: int

GraphQL API imapangidwa yokha ndipo ntchitoyo imakhalapo pakuyimba foni:

query {
   sum(x: 1, y: 2) 
}

Izi zidzayitana wothandizira sumzomwe zidzabweze zotsatira:

3

Mbiri Yamafunso ndi Metrics

Kuti timvetsetse momwe dongosololi limagwirira ntchito komanso zopempha za mbiri, tidakhazikitsa chithandizo cha OpenTracing protocol. Dongosololi limatha kutumiza zidziwitso zofunikila ku zida zomwe zimathandizira protocol iyi, monga Zipkin, zomwe zingakuthandizeni kumvetsetsa momwe pempholo lidachitidwira:

Mwachilengedwe, dongosololi limapereka ma metric amkati omwe amatha kusonkhanitsidwa pogwiritsa ntchito Prometheus ndikuwonera pogwiritsa ntchito Grafana.

Ikani

Tarantool Data Grid ikhoza kutumizidwa kuchokera ku phukusi la RPM kapena posungira, pogwiritsa ntchito zofunikira kuchokera kugawa kapena Ansible, palinso chithandizo cha Kubernetes (Tarantool Kubernetes Operator).

Ntchito yomwe imagwiritsa ntchito malingaliro abizinesi (kusintha, zowongolera) imalowetsedwa mu gulu la Tarantool Data Grid lomwe layikidwa munjira yosungiramo zakale kudzera mu UI kapena kugwiritsa ntchito script kudzera mu API yomwe tapereka.

Zitsanzo Mapulogalamu

Ndi mapulogalamu ati omwe angapangidwe pogwiritsa ntchito Tarantool Data Grid? M'malo mwake, ntchito zambiri zamabizinesi zimakhudzana mwanjira ina ndikukonza, kusunga ndi kupeza ma data. Chifukwa chake, ngati muli ndi mitsinje yayikulu ya data yomwe imayenera kusungidwa bwino ndikufikiridwa, ndiye kuti malonda athu amatha kukupulumutsirani nthawi yambiri yachitukuko ndikuganizira kwambiri malingaliro anu abizinesi.

Mwachitsanzo, tikufuna kusonkhanitsa zambiri za msika wogulitsa nyumba, kotero kuti m'tsogolomu, mwachitsanzo, tidzakhala ndi chidziwitso cha zopereka zabwino kwambiri. M'malo mwake, tikuwonetsani ntchito zotsatirazi:

1. Maloboti omwe amasonkhanitsa zambiri kuchokera kumalo otseguka adzakhala magwero athu a deta. Mutha kuthana ndi vutoli pogwiritsa ntchito mayankho okonzeka kapena kulemba chilankhulo chilichonse.
2. Kenaka, Tarantool Data Grid idzavomereza ndikusunga deta. Ngati mawonekedwe a data kuchokera kumagwero osiyanasiyana ndi osiyana, ndiye kuti mutha kulemba kachidindo mu Lua yomwe ingasinthe kukhala mtundu umodzi. Mukangokonzekera, mudzatha, mwachitsanzo, kusefa zomwe zabwerezedwa kapena kuwonjezera zambiri za othandizira omwe akugwira ntchito pamsika pankhokwe.
3. Tsopano muli ndi kale njira yothetsera scalable mumagulu omwe angathe kudzazidwa ndi deta ndikupanga zosankha za data. Ndiye mukhoza kukhazikitsa ntchito zatsopano, mwachitsanzo, lembani ntchito yomwe idzapempha deta ndikupereka zopindulitsa kwambiri patsiku - izi zidzafuna mizere ingapo mu fayilo yokonzekera ndi kachidindo kakang'ono ka Lua.

Kodi yotsatira?

Cholinga chathu ndikuwongolera kumasuka kwachitukuko pogwiritsa ntchito Tarantool Data Grid. Mwachitsanzo, iyi ndi IDE yokhala ndi chithandizo chowongolera ndi kukonza zolakwika zomwe zikuyenda mu sandbox.

Timasamalanso kwambiri zachitetezo. Pakali pano tikupatsidwa satifiketi ndi FSTEC yaku Russia kuti titsimikizire zachitetezo chambiri ndikukwaniritsa zofunikira pakutsimikizira kwazinthu zamapulogalamu zomwe zimagwiritsidwa ntchito pamakina a chidziwitso chamunthu komanso machitidwe azidziwitso zaboma.

Source: www.habr.com