Uyilo kunye nobuchule beTarantool Data Grid

Ngo-2017, siphumelele ukhuphiswano lokuphuhlisa ingundoqo yentengiselwano yeshishini lotyalo-mali lwe-Alfa-Bank kwaye saqala umsebenzi (kwi-HighLoad ++ 2018 kunye nengxelo malunga neyona nto ingundoqo kwishishini lotyalo-mali. yenziwe UVladimir Drynkin, intloko yentengiselwano engundoqo kwishishini lotyalo-mali lwe-Alfa Bank). Le nkqubo bekufanele ukuba idibanise idatha yentengiselwano evela kwimithombo eyahlukeneyo kwiifomathi ezahlukeneyo, izise idatha kwifom edibeneyo, igcine kwaye inikeze ukufikelela kuyo.

Ngexesha lenkqubo yophuhliso, inkqubo yavela kwaye yafumana ukusebenza, kwaye ngaxa lithile siye saqonda ukuba siqaqambisa into engaphezulu kunesoftware yesicelo eyenzelwe ukusombulula uluhlu olucacileyo lwemisebenzi: siphumelele. inkqubo yokwakha izicelo ezisasazwayo kunye nokugcinwa okuzingileyo. Amava esiwafumeneyo enze isiseko semveliso entsha - Tarantool Data Grid (TDG).

Ndifuna ukuthetha nge-architecture ye-TDG kunye nezisombululo esize kuzo ngexesha lenkqubo yophuhliso, ukukwazisa kumsebenzi oyintloko kwaye ubonise indlela imveliso yethu enokuba sisiseko sokwakha izisombululo ezipheleleyo.

Ngokwezakhiwo, sahlulahlula inkqubo ngokwahlukeneyo iindima, nganye kuzo inoxanduva lokusombulula uluhlu oluthile lweengxaki. Isicelo esisodwa esisebenzayo siphumeza uhlobo lwendima enye okanye ngaphezulu. Kunokubakho iindima ezininzi zohlobo olufanayo kwiqela:

isinxibelelanisi

Umnxibelelanisi unoxanduva lokunxibelelana nehlabathi langaphandle; umsebenzi wayo kukwamkela isicelo, sicazulule, kwaye ukuba oku kuyaphumelela, emva koko thumela idatha yokuqhubela phambili kwiprosesa yegalelo. Sixhasa i-HTTP, i-SOAP, i-Kafka, iifomati ze-FIX. Uyilo lukuvumela ukuba wongeze ngokulula inkxaso yeefomati ezintsha, kunye nenkxaso ye-IBM MQ iyeza kungekudala. Ukuba ulwahlulo lwesicelo aluphumelelanga, umnxibelelanisi uya kubuyisela impazamo; kungenjalo, iya kuphendula ukuba isicelo siqhutywe ngempumelelo, nokuba kwenzeke impazamo ngexesha lokuqhubekeka kwayo. Oku kwenziwa ngokukodwa ukuze kusetyenzwe kunye neenkqubo ezingakwaziyo ukuphinda izicelo - okanye, ngokuchaseneyo, zenze oko ngokuqhubekayo. Ukuze ungalahlekelwa yidatha, umgca wokulungisa usetyenziswa: into yokuqala ingena kuyo kwaye kuphela emva kokuba kuqhutywe ngempumelelo ukususwa kuyo. Umlawuli unokufumana izilumkiso malunga nezinto ezisele kumgca wokulungisa, kwaye emva kokuphelisa impazamo yesoftware okanye ukungaphumeleli kwehardware, zama kwakhona.

Iprosesa yokufaka

I-Input processor ihlela idatha efunyenweyo ngokweempawu zeempawu kwaye ifowunela abaqhubekekisi abafanelekileyo. Abaphangi yikhowudi yeLua esebenza kwibhokisi yesanti, ngoko abanakuchaphazela ukusebenza kwenkqubo. Ngeli nqanaba, idatha ingancitshiswa kwifom efunekayo, kwaye, ukuba kuyimfuneko, inani elingaqhelekanga lemisebenzi lingaqaliswa elinokuphumeza ingqiqo efunekayo. Ngokomzekelo, kwi-MDM (i-Master Data Management) imveliso eyakhiwe kwi-Tarantool Data Grid, xa ufaka umsebenzisi omtsha, ukuze ungacothi ukucutshungulwa kwesicelo, siqalisa ukudala irekhodi yegolide njengomsebenzi ohlukeneyo. Ibhokisi yesanti isekela izicelo zokufunda, ukutshintsha kunye nokongeza idatha, ikuvumela ukuba wenze umsebenzi othile kuzo zonke iindima zohlobo lokugcina kunye nokuhlanganiswa kwesiphumo (imephu / ukunciphisa).

Abaphangi banokuchazwa kwiifayile:

sum.lua

local x, y = unpack(...)
return x + y

Kwaye emva koko, kubhengezwe kuqwalaselo:

functions:
  sum: { __file: sum.lua }

Kutheni uLua? IsiLua lulwimi olulula kakhulu. Ngokusekwe kumava ethu, iiyure ezimbalwa emva kokuyazi, abantu baqala ukubhala ikhowudi esombulula ingxaki yabo. Kwaye aba abaphuhlisi abachwephesha kuphela, kodwa, umzekelo, abahlalutyi. Ukongeza, ngenxa yejit compiler, uLua ubaleka ngokukhawuleza.

yoGcino

Ugcino lugcina idatha eqhubekayo. Ngaphambi kokugcinwa, idatha iqinisekisiwe ngokuchasene ne-schema yedatha. Ukuchaza isekethe sisebenzisa ifomathi eyandisiweyo Apache Avro. Umzekelo:

{
    "name": "User",
    "type": "record",
    "logicalType": "Aggregate",
    "fields": [ 
        { "name": "id", "type": "string"}, 
        {"name": "first_name", "type": "string"}, 
        {"name": "last_name", "type": "string"} 
    ], 
    "indexes": ["id"] 
}

Ngokusekelwe kule nkcazo, i-DDL (uLwimi lweNgcaciso yeDatha) yenziwa ngokuzenzekelayo kwi-Tarantula DBMS kunye IgrafuQL Isicwangciso sofikelelo lwedatha.

Ukuphindaphindwa kwedatha ye-asynchronous kuyaxhaswa (kukho izicwangciso zokongeza i-synchronous enye).

Iprosesa yemveliso

Ngamanye amaxesha kuyimfuneko ukwazisa abathengi bangaphandle malunga nokufika kwedatha entsha; Emva kokugcina idatha, inokudluliselwa kumphathi ohambelanayo (umzekelo, ukuyizisa kwifom efunwa ngumthengi) - kwaye emva koko idluliselwe kwikhonkco ukuze ithunyelwe. Umgca wokulungisa usetyenziswa apha: ukuba akukho mntu wamkela into, umlawuli unokuzama kwakhona kamva.

Ukukala

Isidibanisi, iprosesa yegalelo, kunye neendima zeprosesa zemveliso azinammiselo, zisivumela ukuba silinganise inkqubo ngokuthe tye ngokongeza nje iimeko ezitsha zesicelo kunye nohlobo lwendima olufunekayo olusebenzayo. Ugcino lusetyenziselwa ukulinganisa okuthe tye indlela ukulungelelanisa iqela usebenzisa amabhakethi abonakalayo. Emva kokongeza umncedisi omtsha, ezinye iibhakethi ezivela kwiiseva ezindala zihanjiswa kumncedisi omtsha ngasemva; oku kwenzeka ngokungafihlisiyo kubasebenzisi kwaye akuchaphazeli ukusebenza kwenkqubo yonke.

IiPropati zeDatha

Izinto zinokuba nkulu kakhulu kwaye ziqulathe ezinye izinto. Siqinisekisa i-atomicity yokongeza kunye nokuhlaziya idatha ngokugcina into kunye nabo bonke abaxhomekeke kwibhakethi enye ebonakalayo. Oku kuthintela into ekubeni "isasazeke" kwiiseva ezininzi ezibonakalayo.

Uguqulelo luyaxhaswa: uhlaziyo ngalunye lwento ludala inguqulelo entsha, kwaye sinokuhlala sithatha isilayidi sexesha kwaye sibone ukuba ihlabathi lalijongeka njani ngoko. Ngedatha engadingi mbali ende, sinokulinciphisa inani leenguqulelo okanye sigcine enye kuphela - eyamva nje- oko kukuthi, ngokusisiseko sivale uguqulelo lohlobo oluthile. Ungakwazi kwakhona ukunciphisa imbali ngexesha: umzekelo, cima zonke izinto zohlobo oluthile oludala kunonyaka omnye. Ugcino lwamaxwebhu lukwaxhaswa: sinokukhulula izinto ezindala kunexesha elichaziweyo, sikhulula indawo kwiqela.

Iinjongo

Phakathi kwezinto ezinomdla, kuyafaneleka ukuba uqaphele ukukwazi ukuqalisa imisebenzi kwishedyuli, ngesicelo somsebenzisi, okanye ngokwenkqubo kwibhokisi yesanti:

Apha sibona enye indima - imbaleki. Le ndima ayinammiselo, kwaye iimeko ezongezelelweyo zesicelo kunye nale ndima zinokongezwa kwiqela njengoko kufuneka. Uxanduva lwembaleki kukugqiba imisebenzi. Njengoko kukhankanyiwe, kunokwenzeka ukuvelisa imisebenzi emitsha kwibhokisi yesanti; zigcinwa kumgca kugcino kwaye emva koko zibulawe kwimbaleki. Olu hlobo lomsebenzi lubizwa ngokuba ngumsebenzi. Sikwanalo nodidi lomsebenzi olubizwa ngokuba nguMsebenzi - le yimisebenzi echazwe ngumsebenzisi eqhuba kwishedyuli (usebenzisa i-cron syntax) okanye ngokwemfuno. Ukuqalisa kunye nokulandelela imisebenzi enjalo, sinomphathi wemisebenzi efanelekileyo. Ukuze lo msebenzi ube nokufumaneka, kufuneka uvule indima yomcwangcisi; le nxaxheba inombuso, ngoko ke ayilinganisi, engafunekiyo; ngexesha elifanayo, njengazo zonke ezinye iindima, ingaba ne-replica eqala ukusebenza ukuba inkosi ngokukhawuleza iyala.

Logger

Enye indima ibizwa ngokuba yi-logger. Iqokelela iilogi kuwo onke amalungu eqela kwaye ibonelela nge-interface yokulayisha kunye nokujonga nge-interface yewebhu.

Iinkonzo

Kufanelekile ukukhankanya ukuba inkqubo yenza kube lula ukwenza iinkonzo. Kwifayile yoqwalaselo, ungaxela ukuba zeziphi izicelo ezithunyelwa kumphathi obhalwe ngumsebenzisi osebenza kwibhokisi yesanti. Kulo mphathi, unako, umzekelo, ukuqhuba uhlobo oluthile lombuzo wohlalutyo kwaye ubuyisele isiphumo.

Inkonzo ichazwe kwifayile yoqwalaselo:

services:
   sum:
      doc: "adds two numbers"
      function: sum
      return_type: int
      args:
         x: int
         y: int

IGraphQL API yenziwa ngokuzenzekelayo kwaye inkonzo ifumaneka ngokufowuna:

query {
   sum(x: 1, y: 2) 
}

Oku kuyakubiza umphathi sumeya kubuyisela isiphumo:

3

Umbuzo weProfayili kunye neeMetrics

Ukuqonda ukusebenza kwenkqubo kunye nezicelo zeprofayili, siphumeze inkxaso ye-OpenTracing protocol. Inkqubo inokuthumela ulwazi ngemfuno kwizixhobo ezixhasa le protocol, njengeZipkin, eya kukuvumela ukuba uqonde ukuba isicelo senziwe njani:

Ngokwemvelo, inkqubo ibonelela ngeemetriki zangaphakathi ezinokuqokelelwa kusetyenziswa i-Prometheus kwaye ibonakale isebenzisa iGrafana.

Hambisa

IGridi yeDatha yeTarantool inokuhanjiswa kwiipakethe zeRPM okanye kwindawo yokugcina izinto, kusetyenziswa usetyenziso olusuka kunikezelo okanye iAnsible, kukho nenkxaso yeKubernetes (Tarantool Kubernetes Operator).

Isicelo esisebenzisa ingqiqo yezoshishino (ubumbeko, abaphathi) ilayishwe kwi-Tarantool Data Grid cluster ngendlela yogcino nge-UI okanye kusetyenziswa iskripthi nge-API esiyinikwe sithi.

IiNkqubo zeSampuli

Zeziphi izicelo ezinokudalwa kusetyenziswa iTarantool Data Grid? Ngapha koko, uninzi lwemisebenzi yeshishini ngandlela thile inxulumene nokusetyenzwa, ukugcina kunye nokufikelela ukuhamba kwedatha. Ke ngoko, ukuba unemijelo emikhulu yedatha ekufuneka igcinwe ngokukhuselekileyo kwaye ifikeleleke, ke imveliso yethu inokukugcina ixesha elininzi lophuhliso kwaye ugxininise kwingqiqo yeshishini lakho.

Ngokomzekelo, sifuna ukuqokelela ulwazi malunga neemarike ze-real estate, ukwenzela ukuba kwixesha elizayo, umzekelo, siya kuba nolwazi malunga nezibonelelo ezingcono kakhulu. Kule meko, siya kugxininisa le misebenzi ilandelayo:

1. Iirobhothi eziqokelela ulwazi kwimithombo evulekileyo ziya kuba yimithombo yethu yedatha. Unokusombulula le ngxaki usebenzisa izisombululo esele zenziwe okanye ikhowudi yokubhala kulo naluphi na ulwimi.
2. Okulandelayo, iTarantool Data Grid iya kwamkela kwaye igcine idatha. Ukuba ifomathi yedatha evela kwimithombo eyahlukeneyo ihluke, ngoko ungabhala ikhowudi kwi-Lua eya kwenza ukuguqulwa kwifomathi enye. Kwinqanaba lokulungisa kwangaphambili, uya kuba nakho, umzekelo, ukuhluza unikezelo oluphindwe kabini okanye ukongeza ulwazi malunga neearhente ezisebenza kwimarike kwisiseko sedatha.
3. Ngoku sele unesicombululo esinokukhawuleza kwi-cluster enokuthi izaliswe ngedatha kwaye wenze ukhetho lwedatha. Emva koko unokuphumeza umsebenzi omtsha, umzekelo, bhala inkonzo eya kwenza isicelo sedatha kwaye unike isibonelelo esinenzuzo kakhulu ngosuku - oku kuya kufuna imigca embalwa kwifayile yoqwalaselo kunye nekhowudi encinci yeLua.

Yintoni elandelayo?

Eyona nto iphambili yethu kukuphucula ukusebenziseka lula kophuhliso Tarantool Data Grid. Umzekelo, le yi-IDE enenkxaso yeprofayile kunye nolungiso lweempazamo ezisebenza kwibhokisi yesanti.

Sikwanika ingqwalasela enkulu kwimiba yokhuseleko. Okwangoku siphantsi koqinisekiso lwe-FSTEC yaseRashiya ukuze siqinisekise umgangatho ophezulu wokhuseleko kwaye sihlangabezane neemfuno zoqinisekiso lweemveliso zesoftware ezisetyenziswa kwiinkqubo zolwazi lobuqu kunye neenkqubo zolwazi zikarhulumente.

umthombo: www.habr.com