🥇Tarantool Cartridge: Sharding Lua backend a cikin layi uku

A Rukunin Mail.ru muna da Tarantool - wannan sabar aikace-aikacen ne a cikin Lua, wanda kuma ya ninka azaman bayanan bayanai (ko akasin haka?). Yana da sauri da sanyi, amma iyawar sabar ɗaya har yanzu ba ta da iyaka. Sikeli a tsaye shima ba panacea bane, don haka Tarantool yana da kayan aikin a kwance-modul vshard. [1]. Yana ba ku damar raba bayanai a cikin sabar da yawa, amma dole ne ku yi amfani da shi don saita shi da haɗa dabarun kasuwanci.

Labari mai dadi: mun tattara wasu manyan hotuna (misali [2], [3]) kuma ya ƙirƙiri wani tsarin da zai sauƙaƙa maganin wannan matsala sosai.

Tarantool Cartridge sabon tsari ne don haɓaka hadaddun tsarin rarrabawa. Yana ba ku damar mayar da hankali kan rubuta dabarun kasuwanci maimakon magance matsalolin ababen more rayuwa. A ƙasa da yanke zan gaya muku yadda wannan tsarin ke aiki da yadda ake rubuta ayyukan da aka rarraba ta amfani da shi.

Menene ainihin matsalar?

Muna da tarantula, muna da vshard - menene kuma kuke so?

Na farko, al'amari ne na dacewa. An saita tsarin vshard ta hanyar tebur na Lua. Don tsarin rarraba tsarin tafiyar matakai na Tarantool da yawa don yin aiki daidai, dole ne saitin ya zama iri ɗaya a ko'ina. Babu wanda yake son yin wannan da hannu. Don haka, ana amfani da kowane nau'in rubutun, Mai yiwuwa, da tsarin turawa.

Cartridge da kansa yana sarrafa tsarin vshard, yana yin wannan bisa ga nasa kansa rarraba sanyi. Ainihin fayil ɗin YAML mai sauƙi ne, kwafin wanda aka adana shi a kowane misalin Tarantool. Sauƙaƙe shi ne cewa tsarin da kansa yana kula da tsarin sa kuma yana tabbatar da cewa daidai yake a ko'ina.

Na biyu, shi ne kuma wani al'amari na saukaka. Tsarin vsshard ba shi da alaƙa da haɓaka dabarun kasuwanci kuma kawai ya kawar da mai shirin daga aikinsa. Lokacin da muka tattauna tsarin gine-ginen aikin, yawanci muna magana ne game da abubuwan da aka haɗa da kuma hulɗar su. Ya yi da wuri don yin tunani game da fitar da tari zuwa cibiyoyin bayanai 3.

Mun warware waɗannan matsalolin akai-akai, kuma a wani lokaci mun sami damar haɓaka hanyar da ta sauƙaƙe aiki tare da aikace-aikacen a duk tsawon rayuwarta: halitta, haɓakawa, gwaji, CI / CD, kiyayewa.

Cartridge yana gabatar da manufar rawar kowane tsari na Tarantool. Matsayin ra'ayi ne wanda ke ba mai haɓaka damar mai da hankali kan lambar rubutu. Dukkan ayyukan da ake samu a cikin aikin ana iya gudanar da su akan misalin Tarantool ɗaya, kuma wannan zai isa ga gwaje-gwaje.

Mahimman fasali na Tarantool Cartridge:

ƙungiyar mawaƙa ta atomatik;
fadada ayyukan aikace-aikacen ta amfani da sabbin ayyuka;
samfurin aikace-aikacen don haɓakawa da ƙaddamarwa;
ginanniyar sharding ta atomatik;
haɗin kai tare da tsarin gwajin Luatest;
Gudanar da tari ta amfani da WebUI da API;
marufi da tura kayan aikin.

Sannu Duniya!

Ba zan iya jira don nuna tsarin kanta ba, don haka za mu bar labarin game da gine-gine na gaba kuma mu fara da wani abu mai sauƙi. Idan muka ɗauka cewa Tarantool kanta an riga an shigar dashi, to duk abin da ya rage shine a yi

$ tarantoolctl rocks install cartridge-cli
$ export PATH=$PWD/.rocks/bin/:$PATH

Waɗannan umarni guda biyu za su shigar da kayan aikin layin umarni kuma su ba ku damar ƙirƙirar aikace-aikacenku na farko daga samfuri:

$ cartridge create --name myapp

Kuma wannan shine abin da muke samu:

myapp/
├── .git/
├── .gitignore
├── app/roles/custom.lua
├── deps.sh
├── init.lua
├── myapp-scm-1.rockspec
├── test
│   ├── helper
│   │   ├── integration.lua
│   │   └── unit.lua
│   ├── helper.lua
│   ├── integration/api_test.lua
│   └── unit/sample_test.lua
└── tmp/

Wannan wurin ajiyar git ne tare da shirye-shiryen "Sannu, Duniya!" aikace-aikace. Bari mu yi ƙoƙarin gudu da shi nan da nan, bayan shigar da abubuwan dogaro (ciki har da tsarin kanta):

$ tarantoolctl rocks make
$ ./init.lua --http-port 8080

Don haka, muna da kumburi guda ɗaya wanda ke gudana don aikace-aikacen shared na gaba. Mutumin da ke neman bincike zai iya buɗe mahaɗin yanar gizo nan da nan, ya saita gungu na kumburi ɗaya tare da linzamin kwamfuta kuma ya ji daɗin sakamakon, amma ya yi wuri don murna. Ya zuwa yanzu, aikace-aikacen ba zai iya yin wani abu mai amfani ba, don haka zan gaya muku game da turawa daga baya, amma yanzu lokaci ya yi da za a rubuta lambar.

Ci gaban Aikace-aikacen

Ka yi tunanin, muna tsara aikin da dole ne a karbi bayanai, ajiye shi kuma gina rahoto sau ɗaya a rana.

Za mu fara zana zane kuma mu sanya abubuwa uku akansa: ƙofar, ajiya da mai tsarawa. Muna ci gaba da aikin gine-ginen. Tunda muna amfani da vshard azaman ajiya, muna ƙara vshard-router da vshard-storage zuwa tsarin. Babu ƙofa ko mai tsarawa ba za su shiga wurin ajiyar kai tsaye ba; abin da na'ura mai ba da hanya tsakanin hanyoyin sadarwa ke nan, abin da aka ƙirƙira shi ke nan.

Wannan zane har yanzu ba ya wakiltar ainihin abin da za mu gina a cikin aikin saboda abubuwan da ke tattare da su ba su da tushe. Har yanzu muna buƙatar ganin yadda za a tsara wannan a kan ainihin Tarantool - bari mu haɗa abubuwan da muka haɗa ta tsari.

Akwai ƙaramin ma'ana a kiyaye vshard-router da ƙofa akan al'amura daban-daban. Me yasa muke buƙatar sake hawan hanyar sadarwa idan wannan shine alhakin na'ura mai ba da hanya tsakanin hanyoyin sadarwa? Dole ne a gudanar da su a cikin tsari iri ɗaya. Wato, duka gateway da vshard.router.cfg an fara su ne a cikin tsari ɗaya kuma a bar su su yi hulɗa a cikin gida.

A matakin ƙira, ya dace don aiki tare da sassa uku, amma ni, a matsayin mai haɓakawa, yayin rubuta lambar, ba na son yin tunani game da ƙaddamar da lokuta uku na Tarnatool. Ina buƙatar gudanar da gwaje-gwaje kuma in duba cewa na rubuta ƙofa daidai. Ko watakila ina so in nuna alama ga abokan aiki na. Me yasa zan shiga cikin wahala na tura kwafi uku? Wannan shine yadda aka haifi manufar matsayin. Matsayi shine tsarin luash na yau da kullun wanda Cartridge ke sarrafa tsarin rayuwarsa. A cikin wannan misali akwai hudu daga cikinsu - ƙofa, na'ura mai ba da hanya tsakanin hanyoyin sadarwa, ajiya, tsarawa. Wataƙila akwai ƙari a cikin wani aikin. Ana iya gudanar da dukkan ayyuka a cikin tsari ɗaya, kuma wannan zai isa.

Kuma idan ya zo ga turawa zuwa tsari ko samarwa, to za mu sanya kowane Tarantool aiwatar da nasa tsarin ayyukansa dangane da damar kayan aikin:

Topology management

Dole ne a adana bayanai game da inda ayyukan ke gudana a wani wuri. Kuma wannan "wani wuri" shine tsarin da aka rarraba, wanda na riga na ambata a sama. Abu mafi mahimmanci game da shi shine cluster topology. Anan akwai ƙungiyoyin maimaitawa guda 3 na matakan Tarantool guda 5:

Ba ma son rasa bayanai, don haka muna kula da bayanai game da tafiyar matakai da kulawa. Cartridge yana kiyaye tsarin daidaitawa ta amfani da alƙawarin mataki-biyu. Da zarar muna son sabunta saitin, zai fara bincika cewa duk lamura suna nan kuma a shirye suke don karɓar sabon saitin. Bayan wannan, kashi na biyu yana amfani da saitin. Don haka, ko da kwafin ɗaya ya zama babu shi na ɗan lokaci, babu wani mummunan abu da zai faru. Ba za a yi amfani da tsarin kawai ba kuma za ku ga kuskure a gaba.

Hakanan a cikin sashin topology, ana nuna irin wannan muhimmin ma'auni a matsayin jagoran kowane rukunin kwafi. Yawancin lokaci wannan shine kwafin da ake rikodin. Sauran galibi ana karantawa kawai, kodayake ana iya samun keɓantacce. Wasu lokuta masu haɓaka masu ƙarfin hali ba sa jin tsoron rikice-rikice kuma suna iya rubuta bayanai zuwa kwafi da yawa a layi daya, amma akwai wasu ayyuka waɗanda, ko menene, bai kamata a yi sau biyu ba. Don wannan akwai alamar shugaba.

Rayuwar matsayi

Domin rawar da za ta iya zama a cikin irin wannan gine-gine, dole ne tsarin ya sarrafa su ko ta yaya. A zahiri, sarrafawa yana faruwa ba tare da sake kunna tsarin Tarantool ba. Akwai sake kiran waya guda 4 don gudanar da ayyuka. Cartridge da kansa zai kira su dangane da abin da aka rubuta a cikin tsarin da aka rarraba, don haka amfani da saitin zuwa takamaiman ayyuka.

function init()
function validate_config()
function apply_config()
function stop()

Kowace rawa tana da aiki init. Ana kiran shi sau ɗaya ko dai lokacin da aka kunna rawar ko lokacin da Tarantool ya sake farawa. Ya dace a can, alal misali, don fara box.space.create, ko mai tsarawa zai iya ƙaddamar da wasu fiber na baya wanda zai yi aiki a wasu tazara na lokaci.

Aiki ɗaya init bazai isa ba. Cartridge yana ba da damar matsayi don cin gajiyar tsarin da aka rarraba da yake amfani da shi don adana topology. Za mu iya ayyana sabon sashe a cikin wannan tsari kuma mu adana guntun tsarin kasuwanci a ciki. A cikin misali na, wannan na iya zama tsarin bayanai ko tsarin saituna don rawar mai tsarawa.

Kiran gungu validate_config и apply_config duk lokacin da tsarin da aka rarraba ya canza. Lokacin da aka yi amfani da tsari ta hanyar alƙawarin mataki biyu, gungu yana bincika cewa kowace rawa a shirye take don karɓar wannan sabon tsarin kuma, idan ya cancanta, ta ba da rahoton kuskure ga mai amfani. Lokacin da kowa ya yarda cewa daidaitawar al'ada ne, to apply_config.

Hakanan ayyuka suna da hanya stop, wanda ake buƙata don tsaftace fitar da rawar. Idan muka ce ba a buƙatar mai tsara jadawalin akan wannan uwar garken, zai iya dakatar da waɗannan zaruruwa waɗanda aka fara da su init.

Matsayi na iya hulɗa da juna. An saba rubuta kiran aiki a cikin Lua, amma yana iya faruwa cewa tsarin da aka bayar ba shi da rawar da muke buƙata. Don sauƙaƙe kira akan hanyar sadarwa, muna amfani da rpc (kiran hanya mai nisa) na taimako, wanda aka gina akan madaidaicin akwatin gidan yanar gizon da aka gina cikin Tarantool. Wannan na iya zama da amfani idan, alal misali, ƙofar ku tana son tambayar mai tsarawa kai tsaye don yin aikin a yanzu, maimakon jira kwana ɗaya.

Wani muhimmin batu shine tabbatar da hakuri da kuskure. Cartridge yana amfani da ka'idar SWIM don kula da lafiya [4]. A takaice, matakai suna musayar "jita-jita" da juna akan UDP-kowane tsari yana gaya wa maƙwabtansa sabon labarai, kuma suna amsawa. Idan ba zato ba tsammani amsar ba ta zo ba, Tarantool ya fara zargin wani abu ba daidai ba ne, kuma bayan ɗan lokaci yana karanta mutuwa kuma ya fara gaya wa kowa game da wannan labarin.

Dangane da wannan ƙa'idar, Cartridge yana tsara sarrafa gazawar atomatik. Kowane tsari yana lura da yanayinsa, kuma idan jagora ya daina ba da amsa ba zato ba tsammani, kwafin zai iya ɗaukar nauyin aikinsa, kuma Cartridge yana daidaita ayyukan gudu daidai.

Kuna buƙatar yin hankali a nan, saboda yawan juyawa da baya zai iya haifar da rikice-rikicen bayanai yayin maimaitawa. Tabbas, bai kamata ku kunna gazawar atomatik ba a cikin bazuwar. Dole ne mu fahimci abin da ke faruwa a fili kuma mu tabbata cewa kwafin ba zai karye ba bayan an maido da shugaba kuma an mayar masa da rawani.

Daga duk wannan, zaku iya jin cewa ayyuka suna kama da ƙananan sabis. A wata ma'ana, su ne kawai, kawai a matsayin kayayyaki a cikin ayyukan Tarantool. Amma akwai kuma bambance-bambance na asali. Na farko, duk ayyukan aikin dole ne su rayu a tushen lamba ɗaya. Kuma duk matakan Tarantool ya kamata a ƙaddamar da su daga tushe guda ɗaya, don haka babu wani abin mamaki kamar waɗanda lokacin da muke ƙoƙarin fara tsara tsarin, amma kawai babu shi. Har ila yau, kada ku ƙyale bambance-bambance a cikin nau'ikan code, saboda halayen tsarin a cikin irin wannan yanayin yana da wuyar tsinkaya da kuskure.

Ba kamar Docker ba, ba za mu iya ɗaukar rawar "hoton" kawai ba, ɗauka zuwa wani injin mu gudanar da shi a can. Ayyukanmu ba su keɓe kamar kwantena Docker ba. Hakanan, ba za mu iya gudanar da ayyuka iri ɗaya guda biyu a wani misali ɗaya ba. Rawar ko dai ta wanzu ko babu; a wata ma'ana, guda ce. Kuma na uku, dole ne ayyukan su kasance iri ɗaya a cikin duka rukunin kwafi, saboda in ba haka ba zai zama wauta - bayanan iri ɗaya ne, amma tsarin ya bambanta.

Kayan aikin turawa

Na yi alkawarin nuna yadda Cartridge ke taimakawa tura aikace-aikace. Don sauƙaƙe rayuwa ga wasu, tsarin fakitin RPM:

$ cartridge pack rpm myapp -- упакует для нас ./myapp-0.1.0-1.rpm
$ sudo yum install ./myapp-0.1.0-1.rpm

Kunshin da aka shigar ya ƙunshi kusan duk abin da kuke buƙata: duka aikace-aikacen da abubuwan dogaro da aka shigar. Tarantool kuma zai zo kan sabar a matsayin dogaro na kunshin RPM, kuma sabis ɗinmu yana shirye don ƙaddamarwa. Ana yin wannan ta hanyar tsarin tsarin, amma da farko kuna buƙatar rubuta ɗan ƙaramin tsari. Aƙalla, saka URI na kowane tsari. Uku ya isa misali.

$ sudo tee /etc/tarantool/conf.d/demo.yml <<CONFIG
myapp.router: {"advertise_uri": "localhost:3301", "http_port": 8080}
myapp.storage_A: {"advertise_uri": "localhost:3302", "http_enabled": False}
myapp.storage_B: {"advertise_uri": "localhost:3303", "http_enabled": False}
CONFIG

Akwai nuance mai ban sha'awa a nan. Maimakon ƙayyade kawai tashar yarjejeniya ta binary, mun ƙayyade dukan adireshin jama'a na tsarin ciki har da sunan mai masauki. Wannan ya zama dole don kuɗaɗɗen tari su san yadda ake haɗa juna. Yana da mummunan ra'ayi don amfani da 0.0.0.0 azaman adireshin tallan_uri; yakamata ya zama adireshin IP na waje, ba ɗaure soket ba. Idan ba tare da shi ba, babu abin da zai yi aiki, don haka Cartridge kawai ba zai ƙyale ka ƙaddamar da kumburi tare da advertise_uri mara kyau ba.

Yanzu da sanyi ya shirya, za ku iya fara tafiyar matakai. Tun da naúrar tsarin yau da kullun baya barin tsari fiye da ɗaya farawa, ana shigar da aikace-aikacen akan Cartridge ta abin da ake kira. Raka'o'in nan take masu aiki kamar haka:

$ sudo systemctl start myapp@router
$ sudo systemctl start myapp@storage_A
$ sudo systemctl start myapp@storage_B

A cikin tsarin, mun ƙayyade tashar tashar HTTP wanda Cartridge ke aiki da haɗin yanar gizon - 8080. Bari mu je gare ta mu duba:

Mun ga cewa ko da yake matakai suna gudana, har yanzu ba a daidaita su ba. Harsashin bai riga ya san wanda ya kamata ya yi kwafi da wane ba kuma ba zai iya yanke shawara da kansa ba, don haka yana jiran ayyukanmu. Amma ba mu da zabi mai yawa: rayuwar sabon tari ta fara tare da daidaitawar kumburin farko. Sa'an nan kuma za mu ƙara sauran zuwa gungu, sanya su ayyuka, kuma a wannan lokaci za a iya la'akari da aikin da aka kammala cikin nasara.

Bari mu zuba gilashin abin sha da kuka fi so kuma mu shakata bayan dogon aiki mako. Ana iya amfani da aikace-aikacen.