🥇 Gina tubalan aikace-aikacen da aka rarraba. Kimanta na biyu

Sanarwa

Abokan aiki, a tsakiyar lokacin rani na yi shirin sakin wani jerin labarai game da ƙirar tsarin layi: "Gwajin VTrade" - ƙoƙari na rubuta tsarin tsarin ciniki. Jerin zai bincika ka'idar da aikin gina musayar, gwanjo da kantin sayar da kayayyaki. A ƙarshen labarin, ina gayyatar ku don zaɓar batutuwan da suka fi sha'awar ku.

Wannan shine labarin ƙarshe a cikin jerin kan aikace-aikacen amsawa da aka rarraba a cikin Erlang/Elixir. IN labarin farko za ka iya samun tushen ka'idar reactive gine. Labari na biyu yana kwatanta tsarin asali da hanyoyin gina irin waɗannan tsarin.

A yau za mu gabatar da al'amurran da suka shafi ci gaban code tushe da ayyuka a gaba ɗaya.

Ƙungiyar ayyuka

A rayuwa ta gaske, lokacin haɓaka sabis, galibi dole ne ku haɗa samfuran hulɗa da yawa a cikin mai sarrafawa ɗaya. Misali, sabis na masu amfani, wanda ke magance matsalar sarrafa bayanan mai amfani, dole ne ya amsa buƙatun buƙatun da ba da rahoton sabunta bayanan martaba ta hanyar mashaya-sub. Wannan shari'ar mai sauƙi ce: a bayan saƙo akwai mai sarrafawa guda ɗaya wanda ke aiwatar da dabarun sabis kuma yana buga sabuntawa.

Halin yana ƙara rikitarwa lokacin da muke buƙatar aiwatar da sabis ɗin rarraba mai jure rashin kuskure. Bari mu yi tunanin cewa buƙatun masu amfani sun canza:

yanzu ya kamata sabis ɗin ya aiwatar da buƙatun akan nodes 5,
iya yin ayyukan sarrafa bayanan baya,
sannan kuma ku sami damar sarrafa lissafin biyan kuɗi a hankali don sabunta bayanan martaba.

Sharhi: Ba ma yin la'akari da batun daidaitawar ajiya da kwafin bayanai. Bari mu ɗauka cewa an warware waɗannan batutuwan tun da farko kuma tsarin ya riga ya sami abin dogaro kuma mai daidaitawa, kuma masu sarrafa suna da hanyoyin yin hulɗa da shi.

Babban bayanin sabis na masu amfani ya zama mafi rikitarwa. A mahangar mai shirye-shirye, canje-canje ba su da yawa saboda amfani da saƙo. Don biyan buƙatun farko, muna buƙatar saita daidaitawa a wurin musayar req-resp.

Buƙatar aiwatar da ayyukan bango yana faruwa akai-akai. A cikin masu amfani, wannan na iya zama duba takaddun mai amfani, sarrafa abubuwan da aka zazzage multimedia, ko daidaita bayanai tare da kafofin watsa labarun. hanyoyin sadarwa. Ana buƙatar rarraba waɗannan ayyuka ko ta yaya a cikin gungu kuma a kula da ci gaban aiwatar da aiwatarwa. Sabili da haka, muna da zaɓuɓɓukan mafita guda biyu: ko dai yi amfani da samfurin rarraba ayyuka daga labarin da ya gabata, ko, idan bai dace ba, rubuta mai tsara aikin al'ada wanda zai sarrafa tafkin na'urori masu sarrafawa a hanyar da muke bukata.

Batu 3 yana buƙatar tsawaita samfurin mashaya-sub. Kuma don aiwatarwa, bayan ƙirƙirar wurin musayar mashaya-sub, muna buƙatar bugu da ƙari ƙaddamar da mai sarrafa wannan batu a cikin sabis ɗinmu. Don haka, kamar muna motsa dabaru don sarrafa biyan kuɗi da cire rajista daga layin saƙo zuwa aiwatar da masu amfani.

A sakamakon haka, lalata matsalar ya nuna cewa don biyan buƙatun, muna buƙatar ƙaddamar da lokuta 5 na sabis a kan nodes daban-daban da kuma haifar da ƙarin mahallin - mai kula da mashaya, wanda ke da alhakin biyan kuɗi.
Don gudanar da ma'aikata 5, ba kwa buƙatar canza lambar sabis. Ƙarin ƙarin aikin shine kafa ƙa'idodin daidaitawa a wurin musayar, wanda za mu yi magana game da shi kadan daga baya.
Hakanan akwai ƙarin rikitarwa: mai sarrafa mashaya da mai tsara ɗawainiya na al'ada dole ne suyi aiki a cikin kwafi ɗaya. Bugu da ƙari, sabis na saƙo, a matsayin mai mahimmanci, dole ne ya samar da hanyar zabar jagora.

Zaɓin jagora

A cikin tsarin da aka rarraba, zaɓen shugabanni hanya ce ta nada tsari guda ɗaya da ke da alhakin tsara tsara rarraba wasu kaya.

A cikin tsarin da ba su da sauƙi ga daidaitawa, ana amfani da algorithms na duniya da haɗin kai, irin su paxos ko raft.
Tunda saƙon dillali ne kuma jigon tsakiya, ya san game da duk masu kula da sabis - shugabannin 'yan takara. Saƙo na iya nada shugaba ba tare da jefa ƙuri'a ba.

Bayan farawa da haɗawa zuwa wurin musayar, duk ayyuka suna karɓar saƙon tsarin #'$leader'{exchange = ?EXCHANGE, pid = LeaderPid, servers = Servers}. Idan LeaderPid yayi daidai da pid tsari na yanzu, an nada shi a matsayin jagora, da jerin sunayen Servers ya haɗa da duk nodes da sigoginsu.
A halin yanzu wata sabuwa ta bayyana kuma an katse kumburin gungu mai aiki, duk masu sarrafa sabis suna karɓa #'$slave_up'{exchange = ?EXCHANGE, pid = SlavePid, options = SlaveOpts} и #'$slave_down'{exchange = ?EXCHANGE, pid = SlavePid, options = SlaveOpts} daidai da.

Ta wannan hanyar, duk abubuwan da aka gyara suna sane da duk canje-canje, kuma gungu yana da tabbacin samun jagora ɗaya a kowane lokaci.

Masu shiga tsakani

Don aiwatar da hadaddun tsarin sarrafawa da aka rarraba, da kuma a cikin matsalolin haɓaka gine-ginen da ke akwai, ya dace don amfani da masu tsaka-tsaki.
Don kada a canza lambar sabis da warware, alal misali, matsalolin ƙarin aiki, sarrafa ko saƙon shiga, zaku iya kunna mai sarrafa wakili kafin sabis ɗin, wanda zai yi duk ƙarin aikin.

Misali na yau da kullun na inganta rukunin mashaya shine aikace-aikacen da aka rarraba tare da ainihin kasuwancin da ke haifar da abubuwan sabuntawa, kamar canje-canjen farashi a kasuwa, da layin samun dama - Sabar N waɗanda ke ba da API na socket API don abokan ciniki na yanar gizo.
Idan kun yanke shawara gaba-gaba, to sabis na abokin ciniki yayi kama da haka:

abokin ciniki ya kafa haɗin gwiwa tare da dandamali. A gefen uwar garken da ke dakatar da zirga-zirga, ana ƙaddamar da tsari don hidimar wannan haɗin.
A cikin mahallin tsarin sabis, izini da biyan kuɗin sabuntawa na faruwa. Tsarin yana kiran hanyar biyan kuɗi don batutuwa.
Da zarar an haifar da wani abu a cikin kernel, ana isar da shi zuwa hanyoyin da ke ba da haɗin kai.

Bari mu yi tunanin cewa muna da masu biyan kuɗi 50000 zuwa batun "labarai". Ana rarraba masu biyan kuɗi a ko'ina cikin sabobin 5. Sakamakon haka, kowane sabuntawa, wanda ya isa wurin musayar, za a maimaita shi sau 50000: sau 10000 akan kowane sabar, gwargwadon adadin masu biyan kuɗi akan sa. Ba tsari mai tasiri ba ne, daidai?
Don inganta yanayin, bari mu gabatar da wakili mai suna iri ɗaya da wurin musayar. Mai rejista sunan duniya dole ne ya iya dawo da tsari mafi kusa da suna, wannan yana da mahimmanci.

Bari mu ƙaddamar da wannan wakili akan sabar sabar shiga, kuma duk ayyukanmu da ke hidimar websocket api za su yi rajista da shi, ba zuwa ainihin wurin musayar mashaya-sub a cikin kernel ba. Proxy yana biyan kuɗi zuwa ainihin kawai a yanayin biyan kuɗi na musamman kuma yana maimaita saƙo mai shigowa ga duk masu biyan kuɗin sa.
Sakamakon haka, za a aika saƙonni 5 tsakanin kernel da sabar shiga, maimakon 50000.

Gudanarwa da daidaitawa

Req-Resp

A cikin aiwatar da saƙon na yanzu, akwai dabarun rarraba buƙatun guda 7:

default. Ana aika buƙatar zuwa duk masu sarrafawa.
round-robin. An ƙididdige buƙatun kuma ana rarraba su ta hanyar keke tsakanin masu sarrafawa.
consensus. An raba masu kulawa da ke hidimar sabis ɗin zuwa shugabanni da bayi. Ana aika buƙatun zuwa ga jagora kawai.
consensus & round-robin. Ƙungiyar tana da jagora, amma ana rarraba buƙatun ga duk membobin.
sticky. Ana ƙididdige aikin hash kuma an sanya shi ga takamaiman mai kulawa. Buƙatun na gaba tare da wannan sa hannun suna zuwa ga mai kulawa iri ɗaya.
sticky-fun. Lokacin farawa wurin musayar, lissafin zanta yana aiki don sticky daidaitawa.
fun. Kama da m-fun, kai kaɗai za ka iya kuma turawa, ƙi ko riga-kafi aiwatar da shi.

An saita dabarun rarraba lokacin da aka fara musayar musayar.

Baya ga daidaitawa, saƙo yana ba ku damar yiwa ƙungiyoyi alama. Bari mu kalli nau'ikan tags a cikin tsarin:

Alamar haɗi. Yana ba ku damar fahimtar ta wacce haɗin al'amuran suka zo. Ana amfani da shi lokacin da tsarin sarrafawa ya haɗa zuwa wurin musayar guda ɗaya, amma tare da maɓallan kewayawa daban-daban.
Tambarin sabis. Yana ba ku damar haɗa masu sarrafawa zuwa ƙungiyoyi don sabis ɗaya da faɗaɗa hanyoyin sarrafawa da daidaita iyawa. Don tsarin buƙatar req-resp, kewayawa layi ne. Muna aika buƙatun zuwa wurin musayar, sa'an nan kuma ya wuce zuwa sabis. Amma idan muna buƙatar raba masu sarrafawa zuwa ƙungiyoyi masu ma'ana, to ana yin rarraba ta amfani da tags. Lokacin zayyana alama, za a aika buƙatar zuwa takamaiman rukunin masu sarrafawa.
Neman tag Yana ba ku damar bambanta tsakanin amsoshi. Tunda tsarin mu bai daidaita ba, don aiwatar da martanin sabis muna buƙatar samun damar tantance Tag ɗin Request lokacin aika buƙatu. Daga ciki za mu iya fahimtar amsar wace bukata ta zo mana.

Pub-sub

Don mashaya-sub duk abin da ya fi sauƙi. Muna da wurin musayar inda ake buga saƙonni. Wurin musayar yana rarraba saƙonni tsakanin masu biyan kuɗi waɗanda suka shiga cikin maɓallan kewayawa da suke buƙata (za mu iya cewa wannan yana kama da batutuwa).

Scalability da rashin haƙuri

Matsakaicin tsarin gaba ɗaya ya dogara da matakin scalability na yadudduka da sassan tsarin:

Ana daidaita ayyuka ta ƙara ƙarin nodes zuwa gungu tare da masu sarrafa wannan sabis ɗin. A lokacin aikin gwaji, zaku iya zaɓar madaidaicin manufar daidaitawa.
Sabis ɗin saƙon kanta a cikin gungu daban ana ƙididdige shi gabaɗaya ta hanyar matsar da wuraren musaya musamman masu ɗorewa don raba nodes ɗin gungu, ko ta ƙara matakan wakili zuwa wuraren da aka ɗora musamman na tari.
Ƙaƙƙarfan tsarin gaba ɗaya a matsayin sifa ya dogara ne akan sassauƙa na gine-gine da kuma ikon haɗa nau'i-nau'i guda ɗaya a cikin mahallin ma'ana gama gari.

Nasarar aikin sau da yawa yana dogara ne akan sauƙi da saurin ƙima. Saƙo a cikin sigar sa na yanzu yana girma tare da aikace-aikacen. Ko da ba mu da gungu na injuna 50-60, za mu iya komawa tarayya. Abin takaici, batun tarayya ya wuce iyakar wannan labarin.

Ajiye

Lokacin nazarin daidaita nauyi, mun riga mun tattauna batun rage yawan masu kula da sabis. Koyaya, dole ne kuma a adana saƙon. A cikin lamarin kumburin kumburi ko na'ura, ya kamata saƙo ya dawo ta atomatik, kuma cikin ɗan gajeren lokaci mai yuwuwa.

A cikin ayyukana ina amfani da ƙarin nodes waɗanda ke ɗaukar nauyin idan faɗuwa ya faru. Erlang yana da daidaitaccen aiwatar da yanayin rarrabawa don aikace-aikacen OTP. Yanayin da aka rarraba yana yin murmurewa idan an gaza ta ƙaddamar da aikace-aikacen da ya gaza akan wani kumburin da aka ƙaddamar a baya. Tsarin yana bayyane; bayan gazawar, aikace-aikacen yana motsawa ta atomatik zuwa kumburin gazawa. Kuna iya karanta ƙarin game da wannan aikin a nan.

Yawan aiki

Bari mu gwada aƙalla kwatanta aikin rabbitmq da saƙon mu na al'ada.
Na samu sakamakon hukuma gwajin rabbitmq daga ƙungiyar openstack.

A cikin sakin layi na 6.14.1.2.1.2.2. Takardun asali na nuna sakamakon RPC CAST:

Ba za mu yi wani ƙarin saituna zuwa kernel OS ko erlang VM a gaba ba. Sharuɗɗan gwaji:

erl zaɓi: +A1 +sbtu.
Ana gudanar da gwajin a cikin kumburin erlang guda ɗaya akan kwamfutar tafi-da-gidanka tare da tsohuwar i7 a cikin sigar wayar hannu.
Ana gudanar da gwaje-gwajen tari akan sabar da ke da hanyar sadarwa ta 10G.
Lambar tana aiki a cikin kwantena na docker. Hanyar sadarwa a yanayin NAT.

Lambar gwaji:

req_resp_bench(_) ->
  W = perftest:comprehensive(10000,
    fun() ->
      messaging:request(?EXCHANGE, default, ping, self()),
      receive
        #'$msg'{message = pong} -> ok
      after 5000 ->
        throw(timeout)
      end
    end
  ),
  true = lists:any(fun(E) -> E >= 30000 end, W),
  ok.

Yanayi na 1: Ana gudanar da gwajin akan kwamfutar tafi-da-gidanka tare da tsohuwar sigar wayar hannu ta i7. Ana aiwatar da gwajin, saƙon da sabis ɗin akan kumburi ɗaya a cikin akwati ɗaya na Docker:

Sequential 10000 cycles in ~0 seconds (26987 cycles/s)
Sequential 20000 cycles in ~1 seconds (26915 cycles/s)
Sequential 100000 cycles in ~4 seconds (26957 cycles/s)
Parallel 2 100000 cycles in ~2 seconds (44240 cycles/s)
Parallel 4 100000 cycles in ~2 seconds (53459 cycles/s)
Parallel 10 100000 cycles in ~2 seconds (52283 cycles/s)
Parallel 100 100000 cycles in ~3 seconds (49317 cycles/s)

Hali 2: 3 nodes suna gudana akan injuna daban-daban a ƙarƙashin docker (NAT).

Sequential 10000 cycles in ~1 seconds (8684 cycles/s)
Sequential 20000 cycles in ~2 seconds (8424 cycles/s)
Sequential 100000 cycles in ~12 seconds (8655 cycles/s)
Parallel 2 100000 cycles in ~7 seconds (15160 cycles/s)
Parallel 4 100000 cycles in ~5 seconds (19133 cycles/s)
Parallel 10 100000 cycles in ~4 seconds (24399 cycles/s)
Parallel 100 100000 cycles in ~3 seconds (34517 cycles/s)

A duk lokuta, amfani da CPU bai wuce 250% ba.

Sakamakon

Ina fatan wannan sake zagayowar ba ta yi kama da zubar da hankali ba kuma kwarewata za ta kasance da amfani ga duka masu bincike na tsarin rarrabawa da masu aikin da suke a farkon gina gine-ginen da aka rarraba don tsarin kasuwancin su kuma suna kallon Erlang / Elixir tare da sha'awa. , amma yi shakka yana da daraja ...

Photography @chuttersnap

Masu amfani da rajista kawai za su iya shiga cikin binciken. Shigadon Allah.

Wadanne batutuwa zan yi bayani dalla-dalla a matsayin wani ɓangare na jerin Gwajin VTrade?

Ka'idar: Kasuwa, umarni da lokacinsu: DAY, GTD, GTC, IOC, FOK, MOO, MOC, LOO, LOC
Littafin umarni. Ka'idar da aikin aiwatar da littafi tare da ƙungiyoyi
Halayen ciniki: Ticks, sanduna, shawarwari. Yadda ake adanawa da yadda ake mannewa
Backoffice. Tsare-tsare da haɓakawa. Sa ido da binciken ma'aikata
API. Bari mu gano abubuwan da ake buƙatar musaya da yadda ake aiwatar da su
Adana bayanai: PostgreSQL, Timecale, Tarantool a cikin tsarin ciniki
Reactivity a cikin tsarin ciniki
Sauran. Zan rubuta a cikin sharhi

6 masu amfani sun kada kuri'a. Masu amfani 4 sun kaurace.

source: www.habr.com

Tubalan gini na aikace-aikacen da aka rarraba. Ƙimar ta biyu