TL; DR: Tsanangudzo yemutengi-server architecture yemukati yedu network configuration manejimendi system, QControl. Iyo yakavakirwa pane maviri-layer transport protocol inoshanda negzip-yakazara mameseji pasina decompression pakati pemagumo. Distributed routers uye endpoints inogamuchira zvigadziriso zvekugadzirisa, uye iyo protocol pachayo inobvumira kuisirwa kwenzvimbo yepakati relays. Iyo sisitimu yakavakirwa pamusimboti ("ichangoburwa-yakagadzika", yakatsanangurwa pazasi) uye inoshandisa mutauro wemubvunzo weJMESpath pamwe nejinja yetemplate yeJinja kupa mafaera ekugadzirisa.
Qrator Labs inoshanda pasi rose yakagoverwa kurwisa kuderedza network. Network yedu inoshanda pane anycast musimboti, uye subnets inoshambadzirwa kuburikidza neBGP. Kuve BGP anycast network iri mumatunhu akati wandei ePasi, isu tinogona kugadzirisa uye kusefa zvisiri pamutemo traffic padyo nepakati peInternet - Tier-1 vanoshanda.
Nekune rumwe rutivi, kuve network yakagovaniswa nenzvimbo hakusi nyore. Kukurukurirana pakati petiweki mapoinzi ekuvapo kwakakosha kuti mupi wekuchengetedza sevhisi ave nekugadzirisa kunoenderana kwese network node, kuvagadziridza panguva yakakodzera. Naizvozvo, kuti tipe iyo yepamusoro-soro nhanho yebasa repakati kune mutengi, taifanira kutsvaga nzira yekuyananisa nekuvimbika data yekumisikidza mumakondinendi ese.
Pakutanga Shoko rakanga riripo. Yakakurumidza kuve yekutaurirana protocol inoda kuvandudzwa.
Ibwe rekona rekuvapo kweQControl, uye panguva imwechete chikonzero chikuru chekushandisa yakawanda nguva uye zviwanikwa pakuvaka rudzi urwu rweprotocol, iko kudiwa kwekuwana imwechete ine mvumo sosi yekumisikidza uye, pakupedzisira, kuwiriranisa mapoinzi edu ekuvapo. nayo. Iyo yekuchengetedza pachayo yaingova imwe yezvakawanda zvaidiwa panguva yekuvandudzwa kweQControl. Pamusoro pezvo, isu taida zvakare kubatanidzwa nearipo uye akarongwa masevhisi panzvimbo dzekuvapo (POP), dzakangwara (uye dzinogoneka) nzira dzekusimbisa data, pamwe nekutonga kwekuwana. Kunze kweizvi, isu taidawo kudzora sisitimu yakadai tichishandisa mirairo pane kugadzirisa mafaera. Pamberi peQControl, data yakatumirwa kunzvimbo dzekuvapo dzinenge nemaoko. Kana imwe yemapoinzi ekuvapo yanga isipo uye isu takakanganwa kuigadzirisa gare gare, iyo gadziriso yaizoguma yave kunze kwekuenderana uye isu taizofanira kutambisa nguva kuidzosera uye kushanda.
Nekuda kweizvozvo, takauya nechirongwa chinotevera:
Sevha yekumisikidza ine basa rekusimbisa uye kuchengetedza data; iyo router ine akati wandei ekupedzisira anogashira uye kutepfenyura zvigadziriso zvekugadzirisa kubva kune vatengi uye zvikwata zvekutsigira kune server, uye kubva kuseva kuenda kunzvimbo dzekuvapo.
Hunhu hwekubatanidza paInternet huchiri kusiyanisa zvakanyanya pasirese - kuratidza pfungwa iyi, ngatitarisei iri nyore MTR kubva kuPrague, Czech Republic kuenda kuSingapore neHong Kong.
MTR kubva kuPrague kuenda kuSingapore
Zvimwe chete kuHong Kong
High latency zvinoreva kumhanya kwakaderera. Mukuwedzera, pane kurasikirwa kwepakiti. Chigero chakafara hachitsigire dambudziko iri, iro rinofanirwa kugara richifungwa kana uchivaka masisitimu akasarudzika.
Iyo yakazara yekumisikidzwa kwenzvimbo yekuvapo ndeye huwandu hwakakosha hwe data iyo inofanirwa kutumirwa kune vakawanda vanogamuchira pamusoro pekusavimbika kubatana. Sezvineiwo, kunyangwe iyo gadziriso ichichinja nguva dzose, inoitika mudiki increments.
Yazvino-yakagadzikana dhizaini
Tinogona kutaura kuti kuvaka network yakagoverwa zvichienderana nemusimboti wekuwedzera zvigadziriso imhinduro iri pachena. Asi kune matambudziko mazhinji nekusiyana. Isu tinofanirwa kuchengetedza misiyano yese pakati pemareferenzi mapoinzi, uye zvakare kukwanisa kuitumira zvakare kana mumwe munhu akapotsa chikamu che data. Nzvimbo yega yega inofanira kuashandisa munhevedzano yakanyatsotaurwa. Kazhinji, munyaya yenzvimbo dzinoverengeka, kuvhiyiwa kwakadaro kunogona kutora nguva yakareba. Iye anogamuchira anofanirawo kukwanisa kukumbira zvikamu zvisipo uye, hongu, chikamu chepakati chinofanira kupindura kuchikumbiro chakadaro nenzira kwayo, kutumira chete data isipo.
Nekuda kweizvozvo, takasvika kune imwe inonakidza mhinduro - isu tine imwe chete referensi layer, yakagadziriswa, ngatiidaidze yakagadzikana, uye imwe chete yakasiyana yayo - ichangoburwa. Imwe neimwe ichangoburwa yakavakirwa pane yekupedzisira kugadzirwa yakagadzikana uye inokwana kuvakazve dhizaini yedhizaini. Ichangobva kutsva painongosvika kwainoenda, yekare haichadiwi.
Chinosara ndechekutumira gadziriso nyowani nguva nenguva, semuenzaniso nekuti ichangoburwa yanyanya kukura. Chakakoshawo apa ndechekuti tinotumira zvese izvi zvigadziriso munhepfenyuro/multicast modhi, tisinganetseki nezvevanogamuchira mumwe nemumwe uye kugona kwavo kubatanidza zvidimbu zve data. Kana tangove nechokwadi chekuti munhu wese ane danga chairo, tinongotumira zvitsva. Zvakakosha here kujekesa kuti izvi zvinoshanda? Mabasa. Yakagadzika yakavharirwa pane yekumisikidza server uye vanogamuchira, ichangoburwa inogadzirwa sezvinodiwa.
Architecture yemhando mbiri dzekutakura
Sei takavaka transport yedu pamalevel maviri? Mhinduro iri nyore - isu taida kudhirowa nzira kubva kune yepamusoro-level logic, tichitora kurudziro kubva kuOSI modhi nekutakura kwayo uye mashandisirwo ekushandisa. Isu takashandisa Thrift yebasa reiyo protocol yekufambisa, uye iyo msgpack serialization fomati yemhando yepamusoro-yepamusoro fomati yekutonga mameseji. Ichi ndicho chikonzero iyo router (inoita multicast / nhepfenyuro / relay) isingatarise mukati memsgpack, isingasunungure kana kurongedza zviri mukati, uye inongoendesa mberi data.
Thrift (kubva kuChirungu - "thrift", inodanwa [ΞΈrift]) mutauro wekutsanangura mutauro unoshandiswa kutsanangura nekugadzira masevhisi emitauro yakasiyana-siyana yekuronga. Iyo dhizaini yekuremote maitiro mafoni (RPC). Inosanganisa pombi yesoftware neinjini yekugadzira kodhi kugadzira masevhisi anoshanda zvakanyanya kana zvishoma zvinonyatso uye zviri nyore pakati pemitauro.
Isu takasarudza iyo Thrift chimiro nekuda kweRPC uye rutsigiro rwemitauro yakawanda. Semazuva ese, zvikamu zviri nyore zvaive mutengi uye sevha. Nekudaro, iyo router yakave yakaoma nati kutsemuka, pamwe nekuda kwekushaikwa kweyakagadzirirwa-yakagadziriswa mhinduro panguva yekukura kwedu.
Pane dzimwe sarudzo, senge protobuf / gRPC, zvisinei, patakatanga chirongwa chedu, gRPC yaive nyowani uye hatina kushinga kuitora.
Ehe, isu taigona (uye kutaura zvazviri tinofanira kunge) takavaka bhasikoro redu. Zvingave nyore kugadzira protocol yezvatinoda nekuti mutengi-server architecture iri nyore kuita kana ichienzaniswa nekuvaka router pa Thrift. Imwe nzira kana imwe, pane kurerekera kwechinyakare kumaprotocol anozvinyora uye kuita kwemaraibhurari ane mukurumbira (nechikonzero chakanaka); nekuwedzera, panguva yekukurukurirana mubvunzo unogara uchiuya: "Tichaendesa sei izvi kune mimwe mitauro?" Saka takabva tangorasa pfungwa yebhasikoro.
Msgpack yakafanana neJSON, asi inokurumidza uye idiki. Iyo ibhinari data serialization fomati inobvumira dhata kuchinjaniswa pakati pemitauro yakawanda.
Padanho rekutanga isu tine Thrift ine ruzivo rudiki runodiwa kuti router itumire meseji. Pachikamu chechipiri pane akaiswa msgpack zvimiro.
Isu takasarudza msgpack nekuti inokurumidza uye yakawanda compact ichienzaniswa neJSON. Asi zvinonyanya kukosha, inotsigira tsika dzemhando dzedata, dzichitibvumira kushandisa zvinhu zvinotonhorera sekupfuura mabhinari akaomeswa kana zvinhu zvakakosha zvinoratidza kusavapo kwedata, izvo zvaive zvakakosha kune yedu "yazvino-yakagadzikana" chirongwa.
JMESPath
JMESPath mutauro wemubvunzo weJSON.
Izvi ndizvo chaizvo izvo tsananguro yatinowana kubva kune yepamutemo JMESPath zvinyorwa zvinotaridzika, asi muchokwadi, inoita zvakawanda kupfuura izvozvo. JMESPath inokutendera kuti utsvage uye kusefa subtrees mune inopokana muti chimiro, uye shandisa shanduko kune data panhunzi. Iyo zvakare inobvumidza iwe kuwedzera akakosha mafirita uye data shanduko maitiro. Kunyange zvazvo, chokwadika, zvinoda kushanda nesimba kwouropi kuti uzvinzwisise.
kuJinja
Kune vamwe vatengi, isu tinofanirwa kushandura iyo gadziriso kuita faira - saka isu tinoshandisa template injini uye Jinja ndiyo iri pachena sarudzo. Nerubatsiro rwayo, isu tinogadzira faira yekumisikidza kubva kune template uye data inogamuchirwa kunzvimbo yekuenda.
Kuti tigadzire faira yekumisikidza, tinoda chikumbiro cheJMESPath, template yenzvimbo yefaira muFS, uye template yeiyo config pachayo. Iri zvakare zano rakanaka panguva ino kujekesa mvumo yefaira. Zvese izvi zvakabatanidzwa zvakabudirira mufaira rimwe - isati yatanga template yekumisikidza, tinoisa musoro muYAML fomati inotsanangura zvimwe.
Somuenzaniso:
---
selector: "[@][[email protected]._meta.version == `42`] | items([0].fft_config || `{}`)"
destination_filename: "fft/{{ match[0] }}.json"
file_mode: 0644
reload_daemons: [fft]
...
{{ dict(match[1]) | json(indent=2, sort_keys=True) }}
Kuti tigadzire faira yekumisikidza yebasa idzva, isu tinongowedzera template nyowani faira. Hapana shanduko kune kodhi kodhi kana software pane nzvimbo dzekuvapo inodiwa.
Chii chachinja kubva QControl yakaenda live? Chinhu chekutanga uye chakakosha chinhu chinowirirana uye chakavimbika kuendesa kwekugadzirisa zvigadziriso kune ese nodes mune network. Yechipiri ndeyekugashira chishandiso chine simba chekutarisa kumisikidzwa uye kuita shanduko kwairi nechikwata chedu chekutsigira, pamwe nevatengi vesevhisi.
Isu takakwanisa kuita zvese izvi tichishandisa ichangoburwa-yakagadzika gadziriso chirongwa kurerutsa kutaurirana pakati pesevha yekumisikidza uye vanogashira. Kushandisa maviri-layer protocol kutsigira zvirimo-yakazvimiririra nzira yekufambisa data. Zvakabudirira kubatanidza Jinja-yakavakirwa gadziriso yechizvarwa injini mune yakagoverwa kusefa network. Iyi sisitimu inotsigira nzira dzakasiyana-siyana dzekugadzirisa kune edu akagoverwa uye ane heterogeneous peripherals.
Ndinotenda nerubatsiro rwenyu mukunyora zvinyorwa. , , .
post.
Source: www.habr.com