Yintoni enokunyanzela inkampani enkulu njengeLamoda enenkqubo elungelelanisiweyo kunye neenkonzo ezininzi ezinxibeleleneyo ukuba zitshintshe kakhulu indlela yayo? Inkuthazo inokwahluka ngokupheleleyo: ukusuka kumthetho ukuya kumnqweno wokulinga into ekhoyo kubo bonke abadwelisi benkqubo.
Kodwa oku akuthethi ukuba awukwazi ukubala kwizibonelelo ezongezelelweyo. USergey Zaika uya kukuxelela ukuba yintoni kanye kanye onokuthi uyiphumelele ukuba uphumeza i-API eqhutywa yiziganeko kwi-Kafka (). Ngokuqinisekileyo kuya kubakho amaqhuqhuva kunye nezinto ezifunyanisiweyo ezinomdla nazo-uvavanyo alunakwenza ngaphandle kwabo.
Ukuziphendulela: Eli nqaku lisekelwe kwizinto ezivela kwintlanganiso eyabanjwa nguSergey ngoNovemba ka-2018 kwi-HighLoad ++. Amava kaLamoda asebenza ngokusebenza noKafka atsala abaphulaphuli abangekho ngaphantsi kwezinye iingxelo ezikwishedyuli. Sicinga ukuba lo ngumzekelo obalaseleyo wokuba unako kwaye kufuneka uhlale ufumana abantu abanengqondo efanayo, kwaye abaququzeleli be-HighLoad ++ baya kuqhubeka nokuzama ukudala umoya olungele oku.
Malunga nenkqubo
I-Lamoda yindawo enkulu ye-e-commerce eneziko layo loqhagamshelwano, inkonzo yokuhanjiswa (kunye namahlakani amaninzi), isitudiyo seefoto, indawo yokugcina impahla kwaye konke oku kuqhuba isoftware yayo. Kukho iindlela ezininzi zokuhlawula, amaqabane e-b2b anokusebenzisa ezinye okanye zonke ezi nkonzo kwaye afune ukwazi ulwazi lwangoku kwiimveliso zabo. Ukongeza, iLamoda isebenza kumazwe amathathu ngaphandle kweRussian Federation kwaye yonke into yahluke kancinci apho. Lilonke, mhlawumbi kukho iindlela ezingaphezu kwekhulu zokumisela umyalelo omtsha, omele uqhutywe ngendlela yawo. Konke oku kusebenza ngoncedo lweenkonzo ezininzi ezithi ngamanye amaxesha zinxibelelane ngeendlela ezingacacanga. Kukwakho nenkqubo esembindini emsebenzi wayo ophambili zii-odolo. Simbiza BOB, ndisebenza naye.
Isixhobo sembuyekezo nge-API eqhutywa yimicimbi
Igama elithi i-events-driven is hackneyed; kancinci kancinci siza kuchaza ngokweenkcukacha ezithe kratya ukuba kuthethwa ntoni koku. Ndiza kuqala ngomxholo apho sigqibe kwelokuba sizame indlela ye-API eqhutywa yiminyhadala eKafka.
Kuyo nayiphi na ivenkile, ngaphezu kwee-odolo apho abathengi bahlawula khona, kukho amaxesha apho ivenkile kufuneka ibuyisele imali ngenxa yokuba imveliso ayizange ihambisane nomthengi. Le yinkqubo emfutshane kakhulu: sicacisa ulwazi, ukuba kuyimfuneko, kwaye sidlulisela imali.
Kodwa imbuyekezo yaba nzima ngakumbi ngenxa yotshintsho kumthetho, kwaye kuye kwafuneka ukuba siphumeze i-microservice eyahlukileyo kuyo.
Inkuthazo yethu:
- Umthetho FZ-54 - ngokufutshane, umthetho ufuna ukunika ingxelo kwi-ofisi yerhafu malunga nentengiselwano nganye yemali, nokuba yimbuyekezo okanye irisithi, ngaphakathi kwe-SLA emfutshane yemizuzu embalwa. Thina, njengenkampani ye-e-commerce, senza imisebenzi emininzi. Ngokobuchwephesha, oku kuthetha uxanduva olutsha (oluthetha inkonzo entsha) nophuculo kuzo zonke iinkqubo ezibandakanyekayo.
- BOB yahlukana - iprojekthi yangaphakathi yenkampani ukukrazula i-BOB inani elikhulu leemfanelo ezingekho ngundoqo kunye nokunciphisa ubunzima bayo bonke.
Lo mzobo ubonisa iinkqubo eziphambili zeLamoda. Ngoku uninzi lwazo lufana ngakumbi i-constellation ye-5-10 microservices ejikeleze i-monolith eyanciphayo. Bakhula ngokukhawuleza, kodwa sizama ukubenza bancinci, kuba ukuhambisa iqhekeza elikhethiweyo phakathi kuyoyikisa - asinakuyivumela ukuba siwe. Sinyanzeliswa ukuba sigcine lonke utshintshiselwano (iintolo) kwaye sithathele ingqalelo into yokuba nayiphi na kuzo inokujika ingafumaneki.
Kukwakhona utshintshiselwano oluninzi kwi-BOB: iinkqubo zokuhlawula, iinkqubo zokuhanjiswa, iinkqubo zezaziso, njl.
Ngokobuchwephesha iBOB yile:
- ~ 150k imigca yekhowudi + ~ 100k imigca yovavanyo;
- php7.2 + Zend 1 & Symfony Amacandelo 3;
- > 100 APIs & ~50 udibaniso oluphuma ngaphandle;
- Amazwe ama-4 anengqiqo yoshishino lwawo.
Ukuhambisa i-BOB kuyabiza kwaye kubuhlungu, inani lekhowudi kunye neengxaki ezisombululayo kukuba akukho mntu unokugubungela intloko yakhe ngokupheleleyo. Ngokubanzi, zininzi izizathu zokuyenza ibe lula.
Inkqubo yokuBuyisa
Ekuqaleni, iinkqubo ezimbini zibandakanyeka kwinkqubo: BOB kunye neNtlawulo. Ngoku kuvela ezinye ezimbini:
- Inkonzo yeFiscalisation, eya kuhoya iingxaki ngefiscalization kunye nonxibelelwano kunye neenkonzo zangaphandle.
- IsiXhobo seMbuyiselo, apho utshintshiselwano olutsha ludityaniswe ngokulula ukuze ungafaki i-BOB.
Ngoku inkqubo ibonakala ngolu hlobo:
- I-BOB ifumana isicelo sokubuyiselwa imali.
- UBOB uthetha ngesi sixhobo seMbuyiselo.
- IsiXhobo seMbuyiselo sithi kwiNtlawulo: "Buyisela imali."
- Intlawulo ibuyisela imali.
- Isixhobo sembuyekezo kunye ne-BOB zilungelelanisa iimeko kunye, kuba okwangoku ziyayifuna bobabini. Asikakulungeli ukutshintshela ngokupheleleyo kwiSixhobo sokuBuyiselwa kweMbuyiselo, ekubeni i-BOB ine-UI, iingxelo zokubala, kwaye ngokubanzi idatha eninzi engenakudluliselwa ngokulula. Kufuneka uhlale kwizitulo ezibini.
- Isicelo sokuxhaswa ngemali siyahamba.
Ngenxa yoko, senze uhlobo lwebhasi yesiganeko eKafka - ibhasi yesiganeko, apho yonke into yaqala khona. Hurray, ngoku sinenqaku elinye lokusilela (i-sarcasm).
Izinto ezilungileyo kunye nezibi zicacile. Senze ibhasi, nto leyo ethetha ukuba ngoku zonke iinkonzo zixhomekeke kuyo. Oku kwenza uyilo lube lula, kodwa yazisa inqaku elinye lokusilela kwinkqubo. I-Kafka iya kuphazamiseka, inkqubo iya kuyeka.
Yintoni i-API eqhutywa yiziganeko
Kukho impendulo efanelekileyo kulo mbuzo kwingxelo kaMartin Fowler (GOTO 2017) .
Ngokufutshane into esiyenzileyo:
- Gquba lonke utshintshiselwano lwe-asynchronous nge ugcino lweziganeko. Esikhundleni sokwazisa wonke umthengi onomdla malunga nokutshintsha kwesimo kwinethiwekhi, sibhala isiganeko malunga nokutshintsha kwesimo kwindawo yokugcina indawo, kunye nabathengi abanomdla kwisihloko bafunde yonke into evela apho.
- Isiganeko kulo mzekelo sisaziso (izaziso) malunga nokuba kukho into etshintshileyo kwenye indawo. Umzekelo, imeko ye-odolo itshintshile. Umthengi onomdla kwidatha ethile ehamba notshintsho kwisimo kwaye engabandakanywanga kwisaziso unokufumanisa isimo sakhe ngokwakhe.
- Olona khetho luphezulu kukufumana isiganeko esipheleleyo, udluliselo lukarhulumente, apho isiganeko siqulethe yonke ingcaciso eyimfuneko yokucubungula: apho ivela khona kwaye yeyiphi imeko eya kuyo, njani ngokuthe ngqo idatha yatshintshile, njl. Umbuzo kuphela yinto enokwenzeka kunye nesixa solwazi onokukwazi ukuyigcina.
Njengenxalenye yokuqaliswa kweSixhobo seMbuyiselo, sisebenzise ukhetho lwesithathu. Le nkqubo yenziwa lula kuba bekungekho mfuneko yokukhupha ulwazi oluneenkcukacha, kwaye yasusa imeko apho umsitho omtsha ngamnye uvelisa ukucaciswa kwezicelo ezivela kubathengi.
Inkonzo yeSixhobo sembuyekezo ayilayishwanga, ngoko i-Kafka kukho incasa yepeni ngaphezu kwemfuneko. Andicingi ukuba ukuba inkonzo yokubuyisela imali ibe yiprojekthi yomthwalo ophezulu, ishishini liya kuvuya.
Utshintshiselwano lwe-Async NJENGOKO
Ukutshintshana kwe-asynchronous, isebe le-PHP lidla ngokusebenzisa i-RabbitMQ. Siqokelele idatha yesicelo, sayibeka emgceni, kwaye umthengi wenkonzo efanayo wayiqwalasela kwaye wayithumela (okanye akazange ayithumele). Kwi-API ngokwayo, iLamoda isebenzisa ngokusebenzayo iSwagger. Siyila i-API, siyichaza kwi-Swagger, kwaye sivelise umxhasi kunye nekhowudi yeseva. Sikwasebenzisa i-JSON RPC 2.0 ephuculwe kancinci.
Kwezinye iindawo kusetyenziswa iibhasi zeESB, ezinye zihlala kwi-activeMQ, kodwa, ngokubanzi, RabbitMQ ngumgangatho.
Utshintshiselwano lwe-Async UKUBA
Xa uyila utshintshiselwano ngeziganeko-ibhasi, isifaniso sinokulandelwa. Ngendlela efanayo, sichaza utshintshiselwano lwedatha yexesha elizayo ngeenkcazo zesakhiwo sesiganeko. Ifomathi ye-yaml, kwafuneka senze ukuveliswa kwekhowudi ngokwethu, i-generator idala i-DTOs ngokwenkcazo kwaye ifundise abathengi kunye neeseva ukusebenza nabo. Isizukulwana ngeelwimi ezimbini: golang kunye nephp. Oku kunceda ukugcina amathala angatshintshi. Ijenereyitha ibhalwe ngegolang, yiyo loo nto ifumene igama elithi gogi.
Ukufumana umsitho kwiKafka yinto eqhelekileyo. Kukho isisombululo esivela kuhlobo oluphambili lweshishini leKafka Confluent, kukho , isisombululo esivela kubazalwana bethu besizinda uZalando. Yethu inkuthazo ukuqala nge vanilla Kafka -oku kukushiya isisombululo simahla de sigqibe ekubeni siza kusisebenzisa kuyo yonke indawo, kwaye sizishiyele indawo yokwenza kunye nokuphucula: sifuna inkxaso yethu. I-JSON RPC 2.0, iijenereyitha zeelwimi ezimbini kwaye makhe sibone enye into.
Iyamangalisa into yokuba nakwimeko enethamsanqa kukho ishishini eliphantse lifane, iZalando, elenze isisombululo esiphantse safana naso, asinakusisebenzisa ngokufanelekileyo.
Ipateni yokwakha ekusungulweni ilandelayo: sifunda ngokuthe ngqo ukusuka eKafka, kodwa bhala kuphela ngeziganeko-ibhasi. Zininzi izinto esele zenziwe eKafka ukuze zifundwe: ii-broker, ii-balancers, kwaye ilungile okanye ilungele ukulinganisa okuthe tye, bendifuna ukuyigcina le nto. Ukurekhoda, besifuna ukuhamba ngeGateway aka Iziganeko-ibhasi, kwaye nantsi isizathu.
Iziganeko-ibhasi
Okanye ibhasi yesiganeko. Eli lisango elingenammiselo le-http, elithatha iindima ezininzi ezibalulekileyo:
- Ukuvelisa uQinisekiso - sijonga ukuba iziganeko ziyahlangabezana neenkcukacha zethu.
- Inkqubo eyinkosi yomsitho, oko kukuthi, le yinkqubo ephambili kunye nekuphela kwenkqubo kwinkampani ephendula umbuzo weziphi iziganeko apho izakhiwo zithathwa njengezifanelekileyo. Ukuqinisekiswa kubandakanya iintlobo zedatha kunye neenums ukucacisa ngokungqongqo umxholo.
- Umsebenzi weHash for sharding - isakhiwo somyalezo we-Kafka singundoqo-ixabiso kwaye, ngokusekelwe kwi-hash yesitshixo, kubalwa apho uyibeka khona.
Ngoba
Sisebenza kwinkampani enkulu enenkqubo elungelelanisiweyo. Kutheni utshintsha nantoni na? Olu luvavanyo, kwaye silindele ukuvuna iinzuzo ezininzi.
1:n+1 utshintshiselwano (enye ukuya kwezininzi)
NgeKafka kulula kakhulu ukudibanisa abathengi abatsha kwi-API.
Masithi unoluhlu okufuneka ulugcine usexesheni kwiinkqubo ezininzi ngexesha elinye (kwaye kwezinye ezintsha). Ngaphambili, siye sayila inqwaba ephumeze isethi-API, kwaye yanikezela ngenkqubo ephambili ngeedilesi zabathengi. Ngoku i-master system ithumela ukuhlaziywa kwisihloko, kwaye wonke umntu onomdla uyayifunda. Inkqubo entsha iye yavela - siyisayinele isihloko. Ewe, kwakhona inqwaba, kodwa ilula.
Kwimeko ye-refund-isixhobo, isiqwenga se-BOB, kulungele ukuba sizigcine zihambelana ne-Kafka. Intlawulo ithi imali ibuyiselwe: I-BOB, i-RT yafumanisa ngale nto, yatshintsha isimo sabo, i-Fiscalization Service yafumanisa malunga nale nto kwaye yakhupha isheke.
Sinezicwangciso zokwenza iNkonzo yeZaziso ezidibeneyo eziza kwazisa umxhasi malunga neendaba malunga nomyalelo / ukubuyisela kwakhe. Ngoku olu xanduva lusasazwe phakathi kweenkqubo. Kuya kukwanela ukuba sifundise iNkonzo yeZaziso ukubamba ulwazi olufanelekileyo oluvela eKafka kwaye lusabele kuyo (kwaye ukhubaze ezi zaziso kwezinye iinkqubo). Akukho tshintshiselwano entsha ngokuthe ngqo iya kufuneka.
Ukuqhutywa kwedatha
Ulwazi phakathi kweenkqubo luba selubala- nokuba leliphi na “ishishini elinegazi” onalo kwaye nokuba ukusilela kwakho kuninzi kangakanani na. I-Lamoda inesebe le-Data Analytics eliqokelela idatha kwiinkqubo kwaye izise kwifomu enokuphinda isetyenziswe, zombini kwishishini kunye neenkqubo ezihlakaniphile. I-Kafka ikuvumela ukuba unike ngokukhawuleza idatha eninzi kwaye ugcine olo lwazi luhamba ngoku.
Ilog yokuphindaphinda
Imiyalezo ayipheli emva kokuba ifundiwe, njengakwiRabbitMQ. Xa isiganeko siqulethe ulwazi olwaneleyo lokucubungula, sinembali yotshintsho lwamva nje kwinto, kwaye, ukuba unqwenela, ukukwazi ukusebenzisa olu tshintsho.
Ixesha lokugcinwa kwelogi yokuphindaphinda kuxhomekeke kubukhulu bokurekhoda kwesi sihloko; I-Kafka ikuvumela ukuba ubeke umda ngexesha lokugcina kunye nomthamo wedatha. Kwizihloko ezinzulu, kubalulekile ukuba bonke abathengi babe nexesha lokufunda ulwazi ngaphambi kokuba lunyamalale, nangona kwimeko yokungasebenzi kwexesha elifutshane. Ngokuqhelekileyo kuyenzeka ukugcina idatha iiyunithi zeentsuku, eyanele ngokwaneleyo inkxaso.
Okulandelayo, ukubuyiswa okuncinci kwamaxwebhu, kwabo bangaqhelananga neKafka (umfanekiso nawo uvela kumaxwebhu)
I-AMQP inemigca: sibhala imiyalezo kumgca womthengi. Ngokuqhelekileyo, umgca omnye ucutshungulwa yinkqubo enye enengqiqo yeshishini elifanayo. Ukuba ufuna ukwazisa iinkqubo ezininzi, ungafundisa isicelo ukubhalela imigca emininzi okanye umise utshintshiselwano ngesixhobo se fanout, esizikopelayo ngokwazo.
I-Kafka ine-abstraction efanayo Sihloko, apho ubhala khona imiyalezo, kodwa ayipheli emva kokufunda. Ngokungagqibekanga, xa uqhagamshela kwiKafka, ufumana yonke imiyalezo kwaye unokhetho lokugcina apho uyeke khona. Oko kukuthi, ufunda ngokulandelelanayo, awukwazi ukumakisha umyalezo njengofundiweyo, kodwa gcina i-id apho unokuqhubeka ufunda kamva. I-Id ohleli kuyo ibizwa ngokuba yi-offset, kwaye indlela leyo ibizwa ngokuba yi-commit offset.
Ngokufanelekileyo, ingqiqo eyahlukileyo inokuphunyezwa. Ngokomzekelo, sineBOB kwiimeko ezi-4 kumazwe ahlukeneyo - iLamoda iseRashiya, eKazakhstan, eUkraine, eBelarus. Kuba zisasazwe ngokwahlukeneyo, zinee-configs ezahluke kancinane kunye nengqiqo yeshishini labo. Sibonisa kumyalezo ukuba leliphi ilizwe elibhekisa kulo. Umthengi ngamnye BOB kwilizwe ngalinye ufunda kunye neqelaId eyahlukileyo, kwaye ukuba umyalezo awusebenzi kuye, bayawutsiba, oko kukuthi. ngokukhawuleza yenza i-offset +1. Ukuba iNkonzo yethu yeNtlawulo ifunda isihloko esifanayo, ngoko ikwenza oku kunye neqela elihlukeneyo, kwaye ngoko ke i-offsets ayiphambanisi.
Iimfuno zomsitho:
- Ukugqibelela kwedatha. Ndingathanda umsitho ubenedatha eyaneleyo ukuze iqwalaselwe.
- Ingqibelelo. Sinikezela kwiMisitho-ibhasi itshekhi ukuba umsitho uyahambelana kwaye unokuwuqhuba.
- Umyalelo ubalulekile. Kwimeko yokubuya, sinyanzelekile ukuba sisebenze ngembali. Ngezaziso, umyalelo awubalulekanga, ukuba ngezaziso ze-homogeneous, i-imeyile iya kuba yinto efanayo kungakhathaliseki ukuba yeyiphi iodolo efike kuqala. Kwimeko yokubuyisela imali, kukho inkqubo ecacileyo; ukuba sitshintsha umyalelo, ngoko ngaphandle kuya kuvela, ukubuyisela imali ayiyi kudalwa okanye iqhutywe - siya kuphelela kwisimo esahlukileyo.
- Ukungqinelana. Sinendawo yokugcina, kwaye ngoku senza iziganeko endaweni yee-APIs. Sidinga indlela yokuhambisa ngokukhawuleza nangexabiso eliphantsi ulwazi malunga nemisitho emitsha kunye notshintsho kwezo sele zikhona kwiinkonzo zethu. Oku kuphunyezwa kusetyenziswa ingcaciso eqhelekileyo kwindawo yogcino lwegit eyahlukileyo kunye neejenereyitha zekhowudi. Ke ngoko, abathengi kunye neeseva kwiinkonzo ezahlukeneyo ziyahambelana nathi.
Kafka e Lamoda
Sinofakelo lweKafka ezintathu:
- Iilog;
- I-R&D;
- Iziganeko-ibhasi.
Namhlanje sithetha kuphela ngenqaku lokugqibela. Kwiminyhadala-ibhasi asinakufakela okukhulu kakhulu - abarhwebi aba-3 (abancedisi) kunye nezihloko ezingama-27 kuphela. Njengomthetho, isihloko esinye yinkqubo enye. Kodwa le yingongoma efihlakeleyo, kwaye ngoku siza kuyichukumisa.
Ngentla yigrafu ye-rps. Inkqubo yokubuyisela iphawulwe ngomgca we-turquoise (ewe, lowo ukwi-X-axis), kwaye umgca wepinki yinkqubo yokuhlaziya umxholo.
Ikhathalogu yeLamoda iqulethe izigidi zeemveliso, kwaye idatha ihlaziywa ngalo lonke ixesha. Ezinye ingqokelela ziphuma kwifashoni, ezintsha ziyakhululwa ukuze zitshintshe, kwaye iimodeli ezintsha zihlala zivela kwikhathalogu. Sizama ukuqikelela ukuba yintoni eza kuba nomdla kubathengi bethu ngomso, ke sihlala sithenga izinto ezintsha, sizifota kwaye sihlaziya imeko yokubonisa.
Iincopho zepinki zihlaziyo lwemveliso, oko kukuthi, utshintsho kwiimveliso. Kucacile ukuba abafana bathatha imifanekiso, bathatha imifanekiso, kwaye kwakhona! - ilayishe ipakethi yeziganeko.
Iziganeko zeLamoda zisebenzisa iimeko
Sisebenzisa i-architecture eyakhiwe kule misebenzi ilandelayo:
- Buyisela isimo sokulandelela: ukubizelwa kwisenzo kunye nokulandelwa kwemeko kuzo zonke iinkqubo ezibandakanyekayo. Intlawulo, amanani, imali, izaziso. Apha sizame indlela, senze izixhobo, siqokelele zonke iibhugi, sibhale amaxwebhu kwaye sixelele oogxa bethu indlela yokuyisebenzisa.
- Ukuhlaziya amakhadi emveliso: uqwalaselo, imeta-data, iimpawu. Inkqubo enye iyafundeka (ebonisa), kwaye abaninzi babhala.
- I-imeyile, i-push kunye ne-sms: umyalelo uqokelelwe, umyalelo ufikile, imbuyekezo yamkelwe, njl., ininzi yabo.
- Isitokhwe, uhlaziyo lwesitora - uhlaziyo lobungakanani bezinto, amanani nje: ukufika kwindawo yokugcina impahla, ukubuya. Kuyimfuneko ukuba zonke iinkqubo ezinxulumene nokugcinwa kwempahla zisebenze ngeyona datha yangoku. Okwangoku, inkqubo yohlaziyo lwesitokhwe inzima kakhulu; I-Kafka iya kwenza kube lula.
- Uhlalutyo lweenkcukacha (isebe le-R & D), izixhobo ze-ML, uhlalutyo, izibalo. Sifuna ulwazi lubonakale, nto leyo ilungele iKafka.
Ngoku eyona nxalenye inika umdla malunga namaqhuqhuva amakhulu kunye nezinto ezifunyanisiweyo ezinomdla ezenzeke kwezi nyanga zintandathu zidlulileyo.
Iingxaki zoyilo
Masithi sifuna ukwenza into entsha - umzekelo, ukudlulisela yonke inkqubo yokuhambisa eKafka. Ngoku inxalenye yenkqubo iphunyezwa kwi-Order Processing kwi-BOB. Emva kokudluliselwa komyalelo kwinkonzo yokuhanjiswa, ukunyakaza kwi-warehouse ephakathi, njalo njalo, kukho imodeli yesimo. Kukho i-monolith epheleleyo, nokuba zimbini, kunye neqela le-APIs ezinikezelwe ukuhanjiswa. Bazi okuninzi malunga nokuhanjiswa.
Ezi zibonakala ziyimimandla efanayo, kodwa iimeko zihluke kwi-Order Processing kwi-BOB kunye nenkqubo yokuthumela. Umzekelo, ezinye iinkonzo zekhuriya azithumeli isimo esiphakathi, kodwa kuphela ezokugqibela: "zinikelwe" okanye "zilahlekile". Abanye, ngokuchaseneyo, banika ingxelo ngeenkcukacha ezinkulu malunga nokuhamba kwempahla. Wonke umntu unemithetho yakhe yokuqinisekisa: kwabanye, i-imeyile ivumelekile, oku kuthetha ukuba iya kuqhutyelwa phambili; kwabanye ayisebenzi, kodwa umyalelo usaza kuqhutyelwa phambili kuba kukho inombolo yefowuni yoqhagamshelwano, kwaye abanye baya kuthi loo myalelo awuyi kuqhutyelwa phambili.
Umjelo wedatha
Kwimeko yeKafka, umbuzo wokulungelelanisa ukuhamba kwedatha uvela. Lo msebenzi ubandakanya ukukhetha isicwangciso esisekwe kumanqaku amaninzi; masihambe ngawo onke.
Kwisihloko esinye okanye kwezinye ezahlukeneyo?
Sinenkcukacha zesiganeko. Kwi-BOB sibhala ukuba umyalelo onjalo kunye nonjalo kufuneka uhanjiswe, kwaye ubonise: inombolo ye-oda, imixholo yayo, ezinye ii-SKU kunye neekhowudi zebar, njl. Xa iimpahla zifika kwindawo yokugcina impahla, ukuhanjiswa kuya kukwazi ukufumana i-status, izitampu zexesha kunye nayo yonke into efunekayo. Kodwa ke sifuna ukufumana uhlaziyo kule datha kwi-BOB. Sinenkqubo ebuyela umva yokufumana idatha ekuhanjisweni. Ngaba sisiganeko esinye? Okanye ngaba olu lutshintshiselwano olwahlukileyo olufanele isihloko salo?
Okunokwenzeka ukuba, baya kufana kakhulu, kwaye isilingo sokwenza isihloko esinye asinasiseko, kuba isihloko esahlukileyo sithetha abathengi abahlukeneyo, i-configs eyahlukileyo, isizukulwana esahlukileyo sayo yonke le nto. Kodwa akuyonyani.
Intsimi entsha okanye isiganeko esitsha?
Kodwa ukuba usebenzisa iziganeko ezifanayo, ngoko kuvela enye ingxaki. Umzekelo, ayizizo zonke iinkqubo zokuhambisa ezinokuvelisa uhlobo lwe-DTO enokuthi i-BOB ivelise. Sibathumelela i-id, kodwa abayisigcini kuba abayifuni, kwaye ukusuka kwindawo yokujonga inkqubo yebhasi yomsitho, le ndawo iyafuneka.
Ukuba sazisa umgaqo wesiganeko-ibhasi ukuba le ntsimi iyadingeka, ngoko siphoqeleka ukuba sibeke imithetho eyongezelelweyo yokuqinisekisa kwi-BOB okanye kumphathi wesiganeko sokuqala. Ukuqinisekiswa kuqala ukusasazeka kuyo yonke inkonzo - oku akulula kakhulu.
Enye ingxaki sisilingo sokukhula okunyukayo. Sixelelwa ukuba kukho into efuna ukongezwa kumsitho, kwaye mhlawumbi ukuba sicinga ngawo, bekumele ukuba ibe sisiganeko esahlukileyo. Kodwa kwisicwangciso sethu, isiganeko esahlukileyo sisihloko esahlukileyo. Isihloko esahlukileyo yinkqubo yonke endiyichaze ngasentla. Umphuhlisi uyahendelwa ukuba wongeze nje enye intsimi kwi-schema ye-JSON kwaye ayivelise kwakhona.
Kwimeko yokubuyiselwa imali, safika kwisiganeko kwiinyanga ezintandathu. Sasinesiganeko esinye esibizwa ngokuba kuhlaziyo lwembuyekezo, enohlobo lwendawo echaza ukuba olu hlaziyo lwaluyintoni kanye kanye. Ngenxa yoku, sibe nokutshintsha "okumangalisayo" kunye nabaqinisekisi abasixelele indlela yokuqinisekisa esi siganeko ngolu hlobo.
Uguqulelo lwesiganeko
Ukuqinisekisa imiyalezo eKafka ungasebenzisa , kodwa bekuyimfuneko ukuzibophelela ngokukhawuleza koku kwaye usebenzise i-Confluent. Kwimeko yethu, kufuneka silumke ngoguqulelo. Ayizukuhlala ikwazi ukufunda kwakhona imiyalezo evela kwilog yokuphindaphinda kuba imodeli "ishiye". Ngokusisiseko, kuvela ukuba kwakhiwe iinguqulelo ukuze imodeli ihambelane ngasemva: umzekelo, yenza intsimi okwethutyana. Ukuba ukungafani kunamandla kakhulu, siqala ukubhala kwisihloko esitsha, kwaye sidlulisele abathengi xa begqiba ukufunda endala.
Ulandelelwano lokufunda oluqinisekisiweyo lwezahlulo
Izihloko ngaphakathi kweKafka zohlulwe zibe ngezahlulo. Oku akubalulekanga kakhulu ngelixa siyila amaqumrhu kunye notshintshiselwano, kodwa kubalulekile xa sithatha isigqibo sokuba siza kuyisebenzisa njani kwaye siyilinganise.
Kwimeko eqhelekileyo, ubhala isihloko esinye kwiKafka. Ngokungagqibekanga, isahlulelo esinye sisetyenziswa, kwaye yonke imiyalezo ekwesi sihloko iwela kuyo. Kwaye umthengi ufunda le miyalezo ngokulandelelanayo. Masithi ngoku kufuneka sandise inkqubo ukuze imiyalezo ifundwe ngabathengi ababini abahlukeneyo. Ukuba, umzekelo, uthumela i-SMS, ngoko unokuxelela uKafka ukuba enze isahlulelo esongezelelweyo, kwaye iKafka iya kuqala ukwahlula imiyalezo ibe ngamacandelo amabini-isiqingatha apha, isiqingatha apho.
I-Kafka ibahlula njani? Umyalezo ngamnye unomzimba (apho sigcina i-JSON) kunye nesitshixo. Ungaqhoboshela umsebenzi we-hash kweli qhosha, eliya kumisela ukuba yeyiphi isahlulelo somyalezo oya kungena kuso.
Kwimeko yethu ngokubuyisela imali, oku kubalulekile, ukuba sithatha izahlulo ezimbini, ngoko kukho ithuba lokuba umthengi ohambelanayo uya kucubungula isiganeko sesibini ngaphambi kokuqala kwaye kuya kubakho ingxaki. Umsebenzi we-hash uqinisekisa ukuba imiyalezo eneqhosha elifanayo iphelela kwisahlulelo esifanayo.
Iziganeko vs imiyalelo
Le yenye ingxaki esijongene nayo. Isiganeko sisiganeko esithile: sithetha ukuba kukho into eyenzekayo kwindawo ethile (into_yenzeke), umzekelo, into ethile irhoxisiwe okanye imbuyekezo yenzeke. Ukuba umntu umamele ezi ziganeko, ngoko ke ngokutsho "kwinto ecinyiweyo," iqumrhu lembuyekezo liya kwenziwa, kwaye "imbuyekezo yenzeke" iya kubhalwa kwenye indawo ekucwangcisweni.
Kodwa ngokuqhelekileyo, xa uyila imicimbi, awufuni ukuyibhala ilize - ugcina ibhanki kwinto yokuba umntu uya kuyifunda. Isilingo siphezulu sokubhala hayi into_yenzekile (into_irhoxisiwe, ibuyiswa_ibuyiselwe), kodwa into_kufuneka_ yenziwe. Umzekelo, into ilungele ukubuyiselwa.
Kwelinye icala, icebisa indlela esiza kusetyenziswa ngayo eso siganeko. Kwelinye icala, kuvakala kancinci njengegama lesiganeko esiqhelekileyo. Ngaphandle koko, akukho kude ukusuka apha ukuya kwiqela le-do_something. Kodwa akunasiqinisekiso sokuba kukho umntu osifundileyo esi siganeko; kwaye ukuba uyayifunda, ngoko uyifunde ngempumelelo; kwaye ukuba uyifunda ngempumelelo, ngoko wenze into, kwaye into ethile yaba yimpumelelo. Umzuzwana xa isiganeko sisenza_into ethile, impendulo iba yimfuneko, kwaye yingxaki leyo.
Kwi-asynchronous exchange in RabbitMQ, xa ufunda umyalezo kwaye uye kwi-http, unempendulo - ubuncinane ukuba umyalezo ufunyenwe. Xa ubhalela uKafka, kukho umyalezo owubhalele uKafka, kodwa awazi nto malunga nendlela eqhutywe ngayo.
Ngoko ke, kwimeko yethu, kwafuneka ukuba sazise isiganeko sokuphendula kwaye simise ukubeka iliso ukwenzela ukuba ukuba iziganeko ezininzi zithunyelwe, emva kwexesha elinjalo kunye nexesha elifanayo leziganeko zokuphendula kufuneka zifike. Ukuba oku akwenzeki, ngoko kubonakala ngathi kukho into engahambi kakuhle. Umzekelo, ukuba sithumele isiganeko esithi "item_ready_to_refund", silindele ukuba ukubuyiselwa imali kuya kwenziwa, imali iya kubuyiselwa kumxhasi, kwaye umcimbi othi "money_refunded" uya kuthunyelwa kuthi. Kodwa oku akuchanekanga, ngoko ke ukubekwa esweni kuyafuneka.
Imiba
Kukho ingxaki ecacileyo: ukuba ufunda kwisihloko ngokulandelelana, kwaye unomyalezo ombi, umthengi uya kuwa, kwaye awuyi kuqhubela phambili. Udinga misa bonke abathengi, yenza uhlengahlengiso ngakumbi ukuze uqhubeke ufunda.
Sasisazi ngayo, sabhanka ngayo, kwaye yenzeka. Kwaye oku kwenzeka ngenxa yokuba isiganeko sasisemthethweni ukusuka kwindawo yokujonga iziganeko-ibhasi, isiganeko sasisemthethweni ukusuka kwindawo yokujonga umqinisekisi wesicelo, kodwa yayingavumelekanga ukusuka kwindawo yokujonga i-PostgreSQL, kuba kwinkqubo yethu i-MySQL nge UNSIGNED INT, kwaye kwinkqubo entsha ebhaliweyo inePostgreSQL ngokulula nge INT. Ubungakanani bakhe buncinci, kwaye i-ID ayifanelekanga. U-Symfony wafa ngaphandle kwengxaki. Thina, ewe, sibambe okwahlukileyo kuba besithembele kuyo kwaye siza kuyenza le offset, kodwa ngaphambi koko besifuna ukunyusa ikhawunta yengxaki, kuba umyalezo uqhutyelwe phambili ngempumelelo. Izixhobo zokubala kule projekthi zikwakho kwisiseko sedatha, kwaye i-Symfony sele ivaliwe unxibelelwano kunye nedathabheyisi, kwaye okwesibini kwabulala yonke inkqubo ngaphandle nethuba lokuzibophelela.
Inkonzo yalala ixesha elithile - ngethamsanqa, kunye neKafka ayisoyiki kangako, kuba imiyalezo ihleli. Xa umsebenzi ubuyiselwe, unokugqiba ukuzifunda. Ikhululekile.
I-Kafka inamandla okuseta i-offset engafanelekanga ngokusebenzisa izixhobo. Kodwa ukwenza oku, kufuneka umise bonke abathengi - kwimeko yethu, lungiselela ukukhululwa okwahlukileyo apho kungayi kubakho abathengi, ukuthunyelwa kwakhona. Emva koko i-offset ye-Kafka inokutshintshwa ngokusebenzisa izixhobo, kwaye umyalezo uya kuhamba.
Enye i-nuance yile ilogi yokuphindaphinda vs rdkafka.so - inxulumene neenkcukacha zeprojekthi yethu. Sisebenzisa i-PHP, kwaye kwi-PHP, njengomthetho, onke amathala eencwadi anxibelelana neKafka nge-rdkafka.so repository, kwaye ke kukho uhlobo oluthile lwe-wrapper. Mhlawumbi ezi zingxaki zethu zobuqu, kodwa kwavela ukuba ukufunda nje isiqwenga sento ebesele siyifundile akukho lula. Ngokubanzi, bekukho iingxaki zesoftware.
Ukubuyela kwiinkcukacha zokusebenza kunye nezahlulo, kubhaliwe ngokufanelekileyo kumaxwebhu abathengi >= izahlulo zesihloko. Kodwa ndafumanisa ngale nto emva kwexesha kakhulu kunokuba bendingathanda. Ukuba ufuna ukukala kwaye ube nabathengi ababini, udinga ubuncinci izahlulo ezimbini. Oko kukuthi, ukuba unesahlulelo esinye apho imiyalezo engamawaka angama-20 iqokelelwe, kwaye wenze entsha, inani lemiyalezo aliyi kulinganisa kungekudala. Ngoko ke, ukuze ube nabathengi ababini abafanayo, kufuneka ujongane nezahlulo.
Ukubeka iliso
Ndicinga ukuba indlela esiyijonga ngayo iya kucaca ngakumbi ukuba zeziphi iingxaki ezikhoyo kwindlela ekhoyo.
Ngokomzekelo, sibala ukuba zingaphi iimveliso kwisiseko sedatha ezisandul 'ukutshintsha imeko yazo, kwaye, ngokufanelekileyo, iziganeko kufuneka zenzeke ngokusekelwe kolu tshintsho, kwaye sithumela le nombolo kwinkqubo yethu yokubeka iliso. Emva koko ukusuka eKafka sifumana inombolo yesibini, zingaphi iziganeko ezirekhodwe ngokwenene. Ngokucacileyo, umahluko phakathi kwala manani mabini kufuneka uhlale unguziro.
Ukongeza, kufuneka ubeke iliso kwindlela umvelisi aqhuba ngayo, nokuba iziganeko-ibhasi ifumene imiyalezo, kunye nendlela umthengi aqhuba ngayo. Ngokomzekelo, kwiitshathi ezingezantsi, isiXhobo seMbuyiselo siqhuba kakuhle, kodwa iBOB ngokucacileyo ineengxaki ezithile (iincopho eziluhlaza).
Sele ndiyikhankanyile i-lag yeqela labathengi. Xa sithetha nje, eli linani lemiyalezo engafundwanga. Ngokubanzi, abathengi bethu basebenza ngokukhawuleza, ngoko ke i-lag idla ngokuba ngu-0, kodwa ngamanye amaxesha kunokubakho ixesha elifutshane. I-Kafka inokwenza oku ngaphandle kwebhokisi, kodwa kufuneka usete ikhefu elithile.
Kukho iprojekthi , eya kukunika ulwazi oluninzi nge Kafka. Inika ngokulula imeko ye-API yeqela labathengi, njengokuba eli qela lineshishini. Ukongeza kokuLungileyo kunye nokuFayile, kukho isilumkiso, kwaye uya kuba nakho ukufumanisa ukuba abathengi bakho abanako ukumelana nesantya sokuvelisa - abanalo ixesha lokuhlola kwakhona oko kubhaliweyo. Inkqubo ikrelekrele kwaye kulula ukuyisebenzisa.
Yile ndlela impendulo ye-API ibonakala ngayo. Nali iqela le-bob-live-fifa, i-partition refund.update.v1, isimo silungile, i-lag 0 yi-offset yokugqibela yokugqibela ethi kunye nokunye.
Ukubeka iliso ihlaziywe_kwi-SLA (ibambekile) Senditshilo. Umzekelo, imveliso iye yafudukela kwisimo sokuba ilungele ukubuyiswa. Sifaka i-Cron, ethi ukuba kwimizuzu emi-5 le nto ayizange ibuyele (sibuyisela imali ngeenkqubo zokuhlawula ngokukhawuleza), ngoko ngokuqinisekileyo kukho into ephosakeleyo, kwaye ngokuqinisekileyo le yimeko yokuxhasa. Ke ngoko, sithatha ngokulula iCron, efunda izinto ezinjalo, kwaye ukuba zikhulu kuno-0, ke ithumela isilumkiso.
Ukushwankathela, ukusebenzisa iziganeko kukulungele xa:
- ulwazi olufunekayo kwiinkqubo ezininzi;
- isiphumo sokusebenza asibalulekanga;
- kukho iziganeko ezimbalwa okanye iziganeko ezincinci.
Kubonakala ngathi inqaku linesihloko esicacileyo kakhulu - i-API esynchronous kwi-Kafka, kodwa ngokunxulumene nayo ndingathanda ukucebisa izinto ezininzi ngexesha elinye.
Okokuqala, ngokulandelayo Kufuneka silinde ngoNovemba, ngo-Ephreli kuya kubakho inguqulelo yaseSt. Petersburg, kwaye ngoJuni siza kuthetha ngemithwalo ephezulu eNovosibirsk.
Okwesibini, umbhali wengxelo, uSergei Zaika, ilungu leKomiti yeNkqubo yenkomfa yethu entsha yokulawula ulwazi. . Le nkomfa yosuku olunye, iza kubanjwa ngomhla wamashumi amabini anesithandathu kuTshazimpuzi, kodwa inkqubo yayo ixakeke kakhulu.
Kwaye kuya kuba ngoMeyi и (neDevOpsConf ibandakanyiwe) - unokusicebisa ngesihloko sakho apho, uthethe ngamava akho kwaye ukhalaze malunga namaqhuqhuva akho agxilileyo.
umthombo: www.habr.com