I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu

В isihloko sokugcina sibheke i-RabbitMQ ehlanganisa ukubekezelela amaphutha nokutholakala okuphezulu. Manje ake simbe sijule ku-Apache Kafka.

Lapha iyunithi yokuphindaphinda ihlukanisa. Isihloko ngasinye sinesigaba esisodwa noma ngaphezulu. Isigaba ngasinye sinomholi onabalandeli noma ongenabo. Lapho udala isihloko, ucacisa inani lama-partitions kanye ne-coefficient yokuphindaphinda. Inani elijwayelekile ngu-3, ​​okusho ukuthi izifaniso ezintathu: umholi oyedwa nabalandeli ababili.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 1. Izigaba ezine ezisatshalaliswa phakathi kwabathengi abathathu

Zonke izicelo zokufunda nokubhala ziya kumholi. Abalandeli ngezikhathi ezithile bathumela izicelo kumholi ukuze bathole imilayezo yakamuva. Abathengi abalokothi baphendukele kubalandeli; lokhu kwakamuva kutholakala kuphela ngenxa yokuphindaphinda nokubekezelela amaphutha.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu

Ukuhluleka kokuhlukanisa

Uma umthengisi ehluleka, abaholi bezigaba eziningana bavame ukuhluleka. Kulowo nalowo kubo, umlandeli ovela kwenye indawo uba umholi. Eqinisweni, lokhu akunjalo ngaso sonke isikhathi, njengoba isici sokuvumelanisa siphinde sibe nomthelela: noma ngabe kukhona abalandeli abavumelanisiwe, futhi uma kungenjalo, khona-ke ukuthi ukushintshela ku-replica engavumelanisiwe kuvunyelwe yini. Kodwa masingaxabanisi izinto okwamanje.

U-Broker 3 ushiya inethiwekhi, futhi umholi omusha ukhethwa esigabeni 2 ku-broker 2.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 2. U-Broker 3 uyashona futhi umlandeli wakhe ku-broker 2 ukhethwa njengomholi omusha we-partition 2

Ngemuva kwalokho umthengisi 1 uyashiya futhi isigaba 1 siphinde silahlekelwe umholi, indima yakhe idlulela ku-broker 2.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 3. Kusele umthengisi oyedwa. Bonke abaholi bakwi-broker eyodwa futhi akukho ukudunyiswa kwemisebenzi

Lapho umthengisi 1 ebuya ku-inthanethi, wengeza abalandeli abane, okuhlinzeka ngokuphindaphinda okuthile ekuhlukaniseni ngakunye. Kodwa bonke abaholi basahlala ku-broker 2.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 4. Abaholi bahlala ku-broker 2

Uma umthengisi 3 eqhamuka, sibuyela kumakhophi amathathu ngokuhlukaniswa ngakunye. Kepha bonke abaholi basakwi-broker 2.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 5. Ukubekwa kwabaholi okungenakulinganisela ngemva kokubuyiselwa kwabadayisi 1 no-3

I-Kafka inethuluzi lokuqondisa umholi ongcono kunoRabbitMQ. Lapho, bekumele usebenzise i-plugin yenkampani yangaphandle noma iskripthi esiguqule izinqubomgomo zokuthutha i-master node ngokunciphisa ukuphinda kusetshenziswe ngesikhathi sokuthutha. Ngaphezu kwalokho, kolayini abakhulu bekufanele samukele ukungatholakali ngesikhathi sokuvumelanisa.

I-Kafka inomqondo "wezithombe ezithandwayo" zendima yomholi. Lapho ukuhlukaniswa kwesihloko kwakhiwa, i-Kafka izama ukusabalalisa abaholi ngokulinganayo kuzo zonke izindawo futhi imaka labo baholi bokuqala njengokuthandayo. Ngokuhamba kwesikhathi, ngenxa yokuqaliswa kabusha kweseva, ukwehluleka, kanye nokwephuka kokuxhumana, abaholi bangase bagcine bephelele kwamanye ama-node, njengasesimweni esibi kakhulu esichazwe ngenhla.

Ukulungisa lokhu, i-Kafka inikeza izinketho ezimbili:

  • Inketho auto.leader.rebalance.enable=true ivumela i-node yesilawuli ukuthi inikeze abaholi kabusha ngokuzenzakalelayo kuma-replicas akhethwayo futhi ngaleyo ndlela ibuyisele ukusatshalaliswa okufanayo.
  • Umlawuli angaqalisa iskripthi kafka-preferred-replica-election.sh ngokunikezwa kabusha mathupha.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 6. Izifaniso ngemva kokulinganisa kabusha

Lena kwakuyinguqulo eyenziwe lula yokwehluleka, kodwa iqiniso liyinkimbinkimbi kakhulu, nakuba kungekho lutho oluyinkimbinkimbi kakhulu lapha. Konke kuza kuma-replicas avumelanisiwe (In-Sync Replicas, ISR).

I-synchronized Replicas (ISR)

I-ISR iyisethi yezifaniso zesahlukaniso esithathwa “njengesivunyelanisiwe” (ekuvunyelanisweni). Ukhona umholi, kodwa kungase kungabi khona abalandeli. Umlandeli uthathwa njengovunyelanisiwe uma enze amakhophi aqondile ayo yonke imilayezo yomholi ngaphambi kokuphelelwa yisikhathi. replica.lag.time.max.ms.

Umlandeli uyasuswa kusethi ye-ISR uma:

  • akenzanga isicelo sokukhetha isikhawu replica.lag.time.max.ms (kucatshangwa ukuthi ufile)
  • ayikwazanga ukuvuselela ngesikhathi sokuphumula replica.lag.time.max.ms (kucatshangwa ukuthi kuhamba kancane)

Abalandeli benza izicelo zesampula ngesikhathi sokuphumula replica.fetch.wait.max.ms, okuzenzakalelayo ku-500ms.

Ukuze sichaze ngokucacile inhloso ye-ISR, sidinga ukubheka iziqinisekiso ezivela kumkhiqizi kanye nezimo ezithile zokuhluleka. Abakhiqizi bangakhetha lapho umthengisi ethumela isiqinisekiso:

  • acks=0, isiqinisekiso asithunyelwanga
  • acks=1, isiqinisekiso sithunyelwa ngemva kokuba umholi ebhale umlayezo kulogi yangakini
  • acks=konke, isiqinisekiso sithunyelwa ngemva kokuthi zonke izifaniso ku-ISR zibhale umlayezo ezingodweni zendawo

Kumagama e-Kafka, uma i-ISR igcine umlayezo, "izibophezele". I-Acks=konke iyindlela ephephe kunazo zonke, kodwa futhi yengeza ukubambezeleka okwengeziwe. Ake sibheke izibonelo ezimbili zokwehluleka nokuthi izinketho ezihlukile 'ze-acks' zihlangana kanjani nomqondo we-ISR.

Ama-Acks=1 kanye ne-ISR

Kulesi sibonelo, sizobona ukuthi uma umholi engalindi ukuthi yonke imilayezo evela kubo bonke abalandeli igcinwe, khona-ke ukulahleka kwedatha kungenzeka uma umholi ehluleka. Ukuzulazulela kumlandeli ongavumelanisiwe kunganikwa amandla noma kukhutshazwe ngokusetha ongcolile.ukhetho.lomholi.Vumela.

Kulesi sibonelo, umenzi unenani elithi acks=1. Isigaba sisatshalaliswa kubo bonke abadayisi abathathu. I-Broker 3 ingemuva, ivunyelaniswe nomholi emizuzwaneni eyisishiyagalombili edlule futhi manje isisemuva kwemilayezo engu-7456. U-Broker 1 ubengemuva ngesekhondi elilodwa kuphela. Umkhiqizi wethu uthumela umlayezo futhi ngokushesha wamukele i-ack back, ngaphandle kwe-overhead yabalandeli abahamba kancane noma abafile umholi angayilindile.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 7. I-ISR enezifaniso ezintathu

I-Broker 2 yehluleka futhi umkhiqizi uthola iphutha lokuxhuma. Ngemuva kokuthi ubuholi budlulele ku-broker 1, silahlekelwa imilayezo engu-123. Umlandeli ku-broker 1 ubeyingxenye ye-ISR, kodwa akazange avunyelaniswe ngokugcwele nomholi ngesikhathi ewa.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 8. Imilayezo iyalahleka uma iphahlazeka

Ekucushweni bootstrap.servers Umkhiqizi unabathengisi abambalwa abasohlwini futhi angabuza omunye umthengisi ongumholi wesigaba esisha. Ibese isungula ukuxhumana nomthengisi 1 futhi iqhubeke nokuthumela imilayezo.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 9. Ukuthumela imilayezo kuyaqala futhi ngemva kwekhefu elifushane

U-Broker 3 usemuva kakhulu. Yenza izicelo zokulanda kodwa ayikwazi ukuvumelanisa. Lokhu kungenzeka kungenxa yokuxhumana okunensayo kwenethiwekhi phakathi kwabadayisi, inkinga yesitoreji, njll. Kususiwe ku-ISR. Manje i-ISR iqukethe ikhophi eyodwa - umholi! Umenzi uyaqhubeka nokuthumela imilayezo futhi athole iziqinisekiso.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 10. Umlandeli ku-broker 3 uyasuswa ku-ISR

U-Broker 1 uyehla futhi indima yobuholi iya ku-broker 3 ngokulahlekelwa kwemilayezo engu-15286! Umkhiqizi uthola umlayezo wephutha lokuxhumana. Ukushintshela kumholi ongaphandle kwe-ISR kwenzeke kuphela ngenxa yesilungiselelo okungcolile.ukhetho.umholi.vumela=iqiniso. Uma ifakwe ku bamanga, khona-ke ukuguqulwa bekungeke kwenzeke futhi zonke izicelo zokufunda nokubhala zizonqatshwa. Kulokhu, silinda umthengisi 1 ukuthi abuye nedatha yakhe esesimweni esifana nesesithombeni, esizophinde athathe ubuholi.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 11. U-Broker 1 uyawa. Uma kwenzeka ukwehluleka, inani elikhulu lemilayezo liyalahleka

Umdidiyeli usungula ukuxhumana nomthengisi wokugcina futhi ubona ukuthi manje usengumholi wesigaba. Uqala ukuthumela imiyalezo ku-broker 3.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 12. Ngemva kwekhefu elifushane, imiyalezo ithunyelwa futhi esigabeni 0

Sibonile ukuthi, ngaphandle kweziphazamiso ezimfushane zokusungula ukuxhumana okusha nokucinga umholi omusha, umenzi wayehlala ethumela imilayezo. Lokhu kulungiselelwa kuqinisekisa ukutholakala ngezindleko zokuvumelana (ukuphepha kwedatha). U-Kafka ulahlekelwe izinkulungwane zemilayezo kodwa waqhubeka nokwamukela imibhalo emisha.

Acks=konke kanye ne-ISR

Asiphinde lesi simo futhi, kodwa nge imikhovu=konke. I-Broker 3 ine-latency emaphakathi yamasekhondi amane. Umkhiqizi uthumela umlayezo nge imikhovu=konke, futhi manje ayitholi impendulo esheshayo. Umholi ulinda ukuthi umlayezo ulondolozwe yizo zonke izifaniso ku-ISR.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 13. I-ISR enezifaniso ezintathu. Okukodwa kuhamba kancane, okuholela ekubambezelekeni kokurekhoda

Ngemuva kwemizuzwana emine yokulibaziseka okwengeziwe, umthengisi 2 uthumela i-ack. Zonke izifaniso manje sezibuyekezwe ngokugcwele.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 14. Wonke amakhophi agcina imiyalezo futhi athumele i-ack

U-Broker 3 manje usemuva kakhulu futhi ususiwe ku-ISR. Ukubambezeleka kwehliswe kakhulu ngoba azikho izifaniso ezihamba kancane ezisele ku-ISR. U-Broker 2 manje ulinde u-broker 1 kuphela, futhi une-average lag engu-500 ms.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 15. Okufanayo ku-broker 3 kuyasuswa ku-ISR

Ngemuva kwalokho u-broker 2 uyawa futhi ubuholi budlulela ku-broker 1 ngaphandle kokulahlekelwa yimilayezo.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 16. U-Broker 2 uyawa

Umkhiqizi uthola umholi omusha futhi uqala ukuthumela imiyalezo kuye. Ukubambezeleka kuphinde kuncishiswe ngoba i-ISR manje isiqukethe isifaniso esisodwa! Ngakho-ke inketho imikhovu=konke akungezi ukuphindaphinda.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 17. I-Replica ku-broker 1 iyahola ngaphandle kokulahlekelwa imilayezo

Khona-ke u-broker 1 uphahlazeka futhi ukuhola kuya ku-broker 3 ngokulahlekelwa kwemilayezo ye-14238!

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 18. U-Broker 1 uyafa futhi ukuguquka kobuholi ngesilungiselelo esingcolile kuphumela ekulahlekeni kwedatha okubanzi

Asikwazanga ukufaka inketho ongcolile.ukhetho.lomholi.Vumela ibe yincazelo weqiniso. Ngokuzenzakalelayo kuyalingana bamanga. Izilungiselelo imikhovu=konke с okungcolile.ukhetho.umholi.vumela=iqiniso inikeza ukufinyeleleka ngokuphepha okuthile kwedatha eyengeziwe. Kodwa njengoba ubona, sisengalahlekelwa yimilayezo.

Kodwa kuthiwani uma sifuna ukukhulisa ukuphepha kwedatha? Ungabeka okungcolile.umholi.ukhetho.vumela = amanga, kodwa lokhu ngeke kusivikele ekulahlekeni kwedatha. Uma umholi ewa kanzima futhi wathatha idatha ngayo, khona-ke imilayezo isalahlekile, futhi ukutholakala kuyalahleka kuze kube yilapho umlawuli ebuyisela isimo.

Kungcono ukuqinisekisa ukuthi yonke imilayezo ayisebenzi, futhi ulahle okurekhodiwe. Bese, okungenani ngokombono womdayisi, ukulahleka kwedatha kungenzeka kuphela esimweni sokuhluleka okubili noma ngaphezulu ngesikhathi esisodwa.

Acks=konke, min.insync.replicas kanye ne-ISR

Ngokucushwa kwesihloko min.insync.replicas Senyusa izinga lokuvikeleka kwedatha. Ake sidlule engxenyeni yokugcina yesimo esidlule futhi, kodwa kulokhu min.insync.replicas=2.

Ngakho-ke umthengisi 2 unomholi we-replica futhi umlandeli ku-broker 3 uyasuswa ku-ISR.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 19. I-ISR evela kumakhophi amabili

U-Broker 2 uyawa futhi ubuholi budlulela ku-broker 1 ngaphandle kokulahlekelwa yimilayezo. Kodwa manje i-ISR iqukethe ikhophi eyodwa kuphela. Lokhu akuhlangabezani nenani elincane lokuthola amarekhodi, ngakho-ke umthengisi uphendula umzamo wokubhala ngephutha NotEnoughReplicas.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 20. Inombolo yama-ISR ingaphansi ngokukodwa kunaleyo eshiwo kokuthi min.insync.replicas

Lokhu kulungiselelwa kudela ukutholakala kokuvumelana. Ngaphambi kokuvuma umlayezo, siyaqinisekisa ukuthi ubhalelwe okungenani izifaniso ezimbili. Lokhu kunikeza umenzi ukuzethemba okwengeziwe. Lapha, ukulahlekelwa komlayezo kungenzeka kuphela uma izifaniso ezimbili zehluleka ngesikhathi esisodwa esikhathini esifushane kuze kube yilapho umlayezo uphindwaphindwa kumlandeli owengeziwe, okuyinto engenakwenzeka. Kodwa uma u-super paranoid, ungasetha isici sokuphindaphinda sibe ngu-5, futhi min.insync.replicas ngo-3. Lapha abathengisi abathathu kufanele bawe ngesikhathi esisodwa ukuze balahlekelwe irekhodi! Yebo, ukhokhela lokhu kwethembeka ekubambezelekeni okwengeziwe.

Lapho ukufinyeleleka kudingekile ukuze kuvikelwe idatha

Njengangaphakathi icala nge-RabbitMQ, ngezinye izikhathi ukufinyeleleka kuyadingeka ukuze kuvikelwe idatha. Nakhu okudingeka ucabange ngakho:

  • Ingabe umshicileli angavele abuyisele iphutha bese enikeza isevisi ekhuphukayo noma umsebenzisi azame futhi ngemva kwesikhathi?
  • Ingabe umshicileli angawugcina umlayezo endaweni noma kusizindalwazi ukuze azame futhi ngemva kwesikhathi?

Uma impendulo ithi cha, ukwenza ngcono ukutholakala kuthuthukisa ukuphepha kwedatha. Uzolahlekelwa idatha encane uma ukhetha ukutholakala esikhundleni sokungaqophi. Ngakho, konke kufika ekutholeni ibhalansi, futhi isinqumo sincike esimweni esithile.

Incazelo ye-ISR

I-ISR suite ikuvumela ukuthi ukhethe ibhalansi efanele phakathi kokuphepha kwedatha nokubambezeleka. Isibonelo, qinisekisa ukutholakala uma kwenzeka ukwehluleka kwezifaniso eziningi, unciphisa umthelela wama-replicas afile noma ahamba kancane ngokuya ngokubambezeleka.

Sizikhethela incazelo ngokwethu replica.lag.time.max.ms ngokwezidingo zakho. Empeleni, le pharamitha isho ukuthi kungakanani ukubambezeleka esizimisele ukukwamukela nini imikhovu=konke. Inani elizenzakalelayo liyimizuzwana eyishumi. Uma lokhu kukude kakhulu, ungakunciphisa. Khona-ke imvamisa yezinguquko ku-ISR izokhula, njengoba abalandeli bazosuswa futhi bangezwe kaningi.

I-RabbitMQ imane iyisethi yezibuko ezidinga ukuphindwa. Izibuko ezihamba kancane zethula ukubambezeleka okwengeziwe, futhi izibuko ezifile zingalinda kuze kube yilapho amaphakethe ahlola ukutholakala kwenodi ngayinye (uphawu lwenetha) ukuze aphendule. I-ISR iyindlela ethokozisayo yokugwema lezi zinkinga zokubambezeleka. Kodwa sizibeka engcupheni yokulahlekelwa ukuphindiselwa kabusha njengoba i-ISR ingancipha kuphela kumholi. Ukuze ugweme le ngozi, sebenzisa isilungiselelo min.insync.replicas.

Isiqinisekiso sokuxhuma kweklayenti

Kuzilungiselelo bootstrap.servers umkhiqizi nomthengi bangacacisa abathengisi abaningi bokuxhuma amaklayenti. Umqondo uwukuthi lapho i-node eyodwa yehla, kunezimbalwa ezimbalwa ezisele lapho iklayenti lingavula khona uxhumano. Laba akubona ngempela abaholi bezigaba, kodwa bamane nje bayibhodi lokuqala lokulayisha. Iklayenti lingalibuza ukuthi iyiphi i-node esingethe umholi wesigaba sokufunda/sokubhala.

Ku-RabbitMQ, amaklayenti angaxhuma kunoma iyiphi indawo, futhi umzila wangaphakathi uthumela isicelo lapho kudingeka siye khona. Lokhu kusho ukuthi ungakwazi ukufaka isilinganisi somthwalo phambi kwe-RabbitMQ. I-Kafka idinga amaklayenti ukuthi axhume endaweni esingethe umholi ohambisanayo wokuhlukanisa. Esimeni esinjalo, awukwazi ukufaka isilinganisi somthwalo. Uhlu bootstrap.servers Kubalulekile ukuthi amaklayenti akwazi ukufinyelela futhi athole ama-node alungile ngemuva kokwehluleka.

I-Kafka Consensus Architecture

Kuze kube manje, asikacabangi ukuthi iqoqo lifunda kanjani ngokuwa komthengisi nokuthi umholi omusha ukhethwa kanjani. Ukuze uqonde ukuthi i-Kafka isebenza kanjani ngokuhlukaniswa kwenethiwekhi, udinga kuqala ukuqonda ukwakheka kokuvumelana.

Iqoqo ngalinye le-Kafka lisetshenziswa kanye neqoqo le-Zookeeper, okuyisevisi yokuvumelana esabalalisiwe evumela isistimu ukuthi ifinyelele esivumelwaneni esithile esimweni esithile, ibeka phambili ukuvumelana ngaphezu kokutholakala. Imvume yeningi lama-Zookeeper node iyadingeka ukuze kugunyazwe ukusebenza kokufunda nokubhala.

I-Zookeeper igcina isimo seqoqo:

  • Uhlu lwezihloko, izigaba, ukumisa, izifaniso zamanje zomholi, izifaniso ezikhethwayo.
  • Amalungu eqoqo. Umthengisi ngamunye uphinga iqoqo le-Zookeeper. Uma ingatholi i-ping phakathi nenkathi ethile yesikhathi, i-Zookeeper irekhoda umthengisi njengongatholakali.
  • Ukukhetha ama-node amakhulu nama-spare wesilawuli.

Indawo yesilawuli ingenye yabathengi be-Kafka abanesibopho sokukhetha abaholi be-replica. I-Zookeeper ithumela izaziso kusilawuli mayelana nobulungu beqembu nezinguquko zesihloko, futhi umlawuli kufanele enze lezi zinguquko.

Isibonelo, ake sithathe isihloko esisha esinama-partitions ayishumi kanye ne-replication factor engu-3. Umlawuli kufanele akhethe umholi wengxenye ngayinye, ezama ukusabalalisa abaholi phakathi kwama-broker.

Ngesilawuli sesigaba ngasinye:

  • ibuyekeza ulwazi ku-Zookeeper mayelana ne-ISR nomholi;
  • Ithumela i-LeaderAndISRCommand kumthengisi ngamunye osingatha isifaniso salokhu kwahlukaniswa, ukwazisa abathengi nge-ISR kanye nomholi.

Uma umthengisi onomholi ewa, i-Zookeeper ithumela isaziso kumlawuli, bese ikhetha umholi omusha. Futhi, isilawuli siqale sibuyekeze i-Zookeeper bese sithumela umyalo kumthengisi ngamunye emazisa ngoshintsho lobuholi.

Umholi ngamunye unesibopho sokuqasha ama-ISR. Izilungiselelo replica.lag.time.max.ms inquma ukuthi ubani ozongena lapho. Lapho i-ISR ishintsha, umholi udlulisela ulwazi olusha ku-Zookeeper.

U-Zookeeper uhlale ekwaziswa nganoma yiziphi izinguquko ukuze uma kwenzeka ukwehluleka, abaphathi bashintshele kumholi omusha ngokushelela.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 21. Kafka Consensus

Iphrothokholi yokuphindaphinda

Ukuqonda imininingwane yokuphindaphinda kukusiza ukuthi uqonde kangcono izimo zokulahleka kwedatha ezingaba khona.

Imibuzo yamasampula, i-Log End Offset (LEO) kanye ne-Highwater Mark (HW)

Sicabange ukuthi abalandeli ngezikhathi ezithile bathumela izicelo zokulanda kumholi. Isikhawu esizenzakalelayo singu-500ms. Lokhu kwehlukile ku-RabbitMQ ngoba ukuphindaphinda kwe-RabbitMQ akuqalwa yisibuko esikumugqa kodwa kuqalwa umnikazi. Inkosi iphusha izinguquko ezibukweni.

Umholi nabo bonke abalandeli balondoloza ilebula ye-Log End Offset (LEO) kanye ne-Highwater (HW). Uphawu lwe-LEO lugcina ukuguqulwa komlayezo wokugcina kumfanekiso wasendaweni, futhi i-HW ibambe ingxenye yesivumelwano sokugcina. Khumbula ukuthi ngesimo sokuzibophezela, umlayezo kufanele uqhutshwe kuzo zonke izifaniso ze-ISR. Lokhu kusho ukuthi i-LEO ivamise ukuba ngaphambi kancane kune-HW.

Uma umholi ethola umyalezo, uwugcina endaweni. Umlandeli wenza isicelo sokulanda ngokudlulisa i-LEO yakhe. Umholi ube esethumela inqwaba yemilayezo eqala kule LEO futhi aphinde adlulisele i-HW yamanje. Uma umholi ethola ulwazi lokuthi zonke izifaniso zigcine umlayezo endaweni enikeziwe, ihambisa umaki we-HW. Umholi kuphela ongahambisa i-HW, ngakho bonke abalandeli bazokwazi inani lamanje ezimpendulweni zesicelo sabo. Lokhu kusho ukuthi abalandeli bangase basalele ngemuva komholi kukho kokubili umlayezo kanye nolwazi lwe-HW. Abathengi bathola imilayezo kuphela kuze kufike ku-HW yamanje.

Qaphela ukuthi "kuphikelela" kusho ukuthi kubhalwe kumemori, hhayi kudiski. Ngokusebenza, i-Kafka ivumelanisa kudiski ngesikhathi esithile. I-RabbitMQ nayo inesikhawu esinjalo, kodwa izothumela ukuvuma kumshicileli kuphela ngemva kokuba umphathi nazo zonke izibuko zibhale umlayezo kudiski. Abathuthukisi be-Kafka, ngenxa yezizathu zokusebenza, banqume ukuthumela i-ack ngokushesha nje lapho umlayezo ubhalwa enkumbulweni. I-Kafka ibhejela ukuthi ukunganakwa kuqeda ubungozi bokugcina kafushane imilayezo eyamukelwe enkumbulweni kuphela.

Ukwehluleka komholi

Uma umholi ewa, i-Zookeeper yazisa isilawuli, bese ikhetha isifaniso somholi omusha. Umholi omusha usetha umaki omusha we-HW ngokusho kwe-LEO yakhe. Abalandeli babe sebethola ulwazi ngomholi omusha. Ngokuya ngenguqulo ye-Kafka, umlandeli uzokhetha esinye sezimo ezimbili:

  1. Izonciphisa ilogu yendawo ku-HW eyaziwayo futhi ithumele isicelo kumholi omusha semilayezo ngemva kwalolu phawu.
  2. Uzothumela isicelo kumholi ukuze athole i-HW ngesikhathi eqokwa njengomholi, bese enciphisa ilogi kulokhu. Izoqala-ke ukwenza izicelo zokulanda ngezikhathi ezithile iqala kulokhu kususwa.

Umlandeli angase adinge ukunqamula ilogu ngezizathu ezilandelayo:

  • Uma umholi ehluleka, umlandeli wokuqala kusethi ye-ISR ebhaliswe ne-Zookeeper uyawina ukhetho futhi abe umholi. Bonke abalandeli ku-ISR, nakuba kucatshangelwa “ekuvunyelanisweni,” kungenzeka abatholanga amakhophi ayo yonke imilayezo evela kumholi wangaphambili. Kungenzeka ngokuphelele ukuthi umlandeli ofakiwe akanayo ikhophi yakamuva kakhulu. I-Kafka iqinisekisa ukuthi akukho mehluko phakathi kwama-replicas. Ngakho, ukuze kugwenywe ukungqubuzana, umlandeli ngamunye kufanele anciphise ilogi yakhe enanini lwe-HW lomholi omusha ngesikhathi sokukhethwa kwakhe. Lesi esinye isizathu esenza ukusetha imikhovu=konke kubaluleke kakhulu ukungaguquguquki.
  • Imilayezo ibhalwa ngezikhathi ezithile kudiski. Uma wonke ama-cluster node ehluleka ngesikhathi esifanayo, ama-replicas ane-offsets ahlukene azogcinwa kumadiski. Kungenzeka ukuthi uma ama-broker ebuya ku-inthanethi, umholi omusha okhethiwe uzoba ngemuva kwabalandeli bakhe ngoba ugcinwe kudiski ngaphambi kwabanye.

Ukuhlangana kabusha neqoqo

Lapho ijoyina futhi iqoqo, izifaniso zenza okufanayo nalapho umholi ehluleka: zihlola umfanekiso womholi futhi zinciphise ilogi yazo zibe yi-HW yayo (ngesikhathi sokhetho). Uma kuqhathaniswa, i-RabbitMQ iphatha ngokulinganayo ama-node ahlangene njengamasha ngokuphelele. Kuzo zombili izimo, umdayisi ulahla noma yisiphi isimo esikhona. Uma kusetshenziswa ukuvumelanisa okuzenzakalelayo, khona-ke umnikazi kufanele aphindaphinde ngokuphelele konke okuqukethwe kwamanje esibukweni esisha ngendlela "yeka umhlaba wonke ulinde". Umphathi akakwamukeli noma yimiphi imisebenzi yokufunda noma yokubhala phakathi nalo msebenzi. Le ndlela idala izinkinga kolayini abakhulu.

I-Kafka iyilogi esabalalisiwe futhi ngokuvamile igcina imilayezo eminingi kunomugqa we-RabbitMQ, lapho idatha ikhishwa khona kulayini ngemva kokuba ifundiwe. Olayini abasebenzayo kufanele bahlale bebancane uma kuqhathaniswa. Kodwa i-Kafka iyilogi enenqubomgomo yayo yokugcina, engabeka isikhathi sezinsuku noma amasonto. Ukuvinjwa komugqa nendlela yokuvumelanisa ngokugcwele akwamukelekile kulogi esabalalisiwe. Esikhundleni salokho, abalandeli be-Kafka bavele banciphise ilogi yabo ku-HW yomholi (ngesikhathi sokhetho lwakhe) uma ikhophi yabo ingaphambi komholi. Esimeni esingase sibe khona, lapho umlandeli engemuva, uvele aqale ukwenza izicelo zokulanda eqala nge-LEO yayo yamanje.

Abalandeli abasha noma abajoyinile kabusha baqala ngaphandle kwe-ISR futhi abahlanganyeli kuzibophezelo. Bamane basebenze kanye neqembu, bethola imilayezo ngokushesha ngangokunokwenzeka baze bahlangane nomholi futhi bangene ku-ISR. Akukho ukukhiya futhi asikho isidingo sokulahla yonke idatha yakho.

Ukulahleka kokuxhumana

I-Kafka inezingxenye eziningi kune-RabbitMQ, ngakho-ke inesethi yokuziphatha eyinkimbinkimbi lapho iqoqo linqanyulwa. Kodwa i-Kafka ekuqaleni yayiklanyelwe amaqoqo, ngakho-ke izixazululo zicatshangelwe kahle kakhulu.

Ngezansi kunezimo ezimbalwa zokuhluleka kokuxhumana:

  • Isimo 1: Umlandeli akamboni umholi, kodwa usabona uMgcini-Zookeeper.
  • Isimo sesi-2: Umholi akaboni abalandeli, kodwa usabona i-Zookeeper.
  • Isimo sesi-3: Umlandeli ubona umholi, kodwa akaboni uMgcini-Zookeeper.
  • Isimo sesi-4: Umholi ubona abalandeli, kodwa akaboni uMgcini-Zookeeper.
  • Isimo sesi-5: Umlandeli uhluke ngokuphelele kuwo womabili amanodi e-Kafka kanye ne-Zookeeper.
  • Isimo sesi-6: Umholi uhluke ngokuphelele kuwo womabili amanodi e-Kafka kanye ne-Zookeeper.
  • Isimo sesi-7: Inodi yesilawuli se-Kafka ayikwazi ukubona enye inodi ye-Kafka.
  • Isimo 8: Isilawuli se-Kafka asiboni i-Zookeeper.

Isimo ngasinye sinokuziphatha kwaso.

Isimo 1: Umlandeli akamboni umholi, kodwa usabona i-Zookeeper

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 22. Isimo 1: I-ISR yamakhophi amathathu

Ukwehluleka kokuxhumana kwehlukanisa umthengisi 3 kubathengi 1 no-2, kodwa hhayi ku-Zookeeper. U-Broker 3 akasakwazi ukuthumela izicelo zokulanda. Ngemva kwesikhathi sesidlulile replica.lag.time.max.ms iyasuswa ku-ISR futhi ayihlanganyeli ekwenziweni komlayezo. Uma uxhumano selubuyiselwe, luzoqala kabusha ukulanda izicelo futhi lujoyine i-ISR uma itholana nomholi. I-Zookeeper izoqhubeka nokuthola ama-pings futhi icabange ukuthi umthengisi uyaphila futhi uphilile.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 23. Isimo 1: Umdayiseli uyasuswa ku-ISR uma kungekho sicelo sokulanda esitholwa kuyo phakathi nesikhathi se-replica.lag.time.max.ms

Akukho ukuhlukaniswa kobuchopho noma ukumiswa kwe-node njengaku-RabbitMQ. Esikhundleni salokho, ukuphinda kuncishiswe.

Isimo sesi-2: Umholi akakaboni balandeli, kodwa usabona i-Zookeeper

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 24. Isimo 2. Umholi nabalandeli ababili

Ukuphazamiseka kokuxhumana kwenethiwekhi kwehlukanisa umholi kubalandeli, kodwa umthengisi usengakwazi ukubona i-Zookeeper. Njengasesimeni sokuqala, i-ISR iyashwabana, kodwa kulokhu kuphela kumholi njengoba bonke abalandeli beyeka ukuthumela izicelo zokulanda. Futhi, akukho ukwehlukana okunengqondo. Esikhundleni salokho, kukhona ukulahleka kokuphinda kusetshenziswe imilayezo emisha kuze kubuyiselwe ukuxhumana. U-Zookeeper uyaqhubeka nokuthola ama-pings futhi ukholelwa ukuthi umthengisi uyaphila futhi uphilile.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 25. Isimo 2. I-ISR inciphe kumholi kuphela

Isimo 3. Umlandeli ubona umholi, kodwa akaboni uMgcini-Zookeeper

Umlandeli uhlukaniswe noZookeeper, kodwa hhayi kumthengisi onomholi. Njengomphumela, umlandeli uyaqhubeka nokwenza izicelo zokulanda futhi abe yilungu le-ISR. I-Zookeeper ayisawatholi ama-pings futhi ibhalisa ukuphahlazeka komthengisi, kodwa njengoba ingumlandeli kuphela, ayikho imiphumela ngemva kokululama.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 26. Isimo sesi-3: Umlandeli uyaqhubeka nokuthumela izicelo zokulanda kumholi

Isimo 4. Umholi ubona abalandeli, kodwa akaboni u-Zookeeper

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 27. Isimo 4. Umholi nabalandeli ababili

Umholi uhlukene noZookeeper, kodwa hhayi kubathengi abanabalandeli.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 28. Isimo 4: Umholi uhlukaniswa yedwa kuZookeeper

Ngemuva kwesikhathi esithile, i-Zookeeper izobhalisa ukwehluleka komdayisi futhi azise isilawuli ngakho. Uzokhetha umholi omusha phakathi kwabalandeli bakhe. Kodwa-ke, umholi wokuqala uzoqhubeka nokucabanga ukuthi ungumholi futhi uzoqhubeka nokwamukela imingenelo evela amaqebe=1. Abalandeli abasamthumeli ukulanda izicelo, ngakho-ke uzozibheka njengezifile futhi azame ukuzincisha i-ISR ngokwayo. Kodwa njengoba ingenakho ukuxhumana ne-Zookeeper, ngeke ikwazi ukwenza lokhu, futhi ngaleso sikhathi izokwenqaba ukwamukela okunye okufakiwe.

Imilayezo imikhovu=konke ngeke ithole ukuvunywa ngoba i-ISR ivula kuqala zonke izifaniso, futhi imilayezo ayifinyeleli kuzo. Uma umholi wangempela ezama ukuwasusa ku-ISR, ngeke akwazi ukwenza kanjalo futhi uzoyeka ukwamukela noma yimiphi imilayezo nhlobo.

Ngokushesha amaklayenti abona ushintsho kumholi futhi aqala ukuthumela amarekhodi kuseva entsha. Uma inethiwekhi isibuyiselwe, umholi wangempela ubona ukuthi akaseyena umholi futhi unciphisa ilogi yakhe enanini le-HW umholi omusha ayenalo ngesikhathi sokwehluleka ukugwema ukwehlukana kwelogi. Izobe isiqala ukuthumela izicelo zokulanda kumholi omusha. Wonke amarekhodi avela kumholi wasekuqaleni angazange afaniswe nomholi omusha alahlekile. Okusho ukuthi, imilayezo engazange ivunywe umholi wokuqala kuleyo mizuzwana embalwa lapho abaholi ababili besebenza izolahleka.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 29. Isimo 4. Umholi kumthengisi 1 uba umlandeli ngemva kokubuyiselwa kwenethiwekhi

Isimo sesi-5: Umlandeli uhluke ngokuphelele kuwo womabili amanodi e-Kafka kanye ne-Zookeeper

Umlandeli uhlukaniswe ngokuphelele kuzo zombili ezinye izindawo ze-Kafka kanye ne-Zookeeper. Uvele azikhiphe ku-ISR kuze kube yilapho inethiwekhi ibuyiselwa, bese ibamba nabanye.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 30. Isimo 5: Umlandeli oyedwa uyasuswa ku-ISR

Isimo sesi-6: Umholi uhluke ngokuphelele kuwo womabili amanodi e-Kafka kanye ne-Zookeeper

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 31. Isimo 6. Umholi nabalandeli ababili

Umholi uhlukaniswe ngokuphelele nabalandeli bakhe, isilawuli kanye neZookeeper. Isikhathi esifushane izoqhubeka nokwamukela imingenelo evela amaqebe=1.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 32. Isimo 6: Ukuhlukanisa umholi kwamanye amanodi e-Kafka kanye neZookeeper

Njengoba ungazitholanga izicelo ngemuva kokuphelelwa yisikhathi replica.lag.time.max.ms, izozama ukunciphisa i-ISR ngokwayo, kodwa ngeke ikwazi ukwenza kanjalo ngoba akukho ukuxhumana ne-Zookeeper, khona-ke izoyeka ukwamukela ukubhala.

Ngaleso sikhathi, u-Zookeeper uzomaka umthengisi ongayedwana njengofile futhi umlawuli uzokhetha umholi omusha.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 33. Isimo 6. Abaholi ababili

Umholi wangempela angase amukele okufakiwe imizuzwana embalwa, kodwa abese eyeka ukwamukela noma imiphi imilayezo. Amaklayenti abuyekezwa njalo ngemizuzwana engama-60 ngemethadatha yakamuva. Bazokwaziswa ngokushintsha komholi futhi bazoqala ukuthumela imingenelo kumholi omusha.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 34. Isimo 6: Abakhiqizi bashintshela kumholi omusha

Konke okufakiwe okuqinisekisiwe okwenziwe umholi wangempela kusukela ekulahlekeni kokuxhumana kuzolahleka. Uma inethiwekhi isibuyiselwe, umholi wangempela uzothola nge-Zookeeper ukuthi akaseyena umholi. Ngemuva kwalokho izonciphisa ilogi yayo ku-HW yomholi omusha ngesikhathi sokhetho bese iqala ukuthumela izicelo njengomlandeli.

I-RabbitMQ vs Kafka: Ukubekezelela Amaphutha kanye Nokutholakala Okuphezulu
Ilayisi. 35. Isimo 6: Umholi wangempela uba umlandeli ngemva kokubuyiselwa kokuxhumana kwenethiwekhi

Kulesi simo, ukuhlukana okunengqondo kungase kwenzeke isikhathi esifushane, kodwa kuphela uma amaqebe=1 и min.insync.replicas futhi 1. Ukuhlukaniswa okunengqondo kuphela ngokuzenzakalelayo noma ngemva kokubuyiselwa kwenethiwekhi, lapho umholi wangempela eqaphela ukuthi akaseyena umholi, noma lapho wonke amaklayenti ebona ukuthi umholi ushintshile futhi waqala ukubhalela umholi omusha - noma yikuphi okwenzeka kuqala. Kunoma yikuphi, eminye imilayezo izolahleka, kodwa kuphela nge amaqebe=1.

Kukhona okunye okuhlukile kwalesi simo lapho, ngaphambi nje kokuba inethiwekhi ihlukane, abalandeli bawa ngemuva futhi umholi wacindezela i-ISR kuye ngokwakhe. Ibe isihlukaniswa ngenxa yokulahleka kokuxhumana. Umholi omusha uyakhethwa, kodwa umholi wokuqala uyaqhubeka nokwamukela imingenelo, ngisho imikhovu=konke, ngoba akekho omunye ku-ISR ngaphandle kwakhe. Lawa marekhodi azolahleka uma inethiwekhi isibuyiselwe. Okuwukuphela kwendlela yokugwema le nketho min.insync.replicas = 2.

Isimo 7: I-Kafka Controller Node Ayikwazi Ukubona Enye I-Kafka Node

Ngokuvamile, uma uxhumano nenodi ye-Kafka selulahlekile, isilawuli ngeke sikwazi ukudlulisa noma yiluphi ulwazi loshintsho lomholi kuso. Esimeni esibi kakhulu, lokhu kuzoholela ekuhlukaneni okunengqondo kwesikhathi esifushane, njengakusimo sesi-6. Kaningi kunalokho, umthengisi ngeke avele akhethe ubuholi uma lowo kugcina ehluleka.

Isimo 8: Isilawuli se-Kafka asiboni i-Zookeeper

I-Zookeeper ngeke ithole i-ping kusuka kusilawuli esiwile futhi izokhetha inodi entsha ye-Kafka njengesilawuli. Isilawuli sangempela singaqhubeka nokuzethula sinjalo, kodwa asizitholi izaziso ezivela ku-Zookeeper, ngakho ngeke sibe nanoma yimiphi imisebenzi esizoyenza. Uma inethiwekhi isibuyiselwe, uzobona ukuthi akaseyena umlawuli, kodwa usephenduke i-Kafka node evamile.

Iziphetho ezivela kuzimo

Siyabona ukuthi ukulahlekelwa uxhumano lomlandeli akuholeli ekulahlekeni komlayezo, kodwa kumane kunciphisa okwesikhashana ukungasasebenzi kuze kube yilapho inethiwekhi ibuyiselwa. Lokhu, kunjalo, kungaholela ekulahlekeni kwedatha uma i-node eyodwa noma ngaphezulu ilahleka.

Uma umholi ehlukana no-Zookeeper ngenxa yokulahlekelwa ukuxhumana, lokhu kungase kubangele ukuthi imilayezo ilahleke evela ku-Zookeeper. amaqebe=1. Ukuntuleka kokuxhumana noZookeeper kubangela ukuhlukana okufushane okunengqondo nabaholi ababili. Le nkinga ixazululwa yipharamitha imikhovu=konke.

Ipharamitha min.insync.replicas zibe izifaniso ezimbili noma ngaphezulu zinikeza isiqinisekiso esengeziwe sokuthi izimo zesikhathi esifushane ezinjalo ngeke ziphumele emilayezweni elahlekile njengakuSimo sesi-6.

Isifinyezo Semilayezo Elahlekile

Ake sibhale zonke izindlela ongalahlekelwa ngazo idatha e-Kafka:

  • Noma yikuphi ukwehluleka komholi uma imilayezo iqinisekisiwe kusetshenziswa amaqebe=1
  • Noma yikuphi ukuguquka kobuholi obungcolile, okungukuthi, kumlandeli ongaphandle kwe-ISR, ngisho noma imikhovu=konke
  • Ukuhlukanisa umholi ku-Zookeeper uma imilayezo iqinisekisiwe kusetshenziswa amaqebe=1
  • Ukuhlukaniswa okuphelele komholi osevele wehlisela kuyena iqembu le-ISR. Yonke imilayezo izolahleka, ngisho imikhovu=konke. Lokhu kuyiqiniso kuphela uma min.insync.replicas=1.
  • Ukwehluleka ngesikhathi esisodwa kwawo wonke ama-partition node. Ngenxa yokuthi imilayezo yamukelwa ngekhanda, eminye ingase ingakabhalwa kudiski. Ngemva kokuqalisa kabusha iziphakeli, eminye imilayezo ingase ingekho.

Izinguquko ezingcolile zobuholi zingagwenywa ngokuzivimbela noma ngokuqinisekisa okungenani ukuxoshwa okubili. Ukucushwa okuhlala isikhathi eside kuyinhlanganisela imikhovu=konke и min.insync.replicas phezu 1.

Ukuqhathanisa okuqondile kokuthembeka kwe-RabbitMQ ne-Kafka

Ukuqinisekisa ukwethembeka nokutholakala okuphezulu, zombili izinkundla zisebenzisa uhlelo lokuphindaphinda oluyisisekelo nolwesibili. Nokho, i-RabbitMQ inesithende se-Achilles. Lapho uxhuma kabusha ngemva kokwehluleka, amanodi alahla idatha yawo futhi ukuvumelanisa kuyavinjwa. Lokhu kuthuka kabili kwenza kungabaza ubude besikhathi solayini abakhulu ku-RabbitMQ. Kuzodingeka wamukele noma ukuncishiswa ukuphinda uthole izikhathi ezinde zokuvinjwa. Ukunciphisa ukungasasebenzi kwandisa ingozi yokulahleka kwedatha okukhulu. Kodwa uma imigqa mincane, ngakho-ke ngenxa yokuphindaphinda, izikhathi ezimfushane zokungatholakali (imizuzwana embalwa) zingabhekwana nazo ngokusebenzisa imizamo yokuxhumana ephindaphindiwe.

I-Kafka ayinayo le nkinga. Ilahla idatha kuphela ephuzwini lokuhlukana phakathi komholi nomlandeli. Yonke idatha eyabiwe ilondoloziwe. Ngaphezu kwalokho, ukuphindaphinda akuvimbi uhlelo. Umholi uyaqhubeka nokwamukela okuthunyelwe ngenkathi umlandeli omusha ebamba, ngakho-ke kuma-devops, ukujoyina noma ukujoyina kabusha iqoqo kuba umsebenzi omncane. Yiqiniso, kusenezinkinga ezifana nomkhawulokudonsa wenethiwekhi ngesikhathi sokuphindaphinda. Uma ungeza abalandeli abaningi ngesikhathi esisodwa, ungase uhlangabezane nomkhawulo womkhawulokudonsa.

I-RabbitMQ iphakeme kune-Kafka ngokwethembeka lapho amaseva amaningi kuqoqo ehluleka ngesikhathi esisodwa. Njengoba sesishilo kakade, i-RabbitMQ ithumela isiqinisekiso kumshicileli kuphela ngemva kokuba umlayezo ubhalwe ku-disk yinkosi nazo zonke izibuko. Kodwa lokhu kwengeza ukubambezeleka okwengeziwe ngenxa yezizathu ezimbili:

  • fsync njalo ngama-millisecond angamakhulu ambalwa
  • Ukwehluleka kwesibuko kungaqashelwa kuphela ngemva kokuba isikhathi sokuphila samaphakethe ahlola ukutholakala kwe-node ngayinye (ithikhi inetha) isiphelelwe yisikhathi. Uma isibuko sinciphisa noma siwa, lokhu kwengeza ukubambezeleka.

Ukubheja kuka-Kafka ukuthi uma umlayezo ugcinwe ezindaweni eziningi, ungavuma imilayezo ngokushesha nje lapho ufika kumemori. Ngenxa yalokhu, kunengozi yokulahlekelwa yimilayezo yanoma yiluphi uhlobo (ngisho imikhovu=konke, min.insync.replicas=2) uma kwenzeka ukwehluleka kanyekanye.

Sekukonke, i-Kafka ibonisa ukusebenza okungcono kwesofthiwe futhi yakhelwe kusukela phansi ukuya phezulu ukuze ibe ngamaqoqo. Inani labalandeli linganyuswa libe yi-11 uma kunesidingo ukuze kuthembeke. Isici esiphindaphindayo esingu-5 kanye nenani elincane lezifaniso ekuvumelaniseni min.insync.replicas=3 izokwenza ukulahlekelwa komlayezo kube isenzakalo esiyivelakancane kakhulu. Uma ingqalasizinda yakho ingakwazi ukusekela lesi silinganiso sokuphindaphinda kanye nezinga lokuphindaphinda, ungakhetha le nketho.

Ukuhlanganisa i-RabbitMQ kuhle kolayini abancane. Kodwa ngisho nolayini abancane bangakhula ngokushesha uma kunezimoto eziningi. Uma imigqa iba mikhulu, kuzodingeka wenze izinqumo ezinzima phakathi kokutholakala nokwethembeka. Iqoqo le-RabbitMQ lifaneleka kakhulu ezimweni ezingajwayelekile lapho izinzuzo zokuguquguquka kwe-RabbitMQ zidlula noma yikuphi ukonakala kokuhlanganisa kwayo.

Ikhambi elilodwa lokuba sengozini kwe-RabbitMQ kolayini abakhulu ukuwahlukanisa abe olayini abaningi abancane. Uma ungadingi ukuhleleka okuphelele kwawo wonke ulayini, kodwa imilayezo efanele kuphela (isibonelo, imilayezo evela kuklayenti elithile), noma unga-odi lutho nhlobo, khona-ke le nketho iyamukeleka: bheka iphrojekthi yami. I-Rebalancer ukuhlukanisa ulayini (iphrojekthi isesigabeni sokuqala).

Okokugcina, ungakhohlwa ngenani leziphazamisi kuzinqubo zokuhlanganisa nokuphindaphinda kwakho kokubili i-RabbitMQ ne-Kafka. Ngokuhamba kwesikhathi, amasistimu asekhulile futhi azinzile, kodwa awukho umlayezo oyoke uphephe ngo-100% ekulahlekeni! Ngaphezu kwalokho, izingozi ezinkulu zenzeka ezikhungweni zedatha!

Uma ngiphuthelwe okuthile, ngenza iphutha, noma ungavumelani nanoma yimaphi amaphuzu, zizwe ukhululekile ukubhala amazwana noma ungithinte.

Ngivame ukubuzwa: "Yini engingayikhetha, i-Kafka noma i-RabbitMQ?", "Iyiphi ipulatifomu engcono?". Iqiniso liwukuthi kuncike ngempela esimweni sakho, okuhlangenwe nakho kwamanje, njll. Ngiyangabaza ukunikeza umbono wami ngoba kungaba ukwenza lula kakhulu ukuncoma inkundla eyodwa yazo zonke izimo zokusebenzisa kanye nemikhawulo engenzeka. Ngibhale lolu chungechunge lwezihloko ukuze nizenzele owakho umbono.

Ngifuna ukusho ukuthi zombili lezi zinhlelo zingabaholi kule ndawo. Ngingase ngicheme kancane ngoba kokuhlangenwe nakho kwami ​​ngamaphrojekthi ngivame ukwazisa izinto ezifana noku-oda komlayezo okuqinisekisiwe nokuthembeka.

Ngibona obunye ubuchwepheshe obuntula lokhu kwethembeka noku-oda okuqinisekisiwe, bese ngibheka i-RabbitMQ ne-Kafka bese ngiqaphela inani elimangalisayo lazo zombili lezi zinhlelo.

Source: www.habr.com

Engeza amazwana