{"id":52541,"date":"2019-11-11T00:00:00","date_gmt":"2019-11-10T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost"},"modified":"2020-02-18T14:00:17","modified_gmt":"2020-02-18T11:00:17","slug":"rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost","title":{"rendered":"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/6557ce69c2b3070622feca8b3c228b09.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/itsumma\/blog\/471858\/\">artikullin e kaluar.<\/a><\/noindex> Ne kemi shqyrtuar klasterizimin RabbitMQ p\u00ebr t\u00eb siguruar q\u00ebndrueshm\u00ebrin\u00eb dhe disponueshm\u00ebrin\u00eb e lart\u00eb. Tani do t\u00eb thellohemi n\u00eb Apache Kafka.<\/p>\n<p>K\u00ebtu, nj\u00ebsi e replikimit \u00ebsht\u00eb ndarja (partition). \u00c7do tem\u00eb ka nj\u00eb ose m\u00eb shum\u00eb ndarje. N\u00eb \u00e7do ndarje ka nj\u00eb lider me ose pa ndjek\u00ebs. Kur krijohet nj\u00eb tem\u00eb, p\u00ebrcaktohet numri i ndarjeve dhe coefficienti i replikimit. Vlera e zakonshme \u00ebsht\u00eb 3, q\u00eb do t\u00eb thot\u00eb tri replika: nj\u00eb lider dhe dy ndjek\u00ebs.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/36deb6d506223c2ee2ec147d5c215789.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 1. Kat\u00ebr ndarjet jan\u00eb shp\u00ebrndar\u00eb midis tre broker\u00ebve<\/i><\/p>\n<p>T\u00eb gjitha k\u00ebrkesat p\u00ebr lexim dhe shkruajm\u00eb i drejtohen liderit. Ndjek\u00ebsit d\u00ebrgojn\u00eb rregullisht liderit k\u00ebrkesa p\u00ebr t\u00eb marr\u00eb mesazhet m\u00eb t\u00eb fundit. P\u00ebrdoruesit nuk i drejtohen ndjek\u00ebsve; k\u00ebta ekzistojn\u00eb vet\u00ebm p\u00ebr p\u00ebrbashk\u00ebsi dhe q\u00ebndrueshm\u00ebri.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/715c51d75cb45863cfad4b37e81b9a9a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h1>D\u00ebshtimi i ndarjes<\/h1>\n<p>\nKur nj\u00eb broker bie, shpesh ndodhin d\u00ebshtime t\u00eb lider\u00ebve t\u00eb disa ndarjeve. N\u00eb secil\u00ebn prej tyre, nj\u00eb ndjek\u00ebs nga nj\u00eb nyje tjet\u00ebr b\u00ebhet lider. N\u00eb t\u00eb v\u00ebrtet\u00eb, kjo nuk ndodh gjithmon\u00eb, pasi ndikon edhe faktori i sinkronizimit: a ka ndjek\u00ebs t\u00eb sinkronizuar, dhe n\u00ebse jo, a lejohet kalimi n\u00eb nj\u00eb replik\u00eb t\u00eb nesinkronizuar. Por p\u00ebr momentin, mos e kompliko.<\/p>\n<p>Nj\u00eb broker 3 del nga rrjeti \u2014 dhe p\u00ebr ndarjen 2, nj\u00eb lider i ri zgjidhet n\u00eb brokerin 2.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/b8cfce64cf6b848b0659de023f9d9b63.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 2. Brokeri 3 \u00ebsht\u00eb i vdekur, dhe ndjek\u00ebsi i tij n\u00eb brokerin 2 zgjidhet lider i ri p\u00ebr ndarjen 2<\/i><\/p>\n<p>Pastaj bie brokeri 1 dhe ndarja 1 gjithashtu humbet liderin e saj, roli i t\u00eb cilit kalon n\u00eb brokerin 2.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/9f5409c723da7ff059b93f9b23e7d5f6.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 3. Ka mbetur vet\u00ebm nj\u00eb broker. T\u00eb gjitha lider\u00ebt jan\u00eb n\u00eb nj\u00eb broker me zero mbivendosje<\/i><\/p>\n<p>Kur brokeri 1 kthehet n\u00eb rrjet, ai shton kat\u00ebr ndjek\u00ebs, duke siguruar nj\u00eb far\u00eb mbivendosjeje p\u00ebr \u00e7do ndarje. Por t\u00eb gjitha lider\u00ebt akoma q\u00ebndrojn\u00eb n\u00eb brokerin 2.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/f9c8e42f51258b7fd7f6f35dc7b8cc3b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 4. Lider\u00ebt mbeten n\u00eb brokerin 2<\/i><\/p>\n<p>Kur brokeri 3 ngrihet, ne kthehemi n\u00eb tre replika n\u00eb ndarje. Por t\u00eb gjitha lider\u00ebt akoma jan\u00eb n\u00eb brokerin 2.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/9c8c02be4161f8468b5dbc1e3f6ecb3a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 5. Vendosja e paekuilibruar e lider\u00ebve pas ristrukturimit t\u00eb broker\u00ebve 1 dhe 3<\/i><\/p>\n<p>Kafka ka nj\u00eb mjet p\u00ebr nj\u00eb ristrukturim m\u00eb t\u00eb mir\u00eb t\u00eb lider\u00ebve sesa RabbitMQ. Atje duhej t\u00eb p\u00ebrdorej nj\u00eb plugin ose skript i jasht\u00ebm, i cili ndryshonte politikat p\u00ebr migrimin e nyj\u00ebs kryesore duke ulur mbivendosjen gjat\u00eb migrimit. P\u00ebr m\u00eb tep\u00ebr, p\u00ebr kolonat m\u00eb t\u00eb m\u00ebdha, duhej t\u00eb pranohej pap\u00ebrshkrueshm\u00ebria gjat\u00eb sinkronizimit.<\/p>\n<p>Kafka ka konceptin e \"replikave t\u00eb preferuara\" p\u00ebr rolin e liderit. Kur krijohen ndarjet e tem\u00ebs, Kafka p\u00ebrpiqet t\u00eb shp\u00ebrndaj\u00eb lider\u00ebt baraz n\u00eb nyje dhe i sh\u00ebnon k\u00ebta lider\u00ebt e par\u00eb si t\u00eb preferuar. Me kalimin e koh\u00ebs, p\u00ebr shkak t\u00eb rifillimeve t\u00eb server\u00ebve, d\u00ebshtimeve dhe nd\u00ebrprerjeve t\u00eb lidhjeve, lider\u00ebt mund t\u00eb p\u00ebrfundojn\u00eb n\u00eb nyje t\u00eb tjera, si\u00e7 ndodhi n\u00eb rastin ekstrem t\u00eb p\u00ebrmendur m\u00eb sip\u00ebr.<\/p>\n<p>P\u00ebr t\u00eb rregulluar k\u00ebt\u00eb, Kafka ofron dy opsione:<\/p>\n<ul>\n<li>Opcioni <i>auto.leader.rebalance.enable=true<\/i> lejon q\u00eb nyja kontrolluese t\u00eb rinominoj\u00eb automatikisht lider\u00ebt mbrapsht n\u00eb replikat e preferuara dhe k\u00ebshtu t\u00eb rikthej\u00eb shp\u00ebrndarjen e barabart\u00eb.\n<\/li>\n<li>Administratori mund t\u00eb ekzekutoj\u00eb skriptin <i>kafka-preferred-replica-election.sh<\/i> p\u00ebr t\u00eb rinovuar manualisht.<\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/831930b06809637018fecb6d7d5a558d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 6. Replikat pas ristrukturimit<\/i><\/p>\n<p>Kjo ishte nj\u00eb version i thjeshtuar i d\u00ebshtimit, por realiteti \u00ebsht\u00eb m\u00eb i nd\u00ebrlikuar, edhe pse nuk ka asgj\u00eb t\u00eb komplikuar k\u00ebtu. E gjith\u00eb kjo lidhet me replikat e sinkronizuara (In-Sync Replicas, ISR).<\/p>\n<h1>Replikat e sinkronizuara (ISR)<\/h1>\n<p>\nISR \u00ebsht\u00eb nj\u00eb grup replikash t\u00eb ndarjes, i cili konsiderohet \"i sinkronizuar\" (in-sync). Ka nj\u00eb lider dhe ndjek\u00ebsit mund t\u00eb mos jen\u00eb t\u00eb pranish\u00ebm. Nj\u00eb ndjek\u00ebs konsiderohet i sinkronizuar n\u00ebse ka b\u00ebr\u00eb kopje t\u00eb sakta t\u00eb t\u00eb gjitha mesazheve t\u00eb liderit para se t\u00eb skadoj\u00eb interesi <i>replica.lag.time.max.ms<\/i>.<\/p>\n<p>Nj\u00eb ndjek\u00ebs largohet nga grupi ISR n\u00ebse ai:<\/p>\n<ul>\n<li>nuk ka k\u00ebrkuar p\u00ebrzgjedhjen p\u00ebr brenda intervalit <i>replica.lag.time.max.ms<\/i> (konsiderohet i vdekur)\n<\/li>\n<li>nuk arriti t\u00eb p\u00ebrdit\u00ebsohet n\u00eb interval <i>replica.lag.time.max.ms<\/i> (konsiderohet i ngadalsh\u00ebm)<\/li>\n<\/ul>\n<p>\nNdjek\u00ebsit b\u00ebjn\u00eb k\u00ebrkesa p\u00ebrzgjedhjeje brenda intervalit <i>replica.fetch.wait.max.ms<\/i>, i cili p\u00ebrshekull \u00ebsht\u00eb 500 ms.<\/p>\n<p>P\u00ebr t\u00eb shpjeguar qart\u00ebsisht q\u00ebllimin e ISR, duhet t\u00eb shqyrtojm\u00eb konfirmimet nga prodhuesi dhe disa skenar\u00eb d\u00ebshtimi. Prodhuesit mund t\u00eb zgjedhin se kur brokeri d\u00ebrgon konfirmim:<\/p>\n<ul>\n<li>acks=0, nuk d\u00ebrgohet konfirmim\n<\/li>\n<li>acks=1, konfirmimi d\u00ebrgohet pasi lideri ka regjistruar mesazhin n\u00eb logun e tij lokal\n<\/li>\n<li>acks=all, konfirmimi d\u00ebrgohet pasi t\u00eb gjitha replikat n\u00eb ISR regjistrojn\u00eb mesazhin n\u00eb log\u00ebt e tyre lokale<\/li>\n<\/ul>\n<p>\nN\u00eb terminologjin\u00eb e Kafka-s, n\u00ebse ISR ruan nj\u00eb mesazh, ndodh \"komitimi\" i tij. Acks=all \u00ebsht\u00eb opsioni m\u00eb i sigurt, por sjell gjithashtu nj\u00eb vones\u00eb shtes\u00eb. Le t\u00eb shqyrtojm\u00eb dy shembuj d\u00ebshtimi dhe si opsionet e ndryshme t\u00eb 'acks' nd\u00ebrveprojn\u00eb me konceptin e ISR.<\/p>\n<h3>Acks=1 dhe ISR<\/h3>\n<p>\nN\u00eb k\u00ebt\u00eb shembull, ne do t\u00eb shohim se n\u00ebse lideri nuk pret t\u00eb ruaj\u00eb \u00e7do mesazh nga t\u00eb gjith\u00eb ndjek\u00ebsit, at\u00ebher\u00eb n\u00eb rast t\u00eb d\u00ebshtimit t\u00eb liderit, mund t\u00eb ndodhi humbja e t\u00eb dh\u00ebnave. Kalimi te ndjek\u00ebsi jo t\u00eb sinkronizuar mund t\u00eb lejohet ose ndalohet nga konfigurimi. <i>unclean.leader.election.enable<\/i>.<\/p>\n<p>N\u00eb k\u00ebt\u00eb shembull, prodhuesi ka vendosur vler\u00ebn acks=1. N\u00ebnshkrimi \u00ebsht\u00eb shp\u00ebrndar\u00eb n\u00eb t\u00eb tre broker\u00ebt. Brokeri 3 \u00ebsht\u00eb duke pritur, ai \u00ebsht\u00eb sinkronizuar me liderin tet\u00eb sekonda m\u00eb par\u00eb dhe tani \u00ebsht\u00eb prapa me 7456 mesazhe. Brokeri 1 \u00ebsht\u00eb prapa vet\u00ebm nj\u00eb sekond\u00eb. Prodhuesi yn\u00eb d\u00ebrgon nj\u00eb mesazh dhe merr shpejt nj\u00eb konfirmim, pa ngarkes\u00eb p\u00ebr ndjek\u00ebsit e ngadalsh\u00ebm ose t\u00eb vdekur, t\u00eb cil\u00ebt lideri nuk i pret.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/f194e6309b732b772574392eff482a8b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 7. ISR me tre replica<\/i><\/p>\n<p>Brokeri 2 d\u00ebshtoi, dhe prodhuesi merr nj\u00eb gabim lidhjeje. Pas kalimit t\u00eb liderit te brokeri 1, ne humbim 123 mesazhe. Ndjek\u00ebsi n\u00eb brokerin 1 ishte n\u00eb ISR, por nuk ishte sinkronizuar plot\u00ebsisht me liderin kur ai ra.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/367f219daebafefe585059f67fdedf6f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 8. Mesazhet humbasin kur ndodhi d\u00ebshtimi<\/i><\/p>\n<p>N\u00eb konfiguracionin <i>bootstrap.servers<\/i> prodhuesi rendit disa broker\u00eb dhe ai mund t\u00eb pyes\u00eb nj\u00eb broker tjet\u00ebr se kush e ka marr\u00eb rolin e liderit t\u00eb n\u00ebnshkrimit. Pastaj ai vendos nj\u00eb lidhje me brokerin 1 dhe vazhdon t\u00eb d\u00ebrgoj\u00eb mesazhe.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/ba45c51f4ce1f63f10b2fa0e1487b223.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 9. D\u00ebrgimi i mesazheve rinis pas nj\u00eb pauze t\u00eb shkurt\u00ebr<\/i><\/p>\n<p>Brokeri 3 \u00ebsht\u00eb prapa edhe m\u00eb shum\u00eb. Ai b\u00ebn k\u00ebrkesa p\u00ebr t\u00eb t\u00ebrhequr mesazhe, por nuk mund t\u00eb sinkronizohet. Kjo mund t\u00eb jet\u00eb p\u00ebr shkak t\u00eb nj\u00eb lidhjeje rrjeti t\u00eb ngadalshme midis broker\u00ebve, nj\u00eb problem ruajtjeje, etj. Ai \u00ebsht\u00eb larguar nga ISR. Tani ISR p\u00ebrb\u00ebhet nga nj\u00eb replike \u2014 lideri! Prodhuesi vazhdon t\u00eb d\u00ebrgoj\u00eb mesazhe dhe merr konfirmime.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/592c74632e3a8d80c2b4fc15bde5d401.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 10. Ndjek\u00ebsi n\u00eb brokerin 3 largohet nga ISR<\/i><\/p>\n<p>Brokeri 1 bie, dhe roli i liderit kalon te brokeri 3 me humbje 15286 mesazhesh! Prodhuesi merr nj\u00eb mesazh gabimi lidhjeje. Kalimi te lideri jasht\u00eb ISR ishte i mundur vet\u00ebm p\u00ebr shkak t\u00eb konfigurimit <i>unclean.leader.election.enable=true<\/i>. N\u00ebse \u00ebsht\u00eb vendosur n\u00eb <i>false<\/i>, at\u00ebher\u00eb kalimi nuk do t\u00eb ndodhte, dhe t\u00eb gjitha k\u00ebrkesat p\u00ebr lexim dhe shkruaj do t\u00eb refuzoheshin. N\u00eb k\u00ebt\u00eb rast, ne presim rikthimin e brokerit 1 me t\u00eb dh\u00ebnat e tij t\u00eb paprekura n\u00eb repliken q\u00eb do t\u00eb marr\u00eb s\u00ebrish rolin e liderit.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/30f01ca715ccc194890ae061a3ef7396.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 11. Brokeri 1 bie. Gjat\u00eb d\u00ebshtimit humbet nj\u00eb num\u00ebr t\u00eb madh mesazhesh<\/i><\/p>\n<p>Prodhuesi vendos lidhje me brokerin e fundit dhe sheh se ai tani \u00ebsht\u00eb lider i n\u00ebnshkrimit. Ai fillon t\u00eb d\u00ebrgoj\u00eb mesazhe n\u00eb brokerin 3.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/986532d9b22b6514b6039c8460b1beb4.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 12. Pas nj\u00eb pauze t\u00eb shkurt\u00ebr, d\u00ebrgimi i mesazheve rinis n\u00eb n\u00ebnshkrimin 0<\/i><\/p>\n<p>Kemi par\u00eb se p\u00ebrve\u00e7 ndalimeve t\u00eb shkurtra p\u00ebr t\u00eb vendosur lidhje t\u00eb reja dhe p\u00ebr t\u00eb gjetur nj\u00eb lider t\u00eb ri, prodhuesi vazhdimisht d\u00ebrgonte mesazhe. Nj\u00eb konfigurim i till\u00eb siguron disponueshm\u00ebri p\u00ebrmes konsistenc\u00ebs (siguris\u00eb s\u00eb t\u00eb dh\u00ebnave). Kafka humbi mij\u00ebra mesazhe, por vazhdoi t\u00eb pranoj\u00eb regjistrime t\u00eb reja.<\/p>\n<h3>Acks=all dhe ISR<\/h3>\n<p>\nLe t\u00eb p\u00ebrs\u00ebrisim k\u00ebt\u00eb skenar nj\u00eb her\u00eb tjet\u00ebr, por me <i>acks=all<\/i>. Vonesa e brokerit 3 mesatarisht kat\u00ebr sekonda. Prodhuesi d\u00ebrgon nj\u00eb mesazh me <i>acks=all<\/i>, dhe tani nuk merr nj\u00eb p\u00ebrgjigje t\u00eb shpejt\u00eb. Lideri pret derisa mesazhi t\u00eb ruhet nga t\u00eb gjith\u00eb replikat n\u00eb ISR.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/95b1abdc92e699f7bc41ebab249a2e08.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 13. ISR me tre replica. Nj\u00eb punon ngadal\u00eb, duke shkaktuar vonesa n\u00eb shkruarje<\/i><\/p>\n<p>Pas kat\u00ebr sekondash vones\u00eb shtes\u00eb, brokeri 2 d\u00ebrgon nj\u00eb konfirmim. T\u00eb gjitha replikat tani jan\u00eb plot\u00ebsisht t\u00eb p\u00ebrdit\u00ebsuara.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/64356dc8d2641a3e22947126dd047f39.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 14. T\u00eb gjitha replikat ruajn\u00eb mesazhet dhe d\u00ebrgohet konfirmimi<\/i><\/p>\n<p>Brokeri 3 tani \u00ebsht\u00eb prapa edhe m\u00eb shum\u00eb dhe largohet nga ISR. Vonesa zvog\u00eblohet ndjesh\u00ebm, pasi nuk ka mbetur replik\u00eb t\u00eb ngadalshme n\u00eb ISR. Brokeri 2 tani pret vet\u00ebm brokerin 1, i cili ka nj\u00eb vones\u00eb mesatare prej 500 ms.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/5d3bd67dc53cb868fa40b358ed483a1d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 15. Replica n\u00eb brokerin 3 largohet nga ISR<\/i><\/p>\n<p>Pastaj bie brokeri 2, dhe lideri kalon te brokeri 1 pa humbje mesazhesh.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/e3f0417aa7714ea5eae8a8e8d61662bd.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 16. Brokeri 2 bie<\/i><\/p>\n<p>Prodhuesi gjen nj\u00eb lider t\u00eb ri dhe fillon t\u2019i d\u00ebrgoj\u00eb atij mesazhe. Vonesa vazhdon t\u00eb zvog\u00eblohet, sepse tani ISR p\u00ebrb\u00ebhet vet\u00ebm nga nj\u00eb replike! Prandaj, opsioni <i>acks=all<\/i> nuk shton mbivendosje.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/fec9aae9977757d5f61c2909ceae0536.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 17. Replica n\u00eb brokerin 1 merr rolin e liderit pa humbje mesazhesh<\/i><\/p>\n<p>Pastaj bie brokeri 1, dhe roli i liderit kalon te brokeri 3 me humbje 14238 mesazhesh!<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/730b3951fb75ab3a782fc6d064615212.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 18. Brokeri 1 vdes, dhe kalimi i liderit me konfigurimin e papast\u00ebrtis\u00eb \u00e7on n\u00eb humbje t\u00eb m\u00ebdha t\u00eb t\u00eb dh\u00ebnave<\/i><\/p>\n<p>Ne mund t\u00eb mos vendosim opsionin <i>unclean.leader.election.enable<\/i> n\u00eb vler\u00eb <i>e v\u00ebrtet\u00eb<\/i>. M\u00ebnyra e paracaktuar \u00ebsht\u00eb <i>false<\/i>. Konfigurimi <i>acks=all<\/i> me <i>unclean.leader.election.enable=true<\/i> siguron disponueshm\u00ebri me disa siguri shtes\u00eb p\u00ebr t\u00eb dh\u00ebnat. Por, si\u00e7 e shihni, ne ende mund t\u00eb humbim mesazhe.<\/p>\n<p>Por \u00e7far\u00eb n\u00ebse duam t\u00eb rrisim sigurin\u00eb e t\u00eb dh\u00ebnave? Mund t\u00eb vendosim <i>unclean.leader.election.enable = false<\/i>, por kjo nuk do t\u00eb na mbroj\u00eb domosdoshm\u00ebrisht nga humbja e t\u00eb dh\u00ebnave. N\u00ebse lideri ra r\u00ebnd\u00eb dhe e mori me vete t\u00eb dh\u00ebnat, at\u00ebher\u00eb mesazhet p\u00ebrs\u00ebri humbasin, plus humbet disponueshm\u00ebria, derisa administratori t\u00eb rikthej\u00eb situat\u00ebn.<\/p>\n<p>M\u00eb mir\u00eb t\u00eb garantoni redundanc\u00ebn e t\u00eb gjitha mesazheve, n\u00eb t\u00eb kund\u00ebrt do t\u00eb hiqni dor\u00eb nga regjistrimi. K\u00ebshtu, nga pik\u00ebpamja e brokerit, humbja e t\u00eb dh\u00ebnave \u00ebsht\u00eb e mundur vet\u00ebm n\u00eb rast t\u00eb dy ose m\u00eb shum\u00eb d\u00ebshtimeve t\u00eb nj\u00ebkohshme.<\/p>\n<h3>Acks=all, min.insync.replicas dhe ISR<\/h3>\n<p>\nMe konfigurimin e tem\u00ebs <i>min.insync.replicas<\/i> ne rrisim nivelin e siguris\u00eb s\u00eb t\u00eb dh\u00ebnave. Le t\u00eb kalojm\u00eb s\u00ebrish n\u00eb pjes\u00ebn e fundit t\u00eb skenarit t\u00eb kaluar, por k\u00ebt\u00eb her\u00eb me <i>min.insync.replicas=2<\/i>.<\/p>\n<p>Pra, brokeri 2 ka liderin e replikuar, nd\u00ebrsa ndjek\u00ebsi n\u00eb brokerin 3 \u00ebsht\u00eb hequr nga ISR.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/5831463293f1837d3232756894e97162.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 19. ISR nga dy replika<\/i><\/p>\n<p>Brokeri 2 bie, dhe udh\u00ebheqja kalon n\u00eb brokerin 1 pa humbje mesazhesh. Por tani ISR p\u00ebrb\u00ebhet vet\u00ebm nga nj\u00eb replik\u00eb. Kjo nuk i p\u00ebrmbush numrin minimal p\u00ebr regjistrim, dhe prandaj brokeri p\u00ebrgjigjet me nj\u00eb gabim n\u00eb p\u00ebrpjekjen p\u00ebr regjistrim. <i>NotEnoughReplicas<\/i>.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/6b2ee477f4c33ec84f3068b792814e5d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 20. Numri i ISR \u00ebsht\u00eb nj\u00eb m\u00eb i ul\u00ebt se sa e p\u00ebrcaktuar n\u00eb min.insync.replicas<\/i><\/p>\n<p>Kjo konfigurim i sakrifikon disponueshm\u00ebrin\u00eb p\u00ebr konsistenc\u00eb. Para se t\u00eb konfirmoni mesazhin, ne garanti q\u00eb ajo regjistrohet n\u00eb t\u00eb pakt\u00ebn dy replika. Kjo i jep prodhuesit shum\u00eb m\u00eb shum\u00eb siguri. K\u00ebtu humbja e mesazheve \u00ebsht\u00eb e mundur vet\u00ebm n\u00eb rast t\u00eb d\u00ebshtimit t\u00eb dy replika n\u00eb nj\u00eb periudh\u00eb t\u00eb shkurt\u00ebr, derisa mesazhi t\u00eb miratohet te nj\u00eb ndjek\u00ebs tjet\u00ebr, q\u00eb \u00ebsht\u00eb e pamundur. Por n\u00ebse je shum\u00eb paranojak, mund t\u00eb vendos\u00ebsh nj\u00eb faktor replikimi 5, dhe <i>min.insync.replicas<\/i> n\u00eb 3. K\u00ebshtu k\u00ebtu duhet t\u00eb bien tre broker\u00eb nj\u00ebkoh\u00ebsisht p\u00ebr t\u00eb humbur regjistrimin! Sigurisht, p\u00ebr nj\u00eb besueshm\u00ebri t\u00eb till\u00eb do t\u00eb paguani me vones\u00eb t\u00eb shtuar.<\/p>\n<h1>Kur disponueshm\u00ebria \u00ebsht\u00eb e nevojshme p\u00ebr sigurin\u00eb e t\u00eb dh\u00ebnave<\/h1>\n<p>\nAs in <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/itsumma\/blog\/471858\/\">n\u00eb rastin e RabbitMQ<\/a><\/noindex>, ndonj\u00ebher\u00eb disponueshm\u00ebria \u00ebsht\u00eb e nevojshme p\u00ebr sigurin\u00eb e t\u00eb dh\u00ebnave. Ju duhet t\u00eb mendoni p\u00ebr k\u00ebt\u00eb:<\/p>\n<ul>\n<li>A mund t\u00eb kthej\u00eb vet\u00ebm publiku nj\u00eb gabim, dhe sh\u00ebrbimi i lart\u00eb ose p\u00ebrdoruesi ta provoni p\u00ebrs\u00ebri m\u00eb von\u00eb?\n<\/li>\n<li>A mund t\u00eb ruaj\u00eb botuesi mesazhin lokalisht ose n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave p\u00ebr ta provuar m\u00eb von\u00eb?<\/li>\n<\/ul>\n<p>\nN\u00ebse p\u00ebrgjigjja \u00ebsht\u00eb negative, at\u00ebher\u00eb optimizimi i disponueshm\u00ebris\u00eb rrit sigurin\u00eb e t\u00eb dh\u00ebnave. Do t\u00eb humbni m\u00eb pak t\u00eb dh\u00ebna n\u00ebse zgjidhni disponueshm\u00ebri mbi refuzimin e regjistrimit. Pra, gjith\u00e7ka varet nga gjetja e nj\u00eb balancimi, dhe zgjidhja varet nga situata specifike.<\/p>\n<h1>Kuptimi i ISR<\/h1>\n<p>\nGrupi i ISR lejon t\u00eb zgjidhni balancimin optimal midis siguris\u00eb s\u00eb t\u00eb dh\u00ebnave dhe vones\u00ebs. P\u00ebr shembull, sigurojn\u00eb disponueshm\u00ebri n\u00eb kushte t\u00eb d\u00ebshtimit t\u00eb shumic\u00ebs s\u00eb replika, duke minimalizuar ndikimin e replika t\u00eb vdekura ose t\u00eb ngadalta n\u00eb aspektin e vones\u00ebs.<\/p>\n<p>Ne vet\u00eb zgjedhim vler\u00ebn <i>replica.lag.time.max.ms<\/i> sipas nevojave tona. N\u00eb thelb, ky paramet\u00ebr tregon se sa vones\u00eb jemi t\u00eb gatsh\u00ebm t\u00eb pranojm\u00eb n\u00eb <i>acks=all<\/i>. Vlera e paracaktuar \u00ebsht\u00eb dhjet\u00eb sekonda. N\u00ebse p\u00ebr ju kjo \u00ebsht\u00eb shum\u00eb e gjat\u00eb, mund ta reduktoni. K\u00ebshtu, do t\u00eb rritet frekuenca e ndryshimeve n\u00eb ISR, pasi ndjek\u00ebsit do t\u00eb hiqen dhe shtohen m\u00eb shpesh.<\/p>\n<p>N\u00eb RabbitMQ ka thjesht nj\u00eb grup pasqyrash q\u00eb duhet t\u00eb replikohen. Pasqyrat e ngadalta sjellin vones\u00eb shtes\u00eb, dhe p\u00ebr p\u00ebrgjigjet e pasqyrave t\u00eb vdekura mund t\u00eb pritet deri n\u00eb skadimin e koh\u00ebs s\u00eb paketave q\u00eb kontrollojn\u00eb disponueshm\u00ebrin\u00eb e \u00e7do nodi (net tick). ISR \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb interesante p\u00ebr t\u00eb shmangur k\u00ebto probleme me rritjen e vones\u00ebs. Por rrezikojm\u00eb t\u00eb humbasim redundanc\u00ebn, pasi ISR mund t\u00eb zvog\u00eblohet vet\u00ebm deri te lideri. P\u00ebr t\u00eb shmangur k\u00ebt\u00eb rrezik, p\u00ebrdorni konfigurimin <i>min.insync.replicas<\/i>.<\/p>\n<h1>Garancia e lidhjes s\u00eb klient\u00ebve<\/h1>\n<p>\nN\u00eb cil\u00ebsimet <i>bootstrap.servers<\/i> prodhuesit dhe konsumatorit mund t\u00eb specifikojn\u00eb disa broker\u00eb p\u00ebr t'u lidhur me klient\u00ebt. Ideja \u00ebsht\u00eb q\u00eb n\u00eb rast t\u00eb ndarjes s\u00eb nj\u00eb nodi, t\u00eb mbeten disa rezerv\u00eb me t\u00eb cilat klienti mund t\u00eb hap\u00eb nj\u00eb lidhje. Kjo nuk \u00ebsht\u00eb domosdoshm\u00ebrisht lider\u00ebt e ndarjeve, por thjesht nj\u00eb baz\u00eb p\u00ebr ngarkim fillestar. Klienti mund t'i pyes\u00eb ata se n\u00eb cilin nod \u00ebsht\u00eb vendosur lideri i ndarjes p\u00ebr lexim\/shkrim.<\/p>\n<p>N\u00eb RabbitMQ klient\u00ebt mund t\u00eb lidhen me \u00e7do nod, dhe routing i brendsh\u00ebm d\u00ebrgon k\u00ebrkes\u00ebn aty ku duhet. Kjo do t\u00eb thot\u00eb se mund t\u00eb vendosni nj\u00eb balancues ngarkese para RabbitMQ. Kafka k\u00ebrkon q\u00eb klient\u00ebt t\u00eb lidhen me nodin ku ndodhet lideri i ndarjes p\u00ebrkat\u00ebse. N\u00eb nj\u00eb situat\u00eb t\u00eb till\u00eb, nuk \u00ebsht\u00eb e mundur t\u00eb vendosni nj\u00eb balancues ngarkese. Lista <i>bootstrap.servers<\/i> \u00ebsht\u00eb kritike q\u00eb klient\u00ebt t\u00eb mund t'i drejtohen nod\u00ebve t\u00eb nevojsh\u00ebm dhe t'i gjejn\u00eb ato pas d\u00ebshtimit.<\/p>\n<h1>Arkitektura e konsensusit t\u00eb Kafka<\/h1>\n<p>\nDerisa ne nuk kemi shqyrtuar ende se si klasteri e merr vesh p\u00ebr r\u00ebnien e brokerit dhe si zgjidhet nj\u00eb lider i ri. P\u00ebr t\u00eb kuptuar se si Kafka funksionon me ndarjet n\u00eb rrjet, \u00ebsht\u00eb e nevojshme t\u00eb kuptoni fillimisht arkitektur\u00ebn e konsensusit.<\/p>\n<p>\u00c7do klaster i Kafka sh\u00ebrbehet s\u00eb bashku me nj\u00eb klaster Zookeeper \u2014 kjo \u00ebsht\u00eb nj\u00eb sh\u00ebrbim i konsensusit t\u00eb shp\u00ebrndar\u00eb q\u00eb lejon sistemin t\u00eb arrij\u00eb konsensus mbi nj\u00eb gjendje t\u00eb caktuar me prioritet p\u00ebr konsistenc\u00ebn mbi disponueshm\u00ebrin\u00eb. P\u00ebr miratimin e operacioneve t\u00eb leximit dhe shkrimit k\u00ebrkohet dakord\u00ebsia e shumic\u00ebs s\u00eb nod\u00ebve t\u00eb Zookeeper.<\/p>\n<p>Zookeeper ruan gjendjen e klasterit:<\/p>\n<ul>\n<li>Lista e temave, seksioneve, konfigurimi, replikat aktuale t\u00eb liderit, replikat e preferuara.\n<\/li>\n<li>An\u00ebtar\u00ebt e klasterit. \u00c7do broker pingon n\u00eb klasterin Zookeeper. N\u00ebse ai nuk merr ping p\u00ebr nj\u00eb periudh\u00eb t\u00eb caktuar kohe, Zookeeper e regjistron brokerin si t\u00eb paaksesuesh\u00ebm.\n<\/li>\n<li>Zgjedhja e nyjeve kryesore dhe rezerv\u00eb p\u00ebr kontrolluesin.<\/li>\n<\/ul>\n<p>\nNyja e kontrolluesit \u2014 nj\u00eb nga broker\u00ebt Kafka, i cili \u00ebsht\u00eb p\u00ebrgjegj\u00ebs p\u00ebr zgjedhjen e lider\u00ebve t\u00eb replikave. Zookeeper i d\u00ebrgon kontrolluesit njoftime mbi an\u00ebtar\u00ebsimin n\u00eb klaster dhe ndryshimet n\u00eb tem\u00eb, dhe kontrolluesi duhet t\u00eb veproj\u00eb n\u00eb p\u00ebrputhje me k\u00ebto ndryshime.<\/p>\n<p>P\u00ebr shembull, le t\u00eb marrim nj\u00eb tem\u00eb t\u00eb re me dhjet\u00eb seksione dhe faktor riplikimi 3. Kontrolluesi duhet t\u00eb zgjedh\u00eb liderin e \u00e7do seksioni, duke u p\u00ebrpjekur t\u00eb shp\u00ebrndaj\u00eb optimalisht lider\u00ebt midis broker\u00ebve. <\/p>\n<p>P\u00ebr \u00e7do seksion kontrolluesi:<\/p>\n<ul>\n<li>p\u00ebrdit\u00ebson informacionin n\u00eb Zookeeper mbi ISR dhe liderin;\n<\/li>\n<li>d\u00ebrgon komand\u00ebn LeaderAndISRCommand p\u00ebr \u00e7do broker q\u00eb vendos replik\u00ebn e k\u00ebtij seksioni, duke i informuar broker\u00ebt mbi ISR dhe liderin.<\/li>\n<\/ul>\n<p>\nKur bie nj\u00eb broker me liderin, Zookeeper i d\u00ebrgon nj\u00eb njoftim kontrolluesit, dhe ai zgjedh nj\u00eb lider t\u00eb ri. P\u00ebrs\u00ebri, kontrolluesi s\u00eb pari p\u00ebrdit\u00ebson Zookeeper, dhe pastaj d\u00ebrgon komand\u00ebn p\u00ebr \u00e7do broker, duke i njoftuar ata p\u00ebr ndryshimin e liderit.<\/p>\n<p>\u00c7do lider \u00ebsht\u00eb p\u00ebrgjegj\u00ebs p\u00ebr grupin ISR. Konfigurimi <i>replica.lag.time.max.ms<\/i> p\u00ebrcakton se kush do t\u00eb hyj\u00eb aty. Kur ndryshon ISR, lideri e kalon informacionin e ri n\u00eb Zookeeper.<\/p>\n<p>Zookeeper \u00ebsht\u00eb gjithmon\u00eb n\u00eb dijeni t\u00eb \u00e7do ndryshimi, q\u00eb n\u00eb rast d\u00ebshtimi, udh\u00ebheqja t\u00eb kaloj\u00eb pa probleme te nj\u00eb lider i ri.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/3adb1d204b28e3ed85bef3530fa2b738.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 21. Konsensusi Kafka<\/i><\/p>\n<h1>Protokolli i riplikimit<\/h1>\n<p>\nT\u00eb kuptuarit e detajeve t\u00eb riplikimit ndihmon n\u00eb kuptimin m\u00eb t\u00eb mir\u00eb t\u00eb skenar\u00ebve t\u00eb mundsh\u00ebm t\u00eb humbjes s\u00eb t\u00eb dh\u00ebnave.<\/p>\n<h3>K\u00ebrkesat p\u00ebr t\u00ebrheqje, Log End Offset (LEO) dhe Highwater Mark (HW)<\/h3>\n<p>\nNe kemi shqyrtuar se ndjek\u00ebsit d\u00ebrgojn\u00eb her\u00eb pas here k\u00ebrkesa p\u00ebr t\u00ebrheqje (fetch) liderit. Intervali i paracaktuar \u00ebsht\u00eb 500 ms. Kjo \u00ebsht\u00eb e ndryshme nga RabbitMQ, ku riplikimi niset jo nga pasqyra e radh\u00ebs, por nga masteri. Masteri d\u00ebrgon ndryshimet te pasqyra.<\/p>\n<p>Lideri dhe t\u00eb gjith\u00eb ndjek\u00ebsit ruajn\u00eb Offset-in e fundit t\u00eb logut (Log End Offset, LEO) dhe etiket\u00ebn Highwater (HW). Etiketa LEO ruan offset-in e mesazhit t\u00eb fundit n\u00eb replik\u00ebn lokale, ndersa HW \u00ebsht\u00eb offset i komitimit t\u00eb fundit. Mbani mend se p\u00ebr statusin \"komit\" mesazhi duhet t\u00eb ruhet n\u00eb t\u00eb gjitha replikat ISR. Kjo do t\u00eb thot\u00eb se LEO zakonisht \u00ebsht\u00eb pak p\u00ebrpara HW.<\/p>\n<p>Kur lideri merr nj\u00eb mesazh, ai e ruan at\u00eb lokal. Ndjek\u00ebsi b\u00ebn nj\u00eb k\u00ebrkes\u00eb p\u00ebr t\u00ebrheqje, duke dor\u00ebzuar LEO-n\u00eb e tij. Pastaj lideri d\u00ebrgon nj\u00eb paket\u00eb mesazhesh, duke filluar nga ky LEO, si dhe e transmeton HW-n\u00eb aktuale. Kur lideri merr informacionin se t\u00eb gjitha replikat kan\u00eb ruajtur mesazhin me offset-in e caktuar, ai e l\u00ebviz etiket\u00ebn HW. Vet\u00ebm lideri mund t\u00eb l\u00ebviz\u00eb HW-n\u00eb, dhe k\u00ebshtu t\u00eb gjith\u00eb ndjek\u00ebsit m\u00ebsojn\u00eb vler\u00ebn aktuale n\u00eb p\u00ebrgjigje t\u00eb k\u00ebrkes\u00ebs s\u00eb tyre. Kjo do t\u00eb thot\u00eb se ndjek\u00ebsit mund t\u00eb jen\u00eb pas liderit, si n\u00eb mesazhe ashtu edhe n\u00eb njohjen e HW-s\u00eb. Konsumator\u00ebt marrin mesazhe vet\u00ebm deri n\u00eb HW-n\u00eb aktuale.<\/p>\n<p>Kujdes, \"i ruajtur\" (persisted) do t\u00eb thot\u00eb i shkruar n\u00eb memorje, jo n\u00eb disk. P\u00ebr performanc\u00eb, Kafka b\u00ebn sinkronizimin n\u00eb disk me nj\u00eb interval t\u00eb caktuar. Kjo ndodhet gjithashtu n\u00eb RabbitMQ, por do t\u00eb d\u00ebrgoj\u00eb nj\u00eb konfirmim p\u00ebr publikuesin vet\u00ebm pasi masteri dhe t\u00eb gjitha pasqyrat t\u00eb ken\u00eb regjistruar mesazhin n\u00eb disk. Zhvilluesit e Kafka, p\u00ebr arsye t\u00eb performanc\u00ebs, kan\u00eb vendosur t\u00eb d\u00ebrgojn\u00eb ack, sa her\u00eb q\u00eb mesazhi \u00ebsht\u00eb regjistruar n\u00eb memorje. Kafka beson se tep\u00ebrtia kompenson rrezikun e ruajtjes afatshkurt\u00ebr t\u00eb mesazheve t\u00eb konfirmuara vet\u00ebm n\u00eb memorje.<\/p>\n<h1>D\u00ebshtimi i liderit<\/h1>\n<p>\nKur bie lideri, Zookeeper e njofton kontrolluesin, dhe ai zgjedh nj\u00eb replik\u00eb t\u00eb re lideri. Lideri i ri vendos nj\u00eb etiket\u00eb t\u00eb re HW n\u00eb p\u00ebrputhje me LEO-n\u00eb e tij. Pastaj ndjek\u00ebsit marrin informacionin p\u00ebr liderin e ri. N\u00eb var\u00ebsi t\u00eb versionit t\u00eb Kafka, ndjek\u00ebsi do t\u00eb zgjedh\u00eb nj\u00eb nga dy skenar\u00ebt:<\/p>\n<ol>\n<li>Do t\u00eb shkurtoj\u00eb logun lokal n\u00eb nj\u00eb HW t\u00eb njohur dhe do t'i d\u00ebrgoj\u00eb liderit nj\u00eb k\u00ebrkes\u00eb p\u00ebr mesazhe pas k\u00ebsaj etikete.\n<\/li>\n<li>Do t\u00eb d\u00ebrgoj\u00eb nj\u00eb k\u00ebrkes\u00eb p\u00ebr liderin p\u00ebr t\u00eb m\u00ebsuar HW-n\u00eb n\u00eb momentin e zgjedhjes s\u00eb tij si lider, dhe pastaj do t\u00eb shkurt\u00ebzoj\u00eb logun deri n\u00eb k\u00ebt\u00eb offset. Pastaj do t\u00eb filloj\u00eb t\u00eb d\u00ebrgoj\u00eb k\u00ebrkesa t\u00eb rregullta p\u00ebr t\u00ebrheqje, duke filluar nga ky offset.<\/li>\n<\/ol>\n<p>\nNdjek\u00ebsit mund t\u00eb ken\u00eb nevoj\u00eb t\u00eb shkurt\u00ebzojn\u00eb logun p\u00ebr arsyet e m\u00ebposhtme:<\/p>\n<ul>\n<li>Kur ndodh nj\u00eb d\u00ebshtim lideri, ndihm\u00ebsi i par\u00eb nga grupi ISR, i regjistruar n\u00eb Zookeeper, fiton zgjedhjet dhe b\u00ebhet lider. T\u00eb gjith\u00eb ndihm\u00ebsit n\u00eb ISR, megjith\u00ebse konsiderohen \"t\u00eb sinkronizuar\", mund edhe t\u00eb mos ken\u00eb marr\u00eb kopje t\u00eb t\u00eb gjitha mesazheve nga lideri i m\u00ebparsh\u00ebm. \u00cbsht\u00eb mjaft e mundur q\u00eb ndihm\u00ebsi i zgjedhur t\u00eb mos ket\u00eb kopjen m\u00eb t\u00eb p\u00ebrdit\u00ebsuar. Kafka garanton q\u00eb nuk ka shk\u00ebputje midis kopjeve. K\u00ebshtu, p\u00ebr t\u00eb shmangur shk\u00ebputjen, \u00e7do ndihm\u00ebs duhet t\u00eb shkurtoj\u00eb logun e tij deri n\u00eb vler\u00ebn HW t\u00eb liderit t\u00eb ri n\u00eb momentin e zgjedhjes s\u00eb tij. Kjo \u00ebsht\u00eb nj\u00eb arsye tjet\u00ebr pse konfigurimi <i>acks=all<\/i> \u00ebsht\u00eb kaq i r\u00ebnd\u00ebsish\u00ebm p\u00ebr koherenc\u00ebn.\n<\/li>\n<li>Mesazhet regjistrohen periodikisht n\u00eb disk. N\u00ebse t\u00eb gjith\u00eb nodet e klasterit d\u00ebshtojn\u00eb nj\u00ebkoh\u00ebsisht, at\u00ebher\u00eb n\u00eb disqe do t\u00eb ruhen kopje me offset t\u00eb ndrysh\u00ebm. \u00cbsht\u00eb mjaft e mundur q\u00eb kur broker\u00ebt t\u00eb rikthehen n\u00eb rrjet, lideri i ri q\u00eb do t\u00eb zgjidhet t\u00eb jet\u00eb pas ndihm\u00ebsve t\u00eb tij, sepse \u00ebsht\u00eb ruajtur n\u00eb disk m\u00eb her\u00ebt se ata.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Rikthimi n\u00eb klaster<\/h3>\n<p>\nKur rikthehesh n\u00eb klaster, ndihm\u00ebsit veprojn\u00eb ashtu si\u00e7 veprojn\u00eb n\u00eb rast d\u00ebshtimi lideri: kontrollojn\u00eb kopjen e liderit dhe shkurtojn\u00eb logun e tyre deri n\u00eb HW (n\u00eb momentin e zgjedhjes). P\u00ebr krahasim, RabbitMQ i vler\u00ebson nodet e rikthyer si krejt\u00ebsisht t\u00eb reja. N\u00eb t\u00eb dy rastet, brokeri hedh \u00e7do gjendje ekzistuese. N\u00ebse p\u00ebrdoret sinkronizimi automatik, at\u00ebher\u00eb lideri duhet t\u00eb riprodhoj\u00eb krejt p\u00ebrmbajtjen aktuale n\u00eb pasqyr\u00ebn e re me nj\u00eb metod\u00eb \"dhe le t\u00eb pres\u00eb bota e t\u00ebr\u00eb\". Gjat\u00eb k\u00ebsaj operacioni, lideri nuk pranon asnj\u00eb operacion leximi ose shkruaje. Ky qasje krijon probleme n\u00eb radh\u00ebt e m\u00ebdha.<\/p>\n<p>Kafka \u00ebsht\u00eb nj\u00eb log i shp\u00ebrndar\u00eb, dhe n\u00eb p\u00ebrgjith\u00ebsi ruan m\u00eb shum\u00eb mesazhe se nj\u00eb radh\u00eb RabbitMQ, ku t\u00eb dh\u00ebnat fshihen nga radh\u00eb pas leximit t\u00eb tyre. Radh\u00ebt aktive duhet t\u00eb mbeten relativisht t\u00eb vogla. Por Kafka \u00ebsht\u00eb nj\u00eb log me politik\u00ebn e saj t\u00eb ruajtjes, e cila mund t\u00eb vendos\u00eb nj\u00eb afat deri n\u00eb dit\u00eb ose jav\u00eb. Qasja me bllokimin e radh\u00ebs dhe sinkronizimin e plot\u00eb \u00ebsht\u00eb krejt\u00ebsisht e papranueshme p\u00ebr nj\u00eb log t\u00eb shp\u00ebrndar\u00eb. N\u00eb vend t\u00eb k\u00ebsaj, ndihm\u00ebsit e Kafka thjesht shkurtojn\u00eb logun e tyre deri n\u00eb HW t\u00eb liderit (n\u00eb momentin e zgjedhjes) n\u00eb rastin kur kopja e tyre e tejkalon liderin. N\u00eb nj\u00eb rast m\u00eb t\u00eb mundsh\u00ebm, kur ndihm\u00ebsi \u00ebsht\u00eb pas, ai thjesht fillon t\u00eb b\u00ebj\u00eb k\u00ebrkesa p\u00ebr t\u00eb t\u00ebrhequr, duke filluar nga LEO aktual.<\/p>\n<p>Ndihm\u00ebsit e rinj ose ata q\u00eb rikthehen fillojn\u00eb jasht\u00eb ISR dhe nuk marrin pjes\u00eb n\u00eb komitetet. Ata thjesht punojn\u00eb ngjitur me grupin, duke marr\u00eb mesazhe sa m\u00eb shpejt q\u00eb mundeni, derisa t\u00eb p\u00ebrfundojn\u00eb me liderin dhe t\u00eb hyjn\u00eb n\u00eb ISR. Nuk ka bllokim dhe nuk ka nevoj\u00eb t\u00eb hodhni t\u00eb dh\u00ebna.<\/p>\n<h1>Shkeputja e lidhshm\u00ebris\u00eb<\/h1>\n<p>\nKafka ka m\u00eb shum\u00eb componente se RabbitMQ, k\u00ebshtu q\u00eb k\u00ebtu ka nj\u00eb set m\u00eb t\u00eb komplikuar t\u00eb sjelljeve kur lidhshm\u00ebria e klasterit prish\u00ebt. Por Kafka fillimisht \u00ebsht\u00eb projektuar p\u00ebr klaster\u00eb, k\u00ebshtu q\u00eb zgjidhjet jan\u00eb shum\u00eb mir\u00eb t\u00eb menduara.<\/p>\n<p>M\u00eb posht\u00eb jan\u00eb disa skenar\u00eb t\u00eb shkeljes s\u00eb lidhshm\u00ebris\u00eb:<\/p>\n<ul>\n<li>Skenari 1. Ndihm\u00ebsi nuk e sheh liderin, por akoma e sheh Zookeeper.\n<\/li>\n<li>Skenari 2. Lideri nuk sheh asnj\u00eb ndihm\u00ebs, por akoma e sheh Zookeeper.\n<\/li>\n<li>Skenari 3. Ndihm\u00ebsi e sheh liderin, por nuk e sheh Zookeeper.\n<\/li>\n<li>Skenari 4. Lideri i sheh ndihm\u00ebsit, por nuk e sheh Zookeeper.\n<\/li>\n<li>Skenari 5. Ndihm\u00ebsi \u00ebsht\u00eb plot\u00ebsisht i ndar\u00eb nga nodet e tjera t\u00eb Kafka dhe nga Zookeeper.\n<\/li>\n<li>Skenari 6. Lideri \u00ebsht\u00eb plot\u00ebsisht i ndar\u00eb nga nodet e tjera t\u00eb Kafka dhe nga Zookeeper.\n<\/li>\n<li>Skenari 7. Nodi kontrollues i Kafka nuk e sheh ndonj\u00eb nod tjet\u00ebr t\u00eb Kafka.\n<\/li>\n<li>Skenari 8. Kontrolluesi i Kafka nuk e sheh Zookeeper.<\/li>\n<\/ul>\n<p>\nP\u00ebr \u00e7do skenar \u00ebsht\u00eb parashikuar nj\u00eb sjellje e saj.<\/p>\n<h3>Skenari 1. Ndihm\u00ebsi nuk e sheh liderin, por akoma e sheh Zookeeper<\/h3>\n<p>\n<img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/5f51ae679933b2b5a69e21831465d121.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 22. Skenari 1. ISR nga tre kopje<\/i><\/p>\n<p>Shk\u00ebputja e lidhshm\u00ebris\u00eb ndan brokerin 3 nga broker\u00ebt 1 dhe 2, por jo nga Zookeeper. Brokeri 3 nuk mund m\u00eb t\u00eb b\u00ebj\u00eb k\u00ebrkesa p\u00ebr t\u00eb t\u00ebrhequr. Pas kalimit t\u00eb koh\u00ebs <i>replica.lag.time.max.ms<\/i> ai hiqet nga ISR dhe nuk merr pjes\u00eb n\u00eb komitetet e mesazheve. Sapo lidhshm\u00ebria rikthehet, ai do t\u00eb rinovoj\u00eb k\u00ebrkesat p\u00ebr t\u00eb t\u00ebrhequr dhe do t'i bashkohet ISR, kur t\u00eb kap\u00eb liderin. Zookeeper do t\u00eb vazhdoj\u00eb t\u00eb marr\u00eb ping dhe t\u00eb mendoj\u00eb se brokeri \u00ebsht\u00eb i gjall\u00eb dhe i sh\u00ebndosh\u00eb.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/976b7ff080ee6a6d76a165f614f40e95.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 23. Skenari 1. Brokeri hiqet nga ISR, n\u00ebse nuk merr nj\u00eb k\u00ebrkes\u00eb p\u00ebr t\u00eb t\u00ebrhequr brenda intervalit replica.lag.time.max.ms<\/i><\/p>\n<p>Nuk ka asnj\u00eb ndarje logjike (split-brain) ose pezullim nodi si n\u00eb RabbitMQ. N\u00eb vend t\u00eb k\u00ebsaj, reduktohet mbivendosja. <\/p>\n<h3>Skenari 2. Lideri nuk e sheh asnj\u00eb ndihm\u00ebs, por akoma e sheh Zookeeper.<\/h3>\n<p>\n<img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/ff6aaa053d73e8b7c97111979bcf97aa.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 24. Skenari 2. Lideri dhe dy ndihm\u00ebs<\/i><\/p>\n<p>Shk\u00ebputja e lidhjes rrjetike ndan liderin nga ndjek\u00ebsit, por brokeri ende e sheh Zookeeper. Si n\u00eb skenarin e par\u00eb, ISR ngushtohet, por k\u00ebsaj here vet\u00ebm deri te lideri, pasi t\u00eb gjith\u00eb ndjek\u00ebsit ndalojn\u00eb s\u00eb d\u00ebrguari k\u00ebrkesat p\u00ebr t\u00ebrheqje. S\u00ebrish, nuk ka ndarje logjike. N\u00eb vend t\u00eb k\u00ebsaj, ndodhi humbja e tepric\u00ebs p\u00ebr mesazhet e reja, derisa lidhja t\u00eb rikuperohet. Zookeeper vazhdon t\u00eb marr\u00eb ping dhe konsideron se brokeri \u00ebsht\u00eb i gjall\u00eb dhe n\u00eb form\u00eb.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/3c69e8327da241407f76719fc022c37a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 25. Skenari 2. ISR ngushtohet vet\u00ebm deri te lideri<\/i><\/p>\n<h3>Skenari 3. Ndjek\u00ebsi e sheh liderin, por nuk e sheh Zookeeper<\/h3>\n<p>\nNdjek\u00ebsi ndahen nga Zookeeper, por jo nga brokeri me liderin. Si rezultat, ndjek\u00ebsi vazhdon t\u00eb d\u00ebrgoj\u00eb k\u00ebrkesa p\u00ebr t\u00ebrheqje dhe t\u00eb jet\u00eb an\u00ebtar i ISR. Zookeeper nuk merr m\u00eb ping dhe regjistron r\u00ebnien e brokerit, por pasi \u00ebsht\u00eb vet\u00ebm nj\u00eb ndjek\u00ebs, nuk ka asnj\u00eb pasoj\u00eb pas rikuperimit.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/1183bcfe45a3225ae3ef1a0651f61350.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 26. Skenari 3. Ndjek\u00ebsi vazhdon t\u00eb d\u00ebrgoj\u00eb k\u00ebrkesa p\u00ebr t\u00ebrheqje te lideri<\/i><\/p>\n<h3>Skenari 4. Lideri sheh ndjek\u00ebsit, por nuk e sheh Zookeeper<\/h3>\n<p>\n<img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/b3bf27fcb0eb806b27ad00f098e50a26.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 27. Skenari 4. Lideri dhe dy ndjek\u00ebs<\/i><\/p>\n<p>Lideri \u00ebsht\u00eb i ndar\u00eb nga Zookeeper, por jo nga brokerat me ndjek\u00ebs. <\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/82d9f2e5868566678befc414d1885bfd.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 28. Skenari 4. Lideri \u00ebsht\u00eb i izoluar nga Zookeeper<\/i><\/p>\n<p>Pas nj\u00eb kohe, Zookeeper do t\u00eb regjistroj\u00eb r\u00ebnien e brokerit dhe do ta njoftoj\u00eb kontrolluesin. Ai do t\u00eb zgjedh\u00eb nj\u00eb lider t\u00eb ri nga ndjek\u00ebsit. Megjithat\u00eb, lideri origjinal do t\u00eb vazhdoj\u00eb t\u00eb mendoj\u00eb se \u00ebsht\u00eb lider dhe do t\u00eb vazhdoj\u00eb t\u00eb pranoj\u00eb regjistrime me <i>acks=1<\/i>. Ndjek\u00ebsit nuk i d\u00ebrgojn\u00eb m\u00eb atij k\u00ebrkesa p\u00ebr t\u00ebrheqje, k\u00ebshtu ai do t'i konsideroj\u00eb ata t\u00eb vdekur dhe do t\u00eb p\u00ebrpiqet ta ngushtoj\u00eb ISR deri te vetja. Por, pasi nuk ka lidhje me Zookeeper, nuk do t\u00eb mundet ta b\u00ebj\u00eb k\u00ebt\u00eb dhe n\u00eb at\u00eb moment do t\u00eb heq\u00eb dor\u00eb nga pranimi i m\u00ebtejsh\u00ebm t\u00eb regjistrimeve. <\/p>\n<p>Mesazhet <i>acks=all<\/i> nuk do t\u00eb marrin konfirmime, sepse n\u00eb fillim ISR p\u00ebrfshin t\u00eb gjitha replikat, dhe para se t\u00eb arrijn\u00eb te ato, mesazhet nuk kalojn\u00eb. Kur lideri fillestar t\u00eb p\u00ebrpiqet t'i heq\u00eb ato nga ISR, nuk do t\u00eb mund ta b\u00ebj\u00eb k\u00ebt\u00eb dhe do t\u00eb ndaloj\u00eb gjithashtu pranimin e ndonj\u00eb mesazhi.<\/p>\n<p>Konsumator\u00ebt shpejt v\u00ebrejn\u00eb nd\u00ebrrimin e liderit dhe fillojn\u00eb t\u00eb d\u00ebrgojn\u00eb regjistrime n\u00eb serverin e ri. Pas rikuperimit t\u00eb rrjetit, lideri origjinal sheh se nuk \u00ebsht\u00eb m\u00eb lider dhe e pranon logun e tij n\u00eb vler\u00ebn HW q\u00eb kishte lideri i ri n\u00eb momentin e d\u00ebshtimit, p\u00ebr t\u00eb shmangur \u00e7arjen e logjeve. Pastaj fillon t'u d\u00ebrgoj\u00eb k\u00ebrkesa p\u00ebr t\u00ebrheqje liderit t\u00eb ri. Do t\u00eb humbasin t\u00eb gjitha regjistrimet e liderit origjinal, t\u00eb pa replikuara te lideri i ri. Do t\u00eb thot\u00eb se mesazhet e pa konfirmuara nga lideri fillestar do t\u00eb humbasin n\u00eb ato disa sekonda, kur dy lider\u00eb ishin aktiv\u00eb.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/d88d6f33c13dccb814c0f206ec72512b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 29. Skenari 4. Lideri n\u00eb brokerin 1 b\u00ebhet ndjek\u00ebs pas rikuperimit t\u00eb rrjetit<\/i><\/p>\n<h3>Skenari 5. Ndjek\u00ebsi \u00ebsht\u00eb plot\u00ebsisht i ndar\u00eb nga t\u00eb tjer\u00ebt nodet Kafka dhe nga Zookeeper<\/h3>\n<p>\nNdjek\u00ebsi \u00ebsht\u00eb plot\u00ebsisht i izoluar nga t\u00eb tjer\u00ebt nodet Kafka dhe nga Zookeeper. Ai thjesht hiqet nga ISR, derisa t\u00eb rikuperohet rrjeti, dhe pastaj e ndjek t\u00eb tjer\u00ebt.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/b092607f777014fd945f18734dd7f4e7.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 30. Skenari 5. Ndjek\u00ebsi i izoluar hiqet nga ISR<\/i><\/p>\n<h3>Skenari 6. Lideri \u00ebsht\u00eb plot\u00ebsisht i ndar\u00eb nga t\u00eb tjer\u00ebt nodet Kafka dhe nga Zookeeper<\/h3>\n<p>\n<img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/bb808d534dbf65748eb0b481f86b4926.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 31. Skenari 6. Lideri dhe dy ndjek\u00ebs<\/i><\/p>\n<p>Lideri \u00ebsht\u00eb plot\u00ebsisht i izoluar nga ndjek\u00ebsit e tij, kontrolluesi dhe Zookeeper. P\u00ebr nj\u00eb periudh\u00eb t\u00eb shkurt\u00ebr, ai do t\u00eb vazhdoj\u00eb t\u00eb pranoj\u00eb regjistrime me <i>acks=1<\/i>.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/6fda78bcd26ef6b916579d447e9fb1f0.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 32. Skenari 6. Izolimi i liderit nga nodet e tjera Kafka dhe Zookeeper<\/i><\/p>\n<p>Duke mos marr\u00eb k\u00ebrkesa pas <i>replica.lag.time.max.ms<\/i>, ai do t\u00eb p\u00ebrpiqet ta ngushtoj\u00eb ISR deri te vetja, por nuk do t\u00eb mundet ta b\u00ebj\u00eb k\u00ebt\u00eb, pasi nuk ka lidhje me Zookeeper, k\u00ebshtu ai do t\u00eb ndaloj\u00eb pranimin e regjistrimeve. <\/p>\n<p>N\u00eb nd\u00ebrkoh\u00eb, Zookeeper do ta sh\u00ebnoj\u00eb brokerin e izoluar si t\u00eb vdekur, dhe kontrolluesi do t\u00eb zgjedh\u00eb nj\u00eb lider t\u00eb ri.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/73d611a411fc64d835af2c714e9eba0a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 33. Skenari 6. Dy lider\u00eb<\/i><\/p>\n<p>Lideri origjinal mund t\u00eb pranoj\u00eb regjistrime p\u00ebr disa sekonda, por pastaj ndalon pranimin e ndonj\u00eb mesazhi. Konsumator\u00ebt p\u00ebrdit\u00ebsohen \u00e7do 60 sekonda me metadatat e fundit. Ata do t\u00eb informohen p\u00ebr nd\u00ebrrimin e liderit dhe do t\u00eb fillojn\u00eb t\u00eb d\u00ebrgojn\u00eb regjistrime te lideri i ri.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/ae99380c8a84fac15f440b699b9c552b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 34. Skenari 6. Prodhuesit kalojn\u00eb te lideri i ri<\/i><\/p>\n<p>Do t\u00eb humbasin t\u00eb gjitha regjistrimet e konfirmuara, t\u00eb b\u00ebra nga lideri origjinal q\u00eb nga humbja e lidhjes. Pasi rrjeti t\u00eb rikuperohet, lideri origjinal p\u00ebrmes Zookeeper do t\u00eb zbuloj\u00eb se nuk \u00ebsht\u00eb m\u00eb lider. At\u00ebher\u00eb do t\u00eb pres\u00eb logun e tij deri n\u00eb HW t\u00eb liderit t\u00eb ri n\u00eb momentin e zgjedhjes dhe do t\u00eb filloj\u00eb t\u00eb d\u00ebrgoj\u00eb k\u00ebrkesa si ndjek\u00ebs.<\/p>\n<p><img decoding=\"async\" alt=\"RabbitMQ kund\u00ebr Kafka: q\u00ebndrueshm\u00ebria dhe disponueshm\u00ebria e lart\u00eb\" src=\"\/wp-content\/uploads\/2019\/11\/10f0983a829b03ca9bbc62e12cbf8a5d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Fig. 35. Skenari 6. Lideri origjinal b\u00ebhet ndjek\u00ebs pas rikuperimit t\u00eb lidhjes s\u00eb rrjetit<\/i><\/p>\n<p>N\u00eb k\u00ebt\u00eb situat\u00eb, p\u00ebr nj\u00eb periudh\u00eb t\u00eb shkurt\u00ebr mund t\u00eb ket\u00eb nj\u00eb ndarje logjike, por vet\u00ebm n\u00ebse <i>acks=1<\/i> dhe <i>min.insync.replicas<\/i> Po gjithashtu 1. Ndarja logjike p\u00ebrfundon automatikisht ose pas rikthimit t\u00eb rrjetit, kur lideri origjinal e kupton se nuk \u00ebsht\u00eb m\u00eb lider, ose kur t\u00eb gjith\u00eb klient\u00ebt kuptojn\u00eb se lideri ka ndryshuar dhe fillojn\u00eb t\u00eb shkruajn\u00eb te lideri i ri \u2014 n\u00eb var\u00ebsi t\u00eb asaj se \u00e7far\u00eb ndodh m\u00eb par\u00eb. N\u00eb \u00e7do rast do t\u00eb ndodhi humbja e disa mesazheve, por vet\u00ebm me <i>acks=1<\/i>.<\/p>\n<p>Ekziston nj\u00eb variant tjet\u00ebr i k\u00ebtij skenari, kur menj\u00ebher\u00eb para ndarjes s\u00eb rrjetit, ndjek\u00ebsit kan\u00eb mbetur pas, dhe lideri e ka ngushtuar ISR n\u00eb vet\u00ebm veten. Pastaj ai izolohet p\u00ebr shkak t\u00eb humbjes s\u00eb lidhshm\u00ebris\u00eb. Zgjedhet nj\u00eb lider i ri, por lideri fillestar vazhdon t\u00eb pranoj\u00eb regjistrime, madje <i>acks=all<\/i>, sepse n\u00eb ISR p\u00ebrve\u00e7 tij nuk ka askush tjet\u00ebr. K\u00ebto regjistrime do t\u00eb humbasin pas rikthimit t\u00eb rrjetit. M\u00ebnyra e vetme p\u00ebr t\u00eb shmangur nj\u00eb variant t\u00eb till\u00eb \u00ebsht\u00eb <i>min.insync.replicas = 2<\/i>.<\/p>\n<h3>Skenari 7. Nj\u00eb nyje e kontrolluesit Kafka nuk e sheh nj\u00eb nyje tjet\u00ebr Kafka<\/h3>\n<p>\nN\u00eb p\u00ebrgjith\u00ebsi, pas humbjes s\u00eb lidhjes me nj\u00eb nyje Kafka, kontroluesi nuk do t\u00eb mund t'i d\u00ebrgoj\u00eb atij asnj\u00eb informacion n\u00eb lidhje me ndryshimin e liderit. N\u00eb rastin m\u00eb t\u00eb keq, kjo do t\u00eb \u00e7oj\u00eb n\u00eb nj\u00eb ndarje logjike t\u00eb shkurt\u00ebr, si n\u00eb skenarin 6. Shpesh, brokeri thjesht nuk do t\u00eb jet\u00eb nj\u00eb kandidat p\u00ebr lider\u00ebsi n\u00eb rastin e d\u00ebshtimit t\u00eb fundit.<\/p>\n<h3>Skenari 8. Kontrolluesi Kafka nuk e sheh Zookeeper<\/h3>\n<p>\nKontrolluesi i humbur nuk do t\u00eb marr\u00eb ping nga Zookeeper dhe do t\u00eb zgjedh\u00eb nj\u00eb nyje t\u00eb re Kafka si kontrollues. Kontrolluesi origjinal mund t\u00eb vazhdoj\u00eb t\u00eb veproj\u00eb si i till\u00eb, por ai nuk merr njoftime nga Zookeeper, prandaj nuk do t\u00eb ket\u00eb asnj\u00eb detyr\u00eb p\u00ebr t\u00eb p\u00ebrfunduar. Sap o rrjeti t\u00eb rikthehet, ai do t\u00eb kuptoj\u00eb se nuk \u00ebsht\u00eb m\u00eb kontrollues, por \u00ebsht\u00eb b\u00ebr\u00eb nj\u00eb nyje e zakonshme Kafka.<\/p>\n<h3>P\u00ebrfundimet nga skenar\u00ebt<\/h3>\n<p>\nNe shohim se humbja e lidhshm\u00ebris\u00eb s\u00eb ndjek\u00ebsve nuk \u00e7on n\u00eb humbjen e mesazheve, por thjesht redukton p\u00ebrkoh\u00ebsisht tepric\u00ebn, derisa rrjeti t\u00eb rikthehet. Kjo, sigurisht, mund t\u00eb \u00e7oj\u00eb n\u00eb humbje t\u00eb t\u00eb dh\u00ebnave, n\u00ebse nj\u00eb ose disa nyje humbin.<\/p>\n<p>N\u00ebse p\u00ebr shkak t\u00eb humbjes s\u00eb lidhshm\u00ebris\u00eb lideri \u00ebsht\u00eb ndar\u00eb nga Zookeeper, kjo mund t\u00eb \u00e7oj\u00eb n\u00eb humbje mesazhesh me <i>acks=1<\/i>. Mungesa e lidhjes me Zookeeper shkakton nj\u00eb ndarje logjike t\u00eb shkurt\u00ebr me dy lider\u00eb. Ky problem zgjidhet nga parametrat <i>acks=all<\/i>.<\/p>\n<p>Parametri <i>min.insync.replicas<\/i> p\u00ebr dy ose m\u00eb shum\u00eb replika ofron garanci t\u00eb shtes\u00eb q\u00eb skenar\u00ebt e till\u00eb afatshkurt\u00ebr nuk do t\u00eb \u00e7ojn\u00eb n\u00eb humbjen e mesazheve, si n\u00eb skenarin 6.<\/p>\n<h1>P\u00ebrmbledhje mbi humbjen e mesazheve<\/h1>\n<p>\nLe t\u00eb rendisim t\u00eb gjitha m\u00ebnyrat se si mund t\u00eb humbni t\u00eb dh\u00ebna n\u00eb Kafka:<\/p>\n<ul>\n<li>\u00c7do d\u00ebshtim i liderit, n\u00ebse mesazhet jan\u00eb konfirmuar duke p\u00ebrdorur <i>acks=1<\/i>\n<\/li>\n<li>\u00c7do kalim i papast\u00ebr (unclean) n\u00eb lider\u00ebsi, do t\u00eb thot\u00eb te ndjek\u00ebsi jasht\u00eb ISR, madje edhe me <i>acks=all<\/i>\n<\/li>\n<li>Izolimi i liderit nga Zookeeper, n\u00ebse mesazhet jan\u00eb konfirmuar duke p\u00ebrdorur <i>acks=1<\/i>\n<\/li>\n<li>Izolimi i plot\u00eb i liderit, i cili tashm\u00eb e ka ngushtuar grupin ISR n\u00eb veten e tij. T\u00eb gjitha mesazhet do t\u00eb humbasin, madje <i>acks=all<\/i>. Kjo \u00ebsht\u00eb e v\u00ebrtet\u00eb vet\u00ebm n\u00eb rast se <i>min.insync.replicas=1<\/i>.\n<\/li>\n<li>D\u00ebshtime t\u00eb menj\u00ebhershme t\u00eb t\u00eb gjitha nyjeve t\u00eb nj\u00eb seksioni. Duke qen\u00eb se mesazhet konfirmohen nga memoria, disa mund t\u00eb mos jen\u00eb regjistruar ende n\u00eb disk. Pas rinisjes s\u00eb server\u00ebve, disa mesazhe mund t\u00eb mungojn\u00eb.<\/li>\n<\/ul>\n<p>\nKalimet e papastra n\u00eb lider\u00ebsi mund t\u00eb shmangen, ose duke i ndaluar ato, ose duke siguruar tepric\u00eb prej t\u00eb pakt\u00ebn dy. Konfigurimi m\u00eb i fort\u00eb \u00ebsht\u00eb nj\u00eb kombinim <i>acks=all<\/i> dhe <i>min.insync.replicas<\/i> m\u00eb shum\u00eb se 1.<\/p>\n<h1>Krahasimi i drejtp\u00ebrdrejt\u00eb i q\u00ebndrueshm\u00ebris\u00eb s\u00eb RabbitMQ dhe Kafka<\/h1>\n<p>\nP\u00ebr t\u00eb siguruar q\u00ebndrueshm\u00ebrin\u00eb dhe disponueshm\u00ebrin\u00eb e lart\u00eb, t\u00eb dy platformat zbatojn\u00eb nj\u00eb sistem replikimi primar dhe sekondar. Megjithat\u00eb, RabbitMQ ka nj\u00eb ahile t\u00eb vet. Gjat\u00eb ri-ngjitjes pas nj\u00eb d\u00ebshtimi, nyjet hedhin t\u00eb dh\u00ebnat e tyre, dhe sinkronizimi bllokohet. Ky dyfishim rrezikon q\u00ebndrueshm\u00ebrin\u00eb e rreshtave t\u00eb m\u00ebdha n\u00eb RabbitMQ. Ju do t\u00eb duhet t\u00eb pajtoheni ose me reduktimin e tepric\u00ebs, ose me bllokime t\u00eb zgjatura. Reduktimi i tepric\u00ebs rrit rrezikun e humbjeve masive t\u00eb t\u00eb dh\u00ebnave. Por n\u00ebse rreshtat jan\u00eb t\u00eb vegj\u00ebl, p\u00ebr q\u00ebllime t\u00eb tepric\u00ebs me periudha t\u00eb shkurtra t\u00eb pakoh\u00ebsis\u00eb (p\u00ebr disa sekonda) mund t\u00eb arrihet p\u00ebrmes p\u00ebrpjekjeve p\u00ebr lidhje.<\/p>\n<p>N\u00eb Kafka nuk ka nj\u00eb problem t\u00eb till\u00eb. Ajo hedh t\u00eb dh\u00ebnat vet\u00ebm nga pika e shk\u00ebputjes mes liderit dhe ndjek\u00ebsit. T\u00eb gjitha t\u00eb dh\u00ebnat e zakonshme ruhen. P\u00ebr m\u00eb tep\u00ebr, replikimi nuk bllokon sistemin. Lideri vazhdon t\u00eb pranoj\u00eb regjistrime, nd\u00ebrsa ndjek\u00ebsi i ri e p\u00ebrfshin, k\u00ebshtu q\u00eb p\u00ebr dev.ops, bashkimi ose ri-bashkimi me klasterin b\u00ebhet nj\u00eb detyr\u00eb triviale. Sigurisht, ende mbeten probleme, si kapaciteti i rrjetit gjat\u00eb replikimit. N\u00ebse disa ndjek\u00ebs shtohen n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, mund t\u00eb hasni kufirin e kapacitetit.<\/p>\n<p>RabbitMQ ofron besueshm\u00ebri m\u00eb t\u00eb lart\u00eb se Kafka kur ndodhin d\u00ebshtime t\u00eb shumt\u00eb server\u00ebsh n\u00eb klaster. Si\u00e7 e p\u00ebrmendem, RabbitMQ d\u00ebrgon nj\u00eb konfirmim ndaj botuesit vet\u00ebm pasi mesazhi t\u00eb regjistrohet n\u00eb disk te mjeshtri dhe t\u00eb gjitha pasqyrat. Por kjo shton nj\u00eb vones\u00eb t\u00eb shtuar p\u00ebr dy arsye:<\/p>\n<ul>\n<li>fsync \u00e7do disa qindra milisekonda\n<\/li>\n<li>D\u00ebshtimi i pasqyr\u00ebs mund t\u00eb vihet re vet\u00ebm pas p\u00ebrfundimit t\u00eb jet\u00ebs s\u00eb pakove, t\u00eb cilat kontrollojn\u00eb disponibilitetin e secilit nyje (net tick). N\u00ebse pasqyra ngec apo d\u00ebshton, kjo shton vones\u00ebn.<\/li>\n<\/ul>\n<p>\nKafka beson se, n\u00ebse mesazhi ruhet n\u00eb disa nyje, mund t\u00eb konfirmohen mesazhet sapo ato futen n\u00eb kujtes\u00eb. P\u00ebr k\u00ebt\u00eb shkak, ekziston rreziku i humbjes s\u00eb mesazheve t\u00eb \u00e7do lloji (edhe <i>acks=all<\/i>, <i>min.insync.replikat=2<\/i>) n\u00eb rast t\u00eb d\u00ebshtimit t\u00eb menj\u00ebhersh\u00ebm.<\/p>\n<p>N\u00eb p\u00ebrgjith\u00ebsi, Kafka tregon performanc\u00eb m\u00eb t\u00eb lart\u00eb dhe \u00ebsht\u00eb projektuar fillimisht p\u00ebr klastere. Numri i ndjek\u00ebsve mund t\u00eb rritet deri n\u00eb 11, n\u00ebse \u00ebsht\u00eb e nevojshme p\u00ebr besueshm\u00ebri. Faktor\u00ebt e replikimit 5 dhe numri minimal i replikave n\u00eb gjendje t\u00eb sinkronizuar <i>min.insync.replicas=3<\/i> do ta b\u00ebjn\u00eb humbjen e mesazheve nj\u00eb ngjarje shum\u00eb t\u00eb rrall\u00eb. N\u00ebse infrastruktura juaj \u00ebsht\u00eb n\u00eb gjendje t\u00eb ofroj\u00eb nj\u00eb faktor replikimi dhe nj\u00eb nivel t\u00eb tep\u00ebrt, at\u00ebher\u00eb mund t\u00eb zgjidhni k\u00ebt\u00eb mund\u00ebsi.<\/p>\n<p>Klasterizimi i RabbitMQ \u00ebsht\u00eb i mir\u00eb p\u00ebr radh\u00ebt e vogla. Por, edhe radh\u00ebt e vogla mund t\u00eb rriten shpejt me trafikun e madh. Sapo radh\u00ebt b\u00ebhen t\u00eb m\u00ebdha, do t\u00eb duhet t\u00eb b\u00ebni nj\u00eb zgjedhje t\u00eb ashp\u00ebr midis disponueshm\u00ebris\u00eb dhe besueshm\u00ebris\u00eb. Klasterizimi i RabbitMQ \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr situata m\u00eb t\u00eb pazakonta, ku p\u00ebrfitimet e fleksibilitetit t\u00eb RabbitMQ e tejkalojn\u00eb \u00e7do disavantazh t\u00eb klasterizimit t\u00eb tij.<\/p>\n<p>Nj\u00eb nga antidotet ndaj dob\u00ebsis\u00eb s\u00eb RabbitMQ n\u00eb lidhje me radh\u00ebt e m\u00ebdha \u00ebsht\u00eb t'i ndash ato n\u00eb shum\u00eb m\u00eb t\u00eb vogla. N\u00ebse nuk k\u00ebrkohet renditje e plot\u00eb e gjith\u00eb radh\u00ebs, por vet\u00ebm e mesazheve p\u00ebrkat\u00ebse (p.sh. mesazhet e nj\u00eb klienti t\u00eb caktuar), ose asgj\u00eb t\u00eb renditur fare, at\u00ebher\u00eb kjo mund\u00ebsi \u00ebsht\u00eb e pranueshme: shihni projektin tim <noindex><a rel=\"nofollow\" href=\"https:\/\/jack-vanlightly.com\/blog\/2018\/7\/22\/creating-consumer-groups-in-rabbitmq-with-rebalanser-part-1\">Rebalanser<\/a><\/noindex> p\u00ebr ndarjen e radh\u00ebs (projekti \u00ebsht\u00eb ende n\u00eb faz\u00eb t\u00eb hershme). <\/p>\n<p>N\u00eb fund, mos harroni p\u00ebr nj\u00eb s\u00ebr\u00eb problemesh n\u00eb mekanizmat e klasterizimit dhe replikimit si te RabbitMQ ashtu edhe te Kafka. Me kalimin e koh\u00ebs sistemet jan\u00eb b\u00ebr\u00eb m\u00eb t\u00eb pjekura dhe t\u00eb q\u00ebndrueshme, por asnj\u00eb mesazh nuk do t\u00eb jet\u00eb ndonj\u00ebher\u00eb 100% i mbrojtur nga humbja! P\u00ebr m\u00eb tep\u00ebr, n\u00eb qendrat e t\u00eb dh\u00ebnave ndodhin katastrofa masive!<\/p>\n<p>N\u00ebse kam l\u00ebn\u00eb di\u00e7ka jasht\u00eb, kam b\u00ebr\u00eb nj\u00eb gabim, ose nuk jeni dakord me ndonj\u00eb nga tez\u00ebn, mos ngurroni t\u00eb shkruani nj\u00eb koment apo t\u00eb m\u00eb kontaktoni.<\/p>\n<p>M\u00eb pyesin shpesh: \u201c\u00c7far\u00eb t\u00eb zgjedh, Kafka apo RabbitMQ?\u201d, \u201cCili \u00ebsht\u00eb sistemi m\u00eb i mir\u00eb?\u201d. E v\u00ebrteta \u00ebsht\u00eb se, kjo varet v\u00ebrtet nga situata juaj, eksperienca aktuale etj. Nuk guxoj t\u00eb shpreh mendimin tim, pasi do t\u00eb ishte nj\u00eb thjeshtim shum\u00eb i madh t\u00eb rekomandoj nj\u00eb platform\u00eb t\u00eb vetme p\u00ebr t\u00eb gjitha p\u00ebrdorimet e mundshme dhe kufizimet. Kam shkruar k\u00ebt\u00eb cik\u00ebl artikujsh, q\u00eb ju mund t\u00eb formoni mendimin tuaj.<\/p>\n<p>Dua t\u00eb them se t\u00eb dy sistemet jan\u00eb lider\u00eb n\u00eb k\u00ebt\u00eb fush\u00eb. Ndoshta jam pak i paragjykuar, pasi p\u00ebrvojat e mia n\u00eb projekte m\u00eb b\u00ebjn\u00eb m\u00eb t\u00eb prirur t\u00eb vler\u00ebsoj gj\u00ebra si renditja e garantuar e mesazheve dhe besueshm\u00ebria. <\/p>\n<p>Shoh teknologji t\u00eb tjera q\u00eb u mungon kjo besueshm\u00ebri dhe renditje e garantuar, pastaj shoh RabbitMQ dhe Kafka \u2014 dhe kuptoj vler\u00ebn e jasht\u00ebzakonshme t\u00eb k\u00ebtyre dy sistemeve.<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/itsumma\/blog\/474984\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u043f\u0440\u043e\u0448\u043b\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043c\u044b \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0435\u043b\u0438 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0438\u0437\u0430\u0446\u0438\u044e RabbitMQ \u0434\u043b\u044f \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0435\u043d\u0438\u044f \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0441\u0442\u0438 \u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0441\u0442\u0438. \u0422\u0435\u043f\u0435\u0440\u044c \u0433\u043b\u0443\u0431\u043e\u043a\u043e \u043f\u043e\u043a\u043e\u043f\u0430\u0435\u043c\u0441\u044f \u0432 Apache Kafka. \u0417\u0434\u0435\u0441\u044c \u0435\u0434\u0438\u043d\u0438\u0446\u0435\u0439 \u0440\u0435\u043f\u043b\u0438\u043a\u0430\u0446\u0438\u0438 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0440\u0430\u0437\u0434\u0435\u043b (partition). \u0423 \u043a\u0430\u0436\u0434\u043e\u0433\u043e \u0442\u043e\u043f\u0438\u043a\u0430 \u043e\u0434\u0438\u043d \u0438\u043b\u0438 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437\u0434\u0435\u043b\u043e\u0432. \u0412 \u043a\u0430\u0436\u0434\u043e\u043c \u0440\u0430\u0437\u0434\u0435\u043b\u0435 \u0435\u0441\u0442\u044c \u043b\u0438\u0434\u0435\u0440 \u0441 \u0444\u043e\u043b\u043b\u043e\u0432\u0435\u0440\u0430\u043c\u0438 \u0438\u043b\u0438 \u0431\u0435\u0437 \u043d\u0438\u0445. \u041f\u0440\u0438 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u0442\u043e\u043f\u0438\u043a\u0430 \u0443\u043a\u0430\u0437\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u043a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u0440\u0430\u0437\u0434\u0435\u043b\u043e\u0432 \u0438 \u043a\u043e\u044d\u0444\u0444\u0438\u0446\u0438\u0435\u043d\u0442 \u0440\u0435\u043f\u043b\u0438\u043a\u0430\u0446\u0438\u0438. \u041e\u0431\u044b\u0447\u043d\u043e\u0435 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0435 3, \u044d\u0442\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-52541","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47RabbitMQ \u043f\u0440\u043e\u0442\u0438\u0432 Kafka: \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0441\u0442\u044c \u0438 \u0432\u044b\u0441\u043e\u043a\u0430\u044f \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0441\u0442\u044c | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-11-10T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T11:00:17+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47RabbitMQ kundrejt Kafka: q\u00ebndrueshm\u00ebri dhe disponueshm\u00ebri e lart\u00eb | ProHoster","description":"N\u00eb","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47RabbitMQ \u043f\u0440\u043e\u0442\u0438\u0432 Kafka: \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0441\u0442\u044c \u0438 \u0432\u044b\u0441\u043e\u043a\u0430\u044f \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0441\u0442\u044c | ProHoster","og:description":"\u0412","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/rabbitmq-protiv-kafka-otkazoustojchivost-i-vysokaya-dostupnost","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-11-10T21:00:00+00:00","article:modified_time":"2020-02-18T11:00:17+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"52541","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 03:57:21","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:41:24","updated":"2026-01-24 03:57:22","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/52541","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=52541"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/52541\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=52541"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=52541"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=52541"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}