{"id":36822,"date":"2019-10-31T22:14:01","date_gmt":"2019-10-31T19:14:01","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kafka-na-kubernetes-eto-horosho\/"},"modified":"2019-10-31T22:14:01","modified_gmt":"2019-10-31T19:14:01","slug":"kafka-na-kubernetes-eto-horosho","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kafka-na-kubernetes-eto-horosho","title":{"rendered":"A \u00ebsht\u00eb Kafka mbi Kubernetes e mir\u00eb?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Mir\u00ebsevini, Habr!<\/p>\n

N\u00eb koh\u00ebn e saj, ne ishim t\u00eb par\u00ebt q\u00eb soll\u00ebm n\u00eb tregun rus tem\u00ebn Kafka<\/a><\/noindex> dhe vazhdojm\u00eb ndjekin<\/a><\/noindex> me zhvillimin e saj. N\u00eb ve\u00e7anti, na dukej interesante tema e nd\u00ebrveprimit mes Kafka dhe Kubernetes<\/a><\/noindex>. Nj\u00eb pasqyr\u00eb (dhe mjaft e kujdesshme) artikulli<\/a><\/noindex> p\u00ebr k\u00ebt\u00eb tem\u00eb u publikua n\u00eb blogun e kompanis\u00eb Confluent q\u00eb n\u00eb tetor t\u00eb vitit t\u00eb kaluar nga autorja Gwen Shapiro. Sot, ne duam t\u00eb t\u00ebrheqim v\u00ebmendjen tuaj n\u00eb nj\u00eb artikull m\u00eb t\u00eb ri, t\u00eb muajit prill, nga Johann Gyger, i cili, edhe pse nuk e evitoi pik\u00ebpyetjen n\u00eb titull, e shqyrton tem\u00ebn me nj\u00eb qasje m\u00eb t\u00eb fokusuara, duke shoq\u00ebruar tekstin me lidhje interesante. Na falni p\u00ebr p\u00ebrkthimin e lir\u00eb t\u00eb \"chaos monkey\", n\u00ebse mundeni!<\/p>\n

\"A<\/a><\/noindex>
\n<\/a><\/noindex> <\/p>\n

Hyrje<\/h4>\n

\nKubernetes \u00ebsht\u00eb i dizajnuar p\u00ebr t\u00eb punuar me ngarkesa q\u00eb nuk ruajn\u00eb gjendjen. N\u00eb p\u00ebrgjith\u00ebsi, k\u00ebto ngarkesa paraqiten n\u00eb form\u00ebn e arkitektur\u00ebs mikros services, jan\u00eb t\u00eb lehta, mir\u00eb p\u00ebrshtaten p\u00ebr shkall\u00ebzim horizontal, i p\u00ebrmbahen parimeve t\u00eb aplikacioneve 12-faktor\u00ebsh, lejojn\u00eb funksionimin me fik\u00ebs automatik\u00eb (circuit breaker) dhe majmun\u00eb kaosi (chaos monkeys).<\/p>\n

Kafka, e vendosur nga ana tjet\u00ebr, n\u00eb thelb funksionon si nj\u00eb baz\u00eb t\u00eb dh\u00ebnash t\u00eb shp\u00ebrndar\u00eb. K\u00ebshtu, gjat\u00eb pun\u00ebs keni t\u00eb b\u00ebni me gjendjen, dhe kjo \u00ebsht\u00eb shum\u00eb m\u00eb e r\u00ebnd\u00eb se nj\u00eb mikros\u00ebrv\u00ebr. Kubernetes mb\u00ebshtet ngarkesa q\u00eb ruajn\u00eb gjendjen, por, si\u00e7 e thekson Kelsey Hightower n\u00eb dy nga tweet-et e tij, duhet t\u00eb jeni t\u00eb kujdessh\u00ebm me to:<\/p>\n

Disa mendojn\u00eb se, n\u00ebse instalohet Kubernetes mbi nj\u00eb ngarkes\u00eb q\u00eb ruan gjendjen, ajo shnd\u00ebrrohet n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash t\u00eb plot\u00eb menaxhuar, e cila mund t\u00eb konkurroj\u00eb me RDS. Kjo nuk \u00ebsht\u00eb e v\u00ebrtet\u00eb. Ndoshta, n\u00ebse punoni mjaft, lidhni komponent\u00eb t\u00eb tjer\u00eb dhe angazhoni nj\u00eb ekip inxhinier\u00ebsh SRE, do t\u00eb jeni n\u00eb gjendje t\u00eb krijoni nj\u00eb RDS mbi Kubernetes.<\/p><\/blockquote>\n

Gjithmon\u00eb rekomandoj kujdes t\u00eb jasht\u00ebzakonsh\u00ebm kur ekzekutoni ngarkesa q\u00eb ruajn\u00eb gjendjen mbi Kubernetes. Shumica e atyre q\u00eb jan\u00eb t\u00eb interesuar, \"mund t\u00eb ekzekutoj ngarkesa q\u00eb ruajn\u00eb gjendjen mbi Kubernetes\" nuk kan\u00eb p\u00ebrvoj\u00eb t\u00eb mjaftueshme n\u00eb pun\u00ebn me Kubernetes, dhe shpesh kan\u00eb munges\u00eb edhe me ngarkes\u00ebn p\u00ebr t\u00eb cil\u00ebn b\u00ebhet fjal\u00eb.<\/p><\/blockquote>\n

Pra ndaj, a duhet t\u00eb startohet Kafka n\u00eb Kubernetes? Nj\u00eb pyetje e kund\u00ebrt: a do ta ket\u00eb Kafka nj\u00eb funksionim m\u00eb t\u00eb mir\u00eb pa Kubernetes? K\u00ebtu \u00ebsht\u00eb pse dua t\u00eb theksoj n\u00eb k\u00ebt\u00eb artikull se si Kafka dhe Kubernetes plot\u00ebsojn\u00eb nj\u00ebri-tjetrin, dhe cilat jan\u00eb pengesat q\u00eb mund t\u00eb hasni gjat\u00eb kombinimit t\u00eb tyre.<\/p>\n

Koha e ekzekutimit<\/h4>\n

\nLe t\u00eb flasim p\u00ebr nj\u00eb gj\u00eb bazike \u2014 ambientin e ekzekutimit si i till\u00eb.<\/p>\n

N\u00eb dit\u00ebn e fillimit t\u00eb modulit, struktura e tij b\u00ebhet e disponueshme. Studimi n\u00eb UoL p\u00ebrb\u00ebhet nga cikli n\u00eb vijim:<\/b><\/i><\/p>\n

Brokerat e Kafka jan\u00eb efikas\u00eb kur punohet me CPU. TLS mund t\u00eb sjell\u00eb disa kosto. Megjithat\u00eb, klient\u00ebt e Kafka mund ta ngarkojn\u00eb m\u00eb shum\u00eb CPU-n\u00eb n\u00ebse p\u00ebrdorin enkriptimin, por kjo nuk ndikon te brokerat.<\/p>\n

Memoria<\/b><\/i><\/p>\n

Brokerat e Kafka marrin shum\u00eb memorje. Madh\u00ebsia e grumbullit t\u00eb JVM zakonisht duhet t\u00eb kufizohet n\u00eb 4\u20135 GB, por gjithashtu do t\u00eb keni nevoj\u00eb p\u00ebr shum\u00eb memorje sistemi, pasi Kafka e p\u00ebrdor shum\u00eb aktivisht cache-n\u00eb e faqeve. N\u00eb Kubernetes, p\u00ebrcaktoni p\u00ebrkat\u00ebsisht kufijt\u00eb e burimeve dhe k\u00ebrkesat e kontejnerit.<\/p>\n

Ruajtja e t\u00eb dh\u00ebnave<\/b><\/i><\/p>\n

Ruajtja e t\u00eb dh\u00ebnave n\u00eb konteiner\u00eb \u00ebsht\u00eb efemere \u2013 t\u00eb dh\u00ebnat humbin gjat\u00eb ribashkimit. P\u00ebr t\u00eb dh\u00ebnat e Kafka, mund t\u00eb p\u00ebrdorni nj\u00eb volum, dhe efekti do t\u00eb jet\u00eb i ngjash\u00ebm: t\u00eb dh\u00ebnat e brokerit tuaj do t\u00eb humbin pas p\u00ebrfundimit. Mesazhet tuaja megjithat\u00eb mund t\u00eb ruhen te brokerat e tjer\u00eb si replikat. Pra, pas ribashkimit, brokeri q\u00eb d\u00ebshtoi duhet t\u00eb riprodhoj\u00eb t\u00eb gjitha t\u00eb dh\u00ebnat si hapin e par\u00eb, dhe ky proces mund t\u00eb marr\u00eb mjaft koh\u00eb. emptyDir<\/code>, dhe efekti do t\u00eb jet\u00eb i ngjash\u00ebm: t\u00eb dh\u00ebnat e brokerit tuaj do t\u00eb humbin pas p\u00ebrfundimit. Mesazhet tuaja megjithat\u00eb mund t\u00eb ruhen te brokerat e tjer\u00eb si replikat. Pra, pas ribashkimit, brokeri q\u00eb d\u00ebshtoi duhet t\u00eb riprodhoj\u00eb t\u00eb gjitha t\u00eb dh\u00ebnat si hapin e par\u00eb, dhe ky proces mund t\u00eb marr\u00eb mjaft koh\u00eb. <\/p>\n

K\u00ebtu \u00ebsht\u00eb pse duhet t\u00eb p\u00ebrdorni ruajtje t\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave. Le t\u00eb jet\u00eb kjo nj\u00eb ruajtje e q\u00ebndrueshme jo-lokale me nj\u00eb sistem skedari XFS apo, m\u00eb sakt\u00ebsisht, ext4. Mos p\u00ebrdorni NFS. T\u00eb kam paralajm\u00ebruar. NFS versionet v3 ose v4 nuk do t\u00eb funksionojn\u00eb. N\u00eb m\u00ebnyr\u00eb t\u00eb p\u00ebrmbledhur, brokeri Kafka do t\u00eb bjer\u00eb, n\u00ebse nuk arrin t\u00eb heq\u00eb dosjen me t\u00eb dh\u00ebna p\u00ebr shkak t\u00eb problemeve me 'rilidhjet e m\u00ebdha', q\u00eb jan\u00eb aktuale n\u00eb NFS. N\u00ebse deri tani nuk ju kam bindur, shikoni me kujdes. lexoni k\u00ebt\u00eb artikull<\/a><\/noindex>. Ruajtja e t\u00eb dh\u00ebnave duhet t\u00eb jet\u00eb jo-lokale, q\u00eb Kubernetes t\u00eb mund t\u00eb zgjedh\u00eb m\u00eb fleksib\u00ebl nj\u00eb nyje t\u00eb re pas ribashkimit ose relokimit.<\/p>\n

Rrjeti<\/b><\/i><\/p>\n

Si n\u00eb rastin e shumic\u00ebs s\u00eb sistemeve t\u00eb shp\u00ebrndara, performanca e Kafka varet shum\u00eb nga minimizimi i vonesave n\u00eb rrjet dhe maksimizimi i gjer\u00ebsis\u00eb s\u00eb brezit. Mos u p\u00ebrpiqni t\u00eb vendosni t\u00eb gjith\u00eb broker\u00ebt n\u00eb nj\u00eb m\u00ebnyr\u00eb t\u00eb nj\u00ebjt\u00eb, sepse kjo do t\u00eb ul\u00eb disponueshm\u00ebrin\u00eb. N\u00ebse nj\u00eb nyje Kubernetes d\u00ebshtojn\u00eb, do t\u00eb d\u00ebshtoj\u00eb gjithashtu e gjith\u00eb klasteri Kafka. Po ashtu, mos e shp\u00ebrndani klasterin Kafka n\u00eb qendra t\u00eb plota t\u00eb t\u00eb dh\u00ebnave. E nj\u00ebjta gj\u00eb vlen p\u00ebr klasterin Kubernetes. Nj\u00eb kompromis i mir\u00eb n\u00eb k\u00ebt\u00eb rast \u00ebsht\u00eb t\u00eb zgjidhni zona t\u00eb ndryshme t\u00eb disponueshm\u00ebris\u00eb.<\/p>\n

Konfigurimi<\/h4>\n

\nManifestet e zakonshme<\/b><\/i><\/p>\n

N\u00eb faqen e internetit t\u00eb Kubernetes ka nj\u00eb udh\u00ebzues shum\u00eb t\u00eb mir\u00eb<\/a><\/noindex> p\u00ebr m\u00ebnyr\u00ebn e konfigurimit t\u00eb ZooKeeper me an\u00eb t\u00eb manifestesh. Duke qen\u00eb se ZooKeeper \u00ebsht\u00eb pjes\u00eb e Kafka, kjo \u00ebsht\u00eb nj\u00eb fillim i mir\u00eb p\u00ebr t\u00eb njohur konceptet q\u00eb zbatohen n\u00eb Kubernetes k\u00ebtu. Pas kuptimit t\u00eb k\u00ebsaj, do t\u00eb jeni n\u00eb gjendje t\u00eb p\u00ebrdorni t\u00eb nj\u00ebjtat koncepte edhe me klasterin Kafka.<\/p>\n