{"id":71213,"date":"2020-02-24T16:09:17","date_gmt":"2020-02-24T13:09:17","guid":{"rendered":"https:\/\/prohoster.info\/blog\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes"},"modified":"2020-03-03T16:14:29","modified_gmt":"2020-03-03T13:14:29","slug":"opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","title":{"rendered":"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><i><b>M\u00e4rkus t\u00f5lke kohta.<\/b>: K\u00e4esolevas artiklis jagab ettev\u00f5te Banzai Cloud n\u00e4idet oma spetsiaalsete utiliitide kasutamisest, et lihtsustada Kafka haldamist Kuberneteses. Esitatud juhised illustreerivad, kuidas m\u00e4\u00e4rata infrastruktuuri optimaalset suurust ja seadistada Kafka, et saavutada vajalik l\u00e4bilaskev\u00f5ime.<\/i><\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/8a598ec7db091c2944a6442f9bc124da.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nApache Kafka on jagatud voogedastuse platvorm usaldusv\u00e4\u00e4rsete, skaleeritavate ja k\u00f5rge j\u00f5udlusega reaalajas voogesituss\u00fcsteemide loomiseks. Selle muljetavaldavaid v\u00f5imalusi saab laiendada Kubernetesega. Selleks oleme arendanud <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/kafka-operator\">Open Source Kafka operaatori<\/a><\/noindex> ja t\u00f6\u00f6riista nimega <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/overview\/\">Supertubes<\/a><\/noindex>. Need v\u00f5imaldavad k\u00e4itada Kafka Kuberneteses ja kasutada selle erinevaid funktsioone, nagu brokerei konfiguratsiooni peenh\u00e4\u00e4lestus, m\u00f5\u00f5tmete p\u00f5hine skaleerimine koos uuesti tasakaalustamisega, rack awareness (riistvara teadlikkus), \"pehme\" <i>(graceful)<\/i> uuenduste rakendamine jne.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<blockquote><p>Proovige Supertubes oma klastris:<\/p>\n<pre><code class=\"bash\">curl https:\/\/getsupertubes.sh | sh ja supertubes install -a --no-democluster --kubeconfig<\/code><\/pre>\n<p>\nV\u00f5i p\u00f6\u00f6rduge <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/cli\/reference\/\">dokumentatsioon<\/a><\/noindex>. Samuti saab lugeda m\u00f5ningatest Kafka funktsioonidest, mille automatiseeritud haldamine toimub Supertubes ja Kafka operaatori kaudu. Oleme juba kirjutanud nendest meie blogis:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-operator\/\">Oh ei! Veel \u00fcks Kafka operaator Kuberneteses<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-alert\/\">Kafa j\u00e4lgimine ja haldamine Prometheuse m\u00f5\u00f5tmete alusel<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-rack-awareness\/\">Kafka rack awareness Kuberneteses<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-on-istio-performance\/\">Apache Kafka k\u00e4itamine Istio -- tulemused<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-topic-user-management\/\">Kasutaja autentitud ja juurdep\u00e4\u00e4suga kontrollitud klastrid Kafka operaatori abil<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-rolling-upgrade\/\">Kafka \u00fcleminek ja d\u00fcnaamiline konfiguratsioon Kuberneteses<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-envoy-protocol-filter\/\">Envoy protokolli filter Kafka jaoks, meshed<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/blockquote>\n<p>\nOtsustades kasvatada Kafka klastri Kuberneteses, seisate t\u00f5en\u00e4oliselt silmitsi probleemiga m\u00e4\u00e4rata baas-infrastruktuuri optimaalset suurust ja vajadusega peenh\u00e4\u00e4lestada Kafka konfiguratsiooni, et rahuldada l\u00e4bilaskev\u00f5ime n\u00f5udeid. Iga brokerei maksimaalne j\u00f5udlus s\u00f5ltub selle aluseks olevate infrastruktuuri komponentide nagu m\u00e4lu, protsessor, ketta kiirus, v\u00f5rgu l\u00e4bilaskev\u00f5ime jne. j\u00f5udlusest.<\/p>\n<p>Ideaalis peaks maakleri konfiguratsioon olema selline, et k\u00f5ik infrastruktuuri elemendid oleksid oma v\u00f5imekuse piiril kasutuses. Kuid reaalses elus on selline seadistus \u00fcsna keeruline. T\u00f5en\u00e4olisem on, et kasutajad seadistavad maaklerite konfiguratsiooni nii, et maksimaalset kasu saadakse \u00fchest v\u00f5i kahest komponendist (ketas, m\u00e4lu v\u00f5i protsessor). \u00dcldiselt n\u00e4itab maakler maksimaalset j\u00f5udlust, kui tema konfiguratsioon v\u00f5imaldab k\u00f5ige aeglasema komponendi t\u00e4ielikku kasutamist. Nii saame ligikaudse \u00fclevaate koormusest, millega \u00fcks maakler hakkama saab.<\/p>\n<p>Teoreetiliselt saame me samuti hinnata maaklerite arvu, mis on vajalik antud koormuse t\u00f6\u00f6tlemiseks. Kuid praktikas on erinevaid seadistusv\u00f5imalusi nii palju, et on keeruline (kui mitte v\u00f5imatu) hinnata mingi konfiguratsiooni potentsiaalset j\u00f5udlust. Teisis\u00f5nu, on v\u00e4ga keeruline planeerida konfiguratsiooni, l\u00e4htudes mingist antud j\u00f5udlusest.<\/p>\n<p>Supertubesi kasutajate jaoks rakendame me tavaliselt j\u00e4rgmist l\u00e4henemist: alustame m\u00f5nest konfiguratsioonist (infrastruktuur + seadistused), seej\u00e4rel m\u00f5\u00f5dame selle j\u00f5udlust, kohandame maakleri seadistusi ja kordame protsessi veel kord. Seda teeme seni, kuni k\u00f5ige aeglasema infrastruktuuri komponendi potentsiaal on t\u00e4ielikult rakendatud.<\/p>\n<p>Selle viisi abil saame selgema arusaama sellest, kui palju maaklereid on klastrile vajalik, et toime tulla m\u00e4\u00e4ratud koormusega (maaklerite arv s\u00f5ltub samuti teistest teguritest, nagu s\u00f5numite minimaalne replikate arv vastupidavuse tagamiseks, partition-juhid jne). Lisaks saame \u00fclevaate sellest, millise infrastruktuuri komponendi puhul on soovitav vertikaalne skaleerimine.<\/p>\n<p>Selles artiklis r\u00e4\u00e4gime sammudest, mida me astume, et \"v\u00e4lja pressida k\u00f5ik\" k\u00f5ige aeglasematest komponentidest algtaseme konfiguratsioonides ja m\u00f5\u00f5ta Kafka klastrite l\u00e4bilaskev\u00f5imet. K\u00f5rge vastupidavusega konfiguratsioon n\u00f5uab v\u00e4hemalt kolme t\u00f6\u00f6tavat maaklerit (<code>min.insync.replicas=3<\/code>), mis jagatakse kolme erineva k\u00e4ttesaadavuse tsooni. Kubernetes'i infrastruktuuri seadistamiseks, skaleerimiseks ja j\u00e4lgimiseks kasutame meie enda konteinerihaldusplatvormi h\u00fcbriidpilvede jaoks \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pipeline\">Pipeline<\/a><\/noindex>. See toetab on-premise (bare metal, VMware) ja viit t\u00fc\u00fcpi pilvi (Alibaba, AWS, Azure, Google, Oracle), samuti nende igasuguseid kombinatsioone.<\/p>\n<h2>M\u00f5tted Kafka infrastruktuuri ja klastrite konfiguratsiooni kohta<\/h2>\n<p>\nAllpool toodud n\u00e4idetes valisime AWS-i pilveteenuse pakkujaks ja EKS-i Kubernetes'i jaotuseks. Sarnast konfiguratsiooni saab rakendada, kasutades <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pke\">PKE<\/a><\/noindex> \u2014 Banzai Cloud'i Kubernetes'i jaotust, mis on CNCF-i poolt sertifitseeritud.<\/p>\n<h3>Kettaruum<\/h3>\n<p>\nAmazon pakub erinevaid <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/ebs-volume-types.html\">EBS-i mahtude t\u00fc\u00fcpe<\/a><\/noindex>. Aluseks <i>gp2<\/i> ja <i>io1<\/i> on SSD-diskid, kuid k\u00f5rge l\u00e4bilaskev\u00f5ime tagamiseks <i>gp2<\/i> kasutab kogutud krediiti <i>I\/O krediidi (I\/O credits)<\/i>, seega eelistasime t\u00fc\u00fcpi <i>io1<\/i>, mis pakub stabiilset k\u00f5rget l\u00e4bilaskev\u00f5imet.<\/p>\n<h3>Instantside t\u00fc\u00fcbid<\/h3>\n<p>\nKafka j\u00f5udlus s\u00f5ltub tugevalt operatsioonis\u00fcsteemi lehtede vahem\u00e4lust, seega vajame piisava m\u00e4luga instantsse, et toita brokereid (JVM) ja lehtede vahem\u00e4lu. Instants <i>c5.2xlarge<\/i> on hea algus, kuna sellel on 16 GB m\u00e4lu ja <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/ebs-optimized.html\">on optimeeritud EBS-i jaoks.<\/a><\/noindex>Selle puuduseks on see, et see suudab tagada maksimaalse j\u00f5udluse mitte kauem kui 30 minutit 24 tunni jooksul. Kui koormus n\u00f5uab maksimaalset j\u00f5udlust pikema aja jooksul, tuleks kaaluda muid instantside t\u00fc\u00fcpe. Just nii me tegime, valides <i>c5.4xlarge.<\/i>See tagab maksimaalse l\u00e4bilaskev\u00f5ime <b>593,75 MB\/s.<\/b>Maksimaalne EBS-i mahtu l\u00e4bilaskev\u00f5ime <i>io1<\/i> on suurem kui instantsil <i>c5.4xlarge.<\/i>, seega tundub, et infrastruktuuri k\u00f5ige aeglasem element on selle instantsi I\/O l\u00e4bilaskev\u00f5ime (mida peavad kinnitama ka meie koormustestide tulemused).<\/p>\n<h3>V\u00f5rk<\/h3>\n<p>\nV\u00f5rgu l\u00e4bilaskev\u00f5ime peaks olema piisavalt suur v\u00f5rreldes VM-i ja ketta j\u00f5udlusega, vastasel juhul muutub v\u00f5rk kitsaskohaks. Meie juhul toetab v\u00f5rgu liides <i>c5.4xlarge.<\/i> kiirus kuni 10 Gbps, mis on oluliselt suurem kui VM-i instantsi I\/O l\u00e4bilaskev\u00f5ime.<\/p>\n<h3>Brokerite juurutamine<\/h3>\n<p>\nBrokereid peavad kasutama (planeerima Kubernetesis) eraldatud s\u00f5lmi, et v\u00e4ltida konkurentsi teiste protsesside eest CPU, m\u00e4lu, v\u00f5rgu ja ketta ressursside osas.<\/p>\n<h3>Java versioon<\/h3>\n<p>\nLoogiliseks valikuks on Java 11, kuna see on koosk\u00f5las Dockeriga, nii et JVM m\u00e4\u00e4rab \u00f5igesti protsessorid ja m\u00e4lu, mis on konteineris, kus broker t\u00f6\u00f6tab, saadaval. Teades, et protsessorite piirangud on olulised, m\u00e4\u00e4rab JVM sisemiselt ja l\u00e4bipaistvalt GC ja JIT kompilaatori kiiruste arvu. Kasutasime Kafka pilti <code>banzaicloud\/kafka:2.13-2.4.0<\/code>, mis sisaldab Kafka versiooni 2.4.0 (Scala 2.13) Java 11 peal.<\/p>\n<blockquote><p>Kui soovite rohkem teada Java\/JVM kohta Kubernetesis, vaadake meie j\u00e4rgmisi v\u00e4ljaandeid:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/java-resource-limits\/\">Miks minu Java rakendus sai OOMKilled<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/java10-container-sizing\/\">Kuidas \u00f5igesti m\u00f5\u00f5ta konteinerite suurust Java 10 rakenduste jaoks<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/blockquote>\n<p><\/p>\n<h3>Brokereid m\u00e4lu seaded<\/h3>\n<p>\nOn kaht aspekti, mida tuleb arvestada vahendaja m\u00e4luseadistustes: JVM-i ja Kubernetes'e pod'i seaded. Pod'ile seatud m\u00e4lupiirang peab olema suurem kui maksimaalne heap'i suurus, et JVM-il j\u00e4\u00e4ks ruumi Java metaruumi jaoks, mis asub eraldi m\u00e4lus, ja operatsioonis\u00fcsteemi lehek\u00fcljekihile, mida Kafka aktiivselt kasutab. Meie testides k\u00e4ivitasime Kafka vahendajad j\u00e4rgmiste parameetritega <code>-Xmx4G -Xms2G<\/code>, ja pod'i m\u00e4lupiirang oli <code>10 Gi<\/code>. Pange t\u00e4hele, et JVM-i m\u00e4lu parameetrid saavad automaatselt tuvastada, kasutades <code>-XX:MaxRAMPercentage<\/code> ja <code>-X:MinRAMPercentage<\/code>, l\u00e4htudes pod'i m\u00e4lupiirangust.<\/p>\n<h3>Brokereid protsessori seaded<\/h3>\n<p>\n\u00dcldiselt on v\u00f5imalik t\u00f5sta j\u00f5udlust, suurendades paralleelsust, suurendades Kafka kasutatavate kiiruselade arvu. Mida rohkem protsessoreid on Kafka jaoks saadaval, seda parem. Meie testis alustasime 6 protsessori piiranguga ja t\u00f5stsime nende arvu j\u00e4rk-j\u00e4rgult (iteraatsioonide kaudu) kuni 15. Lisaks m\u00e4\u00e4rasime <code>num.network.threads=12<\/code> brokeri seadetes, et suurendada kiiruselade arvu, mis saavad andmeid v\u00f5rgust ja saadavad neid. Kohe m\u00e4rkasin, et sekundaarsed brokerid ei suuda repliike piisavalt kiiresti vastu v\u00f5tta, seet\u00f5ttu t\u00f5stsime <code>num.replica.fetchers<\/code> kuni 4, et suurendada kiirus, millega sekundaarsed brokerid replitseerivad s\u00f5numeid liidritelt.<\/p>\n<h3>Koormuse genereerimise t\u00f6\u00f6riist<\/h3>\n<p>\nOluline on veenduda, et validaud koormuse genereerimise t\u00f6\u00f6riist ei ammenduks enne, kui Kafka klaster (millel tehakse benchmark) saavutab oma maksimaalse koormuse. Teisis\u00f5nu, tuleb eelnevalt hinnata koormuse genereerimise t\u00f6\u00f6riista v\u00f5imekust ning valida selle jaoks piisava protsessori- ja m\u00e4luhulgaga instantsid. Sel viisil suudab meie t\u00f6\u00f6riist tekitada rohkem koormust, kui Kafka klaster suudab taluda. Paljude katsete j\u00e4rel valisime kolm instantsi <i>c5.4xlarge.<\/i>, millest iga\u00fches k\u00e4ivitus generaator.<\/p>\n<h2>J\u00f5udluse testimine<\/h2>\n<p>\nJ\u00f5udluse m\u00f5\u00f5tmine on iteratiivne protsess, mis h\u00f5lmab j\u00e4rgmisi etappe:<\/p>\n<ul>\n<li> infrastruktuuri (EKS klaster, Kafka klaster, koormusgeneraator ning Prometheus ja Grafana) seadistamine;<\/li>\n<li> koormuse genereerimine kindla ajaperioodi jooksul, et filtreerida v\u00e4lja juhuslikud k\u00f5ikumised kogutud j\u00f5udlusn\u00e4itajates;<\/li>\n<li> infrastruktuuri ja brokendi seadistuste kohandamine j\u00e4lgitud j\u00f5udlusn\u00e4itajate p\u00f5hjal;<\/li>\n<li> protsessi kordamine, kuni on saavutatud soovitud j\u00f5udluse tase Kafka klastris. Samuti peab see olema stabiilselt korduv ja n\u00e4itama minimaalset j\u00f5udluse k\u00f5ikumist.<\/li>\n<\/ul>\n<p>\nJ\u00e4rgmises osas on kirjeldatud samme, mida tehti testklastri j\u00f5udluse testimise protsessis.<\/p>\n<h3>T\u00f6\u00f6riistad<\/h3>\n<p>\nKiireks baasconfiguratsiooni seadistamiseks, koormuse genereerimiseks ja j\u00f5udluse m\u00f5\u00f5tmiseks kasutati j\u00e4rgmisi t\u00f6\u00f6riistu:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/pipeline\/overview\/\">Banzai Cloud Pipeline<\/a><\/noindex> Amazon EKS klastrite organiseerimiseks koos <noindex><a rel=\"nofollow\" href=\"https:\/\/prometheus.io\/\">Prometheus<\/a><\/noindex> (Kafka ja infrastruktuuri m\u00f5\u00f5dikute kogumiseks) ja <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/\">Grafana<\/a><\/noindex> (nende m\u00f5\u00f5dikute visualiseerimiseks). Kasutasime <b>integreeritud<\/b> ja <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pipeline\">Pipeline<\/a><\/noindex> teenuseid, mis pakuvad f\u00f6deraalset j\u00e4lgimist, keskset logide kogumist, haavatavuste skannimist, taastumist t\u00f5rgetest, ettev\u00f5tte taseme turvalisust ja palju muud.<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jamiealquiza\/sangrenel\">Sangrenel<\/a><\/noindex> on t\u00f6\u00f6riist Kafka klastri koormustestimiseks.<\/li>\n<li> Grafana paneelid Kafka ja infrastruktuuri m\u00f5\u00f5dikute visualiseerimiseks: <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/grafana\/dashboards\/10123\">Kubernetes Kafka<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/grafana\/dashboards\/1860\">Node Exporter<\/a><\/noindex>.<\/li>\n<li> Supertubes CLI maksimaalse lihtsuse saavutamiseks Kafka klastrite seadistamisel Kuberneteses. Zookeeper, Kafka operaator, Envoy ja mitmed teised komponendid on installitud ja korrektselt konfigureeritud produktsiooniks valmis Kafka klastriks Kuberneteses.\n<ul>\n<li> Paigaldamiseks <i>supertubes CLI<\/i> kasutage antud juhiseid <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/cli\/install\/\">siin<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/23ecf423b1ea66813406943ae7eabf61.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>EKS klaster<\/h3>\n<p>\nValmistage ette EKS klaster eraldatud t\u00f6\u00f6tlust\u00f6\u00f6tlus\u00fclesannetega <i>c5.4xlarge.<\/i> kolmes erinevas k\u00e4ttesaadavuse piirkonnas Kafka vahendajate pod'ide jaoks, samuti eraldatud s\u00f5lmed koormuse genereerijale ja monitooringu infrastruktuurile.<\/p>\n<pre><code class=\"bash\">banzai cluster create -f https:\/\/raw.githubusercontent.com\/banzaicloud\/kafka-operator\/master\/docs\/benchmarks\/infrastructure\/cluster_eks_202001.json<\/code><\/pre>\n<p>\nKui EKS klaster t\u00f6\u00f6tab, aktiviseerige selle integreeritud <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/pipeline\/features\/integrated-services\/\">j\u00e4lgimisteenus<\/a><\/noindex> \u2014 see paigutab Prometheuse ja Grafana klastrisse.<\/p>\n<h3>Kafka s\u00fcsteemikomponendid<\/h3>\n<p>\nPaigaldage Kafka s\u00fcsteemikomponendid (Zookeeper, kafka-operaator) EKS-i supertubes CLI abil:<\/p>\n<pre><code class=\"bash\">supertubes install -a --no-democluster --kubeconfig<\/code><\/pre>\n<p><\/p>\n<h3>Kafka klaster<\/h3>\n<p>\nEKS-is kasutatakse vaikimisi EBS t\u00fc\u00fcpi mahuteid, <i>gp2<\/i>seega tuleb luua eraldi salvestusklass mahutite jaoks <i>io1<\/i> Kafka klastrile:<\/p>\n<pre><code class=\"plaintext\">kubectl create -f - &lt;&lt;EOF\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n  name: fast-ssd\nprovisioner: kubernetes.io\/aws-ebs\nparameters:\n  type: io1\n  iopsPerGB: &quot;50&quot;\n  fsType: ext4\nvolumeBindingMode: WaitForFirstConsumer\nEOF<\/code><\/pre>\n<p>\nM\u00e4\u00e4rake vahendajatele parameeter <code>min.insync.replicas=3<\/code> ja juurutage vahendaja pod'id s\u00f5lmedesse kolmes erinevas k\u00e4ttesaadavuse piirkonnas:<\/p>\n<pre><code class=\"bash\">supertubes cluster create -n kafka --kubeconfig  -f https:\/\/raw.githubusercontent.com\/banzaicloud\/kafka-operator\/master\/docs\/benchmarks\/infrastructure\/kafka_202001_3brokers.yaml --wait --timeout 600<\/code><\/pre>\n<p><\/p>\n<h3>Teemad<\/h3>\n<p>\nK\u00e4ivitame samaaegselt kolm koormuse genereerimise eksemplari. Iga\u00fcks kirjutab oma teemasse, st kokku on meil vaja kolme teemat:<\/p>\n<pre><code class=\"bash\">supertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n  name: perftest1\nspec:\n  name: perftest1\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF\n\nsupertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n    name: perftest2\nspec:\n  name: perftest2\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF\n\nsupertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n  name: perftest3\nspec:\n  name: perftest3\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF<\/code><\/pre>\n<p>\nIga teema replikatsioonifaktor on 3 \u2014 minimaalne soovitatud v\u00e4\u00e4rtus k\u00f5rge k\u00e4ttesaadavusega produktsioonis\u00fcsteemide jaoks.<\/p>\n<h3>Koormuse genereerimise t\u00f6\u00f6riist<\/h3>\n<p>\nK\u00e4ivitasime kolm koormuse genereerimise instantsit (iga\u00fchel eraldi teema kirjutamine). Koormuse genereerimise pod'ide puhul tuleb m\u00e4\u00e4rata node affinity, et need planeeritaks ainult neile eraldatud s\u00f5lmedesse:<\/p>\n<pre><code class=\"plaintext\">apiVersion: extensions\/v1beta1\nkind: Deployment\nmetadata:\n  labels:\n    app: loadtest\n  name: perf-load1\n  namespace: kafka\nspec:\n  progressDeadlineSeconds: 600\n  replicas: 1\n  revisionHistoryLimit: 10\n  selector:\n    matchLabels:\n      app: loadtest\n  strategy:\n    rollingUpdate:\n      maxSurge: 25%\n      maxUnavailable: 25%\n    type: RollingUpdate\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: loadtest\n    spec:\n      affinity:\n        nodeAffinity:\n          requiredDuringSchedulingIgnoredDuringExecution:\n            nodeSelectorTerms:\n            - matchExpressions:\n              - key: nodepool.banzaicloud.io\/name\n                operator: In\n                values:\n                - loadgen\n      containers:\n      - args:\n        - -brokers=kafka-0:29092,kafka-1:29092,kafka-2:29092,kafka-3:29092\n        - -topic=perftest1\n        - -required-acks=all\n        - -message-size=512\n        - -workers=20\n        image: banzaicloud\/perfload:0.1.0-blog\n        imagePullPolicy: Always\n        name: sangrenel\n        resources:\n          limits:\n            cpu: 2\n            memory: 1Gi\n          requests:\n            cpu: 2\n            memory: 1Gi\n        terminationMessagePath: \/dev\/termination-log\n        terminationMessagePolicy: File\n      dnsPolicy: ClusterFirst\n      restartPolicy: Always\n      schedulerName: default-scheduler\n      securityContext: {}\n      terminationGracePeriodSeconds: 30<\/code><\/pre>\n<p>\nM\u00f5ned punktid, millele tasub t\u00e4helepanu p\u00f6\u00f6rata:<\/p>\n<ul>\n<li> Koormuse generaator genereerib 512 baitiseid s\u00f5numeid ja avaldab need Kafka-s 500 s\u00f5numi kaupa.<\/li>\n<li> Argumentidega <code>-required-acks=all<\/code> loetakse avaldamine edukaks, kui k\u00f5ik s\u00fcnkroonitud s\u00f5numi koopiad on saadud ja kinnitatud Kafka brokerite poolt. See t\u00e4hendab, et m\u00f5\u00f5dsime testis mitte ainult liidrite kiirus, kellel on s\u00f5numid, vaid ka nende j\u00e4rglaste kiirus, kes kopeerivad s\u00f5numeid. K\u00f5nealuse testi eesm\u00e4rk ei ole tarbijate <i>(consumers)<\/i> \u00e4rak\u00e4idud s\u00f5numite lugemise kiirus, mis on praegu OS-i lehe vahemikus, ja selle v\u00f5rdlemine s\u00f5numite lugemise kiirusest, mis on salvestatud kettale.<\/li>\n<li> Koormuse genereerija k\u00e4ivitab paralleelselt 20 t\u00f6\u00f6tajat (<code>-workers=20<\/code>). Iga t\u00f6\u00f6taja sisaldab 5 tootjat, kes jagavad t\u00f6\u00f6taja \u00fchendust Kafka klastriga. Seega on igal koormuse genereerijal 100 tootjat, k\u00f5ik saadavad s\u00f5numeid Kafka klastrisse.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Klastri oleku j\u00e4lgimine<\/h3>\n<p>\nKoormustestimise k\u00e4igus j\u00e4lgisime ka Kafka klastri tervist, et veenduda, et pod'e ei taask\u00e4ivitata, replikaid ei ole des\u00fcnkroniseeritud ning maksimaalne l\u00e4bilaskev\u00f5ime oleks minimaalsete k\u00f5ikumistega:<\/p>\n<ul>\n<li> Koormuse generaator salvestab standardseid statistikaid avaldatud s\u00f5numite arvu ja vigade taseme kohta. Vigade protsent peab j\u00e4\u00e4ma <code>0,00%<\/code>.<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/linkedin\/cruise-control\">Cruise Control<\/a><\/noindex>, lahtise kafka-operatoriga, pakub j\u00e4lgimisv\u00f5imet, mille kaudu saame j\u00e4lgida klastrite seisundit. Selle paneeli vaatamiseks tegutsege:\n<pre><code class=\"bash\">supertubes cluster cruisecontrol show -n kafka --kubeconfig<\/code><\/pre>\n<\/li>\n<li> ISR tase <i>(s\u00fcnkroonitud koopiate arv)<\/i> shrink ja expansion on null.<\/li>\n<\/ul>\n<p><\/p>\n<h2>M\u00f5\u00f5tmistulemused<\/h2>\n<p><\/p>\n<h3>3 brokereid, s\u00f5numite suurus - 512 baiti<\/h3>\n<p>\nJaotustega, mis on \u00fchtlaselt jaotatud kolme brokermi vahel, suutsime saavutada tulemuslikkuse <i>~500 Mb\/s (umbes 990 tuhat s\u00f5numit sekundis)<\/i>:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/df65ff6a1a26e28d5e5a7a5ab1a20d98.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/15c5883c0056c651f0a0bd14e9991967.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/9b708fa5ce809b0d059cba9f5edfbb2f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJVM virtuaalmasina m\u00e4lu tarbimine ei \u00fcletanud 2 Gb:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/3320e2eea891055cc0935da492d18ee9.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/6e3976c310309aa8be45fa3fdf547750.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/94baa1b689ae854693fb7cc4ed6e105e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKettal\u00e4bilaskev\u00f5ime saavutas k\u00f5igil kolmel instantsil brokerite t\u00f6\u00f6tamisel maksimumi I\/O kaudu:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/408139ad33cda1294755660df1c128a8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/391d2df0ddfe82f94abdb200370ff775.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/e7e9fd021a2cdc71992ce672363a78ec.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNode\u2019de m\u00e4lu kasutuse andmete p\u00f5hjal vajusid s\u00fcsteemi puhverdamine ja vahem\u00e4lu umbes 10-15 Gb:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/781887507e01a4903b86871961ff9260.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/63f10177f01342039f04192ea93c8bca.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/48b992cc73a968b3d3b1442f96562c4e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>3 brokereid, s\u00f5numite suurus - 100 baiti<\/h3>\n<p>\nS\u00f5numite suuruse v\u00e4henemisega kukub l\u00e4bilaskev\u00f5ime umbes 15-20%: see m\u00f5jutab aega, mis kulub iga s\u00f5numi t\u00f6\u00f6tlemiseks. Lisaks kasvas protsessori koormus peaaegu kahekordseks.<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/2f7dc5a94da371cfe17c9e0fc0ec1941.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/986160a3a4e099a0dbb8aeefaf6f1c2f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/c08244457bf4dbfadf52838d4ea2969e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKuna brokeri s\u00f5lmedes on endiselt kasutamata tuumikud, saab Kafka konfiguratsiooni muutmisega j\u00f5udlust parandada. See ei ole lihtne \u00fclesanne, seet\u00f5ttu on parem saavutada suuremat l\u00e4bilaskev\u00f5imet suuremate s\u00f5numitega.<\/p>\n<h3>4 brokereid, s\u00f5numite suurus - 512 baiti<\/h3>\n<p>\nKliendi soovitud Kafka klastrite j\u00f5udlust saab lihtsalt suurendada, lisades uusi brokereid ja s\u00e4ilitades jaotuste tasakaalu (see tagab koormuse \u00fchtlase jaotamise brokermi vahel). Meie puhul, p\u00e4rast brokerni lisamist, suurenes klastrite l\u00e4bilaskev\u00f5ime kuni <i>~580 Mb\/s (~1,1 miljon s\u00f5numit sekundis)<\/i>. Kasv osutus oodatust v\u00e4iksemaks: seda seletab peamiselt jaotuste tasakaalu puudumine (k\u00f5ik brokerman ei t\u00f6\u00f6ta tipptasemel).<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/8c8d2e70739add3c3f88fe42acf98c46.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/a6e4812270c04487de0ad39b996a13f8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/6162c04cfb87b9d9cd0f3147a4ae1647.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/dae5de3b14be212c679f3627a3b77d7b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJVM-i m\u00e4lutarbimine j\u00e4i alla 2 GB:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/0bca07a817d7da3104b08de10979f049.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/7e1e64f80ec140f13e171cfe3be930ce.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/5e9700b1e54e45f9cd5fa018ec07dba1.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/4ba0c4b59bfe72bcb0d2f22056c47d79.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nBrokermide t\u00f6\u00f6 m\u00f5ju on t\u00f5mmanud jaotuste tasakaalu puudus:<\/p>\n<p><img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/2b8ed3c21adf70c751d1daa6d4c41895.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/77b980869b99903d9de5cd7d43d79291.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/f2f9569a3a1ef0ce35e89b2e52e4a8b2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses\" src=\"\/wp-content\/uploads\/2020\/02\/191d6d97b32b2128c5cfd3914171de15.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h2>J\u00e4reldused<\/h2>\n<p>\n\u00dclal esitletud iteratiivset l\u00e4henemisviisi saab laiendada, et h\u00f5lmata keerukamaid stsenaariume, mis h\u00f5lmavad sadu tarbijaid, jaotuste \u00fcmbersuunamist, rullides uuendusi, podide taask\u00e4ivitamist jne. See k\u00f5ik v\u00f5imaldab meil hinnata Kafka klastrite v\u00f5imeid erinevates tingimustes, tuvastada kitsaskohti ja leida v\u00f5imalusi nende \u00fcletamiseks.<\/p>\n<p>Oleme v\u00e4lja t\u00f6\u00f6tanud Supertubes, et kiiresti ja lihtsalt klastrit \u00fcles seadma, konfigureerima, brokereid ja teemasid lisama\/eemaldama, teavitustele vastama ning tagama Kafka \u00f5ige toimimise Kubernetesis. Meie eesm\u00e4rk on aidata keskenduda p\u00f5hitegevusele (\"toota\" ja \"tarbida\" Kafka s\u00f5numeid), samas kui kogu raske t\u00f6\u00f6 j\u00e4tame Supertubesile ja Kafka operaatorile.<\/p>\n<p>Kui olete huvitatud tehnoloogiatest ja Banzai Cloudi avatud l\u00e4htekoodiga projektidest, siis j\u00e4lgige meid <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\">GitHub<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/www.linkedin.com\/company\/banzaicloud\">LinkedIn<\/a><\/noindex> v\u00f5i <noindex><a rel=\"nofollow\" href=\"https:\/\/twitter.com\/BanzaiCloud\">Twitteris<\/a><\/noindex>.<\/p>\n<h2>P.S. t\u00f5lkijalt<\/h2>\n<p>\nLugege ka meie blogist:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/480722\/\">\u00dcks lugu Redis'i operaatorist K8s-is ja mini\u00fclevaade selle andmebaasi anal\u00fc\u00fcsi t\u00f6\u00f6riistadest<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/450662\/\">Kompleksne RabbitMQ migratsioon Kubernetesesse<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/329224\/\">zetcd CoreOS'ilt: ZooKeeper asendamine... etcd salvestusega<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Allikas: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/488920\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes. \u041f\u0440\u0438\u0432\u043e\u0434\u0438\u043c\u044b\u0435 \u0438\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0438 \u0438\u043b\u043b\u044e\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u043a\u0430\u043a \u043c\u043e\u0436\u043d\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0438\u0442\u044c \u043e\u043f\u0442\u0438\u043c\u0430\u043b\u044c\u043d\u044b\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u044b \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0441\u0430\u043c\u0443 Kafka \u0434\u043b\u044f \u0434\u043e\u0441\u0442\u0438\u0436\u0435\u043d\u0438\u044f \u0442\u0440\u0435\u0431\u0443\u0435\u043c\u043e\u0439 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u0438. Apache Kafka \u2014 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u0430\u044f \u0441\u0442\u0440\u0438\u043c\u0438\u043d\u0433\u043e\u0432\u0430\u044f \u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0430 \u0434\u043b\u044f \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043d\u0430\u0434\u0451\u0436\u043d\u044b\u0445, \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u0443\u0435\u043c\u044b\u0445 \u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u043f\u043e\u0442\u043e\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0440\u0435\u0430\u043b\u044c\u043d\u043e\u0433\u043e \u0432\u0440\u0435\u043c\u0435\u043d\u0438. [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":71214,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-71213","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"et_EE\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u043c \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0449\u0438\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 Kafka \u0432 Kubernetes | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-24T13:09:17+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-03-03T13:14:29+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47M\u00e4\u00e4rame sobiva suuruse Kafka klastrile Kuberneteses | ProHoster","description":"M\u00e4rkus t\u00f5lkijalt: Selles artiklis jagab Banzai Cloud oma spetsiaalsete utiliitide kasutamise n\u00e4idet, et lihtsustada Kafka haldamist Kuberneteses.","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u043c \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0449\u0438\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 Kafka \u0432 Kubernetes | ProHoster","og:description":"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-24T13:09:17+00:00","article:modified_time":"2020-03-03T13:14:29+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"71213","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 19:06:22","updated":"2022-10-04 20:58:00","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/71213","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=71213"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/71213\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/71214"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=71213"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=71213"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=71213"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}