{"id":71213,"date":"2020-02-24T16:09:17","date_gmt":"2020-02-24T13:09:17","guid":{"rendered":"https:\/\/prohoster.info\/blog\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes"},"modified":"2020-03-03T16:14:29","modified_gmt":"2020-03-03T13:14:29","slug":"opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","title":{"rendered":"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><i><b>Nota traduc\u0103torului.<\/b>: \u00cen acest articol, compania Banzai Cloud \u00eemp\u0103rt\u0103\u0219e\u0219te un exemplu de utilizare a utilitarelor sale speciale pentru a facilita operarea Kafka \u00een cadrul Kubernetes. Instruc\u021biunile prezentate ilustreaz\u0103 cum se poate determina dimensiunea optim\u0103 a infrastructurii \u0219i configura Kafka pentru a atinge capacitatea de livrare dorit\u0103.<\/i><\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/8a598ec7db091c2944a6442f9bc124da.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nApache Kafka este o platform\u0103 de streaming distribuit\u0103 pentru crearea de sisteme de streaming de realitate fiabile, scalabile \u0219i cu performan\u021be ridicate. Capabilit\u0103\u021bile sale impresionante pot fi extinse cu ajutorul Kubernetes. Pentru aceasta, am dezvoltat <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/kafka-operator\">operatorul Kafka Open Source<\/a><\/noindex> \u0219i un instrument numit <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/overview\/\">Supertubes<\/a><\/noindex>. Acestea permit rularea Kafka \u00een Kubernetes \u0219i utilizarea diverselor sale func\u021bii, cum ar fi configurarea fin\u0103 a brokerului, scalarea pe baza metricilor cu rebalansare, rack awareness (con\u0219tientizarea resurselor hardware), actualiz\u0103ri cu <i>(graceful)<\/i> \u0219i multe altele.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<blockquote><p>\u00cencerca\u021bi Supertubes \u00een clusterul dumneavoastr\u0103:<\/p>\n<pre><code class=\"bash\">curl https:\/\/getsupertubes.sh | sh \u0219i supertubes install -a --no-democluster --kubeconfig<\/code><\/pre>\n<p>\nSau contacta\u021bi <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/cli\/reference\/\">documentation<\/a><\/noindex>. De asemenea, pute\u021bi citi despre unele dintre func\u021bionalit\u0103\u021bile Kafka, a c\u0103ror operare este automatizat\u0103 cu ajutorul Supertubes \u0219i al operatorului Kafka. Despre acestea am scris deja \u00een blog:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-operator\/\">Oh nu! \u00cenc\u0103 un operator Kafka pentru Kubernetes<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-alert\/\">Monitoriza\u021bi \u0219i opera\u021bi Kafka pe baza metricilor Prometheus<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-rack-awareness\/\">Con\u0219tientizarea rack-ului Kafka \u00een Kubernetes<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-on-istio-performance\/\">Rularea Apache Kafka peste Istio - benchmark<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-topic-user-management\/\">Clustere utilizator authenticate \u0219i controlate cu operatorul Kafka<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-rolling-upgrade\/\">Actualizarea continu\u0103 Kafka \u0219i configura\u021bia dinamic\u0103 \u00een Kubernetes<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/kafka-envoy-protocol-filter\/\">Filtru de protocol Envoy pentru Kafka, meshed<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/blockquote>\n<p>\nDeci, dac\u0103 a\u021bi decis s\u0103 dezv\u0103lui\u021bi un cluster Kafka \u00een Kubernetes, cu siguran\u021b\u0103 ve\u021bi \u00eent\u00e2mpina problema determin\u0103rii dimensiunii optime a infrastructurii de baz\u0103 \u0219i a necesit\u0103\u021bii unei configur\u0103ri fine a Kafka pentru a satisface cerin\u021bele de capacitate. Performan\u021ba maxim\u0103 a fiec\u0103rui broker este determinat\u0103 de performan\u021ba componentelor infrastructurii pe care se bazeaz\u0103, cum ar fi memorie, procesor, vitez\u0103 a discului, l\u0103\u021bimea de band\u0103 a re\u021belei etc.<\/p>\n<p>\u00cen ideal, configura\u021bia brokerului ar trebui s\u0103 fie astfel \u00eenc\u00e2t toate elementele infrastructurii s\u0103 fie utilizate la maximum capacit\u0103\u021bii lor. Totu\u0219i, \u00een via\u021ba real\u0103, o astfel de setare este destul de complicat\u0103. Mai probabil, utilizatorii vor configura brokerii pentru a maximiza utilizarea unuia sau dou\u0103 componente (discuri, memorie sau procesor). \u00cen general, brokerul arat\u0103 cea mai bun\u0103 performan\u021b\u0103 atunci c\u00e2nd configura\u021bia sa permite utilizarea complet\u0103 a celui mai lent component. Astfel, putem ob\u021bine o idee aproximativ\u0103 despre sarcina pe care un broker o poate gestiona.<\/p>\n<p>Teoretic, putem estima, de asemenea, num\u0103rul de brokeri necesari pentru a gestiona o \u00eenc\u0103rcare dat\u0103. Cu toate acestea, \u00een practic\u0103, op\u021biunile de configurare la diferite niveluri sunt at\u00e2t de multe \u00eenc\u00e2t evaluarea poten\u021bialei performan\u021be a unei configura\u021bii devine foarte complicat\u0103 (dac\u0103 nu imposibil\u0103). Cu alte cuvinte, este foarte greu s\u0103 planific\u0103m o configura\u021bie baz\u00e2ndu-ne pe o anumit\u0103 performan\u021b\u0103 dat\u0103.<\/p>\n<p>Pentru utilizatorii Supertubes, de obicei aplic\u0103m urm\u0103toarea abordare: \u00eencepem cu o anumit\u0103 configura\u021bie (infrastructur\u0103 + set\u0103ri), apoi m\u0103sur\u0103m performan\u021ba acesteia, ajust\u0103m set\u0103rile brokerului \u0219i repet\u0103m procesul \u00eenc\u0103 o dat\u0103. Acest lucru se \u00eent\u00e2mpl\u0103 p\u00e2n\u0103 c\u00e2nd poten\u021bialul celui mai lent component al infrastructurii este complet utilizat.<\/p>\n<p>\u00cen acest mod, ob\u021binem o idee mai clar\u0103 despre c\u00e2\u021bi brokeri sunt necesari unui cluster pentru a face fa\u021b\u0103 unei anumite \u00eenc\u0103rc\u0103ri (num\u0103rul de brokeri depinde, de asemenea, de al\u021bi factori, cum ar fi num\u0103rul minim de replici pentru a asigura rezilien\u021ba, num\u0103rul de lideri de partition etc.). \u00cen plus, ob\u021binem o perspectiv\u0103 asupra celui mai dorit component de infrastructur\u0103 pentru scalarea vertical\u0103.<\/p>\n<p>\u00cen acest articol, vom discuta despre pa\u0219ii pe care \u00eei facem pentru a \u201escoate totul\u201d din cele mai lente componente \u00een configura\u021biile ini\u021biale \u0219i a m\u0103sura l\u0103\u021bimea de band\u0103 a clusterului Kafka. O configura\u021bie rezistent\u0103 necesit\u0103 cel pu\u021bin trei brokeri func\u021bionali (<code>min.insync.replicas=3<\/code>), distribuite pe trei zone de disponibilitate diferite. Pentru configurarea, scalarea \u0219i monitorizarea infrastructurii Kubernetes, folosim propria platform\u0103 de gestionare a containerelor pentru clouduri hibride \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pipeline\">Pipeline<\/a><\/noindex>. Aceasta suport\u0103 on-premise (bare metal, VMware) \u0219i cinci tipuri de clouduri (Alibaba, AWS, Azure, Google, Oracle), precum \u0219i orice combina\u021bii ale acestora.<\/p>\n<h2>G\u00e2nduri despre infrastructura \u0219i configurarea clusterului Kafka<\/h2>\n<p>\nPentru exemplele de mai jos, am ales AWS ca furnizor de servicii cloud \u0219i EKS ca distribu\u021bie Kubernetes. O configurare similar\u0103 poate fi realizat\u0103 cu ajutorul <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pke\">PKE<\/a><\/noindex> \u2014 distribu\u021bia Kubernetes de la Banzai Cloud, certificat CNCF.<\/p>\n<h3>Disc<\/h3>\n<p>\nAmazon ofer\u0103 diverse <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/ebs-volume-types.html\">tipuri de volume EBS<\/a><\/noindex>. La baza <i>gp2<\/i> \u0219i <i>io1<\/i> se afl\u0103 SSD-uri, totu\u0219i pentru a asigura un randament mare <i>gp2<\/i> consum\u0103 credite I\/O <i>(I\/O credits)<\/i>, de aceea am preferat un tip <i>io1<\/i>, care ofer\u0103 un randament stabil.<\/p>\n<h3>Tipuri de instan\u021be<\/h3>\n<p>\nPerforman\u021ba Kafka depinde mult de cache-ul pe pagin\u0103 al sistemului de operare, de aceea avem nevoie de instan\u021be cu suficient\u0103 memorie pentru brokeri (JVM) \u0219i cache-ul pe pagin\u0103. Instan\u021ba <i>c5.2xlarge<\/i> \u2014 este un \u00eenceput bun, av\u00e2nd 16 GB de memorie \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/ebs-optimized.html\">optimizat\u0103 pentru EBS<\/a><\/noindex>. Dezavantajul s\u0103u este c\u0103 poate furniza performan\u021be maxime timp de nu mai mult de 30 de minute \u00eentr-o perioad\u0103 de 24 de ore. Dac\u0103 sarcina de lucru necesit\u0103 performan\u021be maxime pentru o perioad\u0103 mai lung\u0103, ar trebui s\u0103 ne uit\u0103m la alte tipuri de instan\u021be. Exact a\u0219a am procedat, aleg\u00e2nd <i>c5.4xlarge<\/i>. Aceasta ofer\u0103 un randament maxim de <b>593,75 MB\/s<\/b>. Randamentul maxim al volumului EBS <i>io1<\/i> este mai mare dec\u00e2t al instan\u021bei <i>c5.4xlarge<\/i>, astfel \u00eenc\u00e2t cel mai lent element al infrastructurii pare a fi randamentul I\/O al acestui tip de instan\u021b\u0103 (ceea ce ar trebui s\u0103 confirme \u0219i rezultatele testelor noastre de \u00eenc\u0103rcare).<\/p>\n<h3>Re\u021bea<\/h3>\n<p>\nRandamentul re\u021belei trebuie s\u0103 fie suficient de mare comparativ cu performan\u021ba instan\u021bei VM \u0219i a discului, altfel re\u021beaua devine un punct nevralgic. \u00cen cazul nostru, interfa\u021ba de re\u021bea <i>c5.4xlarge<\/i> suport\u0103 viteze de p\u00e2n\u0103 la 10 Gb\/s, ceea ce este semnificativ mai mult dec\u00e2t randamentul I\/O al instan\u021bei VM.<\/p>\n<h3>Implementarea brokerilor<\/h3>\n<p>\nBrokerii trebuie s\u0103 fie desf\u0103\u0219ura\u021bi (planifica\u021bi \u00een Kubernetes) pe noduri dedicate pentru a evita competi\u021bia cu alte procese pentru resursele de procesor, memorie, re\u021bea \u0219i disc.<\/p>\n<h3>Versiunea Java<\/h3>\n<p>\nAlegerea logic\u0103 este Java 11, deoarece este compatibil\u0103 cu Docker \u00een sensul c\u0103 JVM determin\u0103 corect procesoarele \u0219i memoria disponibile pentru containerul \u00een care ruleaz\u0103 brokerul. \u0218tiind c\u0103 limitele procesorului sunt importante, JVM stabile\u0219te intern \u0219i transparent num\u0103rul de fire GC \u0219i fire de compilare JIT. Am folosit imaginea Kafka <code>banzaicloud\/kafka:2.13-2.4.0<\/code>, incluz\u00e2nd versiunea Kafka 2.4.0 (Scala 2.13) pe Java 11.<\/p>\n<blockquote><p>Dac\u0103 dori\u021bi s\u0103 afla\u021bi mai multe despre Java\/JVM pe Kubernetes, consulta\u021bi publica\u021biile noastre urm\u0103toare:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/java-resource-limits\/\">De ce aplica\u021bia mea Java este OOMKilled<\/a><\/noindex>;<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/blog\/java10-container-sizing\/\">Cum s\u0103 dimensiona\u021bi corect containerele pentru aplica\u021biile Java 10<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/blockquote>\n<p><\/p>\n<h3>Set\u0103ri de memorie pentru broker<\/h3>\n<p>\nExist\u0103 dou\u0103 aspecte cheie \u00een configurarea memoriei brokerului: set\u0103rile pentru JVM \u0219i pentru pod-ul Kubernetes. Limita de memorie stabilit\u0103 pentru pod trebuie s\u0103 fie mai mare dec\u00e2t dimensiunea maxim\u0103 a heap-ului, astfel \u00eenc\u00e2t s\u0103 existe suficient spa\u021biu pentru metaspatiul Java, care se afl\u0103 \u00een memoria proprie, \u0219i pentru cache-ul pe pagin\u0103 al sistemului de operare, pe care Kafka \u00eel folose\u0219te activ. \u00cen testele noastre, am rulat brokeri Kafka cu parametrii <code>-Xmx4G -Xms2G<\/code>, iar limita de memorie pentru pod era <code>10 Gi<\/code>. Re\u021bine\u021bi c\u0103 set\u0103rile de memorie pentru JVM pot fi ob\u021binute automat prin <code>-XX:MaxRAMPercentage<\/code> \u0219i <code>-X:MinRAMPercentage<\/code>, pe baza limitei de memorie pentru pod.<\/p>\n<h3>Set\u0103rile de procesor ale brokerului<\/h3>\n<p>\n\u00cen general, performan\u021ba poate fi \u00eembun\u0103t\u0103\u021bit\u0103 prin cre\u0219terea paralelismului prin cre\u0219terea num\u0103rului de fire utilizate de Kafka. Cu c\u00e2t mai multe procesoare sunt disponibile pentru Kafka, cu at\u00e2t mai bine. \u00cen testul nostru, am \u00eenceput cu o limit\u0103 de 6 procesoare \u0219i treptat (prin itera\u021bii) am crescut num\u0103rul lor la 15. \u00cen plus, am setat <code>num.network.threads=12<\/code> \u00een set\u0103rile brokerului, pentru a cre\u0219te num\u0103rul de fire care primesc date din re\u021bea \u0219i le trimit. Imediat ce am observat c\u0103 brokerii de replicare nu pot primi replicile suficient de repede, am crescut <code>num.replica.fetchers<\/code> la 4, pentru a cre\u0219te viteza cu care brokerii de replicare replicau mesajele de la lideri.<\/p>\n<h3>Instrument pentru generarea sarcinii<\/h3>\n<p>\nEste necesar s\u0103 ne asigur\u0103m c\u0103 poten\u021bialul generatorului de sarcin\u0103 ales nu se epuizeaz\u0103 p\u00e2n\u0103 c\u00e2nd clusterul Kafka (pentru care se efectueaz\u0103 benchmark-ul) atinge sarcina sa maxim\u0103. Cu alte cuvinte, este necesar s\u0103 evalu\u0103m \u00een avans capacit\u0103\u021bile instrumentului de generare a sarcinii \u0219i s\u0103 alegem tipuri de instan\u021be cu un num\u0103r suficient de procesoare \u0219i memorie. \u00cen acest caz, instrumentul nostru va produce o sarcin\u0103 mai mare dec\u00e2t poate gestiona clusterul Kafka. Dup\u0103 numeroase experimente, ne-am oprit la trei instan\u021be <i>c5.4xlarge<\/i>, fiecare dintre acestea av\u00e2nd un generator pornit.<\/p>\n<h2>Benchmarking<\/h2>\n<p>\nM\u0103surarea performan\u021bei este un proces iterativ care include urm\u0103toarele etape:<\/p>\n<ul>\n<li> configurarea infrastructurii (clusterul EKS, clusterul Kafka, instrumentul de generare a sarcinii, precum \u0219i Prometheus \u0219i Grafana);<\/li>\n<li> generarea sarcinii pe o perioad\u0103 determinat\u0103 pentru a filtra abaterile aleatorii din metricile de performan\u021b\u0103 colectate;<\/li>\n<li> ajustarea infrastructurii \u0219i configura\u021biei brokerului pe baza metricilor de performan\u021b\u0103 observate;<\/li>\n<li> repetarea procesului p\u00e2n\u0103 c\u00e2nd se atinge nivelul dorit de capacitate de band\u0103 al clusterului Kafka. Acesta trebuie s\u0103 fie stabil \u0219i s\u0103 demonstreze varia\u021bii minime ale capacit\u0103\u021bii de band\u0103.<\/li>\n<\/ul>\n<p>\n\u00cen sec\u021biunea urm\u0103toare sunt descri\u0219i pa\u0219ii efectua\u021bi \u00een timpul benchmark-ului clusterului de test.<\/p>\n<h3>Instrumente<\/h3>\n<p>\nPentru desf\u0103\u0219urarea rapid\u0103 a unei configura\u021bii de baz\u0103, generarea sarcinii \u0219i m\u0103surarea performan\u021bei s-au folosit urm\u0103toarele instrumente:<\/p>\n<ul>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/pipeline\/overview\/\">Banzai Cloud Pipeline<\/a><\/noindex> pentru organizarea clusterului EKS de la Amazon \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/prometheus.io\/\">Prometheus<\/a><\/noindex> (pentru colectarea metricilor Kafka \u0219i infrastructurii) \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/\">Grafana<\/a><\/noindex> (pentru vizualizarea acestor metrici). Am folosit <b>servicii integrate<\/b> \u00een <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\/pipeline\">Pipeline<\/a><\/noindex> care ofer\u0103 monitorizare federativ\u0103, colectare centralizat\u0103 a jurnalelor, scanare a vulnerabilit\u0103\u021bilor, recuperare \u00een caz de e\u0219ec, securitate la nivel de \u00eentreprindere \u0219i multe altele.<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jamiealquiza\/sangrenel\">Sangrenel<\/a><\/noindex> este un instrument pentru testarea de sarcin\u0103 a clusterului Kafka.<\/li>\n<li> Panourile Grafana pentru vizualizarea metricilor Kafka \u0219i infrastructurii: <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/grafana\/dashboards\/10123\">Kubernetes Kafka<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/grafana.com\/grafana\/dashboards\/1860\">Node Exporter<\/a><\/noindex>.<\/li>\n<li> Supertubes CLI pentru configurarea extrem de simpl\u0103 a unui cluster Kafka \u00een Kubernetes. Zookeeper, operator Kafka, Envoy \u0219i multe alte componente sunt instalate \u0219i configurate corect pentru a rula un cluster Kafka gata de produc\u021bie \u00een Kubernetes.\n<ul>\n<li> Pentru instalare <i>supertubes CLI<\/i> urmeaz\u0103 instruc\u021biunile furnizate <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/supertubes\/cli\/install\/\">aici<\/a><\/noindex>.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/23ecf423b1ea66813406943ae7eabf61.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Cluster EKS<\/h3>\n<p>\nPreg\u0103te\u0219te clusterul EKS cu noduri de lucru dedicate <i>c5.4xlarge<\/i> \u00een diferite zone de disponibilitate pentru pod-urile cu brokeri Kafka, precum \u0219i noduri dedicate pentru generatorul de sarcin\u0103 \u0219i infrastructura de monitorizare.<\/p>\n<pre><code class=\"bash\">banzai cluster create -f https:\/\/raw.githubusercontent.com\/banzaicloud\/kafka-operator\/master\/docs\/benchmarks\/infrastructure\/cluster_eks_202001.json<\/code><\/pre>\n<p>\nC\u00e2nd clusterul EKS este opera\u021bional, activeaz\u0103 serviciul s\u0103u integrat <noindex><a rel=\"nofollow\" href=\"https:\/\/banzaicloud.com\/docs\/pipeline\/features\/integrated-services\/\">de monitorizare<\/a><\/noindex> \u2014 acesta va lansa Prometheus \u0219i Grafana \u00een cluster.<\/p>\n<h3>Componentele de sistem Kafka<\/h3>\n<p>\nInstaleaz\u0103 componentele de sistem Kafka (Zookeeper, kafka-operator) \u00een EKS folosind supertubes CLI:<\/p>\n<pre><code class=\"bash\">supertubes install -a --no-democluster --kubeconfig<\/code><\/pre>\n<p><\/p>\n<h3>Cluster Kafka<\/h3>\n<p>\nImplicit, \u00een EKS sunt utilizate volume EBS de tip <i>gp2<\/i>, a\u0219a c\u0103 este necesar s\u0103 creezi o clas\u0103 separat\u0103 de stocare bazat\u0103 pe volume <i>io1<\/i> pentru clusterul Kafka:<\/p>\n<pre><code class=\"plaintext\">kubectl create -f - &lt;&lt;EOF\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n  name: fast-ssd\nprovisioner: kubernetes.io\/aws-ebs\nparameters:\n  type: io1\n  iopsPerGB: &quot;50&quot;\n  fsType: ext4\nvolumeBindingMode: WaitForFirstConsumer\nEOF<\/code><\/pre>\n<p>\nSeteaz\u0103 parametrul pentru brokeri <code>min.insync.replicas=3<\/code> \u0219i desf\u0103\u0219ura\u021bi pod-urile brokerilor pe noduri \u00een trei zone diferite de disponibilitate:<\/p>\n<pre><code class=\"bash\">supertubes cluster create -n kafka --kubeconfig  -f https:\/\/raw.githubusercontent.com\/banzaicloud\/kafka-operator\/master\/docs\/benchmarks\/infrastructure\/kafka_202001_3brokers.yaml --wait --timeout 600<\/code><\/pre>\n<p><\/p>\n<h3>Subiecte<\/h3>\n<p>\nAm rulat simultan trei instan\u021be ale generatorului de \u00eenc\u0103rcare. Fiecare scrie \u00een propria subiect, ceea ce \u00eenseamn\u0103 c\u0103 avem nevoie de trei subiecte \u00een total:<\/p>\n<pre><code class=\"bash\">supertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n  name: perftest1\nspec:\n  name: perftest1\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF\n\nsupertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n    name: perftest2\nspec:\n  name: perftest2\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF\n\nsupertubes cluster topic create -n kafka --kubeconfig  -f -&lt;&lt;EOF\napiVersion: kafka.banzaicloud.io\/v1alpha1\nkind: KafkaTopic\nmetadata:\n  name: perftest3\nspec:\n  name: perftest3\n  partitions: 12\n  replicationFactor: 3\n  retention.ms: &#039;28800000&#039;\n  cleanup.policy: delete\nEOF<\/code><\/pre>\n<p>\nPentru fiecare subiect, factorul de replicare este 3 \u2014 valoarea minim\u0103 recomandat\u0103 pentru sistemele de produc\u021bie cu \u00eenalt\u0103 disponibilitate.<\/p>\n<h3>Instrument pentru generarea sarcinii<\/h3>\n<p>\nAm rulat trei instan\u021be ale generatorului de sarcin\u0103 (fiecare scriind \u00eentr-un topic separat). Pentru pod-urile generatorului de sarcin\u0103, trebuie s\u0103 specific\u0103m afinitatea nodurilor, astfel \u00eenc\u00e2t acestea s\u0103 fie planificate doar pe nodurile dedicate lor:<\/p>\n<pre><code class=\"plaintext\">apiVersion: extensions\/v1beta1\nkind: Deployment\nmetadata:\n  labels:\n    app: loadtest\n  name: perf-load1\n  namespace: kafka\nspec:\n  progressDeadlineSeconds: 600\n  replicas: 1\n  revisionHistoryLimit: 10\n  selector:\n    matchLabels:\n      app: loadtest\n  strategy:\n    rollingUpdate:\n      maxSurge: 25%\n      maxUnavailable: 25%\n    type: RollingUpdate\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: loadtest\n    spec:\n      affinity:\n        nodeAffinity:\n          requiredDuringSchedulingIgnoredDuringExecution:\n            nodeSelectorTerms:\n            - matchExpressions:\n              - key: nodepool.banzaicloud.io\/name\n                operator: In\n                values:\n                - loadgen\n      containers:\n      - args:\n        - -brokers=kafka-0:29092,kafka-1:29092,kafka-2:29092,kafka-3:29092\n        - -topic=perftest1\n        - -required-acks=all\n        - -message-size=512\n        - -workers=20\n        image: banzaicloud\/perfload:0.1.0-blog\n        imagePullPolicy: Always\n        name: sangrenel\n        resources:\n          limits:\n            cpu: 2\n            memory: 1Gi\n          requests:\n            cpu: 2\n            memory: 1Gi\n        terminationMessagePath: \/dev\/termination-log\n        terminationMessagePolicy: File\n      dnsPolicy: ClusterFirst\n      restartPolicy: Always\n      schedulerName: default-scheduler\n      securityContext: {}\n      terminationGracePeriodSeconds: 30<\/code><\/pre>\n<p>\nC\u00e2teva aspecte de care ar trebui s\u0103 \u021bine\u021bi cont:<\/p>\n<ul>\n<li> Generatorul de sarcin\u0103 genereaz\u0103 mesaje cu o lungime de 512 octe\u021bi \u0219i le public\u0103 \u00een Kafka \u00een pachete de c\u00e2te 500 de mesaje.<\/li>\n<li> Cu ajutorul argumentului <code>-required-acks=all<\/code> publicarea este considerat\u0103 reu\u0219it\u0103 atunci c\u00e2nd toate replicile sincronizate ale mesajului sunt primite \u0219i confirmate de brokerii Kafka. Aceasta \u00eenseamn\u0103 c\u0103 \u00een benchmark am m\u0103surat nu doar viteza de lucru a liderilor care primesc mesajele, ci \u0219i a succesorilor lor, care replic\u0103 mesajele. Sarcina acestui test nu este de a evalua viteza de citire a consumatorilor <i>(consumatori)<\/i> mesajelor recent acceptate, care r\u0103m\u00e2n deocamdat\u0103 \u00een cache-ul paginilor OS, \u0219i compararea sa cu viteza de citire a mesajelor stocate pe disc.<\/li>\n<li> Generatorul de sarcin\u0103 ruleaz\u0103 simultan 20 de worker-i (<code>-workers=20<\/code>). Fiecare worker con\u021bine 5 producer-i, care \u00eemp\u0103rt\u0103\u0219esc conexiunea worker-ului cu clusterul Kafka. \u00cen total, fiecare generator num\u0103r\u0103 100 de producer-i, iar to\u021bi trimit mesaje \u00een clusterul Kafka.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Monitorizarea st\u0103rii cluster-ului<\/h3>\n<p>\n\u00cen timpul test\u0103rii de sarcin\u0103 a clusterului Kafka, am monitorizat de asemenea s\u0103n\u0103tatea acestuia, pentru a ne asigura c\u0103 nu exist\u0103 reinicializ\u0103ri ale pod-urilor, replici nesincronizate \u0219i capacitate maxim\u0103 cu fluctua\u021bii minime:<\/p>\n<ul>\n<li> Generatorul de \u00eenc\u0103rcare scrie statistici standard despre num\u0103rul de mesaje publicate \u0219i nivelul de erori. Procentul de erori trebuie s\u0103 r\u0103m\u00e2n\u0103 la valoarea <code>0,00%<\/code>.<\/li>\n<li> <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/linkedin\/cruise-control\">Cruise Control<\/a><\/noindex>, desf\u0103\u0219urat cu kafka-operatorul, ofer\u0103 un panou de monitorizare, pe care putem observa starea cluster-ului. Pentru a vizualiza acest panou, executa\u021bi:\n<pre><code class=\"bash\">supertubes cluster cruisecontrol show -n kafka --kubeconfig<\/code><\/pre>\n<\/li>\n<li> Nivelul ISR <i>(num\u0103rul de replici \u201e\u00een-sincron\u201d)<\/i> shrink \u0219i expansion sunt egale cu 0.<\/li>\n<\/ul>\n<p><\/p>\n<h2>Rezultatele m\u0103sur\u0103torilor<\/h2>\n<p><\/p>\n<h3>3 brokeri, dimensiunea mesajelor \u2014 512 byte<\/h3>\n<p>\nCu partition-urile uniform distribuite pe trei brokeri, am reu\u0219it s\u0103 atingem performan\u021ba <i>~500 Mb\/s (aproximativ 990.000 mesaje pe secund\u0103)<\/i>:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/df65ff6a1a26e28d5e5a7a5ab1a20d98.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/15c5883c0056c651f0a0bd14e9991967.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/9b708fa5ce809b0d059cba9f5edfbb2f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nConsumul de memorie al ma\u0219inii virtuale JVM nu a dep\u0103\u0219it 2 Gb:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/3320e2eea891055cc0935da492d18ee9.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/6e3976c310309aa8be45fa3fdf547750.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/94baa1b689ae854693fb7cc4ed6e105e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nCapacitatea de transfer a discului a atins capacitatea maxim\u0103 de I\/O a nodului pe toate cele trei instan\u021be, unde au func\u021bionat brokerii:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/408139ad33cda1294755660df1c128a8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/391d2df0ddfe82f94abdb200370ff775.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/e7e9fd021a2cdc71992ce672363a78ec.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDin datele privind utilizarea memoriei de c\u0103tre noduri, se observ\u0103 c\u0103 bufferizarea \u0219i cache-ul sistemului au ocupat ~10-15 Gb:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/781887507e01a4903b86871961ff9260.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/63f10177f01342039f04192ea93c8bca.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/48b992cc73a968b3d3b1442f96562c4e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>3 brokeri, dimensiunea mesajelor \u2014 100 byte<\/h3>\n<p>\nOdat\u0103 cu reducerea dimensiunii mesajelor, capacitatea de transfer scade cu aproximativ 15-20%: aceasta se datoreaz\u0103 timpului necesar pentru procesarea fiec\u0103rui mesaj. \u00cen plus, \u00eenc\u0103rcarea CPU-ului a crescut aproape de dou\u0103 ori.<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/2f7dc5a94da371cfe17c9e0fc0ec1941.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/986160a3a4e099a0dbb8aeefaf6f1c2f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/c08244457bf4dbfadf52838d4ea2969e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDeoarece nodurile brokerilor au \u00een continuare nuclee neutilizate, performan\u021ba poate fi \u00eembun\u0103t\u0103\u021bit\u0103 prin ajustarea configura\u021biei Kafka. Aceasta este o sarcin\u0103 complex\u0103, a\u0219a c\u0103 pentru a cre\u0219te capacitatea de transfer, este mai bine s\u0103 lucr\u0103m cu mesaje de dimensiuni mai mari.<\/p>\n<h3>4 brokeri, dimensiunea mesajelor \u2014 512 byte<\/h3>\n<p>\nEste u\u0219or s\u0103 cre\u0219te\u021bi performan\u021ba cluster-ului Kafka, ad\u0103ug\u00e2nd pur \u0219i simplu noi brokeri \u0219i men\u021bin\u00e2nd echilibrul partition-urilor (aceasta asigur\u0103 distribu\u021bia uniform\u0103 a \u00eenc\u0103rc\u0103turii \u00eentre brokeri). \u00cen cazul nostru, dup\u0103 ad\u0103ugarea unui broker, l\u0103\u021bimea de band\u0103 a cluster-ului a crescut la <i>~580 Mb\/s (~1,1 milioane mesaje pe secund\u0103)<\/i>. Cre\u0219terea a fost mai mic\u0103 dec\u00e2t ne a\u0219teptam: \u00een principal, acest lucru se explic\u0103 prin dezimbalan\u021bul partition-urilor (nu to\u021bi brokerii lucreaz\u0103 la capacitate maxim\u0103).<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/8c8d2e70739add3c3f88fe42acf98c46.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/a6e4812270c04487de0ad39b996a13f8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/6162c04cfb87b9d9cd0f3147a4ae1647.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/dae5de3b14be212c679f3627a3b77d7b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nConsumul de memorie al ma\u0219inii JVM a r\u0103mas sub 2 GB:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/0bca07a817d7da3104b08de10979f049.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/7e1e64f80ec140f13e171cfe3be930ce.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/5e9700b1e54e45f9cd5fa018ec07dba1.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/4ba0c4b59bfe72bcb0d2f22056c47d79.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nFunc\u021bionarea brokerilor cu stocare a fost afectat\u0103 de dezimbalan\u021bul partition-urilor:<\/p>\n<p><img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/2b8ed3c21adf70c751d1daa6d4c41895.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/77b980869b99903d9de5cd7d43d79291.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/f2f9569a3a1ef0ce35e89b2e52e4a8b2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<img decoding=\"async\" alt=\"Determina\u021bi dimensiunea potrivit\u0103 pentru clusterul Kafka \u00een Kubernetes\" src=\"\/wp-content\/uploads\/2020\/02\/191d6d97b32b2128c5cfd3914171de15.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h2>Conclusions<\/h2>\n<p>\nAbordarea iterativ\u0103 prezentat\u0103 mai sus poate fi extins\u0103 pentru a acoperi scenarii mai complexe, incluz\u00e2nd sute de consumatori, repartitionare, actualiz\u0103ri de tip rolling, reporniri de pod-uri etc. Toate acestea ne permit s\u0103 evalu\u0103m limitele capabilit\u0103\u021bilor cluster-ului Kafka \u00een diferite condi\u021bii, s\u0103 identific\u0103m bottleneck-uri \u00een func\u021bionarea sa \u0219i s\u0103 g\u0103sim modalit\u0103\u021bi de a le dep\u0103\u0219i.<\/p>\n<p>Am dezvoltat Supertubes pentru desf\u0103\u0219urarea rapid\u0103 \u0219i u\u0219oar\u0103 a cluster-ului, configurarea acestuia, ad\u0103ugarea\/\u0219tergerea brokerilor \u0219i a topic-urilor, reac\u021bia la alerte \u0219i asigurarea func\u021bion\u0103rii corecte a Kafka \u00een Kubernetes \u00een general. Scopul nostru este s\u0103 ajut\u0103m utilizatorii s\u0103 se concentreze pe sarcina principal\u0103 (\u201ea genera\u201d \u0219i \u201ea consuma\u201d mesaje Kafka), l\u0103s\u00e2nd toat\u0103 munca grea Supertubes \u0219i kafka operator-ului.<\/p>\n<p>Dac\u0103 sunte\u021bi interesat de tehnologii \u0219i proiecte Open Source Banzai Cloud, urm\u0103ri\u021bi compania pe <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/banzaicloud\">GitHub<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/www.linkedin.com\/company\/banzaicloud\">LinkedIn<\/a><\/noindex> sau <noindex><a rel=\"nofollow\" href=\"https:\/\/twitter.com\/BanzaiCloud\">Twitter<\/a><\/noindex>.<\/p>\n<h2>P.S. de la traduc\u0103tor<\/h2>\n<p>\nCiti\u021bi \u0219i \u00een blogul nostru:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/480722\/\">O poveste despre operatorul Redis \u00een K8s \u0219i un mic rezumat al utilitarilor pentru analizarea datelor acestei baze de date<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/450662\/\">Migrarea complicat\u0103 a RabbitMQ \u00een Kubernetes<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/329224\/\">zetcd de la CoreOS: \u00eenlocuind ZooKeeper cu\u2026 un stocare etcd<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/488920\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes. \u041f\u0440\u0438\u0432\u043e\u0434\u0438\u043c\u044b\u0435 \u0438\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0438 \u0438\u043b\u043b\u044e\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u043a\u0430\u043a \u043c\u043e\u0436\u043d\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0438\u0442\u044c \u043e\u043f\u0442\u0438\u043c\u0430\u043b\u044c\u043d\u044b\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u044b \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0441\u0430\u043c\u0443 Kafka \u0434\u043b\u044f \u0434\u043e\u0441\u0442\u0438\u0436\u0435\u043d\u0438\u044f \u0442\u0440\u0435\u0431\u0443\u0435\u043c\u043e\u0439 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u0438. Apache Kafka \u2014 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u0430\u044f \u0441\u0442\u0440\u0438\u043c\u0438\u043d\u0433\u043e\u0432\u0430\u044f \u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0430 \u0434\u043b\u044f \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043d\u0430\u0434\u0451\u0436\u043d\u044b\u0445, \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u0443\u0435\u043c\u044b\u0445 \u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u043f\u043e\u0442\u043e\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0440\u0435\u0430\u043b\u044c\u043d\u043e\u0433\u043e \u0432\u0440\u0435\u043c\u0435\u043d\u0438. [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":71214,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-71213","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u043c \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0449\u0438\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 Kafka \u0432 Kubernetes | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-24T13:09:17+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-03-03T13:14:29+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Determinarea dimensiunii potrivite pentru cluster-ul Kafka \u00een Kubernetes | ProHoster","description":"Nota trad.: \u00cen acest articol, compania Banzai Cloud \u00ee\u0219i \u00eemp\u0103rt\u0103\u0219e\u0219te exemplul de utilizare a utilitarelor sale speciale pentru a facilita exploatarea Kafka \u00een cadrul Kubernetes.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u043c \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0449\u0438\u0439 \u0440\u0430\u0437\u043c\u0435\u0440 \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 Kafka \u0432 Kubernetes | ProHoster","og:description":"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Banzai Cloud \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u043e\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u0435\u0451 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0445 \u0443\u0442\u0438\u043b\u0438\u0442 \u0434\u043b\u044f \u043e\u0431\u043b\u0435\u0433\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u0438 Kafka \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 Kubernetes.","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/opredelyaem-podhodyashhij-razmer-dlya-klastera-kafka-v-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-24T13:09:17+00:00","article:modified_time":"2020-03-03T13:14:29+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"71213","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 19:06:22","updated":"2022-10-04 20:58:00","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/71213","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=71213"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/71213\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/71214"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=71213"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=71213"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=71213"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}