🥇Kuzindikira kukula koyenera kwa gulu la Kafka ku Kubernetes

Zindikirani. transl.: M'nkhaniyi, Banzai Cloud akugawana chitsanzo cha momwe zida zake zachizolowezi zingagwiritsire ntchito kuti Kafka ikhale yosavuta kugwiritsa ntchito mkati mwa Kubernetes. Malangizo otsatirawa akuwonetsa momwe mungadziwire kukula koyenera kwa zomangamanga zanu ndikukonzekera Kafka yokha kuti mukwaniritse zomwe mukufunikira.

Apache Kafka ndi nsanja yotsatsira yomwe imagawidwa yopanga makina odalirika, owopsa komanso ochita bwino kwambiri munthawi yeniyeni. Mphamvu zake zochititsa chidwi zitha kukulitsidwa pogwiritsa ntchito Kubernetes. Kwa ichi tapanga Wogwiritsa ntchito Open Source Kafka ndi chida chotchedwa Supertubes. Amakulolani kuti muthamangitse Kafka pa Kubernetes ndikugwiritsa ntchito mawonekedwe ake osiyanasiyana, monga kukonza masinthidwe a broker, makulitsidwe otengera ma metric ndikuyikanso, kuzindikira kwa rack, "kufewa" (wachisomo) kutulutsa zosintha, etc.

Yesani ma Supertubes mugulu lanu:
curl https://getsupertubes.sh | sh и supertubes install -a --no-democluster --kubeconfig <path-to-eks-cluster-kubeconfig-file>
Kapena kulumikizana zolemba. Mutha kuwerenganso za kuthekera kwa Kafka, ntchito yomwe imangogwiritsa ntchito Supertubes ndi Kafka. Talemba kale za iwo pabulogu:

Ayi! Wogwiritsa ntchito wina wa Kafka wa Kubernetes;

Yang'anirani ndikugwiritsa ntchito Kafka kutengera ma metric a Prometheus;

Kudziwitsa za Kafka pa Kubernetes;

Kuthamanga Apache Kafka pa Istio - benchmark;

Magulu otsimikiziridwa ndi ogwiritsa ntchito ndi mwayi wofikira ndi wogwiritsa ntchito wa Kafka;

Kafka rolling kukweza ndi kusintha kosinthika pa Kubernetes;

Envoy protocol fyuluta ya Kafka, meshed.

Mukaganiza zotumiza gulu la Kafka ku Kubernetes, mutha kukumana ndi vuto lozindikira kukula kwachitukuko komanso kufunikira kokonza kasinthidwe ka Kafka yanu kuti ikwaniritse zomwe mukufuna. Kuchita kwakukulu kwa broker aliyense kumatsimikiziridwa ndi magwiridwe antchito azinthu zoyambira, monga kukumbukira, purosesa, liwiro la disk, bandwidth network, etc.

Momwemo, kasinthidwe ka broker kuyenera kukhala kotero kuti zida zonse zogwirira ntchito zigwiritsidwe ntchito momwe angathere. Komabe, m'moyo weniweni kukhazikitsidwa uku kumakhala kovuta. Ndizotheka kuti ogwiritsa ntchito azikonza ma broker kuti awonjezere kugwiritsa ntchito gawo limodzi kapena ziwiri (disk, memory, kapena purosesa). Nthawi zambiri, broker amawonetsa magwiridwe antchito apamwamba pomwe kasinthidwe kake kamalola kuti gawo locheperako ligwiritsidwe ntchito mokwanira. Mwanjira iyi titha kukhala ndi lingaliro loyipa la katundu yemwe broker m'modzi atha kunyamula.

Mwamwayi, titha kuyerekezanso kuchuluka kwa ma broker omwe amafunikira kuti agwire ntchito yomwe yaperekedwa. Komabe, muzochita pali zosankha zambiri zokonzekera pamagulu osiyanasiyana kuti ndizovuta kwambiri (ngati sizingatheke) kuyesa ntchito yomwe ingakhalepo ya kasinthidwe kake. Mwa kuyankhula kwina, ndizovuta kwambiri kukonzekera kasinthidwe kutengera magwiridwe antchito ena.

Kwa ogwiritsa ntchito a Supertubes, nthawi zambiri timatenga njira yotsatirayi: timayamba ndi makonzedwe ena (zomangamanga + zoikamo), kenaka muyese ntchito yake, sinthani makonzedwe a broker ndikubwereza ndondomekoyi kachiwiri. Izi zimachitika mpaka chigawo chochepa kwambiri cha zomangamanga chikugwiritsidwa ntchito mokwanira.

Mwanjira imeneyi, timapeza lingaliro lomveka bwino la kuchuluka kwa ma broker omwe gulu limayenera kunyamula katundu wina (chiwerengero cha ma broker chimadaliranso pazinthu zina, monga kuchuluka kwa mauthenga obwereza kuti atsimikizire kulimba, kuchuluka kwa magawo. atsogoleri, etc.). Kuphatikiza apo, timazindikira zomwe zida zogwirira ntchito zimafunikira kuwongolera molunjika.

Nkhaniyi ifotokoza za njira zomwe timatenga kuti tipindule kwambiri ndi magawo ochepera pang'ono pamasinthidwe oyambira ndikuyesa kutulutsa kwa gulu la Kafka. Kukonzekera kokhazikika kumafuna osachepera atatu oyendetsa ma broker (min.insync.replicas=3), zogawidwa m'magawo atatu osiyanasiyana ofikirako. Kuti tikonze, kukulitsa ndi kuyang'anira maziko a Kubernetes, timagwiritsa ntchito nsanja yathu yoyang'anira ziwiya zamtambo wosakanizidwa - Bomba. Imathandizira pamtunda (chitsulo chopanda kanthu, VMware) ndi mitundu isanu ya mitambo (Alibaba, AWS, Azure, Google, Oracle), komanso kuphatikiza kulikonse.

Malingaliro pamagulu amagulu a Kafka ndi kasinthidwe

Pazitsanzo zomwe zili pansipa, tidasankha AWS ngati opereka mtambo ndi EKS ngati kugawa kwa Kubernetes. Kukonzekera kofananako kutha kukhazikitsidwa pogwiritsa ntchito P.K.E. - Kugawa kwa Kubernetes kuchokera ku Banzai Cloud, kutsimikiziridwa ndi CNCF.

litayamba

Amazon imapereka zosiyanasiyana Mitundu ya voliyumu ya EBS. Pakatikati gp2 и io1 pali ma drive a SSD, komabe, kuti muwonetsetse kutulutsa kwakukulu gp2 amadya credits anasonkhanitsa (I/O mbiri), kotero tidakonda mtunduwo io1, zomwe zimapereka kusinthasintha kwakukulu.

Mitundu yachitsanzo

Kuchita kwa Kafka kumadalira kwambiri cache ya tsamba la opareshoni, chifukwa chake timafunikira zochitika zokhala ndi kukumbukira kokwanira kwa otsatsa (JVM) ndi cache yamasamba. Chitsanzo c5.2 kukula - chiyambi chabwino, popeza ili ndi 16 GB ya kukumbukira ndi wokometsedwa kugwira ntchito ndi EBS. Choyipa chake ndikuti chimangotha kupereka magwiridwe antchito osapitilira mphindi 30 maola 24 aliwonse. Ngati ntchito yanu ikufunika kuti mugwire ntchito kwa nthawi yayitali, mungafune kuganizira mitundu ina. Ndizo ndendende zomwe ife tinachita, kuyimapo c5.4 kukula. Imakupatsirani mwayi wolowera kwambiri 593,75 Mbps. Kuchulukitsa kwakukulu kwa voliyumu ya EBS io1 apamwamba kuposa chitsanzo c5.4 kukula, kotero kuti chinthu chochepa kwambiri pazitukuko chikhoza kukhala I / O kupyolera mwa mtundu wa chitsanzo ichi (chomwe mayesero athu a katundu ayenera kutsimikiziranso).

Mtanda

Kutulutsa kwa netiweki kuyenera kukhala kwakukulu kokwanira poyerekeza ndi magwiridwe antchito a VM ndi disk, apo ayi maukonde amakhala otsekereza. Kwathu, mawonekedwe a netiweki c5.4 kukula imathandizira kuthamanga kwa 10 Gb/s, komwe kuli kokwera kwambiri kuposa kutulutsa kwa I/O kwa VM.

Kutumiza kwa Broker

Ma broker akuyenera kutumizidwa (okonzedwa ku Kubernetes) kumalo odzipatulira kuti apewe kupikisana ndi njira zina za CPU, kukumbukira, maukonde, ndi ma disk.

Mtundu wa Java

Kusankha koyenera ndi Java 11 chifukwa imagwirizana ndi Docker m'lingaliro lakuti JVM imasankha molondola mapurosesa ndi kukumbukira komwe kulipo ku chidebe chomwe broker akuyendetsa. Podziwa kuti malire a CPU ndi ofunika, JVM mkati ndi mowonekera imayika chiwerengero cha ulusi wa GC ndi ulusi wa JIT. Tinagwiritsa ntchito chithunzi cha Kafka banzaicloud/kafka:2.13-2.4.0, yomwe ili ndi Kafka version 2.4.0 (Scala 2.13) pa Java 11.

Ngati mukufuna kudziwa zambiri za Java/JVM pa Kubernetes, onani zolemba zathu zotsatirazi:

Chifukwa chiyani pulogalamu yanga ya Java ili OOMKilled;

Momwe mungakulitsire zotengera za Java 10 moyenera.

Zokonda pamtima wa broker

Pali zinthu ziwiri zofunika pakukhazikitsa kukumbukira kwa broker: zoikamo za JVM ndi Kubernetes pod. Malire okumbukira omwe amayikidwa pa pod ayenera kukhala wamkulu kuposa kukula kwake kwa mulu kuti JVM ikhale ndi malo a Java metaspace yomwe imakhala mu kukumbukira kwake komanso cache ya tsamba la opaleshoni yomwe Kafka amagwiritsa ntchito mwakhama. M'mayeso athu tidayambitsa ma broker a Kafka okhala ndi magawo -Xmx4G -Xms2G, ndipo malire a kukumbukira kwa pod anali 10 Gi. Chonde dziwani kuti makonda a kukumbukira a JVM atha kupezeka pogwiritsa ntchito -XX:MaxRAMPercentage и -X:MinRAMPercentage, kutengera malire a kukumbukira kwa pod.

Zokonda pa Broker processor

Nthawi zambiri, mutha kusintha magwiridwe antchito powonjezera kufanana powonjezera kuchuluka kwa ulusi wogwiritsidwa ntchito ndi Kafka. Mapurosesa ambiri omwe amapezeka ku Kafka, ndizabwinoko. M'mayesero athu, tinayamba ndi malire a 6 processors ndipo pang'onopang'ono (kupyolera mu kubwereza) adakweza chiwerengero chawo ku 15. Kuwonjezera apo, timayika. num.network.threads=12 m'makonzedwe a broker kuti muwonjezere chiwerengero cha ulusi umene umalandira deta kuchokera pa intaneti ndikutumiza. Atangozindikira kuti otsatira otsatsa sakanatha kulandira zofananira mwachangu, adadzuka num.replica.fetchers mpaka 4 kuti muwonjezere liwiro lomwe otsatsa amatsanzira mauthenga ochokera kwa atsogoleri.

Load Generation Chida

Muyenera kuwonetsetsa kuti jenereta yonyamula katundu yosankhidwayo simatha mphamvu isanakwane gulu la Kafka (lomwe likuimiridwa) lifika pamtolo wake waukulu. Mwa kuyankhula kwina, m'pofunika kuwunika koyambirira kwa mphamvu ya chida chopangira katundu, ndikusankhanso mitundu yachitsanzo ndi chiwerengero chokwanira cha mapurosesa ndi kukumbukira. Pankhaniyi, chida chathu chidzatulutsa katundu wochuluka kuposa momwe gulu la Kafka lingagwirire. Pambuyo poyesera zambiri, tinakhazikika pa makope atatu c5.4 kukula, chilichonse chinali ndi jenereta yothamanga.

Benchmarking

Muyeso wa magwiridwe antchito ndi njira yobwerezabwereza yomwe ili ndi magawo awa:

kukhazikitsa zomangamanga (gulu la EKS, gulu la Kafka, chida chotulutsa katundu, komanso Prometheus ndi Grafana);
kutulutsa katundu kwa nthawi inayake kuti azisefa zopatuka mwachisawawa muzowonetsa zomwe zasonkhanitsidwa;
kusintha mawonekedwe a broker ndi kasinthidwe kutengera zizindikiro zomwe zimawonedwa;
kubwereza ndondomekoyi mpaka mlingo wofunikira wa masango a Kafka ukwaniritsidwa. Panthawi imodzimodziyo, iyenera kubwerezedwa nthawi zonse komanso kusonyeza kusiyana kochepa pa zomwe zimachitika.

Gawo lotsatira likufotokoza masitepe omwe adachitika panthawi yoyeserera ma benchmarking cluster.

Zida

Zida zotsatirazi zidagwiritsidwa ntchito poyika kasinthidwe koyambira, kupanga zolemetsa, ndikuyesa magwiridwe antchito:

Banzai Cloud Pipeline pokonza gulu la EKS kuchokera ku Amazon c Prometheus (kusonkhanitsa Kafka ndi ma metrics oyambira) ndi grafana (kuti muwone ma metric awa). Tinapezerapo mwayi ophatikizidwa в Bomba ntchito zomwe zimapereka kuwunika kogwirizana, kusonkhanitsa zipika zapakati, kusanthula kwachiwopsezo, kubwezeretsa masoka, chitetezo chamabizinesi ndi zina zambiri.
Sangrenel - chida choyesera katundu wa gulu la Kafka.
Ma dashboard a Grafana owonera ma metric a Kafka ndi zomangamanga: Kubernetes Kafka, Node Exporter.
Supertubes CLI ya njira yosavuta yokhazikitsira gulu la Kafka pa Kubernetes. Zookeeper, Kafka operator, Envoy ndi zina zambiri zimayikidwa ndikukonzedwa bwino kuti ziyendetse gulu la Kafka lokonzekera kupanga pa Kubernetes.
- Kukhazikitsa Zithunzi za CLI gwiritsani ntchito malangizo omwe aperekedwa apa.

Mtengo wa magawo EKS

Konzani gulu la EKS lokhala ndi malo antchito odzipereka c5.4 kukula m'malo osiyanasiyana opezeka ma pod okhala ndi ma broker a Kafka, komanso malo odzipatulira a jenereta yonyamula katundu ndi zowunikira.

banzai cluster create -f https://raw.githubusercontent.com/banzaicloud/kafka-operator/master/docs/benchmarks/infrastructure/cluster_eks_202001.json

Gulu la EKS likangoyamba kugwira ntchito, yambitsani kuphatikizidwa kwake ntchito yowunika - adzatumiza Prometheus ndi Grafana kukhala gulu.

Kafka dongosolo zigawo

Ikani zida za dongosolo la Kafka (Zookeeper, kafka-operator) mu EKS pogwiritsa ntchito ma supertubes CLI:

supertubes install -a --no-democluster --kubeconfig <path-to-eks-cluster-kubeconfig-file>

Kafka Cluster

Mwachikhazikitso, EKS imagwiritsa ntchito mitundu ya EBS yamtundu gp2, kotero muyenera kupanga kalasi yosungiramo yosiyana malinga ndi mavoliyumu io1 kwa gulu la Kafka:

kubectl create -f - <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: io1
  iopsPerGB: "50"
  fsType: ext4
volumeBindingMode: WaitForFirstConsumer
EOF

Khazikitsani magawo a ma broker min.insync.replicas=3 ndikuyika ma pods a broker pa node m'malo atatu osiyanasiyana omwe akupezeka:

supertubes cluster create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f https://raw.githubusercontent.com/banzaicloud/kafka-operator/master/docs/benchmarks/infrastructure/kafka_202001_3brokers.yaml --wait --timeout 600

Mitu

Tinayendetsa maulendo atatu a jenereta mofanana. Aliyense wa iwo amalemba pamutu wawo, ndiye kuti, timafunikira mitu itatu yonse:

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
  name: perftest1
spec:
  name: perftest1
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
    name: perftest2
spec:
  name: perftest2
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
  name: perftest3
spec:
  name: perftest3
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

Pa mutu uliwonse, chobwerezabwereza ndi 3-mtengo wochepera wovomerezeka pamakina opangira omwe amapezeka kwambiri.

Load Generation Chida

Tidayambitsa makope atatu a jenereta yonyamula katundu (iliyonse idalemba pamutu wosiyana). Pazida zonyamula katundu, muyenera kukhazikitsa mgwirizano wa node kotero kuti amangokhazikitsidwa pamanodi omwe aperekedwa kwa iwo:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: loadtest
  name: perf-load1
  namespace: kafka
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: loadtest
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: loadtest
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: nodepool.banzaicloud.io/name
                operator: In
                values:
                - loadgen
      containers:
      - args:
        - -brokers=kafka-0:29092,kafka-1:29092,kafka-2:29092,kafka-3:29092
        - -topic=perftest1
        - -required-acks=all
        - -message-size=512
        - -workers=20
        image: banzaicloud/perfload:0.1.0-blog
        imagePullPolicy: Always
        name: sangrenel
        resources:
          limits:
            cpu: 2
            memory: 1Gi
          requests:
            cpu: 2
            memory: 1Gi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

Mfundo zingapo zofunika kuzikumbukira:

Jenereta yonyamula katundu imapanga mauthenga a 512 byte m'litali ndipo amawasindikiza ku Kafka m'magulu a mauthenga 500.
Kugwiritsa ntchito mkangano -required-acks=all Kusindikizaku kumawonedwa kukhala kopambana pamene zofananira zonse zolumikizidwa za uthengawo zalandiridwa ndikutsimikiziridwa ndi a Kafka broker. Izi zikutanthauza kuti mu benchmark sitinayese liwiro la atsogoleri omwe amalandila mauthenga, komanso otsatira awo akubwereza mauthenga. Cholinga cha mayesowa sikuyesa kuthamanga kwa kuwerenga kwa ogula (ogula) posachedwapa analandira mauthenga amene akadali mu Os tsamba posungira, ndi poyerekeza ndi liwiro kuwerenga mauthenga osungidwa pa litayamba.
Jenereta yonyamula katundu imayendetsa antchito 20 molumikizana (-workers=20). Wogwira ntchito aliyense ali ndi opanga 5 omwe amagawana kulumikizana kwa ogwira ntchito ndi gulu la Kafka. Zotsatira zake, jenereta iliyonse ili ndi opanga 100, ndipo onse amatumiza mauthenga ku gulu la Kafka.

Kuyang'anira thanzi la gulu

Pakuyesa katundu wa gulu la Kafka, tidayang'aniranso thanzi lake kuti tiwonetsetse kuti palibe kuyambiranso kwa pod, palibe zofananira zofananira, komanso kutulutsa kwakukulu komwe kumasinthasintha pang'ono:

Jenereta yonyamula katundu imalemba ziwerengero zokhazikika za kuchuluka kwa mauthenga omwe asindikizidwa komanso kuchuluka kwa zolakwika. Chiwopsezocho chiyenera kukhala chofanana 0,00%.
Kuwongolera Cruise, yoyendetsedwa ndi kafka-operator, imapereka dashboard komwe titha kuyang'aniranso momwe gululi likukhalira. Kuti muwone gulu ili chitani:
```
supertubes cluster cruisecontrol show -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file>
```
Mtengo wapatali wa magawo ISR (chiwerengero cha "in-sync" replicas) kutsika ndi kukulitsa ndizofanana ndi 0.

Zotsatira zoyezera

3 broker, kukula kwa uthenga - 512 bytes

Ndi magawo omwe amagawidwa mofanana pakati pa ogulitsa atatu, tinatha kuchita bwino ~500 Mb/s (pafupifupi mauthenga 990 pamphindikati):

Kugwiritsa ntchito kukumbukira kwa makina a JVM sikunapitirire 2 GB:

Kutulutsa kwa Disk kunafika pamlingo waukulu wa I/O pazochitika zonse zitatu zomwe osinthawo amayendera:

Kuchokera pazomwe zimagwiritsidwa ntchito pamakumbukiro ndi ma node, zimatsata kuti kusungidwa kwadongosolo ndi caching kudatenga ~ 10-15 GB:

3 broker, kukula kwa uthenga - 100 bytes

Pamene kukula kwa uthenga kumachepa, kutulutsa kumatsika pafupifupi 15-20%: nthawi yogwiritsira ntchito uthenga uliwonse imakhudza. Kuphatikiza apo, kuchuluka kwa purosesa kumakhala pafupifupi kawiri.

Popeza ma broker node akadali ndi ma cores osagwiritsidwa ntchito, magwiridwe antchito amatha kusintha mwakusintha kasinthidwe ka Kafka. Iyi si ntchito yophweka, kotero kuti muwonjeze kupititsa patsogolo ndi bwino kugwira ntchito ndi mauthenga akuluakulu.

4 broker, kukula kwa uthenga - 512 bytes

Mutha kuwonjezera magwiridwe antchito a gulu la Kafka pongowonjezera ma broker atsopano ndikusunga magawo angapo (izi zimatsimikizira kuti katunduyo amagawidwa mofanana pakati pa ogulitsa). Kwa ife, titawonjeza broker, kuchuluka kwamagulu kumawonjezeka mpaka ~580 Mb/s (~1,1 miliyoni mauthenga pa sekondi iliyonse). Kukula kunakhala kochepa kuposa momwe amayembekezera: izi zimafotokozedwa makamaka ndi kusalinganika kwa magawo (osati onse ogulitsa amagwira ntchito pachimake cha kuthekera kwawo).

Kugwiritsa ntchito kukumbukira kwa makina a JVM kunakhalabe pansi pa 2 GB:

Ntchito ya ma broker okhala ndi ma drive adakhudzidwa ndi kusalinganika kwa magawo:

anapezazo

Njira yobwerezabwereza yomwe yaperekedwa pamwambapa ikhoza kukulitsidwa kuti ikwaniritse zovuta zambiri zomwe zimakhudza ogula mazana, kugawanitsa, zosintha, kuyambiranso kwa pod, ndi zina zambiri. Zonsezi zimatipangitsa kuwunika malire a kuthekera kwa gulu la Kafka m'mikhalidwe yosiyanasiyana, kuzindikira zopinga pakugwira ntchito kwake ndikupeza njira zothana nazo.

Tidapanga ma Supertubes kuti atumize gulu mwachangu komanso mosavuta, kulikonza, kuwonjezera/kuchotsa ma broker ndi mitu, kuyankha zidziwitso, ndikuwonetsetsa kuti Kafka imagwira ntchito moyenera pa Kubernetes. Cholinga chathu ndikukuthandizani kuyang'ana kwambiri pa ntchito yayikulu ("kupanga" ndi "kuwononga" mauthenga a Kafka), ndikusiya zolimbikira zonse kwa Supertubes ndi oyendetsa Kafka.

Ngati mumakonda matekinoloje a Banzai Cloud ndi ma Open Source mapulojekiti, lembani ku kampaniyo GitHub, LinkedIn kapena Twitter.