Sawubona, Habr!
Ngesinye isikhathi, saba ngabokuqala ukwethula isihloko emakethe yaseRussia
Isingeniso
I-Kubernetes yakhelwe ukuphatha imithwalo yemisebenzi engapheli. Imvamisa, imithwalo yemisebenzi enjalo yethulwa ngendlela ye-microservice architecture, ayisindi, ilinganisa kahle ngokuvundlile, ilandela imigomo yezinhlelo zokusebenza eziyi-12, futhi ingasebenza nama-circuit breaker kanye nezinkawu ze-chaos.
I-Kafka, ngakolunye uhlangothi, isebenza njengesizindalwazi esabalalisiwe. Ngakho-ke, lapho usebenza, kufanele ubhekane nombuso, futhi unzima kakhulu kune-microservice. I-Kubernetes isekela imithwalo yemfanelo, kodwa njengoba u-Kelsey Hightower ebonisa kuma-tweets amabili, kufanele aphathwe ngokucophelela:
Abanye abantu banomuzwa wokuthi uma ufaka i-Kubernetes emthwalweni womsebenzi omuhle, iba isizindalwazi esiphethwe ngokugcwele esiqhudelana ne-RDS. Lokhu akulungile. Mhlawumbe, uma usebenza kanzima ngokwanele, engeza izingxenye ezengeziwe futhi uhehe ithimba lonjiniyela be-SRE, uzokwazi ukwakha i-RDS phezu kwe-Kubernetes.
Ngihlala ngincoma ukuthi wonke umuntu aqaphe ngokwedlulele lapho esebenzisa imithwalo enzima yemisebenzi ku-Kubernetes. Abantu abaningi ababuza ukuthi “ngingakwazi yini ukwenza umsebenzi omningi ku-Kubernetes” abanalo ulwazi olwanele nge-Kubernetes, futhi ngokuvamile ngomthwalo womsebenzi ababuza ngawo.
Ngakho-ke, kufanele usebenzise i-Kafka ku-Kubernetes? Umbuzo ophikisayo: ingabe iKafka izosebenza kangcono ngaphandle kukaKubernetes? Kungakho ngifuna ukugqamisa kulesi sihloko ukuthi uKafka noKubernetes baphelelisana kanjani, nokuthi yiziphi izingibe ezingafika ngokuzihlanganisa.
Isikhathi sokuqeda
Ake sikhulume ngento eyisisekelo - imvelo yesikhathi sokusebenza ngokwayo
Inqubo
Abathengi be-Kafka banobungani be-CPU. I-TLS ingase yethule okunye okungaphezulu. Nokho, amaklayenti e-Kafka angase abe ne-CPU ejulile uma esebenzisa ukubethela, kodwa lokhu akuthinti ama-broker.
inkumbulo
Abathengi be-Kafka badla inkumbulo. Usayizi wenqwaba ye-JVM ngokuvamile ukhawulelwe ku-4-5 GB, kodwa uzodinga futhi inkumbulo yesistimu eningi njengoba i-Kafka isebenzisa inqolobane yekhasi kakhulu. Ku-Kubernetes, setha insiza yesiqukathi bese ucela imikhawulo ngokufanele.
Isitolo sedatha
Ukugcinwa kwedatha ezitsheni kungokwesikhashana - idatha iyalahleka lapho iqalwa kabusha. Ngedatha ye-Kafka ungasebenzisa ivolumu emptyDir
, futhi umphumela uzofana: idatha yakho yomthengisi izolahleka ngemva kokuqedwa. Imilayezo yakho isengagcinwa kwabanye abadayisi njengezifaniso. Ngakho-ke, ngemva kokuqalisa kabusha, umthengisi ohlulekile kufanele aqale aphindaphinde yonke idatha, futhi le nqubo ingathatha isikhathi esiningi.
Yingakho kufanele usebenzise ukugcinwa kwedatha yesikhathi eside. Makube okungesona okwendawo isitoreji sesikhathi eside ngohlelo lwefayela le-XFS noma, ngokuqondile, i-ext4. Ungasebenzisi i-NFS. Ngikuxwayisile. Izinguqulo ze-NFS v3 noma i-v4 ngeke zisebenze. Ngamafuphi, umthengisi we-Kafka uzophahlazeka uma engakwazi ukususa uhla lwedatha ngenxa yenkinga "yokuqamba kabusha oyisiphukuphuku" ku-NFS. Uma ngingakakukholisanga, ngokucophelela
Inethiwekhi
Njengamasistimu amaningi asabalalisiwe, ukusebenza kwe-Kafka kuncike kakhulu ekugcineni ukubambezeleka kwenethiwekhi kubuncane kanye nomkhawulokudonsa ufike phezulu. Ungazami ukusingatha bonke abadayisi endaweni eyodwa, njengoba lokhu kuzonciphisa ukutholakala. Uma i-node ye-Kubernetes ihluleka, lonke iqoqo le-Kafka lizohluleka. Futhi, ungasabalalisi iqoqo le-Kafka kuzo zonke izikhungo zedatha. Okufanayo kuya kwiqoqo le-Kubernetes. Ukuvumelana okuhle kuleli cala ukukhetha izindawo zokutholakala ezihlukene.
Ukucushwa
I-manifestos evamile
Iwebhusayithi ye-Kubernetes ine
- Ngaphansi: I-pod iyunithi encane esebenzisekayo e-Kubernetes. I-pod iqukethe umsebenzi wakho, futhi i-pod ngokwayo ihambisana nenqubo yeqoqo lakho. I-pod iqukethe isitsha esisodwa noma ngaphezulu. Iseva ngayinye ye-ZooKeeper ekuhlanganisweni kanye nomthengisi ngamunye kuqoqo le-Kafka uzosebenza ku-pod ehlukile.
- StatefulSet: I-StatefulSet iyinto ye-Kubernetes ephatha imithwalo yemisebenzi eminingi, futhi imithwalo enjalo idinga ukuxhumana. I-StatefulSets inikeza iziqinisekiso mayelana noku-oda ama-pods kanye nokuhluka kwawo.
- Amasevisi angenakhanda: Amasevisi akuvumela ukuthi ukhiphe ama-pod kumakhasimende usebenzisa igama elinengqondo. U-Kubernetes kuleli cala unesibopho sokulinganisa ukulayisha. Kodwa-ke, lapho usebenzisa imithwalo yemisebenzi emihle, efana ne-ZooKeeper ne-Kafka, amaklayenti adinga ukuxhumana nesenzakalo esithile. Yilapho izinsizakalo ezingenakhanda zisiza khona: kulokhu, iklayenti lisazoba negama elinengqondo, kodwa ngeke kudingeke ukuthi uxhumane ne-pod ngokuqondile.
- Ivolumu yesitoreji eside: Lawa mavolumu ayadingeka ukuze kumiswe isitoreji esiqhubekayo sokuvimba okungesona esasendaweni okukhulunywe ngenhla.
In
Amashadi e-Helm
UHelm ungumphathi wephakheji ye-Kubernetes engafaniswa nabaphathi bephakheji ye-OS njenge-yum, i-apt, i-Homebrew noma i-Chocolatey. Kwenza kube lula ukufaka amaphakheji esoftware achazwe ngaphambilini kumashadi e-Helm. Ishadi le-Helm elikhethwe kahle lenza umsebenzi onzima wokuthi ungamisa kanjani kahle yonke imingcele ukuze usebenzise i-Kafka ku-Kubernetes kalula. Kunemidwebo eminingana ye-Kafka: esemthethweni itholakala
Abasebenzisi
Ngenxa yokuthi iHelm inokushiyeka okuthile, elinye ithuluzi lithola ukuthandwa okukhulu: Ama-opharetha e-Kubernetes. Umsebenzisi akagcini nje ngokupakisha isoftware ye-Kubernetes, kodwa futhi ikuvumela ukuthi usebenzise isoftware enjalo futhi uyilawule.
Ohlwini
Ukukhiqiza
Kubalulekile ukuhlola ukusebenza ngokulinganisa isibonelo sakho se-Kafka. Ukuhlolwa okunjalo kuzokusiza ukuthi uthole izingqinamba ezingase zibe khona ngaphambi kokuba kuvele izinkinga. Ngenhlanhla, i-Kafka isivele ihlinzeka ngamathuluzi amabili okuhlola ukusebenza: kafka-producer-perf-test.sh
и kafka-consumer-perf-test.sh
. Zisebenzise ngokugcwele. Ukuze uthole ireferensi, ungabheka imiphumela echazwe ku
Ukusebenza
Ukuqapha
Ukucaca ohlelweni kubaluleke kakhulu - ngaphandle kwalokho ngeke uqonde ukuthi kwenzekani kulo. Namuhla kunekhithi yamathuluzi eqinile ehlinzeka ngokuqapha okusekelwe kumamethrikhi ngesitayela somdabu samafu. Amathuluzi amabili adumile ale njongo i-Prometheus ne-Grafana. I-Prometheus ingaqoqa amamethrikhi kuzo zonke izinqubo ze-Java (i-Kafka, i-Zookeeper, i-Kafka Connect) isebenzisa isithekelisi se-JMX - ngendlela elula. Uma ungeza amamethrikhi e-cAdvisor, ungaqonda ngokugcwele ukuthi izinsiza zisetshenziswa kanjani ku-Kubernetes.
U-Strimzi unesibonelo esikahle kakhulu sedeshibhodi ye-Grafana ye-Kafka. Ibona ngeso lengqondo amamethrikhi angukhiye, isibonelo, mayelana nemikhakha engaphindaphindeki kangako noma leyo engaxhunyiwe ku-inthanethi. Konke kucace kakhulu lapho. Lawa mamethrikhi ahambisana nokusetshenziswa kwensiza kanye nolwazi lokusebenza, kanye nezinkomba zokuzinza. Ngakho uthola ukuqapha okuyisisekelo kweqoqo le-Kafka mahhala!
Source:
Kungaba kuhle ukwenezela konke lokhu ngokuqapha kwamakhasimende (amamethrikhi kubathengi nabakhiqizi), kanye nokuqapha ukubambezeleka (kwalokhu kukhona
Ukugawula
Ukugawula omunye umsebenzi obalulekile. Qiniseka ukuthi zonke iziqukathi ekufakweni kwakho kwe-Kafka zifakwe kuzo stdout
и stderr
, futhi uqinisekise ukuthi iqoqo lakho le-Kubernetes lihlanganisa wonke amalogi abe nengqalasizinda yokugawulwa kwemithi emaphakathi, isb.
Ukusebenza kokuhlolwa
I-Kubernetes isebenzisa ukuphila nokulungela ukuhlola ukuthi ama-pods akho asebenza ngokujwayelekile yini. Uma ukuhlola ukuphila kwehluleka, i-Kubernetes izomisa leso siqukathi bese isiqala kabusha ngokuzenzakalelayo uma inqubomgomo yokuqalisa kabusha isethwe ngokufanele. Uma ukuhlolwa kokulungela kwehluleka, i-Kubernetes ihlukanisa i-pod ezicelweni zokunikeza isevisi. Ngakho-ke, ezimweni ezinjalo, ukungenelela okwenziwa ngesandla akusadingeki nhlobo, okuyi-plus enkulu.
Ikhipha izibuyekezo
I-StatefulSets isekela izibuyekezo ezizenzakalelayo: uma ukhetha isu le-RollingUpdate, ngalinye elingaphansi kwe-Kafka lizobuyekezwa ngokulandelana. Ngale ndlela, isikhathi sokuphumula singancishiswa sibe ziro.
Ukukala
Ukukala iqoqo le-Kafka akuwona umsebenzi olula. Kodwa-ke, i-Kubernetes yenza kube lula kakhulu ukukala ama-pods enanini elithile le-replicas, okusho ukuthi ungachaza ngokusobala abathengisi abaningi be-Kafka ngendlela othanda ngayo. Into enzima kakhulu kuleli cala ukwaba kabusha imikhakha ngemuva kokwenyusa noma ngaphambi kokwehlisa. Futhi, uKubernetes uzokusiza ngalo msebenzi.
Ukuphatha
Imisebenzi ehlobene nokuphatha iqoqo lakho le-Kafka, njengokudala izihloko nokwabela kabusha imikhakha, ingenziwa kusetshenziswa imibhalo yegobolondo ekhona ngokuvula isixhumi esibonakalayo somugqa womyalo kuma-pods akho. Nokho, lesi sixazululo asisihle kakhulu. I-Strimzi isekela ukuphatha izihloko isebenzisa u-opharetha ohlukile. Kukhona indawo yokuthuthukisa lapha.
Yenza isipele futhi ubuyisele
Manje ukutholakala kwe-Kafka nakho kuzoncika ekutholakaleni kwe-Kubernetes. Uma iqoqo lakho le-Kubernetes lehluleka, khona-ke esimweni esibi kakhulu, iqoqo lakho le-Kafka nalo lizohluleka. Ngokomthetho ka-Murphy, lokhu kuzokwenzeka nakanjani, futhi uzolahlekelwa idatha. Ukunciphisa lolu hlobo lobungozi, yiba nomqondo omuhle wokusekelayo. Ungasebenzisa i-MirrorMaker, enye inketho ukusebenzisa i-S3 yalokhu, njengoba kuchazwe kulokhu
isiphetho
Uma usebenza namaqoqo e-Kafka amancane ukuya kosayizi omaphakathi, kufanelekile impela ukusebenzisa i-Kubernetes njengoba ihlinzeka ngokuguquguquka okwengeziwe futhi yenza kube lula ukuzizwisa komsebenzisi. Uma unezidingo ezibaluleke kakhulu zokubambezeleka ezingasebenzi kanye/noma zokuphuma, kungase kube ngcono ukucabangela enye inketho yokusebenzisa.
Source: www.habr.com