Ba kawai sarrafawa ba: Yadda muka yi bayanan da aka rarraba daga Kafka Streams, da abin da ya zo daga gare ta

Hai Habr!

Muna tunatar da ku cewa bin littafin game da Kafka mun buga wani aiki mai ban sha'awa daidai game da ɗakin karatu API ɗin Kafka Streams.

A yanzu, al'umma suna koyon iyakar wannan kayan aiki mai ƙarfi. Don haka, kwanan nan aka buga labarin, fassarar da muke son gabatar muku da ita. Daga kwarewarsa, marubucin ya gaya yadda za a juya Kafka Streams a cikin ajiyar bayanai da aka rarraba. Ji daɗin karatu!

Apache library Kafka Streams ana amfani da su a duk duniya a cikin masana'antu don rarraba rafi a saman Apache Kafka. Ɗaya daga cikin abubuwan da ba a yaba da wannan tsarin shi ne cewa yana ba ku damar adana jihar gida da aka samar bisa tushen sarrafa zaren.

A cikin wannan labarin, zan gaya muku yadda kamfaninmu ya yi nasarar yin amfani da wannan damar da riba yayin haɓaka samfur don amincin aikace-aikacen girgije. Ta amfani da Kafka Streams, mun ƙirƙiri microservices na jihohi da aka raba, kowannensu yana aiki azaman mai jurewa ga kuskure da samun ingantaccen tushen ingantaccen bayani game da yanayin abubuwa a cikin tsarin. A gare mu, wannan mataki ne na gaba duka dangane da dogaro da sauƙin tallafi.

Idan kuna sha'awar wata hanyar da za ta ba ku damar amfani da rumbun adana bayanai guda ɗaya don tallafawa yanayin abubuwanku na yau da kullun, karanta shi, zai zama mai ban sha'awa ...

Dalilin da ya sa muke tunanin lokaci ya yi da za mu canza yadda muke aiki tare da jihar da aka raba

Muna buƙatar kiyaye yanayin abubuwa daban-daban dangane da rahotannin wakilai (misali: an kai hari a wurin)? Kafin yin ƙaura zuwa Rafukan Kafka, sau da yawa muna dogara da tushen bayanai guda ɗaya (+ API service) don gudanar da jiha. Wannan tsarin yana da nasa drawbacks: kwanan wata m yanayi kiyaye daidaito da aiki tare ya zama ƙalubale na gaske. Database na iya zama ƙulli ko ƙarewa a ciki yanayin tsere kuma suna fama da rashin tabbas.

Hoto na 1: Halin yanayin rarrabuwar kawuna da aka gani kafin sauyawa zuwa
Rafukan Kafka da Kafka: wakilai suna sadar da ra'ayoyinsu ta hanyar API, ana ƙididdige jihar da aka sabunta ta hanyar cibiyar bayanai ta tsakiya

Haɗu da Rafukan Kafka, yana sauƙaƙa ƙirƙirar microservices na jiha

Kimanin shekara guda da ta wuce, mun yanke shawarar yin nazari sosai kan yanayin da muke ciki a jihar don magance wadannan batutuwa. Nan da nan muka yanke shawarar gwada Rafukan Kafka - mun san yadda za'a iya daidaitawa, samuwa sosai da kuma jure rashin kuskure, menene wadataccen aikin yawo da yake da shi (canzawa, gami da na jihohi). Kawai abin da muke buƙata, ba tare da ambaton yadda balagagge da ingantaccen tsarin saƙon ya zama a cikin Kafka ba.

Kowane ɗayan ingantattun ƙananan sabis ɗin da muka ƙirƙira an gina su ne a saman misalin Rafukan Kafka tare da ingantaccen yanayi mai sauƙi. Ya ƙunshi 1) tushen 2) na'ura mai sarrafawa tare da kantin sayar da ƙima mai mahimmanci 3) nutsewa:

Hoto na 2: Tsoffin topology na misalan yawo na mu don ƙananan ayyuka na jihohi. Lura cewa akwai ma wurin ajiya a nan wanda ya ƙunshi tsara bayanai.

A cikin wannan sabuwar hanyar, wakilai suna tsara saƙonnin da aka ciyar da su a cikin batun tushen, kuma masu amfani - sun ce, sabis na sanarwar wasiku - suna karɓar lissafin da aka raba ta hanyar nutsewa (fitowar batu).

Hoto 3: Sabon aikin misali na gudana don yanayin labari tare da microservices masu rabawa: 1) wakili ya haifar da saƙon da ya isa kan batun tushen Kafka; 2) microservice tare da jihar da aka raba (ta amfani da Kafka Streams) yana aiwatar da shi kuma ya rubuta jihar da aka ƙididdige zuwa batun Kafka na ƙarshe; bayan haka 3) masu amfani sun yarda da sabuwar jihar

Hey, wannan ginannen kantin sayar da ƙima a haƙiƙa yana da amfani sosai!

Kamar yadda aka ambata a sama, yanayin yanayin mu na jiharmu yana ƙunshe da mahimmin kantin sayar da ƙima. Mun sami zaɓuɓɓuka da yawa don amfani da shi, kuma an kwatanta biyu daga cikinsu a ƙasa.

Zabin #1: Yi amfani da maɓalli mai ƙima don ƙididdigewa

Shagon mu na maɓalli na farko ya ƙunshi bayanan taimako da muke buƙata don ƙididdigewa. Alal misali, a wasu lokuta an ƙayyade jihar da aka raba bisa ka'idar "mafi rinjaye". Wurin ajiya na iya ɗaukar duk sabbin rahotannin wakili akan matsayin wani abu. Sa'an nan, idan muka sami sabon rahoto daga wani wakili ko wani, za mu iya ajiye shi, maido da rahotanni daga duk sauran wakilai game da yanayin abu ɗaya daga ajiya, kuma maimaita lissafin.
Hoto na 4 da ke ƙasa yana nuna yadda muka fallasa maɓallan maɓalli/darajar ga hanyar sarrafa kayan sarrafawa ta yadda za a iya sarrafa sabon saƙon.

Misali na 4: Muna buɗe damar shiga kantin sayar da ƙima don hanyar sarrafa kayan sarrafawa (bayan wannan, kowane rubutun da ke aiki tare da jihar da aka raba dole ne ya aiwatar da hanyar. doProcess)

Zabin #2: Ƙirƙirar API na CRUD a saman Rafukan Kafka

Bayan kafa tushen aikin mu na yau da kullun, mun fara ƙoƙarin rubuta RESTful CRUD API don ƙananan ayyukan mu na jiha. Muna so mu sami damar dawo da yanayin wasu ko duk abubuwa, da kuma saita ko cire yanayin wani abu (mai amfani don tallafin baya).

Don tallafawa duk Samar da APIs na Jiha, duk lokacin da muke buƙatar sake ƙididdige jihar yayin sarrafawa, mun adana shi a cikin babban maɓalli mai ƙima na dogon lokaci. A wannan yanayin, ya zama mai sauƙi don aiwatar da irin wannan API ta amfani da misali guda ɗaya na Rafukan Kafka, kamar yadda aka nuna a cikin jeri na ƙasa:

Hoto 5: Yin amfani da ginanniyar maɓalli mai ƙima don samun yanayin da aka riga aka ƙirga na abu

Ana ɗaukaka yanayin abu ta hanyar API kuma yana da sauƙin aiwatarwa. Ainihin, duk abin da kuke buƙatar ku yi shine ƙirƙirar mai samar da Kafka kuma kuyi amfani da shi don yin rikodin da ke ɗauke da sabon jihar. Wannan yana tabbatar da cewa duk saƙonnin da aka samar ta hanyar API za a sarrafa su kamar yadda aka karɓa daga wasu furodusoshi (misali wakilai).

Hoto na 6: Kuna iya saita yanayin abu ta amfani da mai samar da Kafka

Ƙananan rikitarwa: Kafka yana da ɓangarori da yawa

Na gaba, muna so mu rarraba nauyin sarrafawa da haɓaka samuwa ta hanyar samar da gungu na ƙananan sabis na jihohi a kowane yanayi. Saitin ya kasance iska: da zarar mun saita duk lokuta don gudana ƙarƙashin ID ɗin aikace-aikacen iri ɗaya (da sabar bootstrap iri ɗaya), kusan komai an yi shi ta atomatik. Mun kuma ayyana cewa kowane jigo na tushen zai ƙunshi ɓangarori da yawa, ta yadda kowane misali za a iya sanya juzu'i na irin waɗannan ɓangarori.

Zan kuma ambaci cewa al'ada ce ta gama gari don yin kwafin ajiyar ajiya na kantin sayar da kayayyaki ta yadda, alal misali, idan an sake dawowa bayan gazawar, canja wurin wannan kwafin zuwa wani misali. Ga kowane kantin sayar da jaha a cikin Rafukan Kafka, an ƙirƙiri wani jigon da aka kwafi tare da rajistan canji (wanda ke bibiyar sabuntawar gida). Don haka, Kafka koyaushe yana tallafawa kantin sayar da jihar. Sabili da haka, a cikin yanayin rashin nasara na ɗaya ko wani misali na Kafka Streams, ana iya dawo da kantin sayar da jihar da sauri a wani misali, inda sassan da suka dace zasu tafi. Gwaje-gwajen da muka yi sun nuna cewa ana yin hakan a cikin daƙiƙa kaɗan, koda kuwa akwai miliyoyin bayanai a cikin shagon.

Motsawa daga microservice guda ɗaya tare da jihar da aka raba zuwa gungu na ƙananan sabis, ya zama ƙasa da ƙarancin aiwatar da Get State API. A cikin sabon halin da ake ciki, kantin sayar da jihar na kowane microservice yana ƙunshe da wani ɓangare na cikakken hoto (waɗannan abubuwan da aka tsara maɓallan su zuwa wani yanki na musamman). Dole ne mu tantance ko wane misali ne ya ƙunshi yanayin abin da muke buƙata, kuma mun yi hakan bisa ga zaren metadata, kamar yadda aka nuna a ƙasa:

Hoto na 7: Yin amfani da metadata na rafi, muna ƙayyade daga wane misali don tambayar yanayin abin da ake so; An yi amfani da irin wannan hanya tare da GET ALL API

Nemo Mabuɗi

Shagunan Jiha a cikin Rafukan Kafka na iya zama bayanan da aka rarraba a zahiri,

• akai-akai a cikin Kafka
• Ana iya gina API na CRUD a saman irin wannan tsarin cikin sauƙi
• Karɓar ɓangarori da yawa ya ɗan fi rikitarwa
• Hakanan yana yiwuwa a ƙara shagunan jihohi ɗaya ko fiye zuwa topology mai yawo don adana bayanan taimako. Ana iya amfani da wannan zaɓi don:
• Adana bayanan dogon lokaci da ake buƙata don ƙididdigewa yayin sarrafa rafi
• Adana bayanai na dogon lokaci wanda zai iya zama da amfani a gaba lokacin da aka samar da misalin yawo
• yafi...

Waɗannan da sauran fa'idodi sun sa Rafukan Kafka ya dace sosai don kiyaye yanayin duniya a cikin tsarin rarraba kamar namu. Kafka Streams ya tabbatar da kasancewa abin dogaro sosai a samarwa (ba mu sami asarar saƙo ba tun lokacin da aka tura shi), kuma muna da tabbacin cewa ƙarfinsa ba zai tsaya a can ba!

source: www.habr.com