Hei Habr!
Tinokuyeuchidza kuti kutevera bhuku nezve takaburitsa basa rinonakidza zvakaenzana nezveraibhurari .
Parizvino, nharaunda iri kungodzidza miganhu yechishandiso chine simba ichi. Saka, chinyorwa chakaburitswa nguva pfupi yadarika, shanduro yatinoda kukuzivisa kwairi. Kubva pane zvakaitika kwaari, munyori anoudza maitiro ekushandura Kafka Streams kuita yakagoverwa yekuchengetedza data. Nakidzwa nekuverenga!
Apache library inoshandiswa pasi rese mumabhizinesi ekuparadzira kurukova kugadzirisa pamusoro peApache Kafka. Chimwe chezvinhu zvisingakoshesirwe chechimiro ichi ndechekuti chinokutendera iwe kuti uchengete yenzvimbo inogadzirwa zvichibva pakugadziriswa kweshinda.
Muchikamu chino, ini ndichakuudza kuti kambani yedu yakakwanisa sei kushandisa mukana uyu zvine pundutso pakugadzira chigadzirwa chekuchengetedza kwegore application. Tichishandisa Kafka Streams, isu takagadzira yakagovaniswa ma microservices ehurumende, imwe neimwe inoshanda senge ine mhosva-inoshivirira uye inowanikwa zvakanyanya sosi yeruzivo rwakavimbika nezve mamiriro ezvinhu muhurongwa. Kwatiri, iyi inhanho mberi zvose maererano nekuvimbika uye nyore kwekutsigirwa.
Kana iwe uchifarira imwe nzira inobvumidza iwe kushandisa imwechete yepakati dhatabhesi kutsigira iyo yakarongwa mamiriro ezvinhu ezvinhu zvako, iverenge, ichave inonakidza ...
Nei takafunga kuti yaive nguva yekuchinja nzira yatinoshanda nayo nenyika yakagovaniswa
Taifanira kuchengetedza mamiriro ezvinhu akasiyana-siyana zvichibva pamishumo yeagent (semuenzaniso: nzvimbo yacho yanga ichirwiswa)? Tisati tatamira kuKafka Streams, taiwanzovimba nedatabase yepakati (+ service API) yekutonga kwenyika. Iyi nzira ine zvipingamupinyi zvayo: kuchengetedza kuenderana uye kuwiriranisa kunova dambudziko chairo. Iyo dhatabhesi inogona kuita bhodhoro kana kupera mukati uye kutambura nekusafungira.
Mufananidzo 1: Yakajairika kupatsanura-nyika mamiriro akaonekwa isati yachinja kuenda
Kafka neKafka Streams: vamiririri vanotaura maonero avo kuburikidza neAPI, yakagadziridzwa mamiriro anoverengerwa kuburikidza nepakati dhatabhesi.
Sangana neKafka Streams, zvichiita kuti zvive nyore kugadzira yakagovaniswa mamiriro mamicroservices
Rinenge gore rapfuura, takasarudza kunyatsotarisa mamiriro edu ehurumende akagovana kugadzirisa nyaya idzi. Isu takabva tangofunga kuyedza Kafka Streams - isu tinoziva kuti inogona scalable, inowanikwa zvakanyanya uye inoshivirira-kukanganisa, ndeyei yakapfuma yekufambisa mashandiro ayo inayo (shanduko, kusanganisira idzo dzakasimba). Izvo zvataida, tisingataure kuti yakakura uye yakavimbika sei sisitimu yekutumira mameseji muKafka.
Imwe neimwe yema microservices yatakagadzira yakavakirwa pamusoro peiyo Kafka Streams muenzaniso ine yakapusa topology. Iyo yaisanganisira 1) sosi 2) processor ine inopfuurira kiyi-yakakosha chitoro 3) singi:
Mufananidzo 2: Iyo yekusarudzika topology yemakesi edu ekutepfenyura ehurumende microservices. Ziva kuti pane zvakare repository pano ine metadata yekuronga.
Mune iyi nzira nyowani, vamiririri vanonyora mameseji anopihwa mumusoro wenyaya, uye vatengi - toti, sevhisi yekuzivisa tsamba - vanogashira iyo computed yakagovaniswa mamiriro kuburikidza nekunyura (inobuda musoro).
Mufananidzo 3: Mutsva wemuenzaniso webasa rinoyerera nokuda kwechiitiko chine microservices yakagoverwa: 1) mumiririri anogadzira mharidzo inosvika paKafka source topic; 2) microservice ine yakagovaniswa nyika (ichishandisa Kafka Streams) inoigadzirisa uye inonyora iyo yakaverengerwa mamiriro kune yekupedzisira Kafka musoro; mushure mezvo 3) vatengi vanogamuchira nyika itsva
Hei, ichi chakavakirwa-mukati kiyi-kukosha chitoro chinonyatso batsira!
Sezvambotaurwa pamusoro, yedu yakagovaniswa state topology ine kiyi-yakakosha chitoro. Takawana sarudzo dzinoverengeka dzekuishandisa, uye mbiri dzadzo dzinotsanangurwa pazasi.
Sarudzo #1: Shandisa kiyi-yakakosha chitoro chekuverenga
Yedu yekutanga kiyi-kukosha chitoro chaive nerubatsiro data rataida pakuverenga. Semuenzaniso, mune zvimwe zviitiko nyika yakagovaniswa yakatemwa nemusimboti we "mavhoti akawanda". Iyo repository inogona kubata ese azvino agent mishumo pachimiro chechimwe chinhu. Zvino, patakagamuchira mushumo mutsva kubva kune mumwe mumiririri kana mumwe, taigona kuichengeta, kutora mishumo kubva kune vamwe vese vamiririri nezve mamiriro echinhu chimwe chete kubva mukuchengetedza, uye kudzokorora kuverenga.
Mufananidzo 4 pazasi unoratidza kufumura kwatakaita kiyi / kukosha chitoro kune processor's yekugadzirisa nzira kuitira kuti meseji nyowani igozogadziriswa.
Mufananidzo 4: Isu tinovhura mukana kune kiyi-yakakosha chitoro cheiyo processor yekugadzirisa nzira (mushure meizvi, yega yega script inoshanda nenyika yakagovaniswa inofanirwa kuita iyo nzira. doProcess)
Sarudzo #2: Kugadzira CRUD API pamusoro peKafka Streams
Mushure mekutangisa basa redu rekutanga kuyerera, takatanga kuyedza kunyora RESTful CRUD API yema microservices edu akagovaniswa. Isu taida kukwanisa kudzoreredza mamiriro ezvimwe kana zvese zvinhu, pamwe nekuseta kana kubvisa mamiriro echinhu (chinobatsira kune backend rutsigiro).
Kutsigira ese Get State APIs, pese pataida kuverengera nyika panguva yekugadziriswa, takaichengeta muchitoro chakavakirwa-mukati kiyi-kukosha kwenguva yakareba. Muchiitiko ichi, zvinova nyore kushandisa API yakadaro uchishandisa imwe muenzaniso yeKafka Streams, sezvakaratidzwa mune rondedzero pazasi:
Mufananidzo 5: Kushandisa yakavakirwa-mukati kiyi-kukosha chitoro kuwana precomputed mamiriro echinhu.
Kuvandudza mamiriro echinhu kuburikidza neAPI zvakare nyore kuita. Chaizvoizvo, zvese zvaunofanirwa kuita kugadzira mugadziri weKafka uye woishandisa kugadzira rekodhi ine nyika nyowani. Izvi zvinova nechokwadi chekuti mameseji ese anogadzirwa kuburikidza neAPI anozogadziriswa nenzira imwechete neaya anogamuchirwa kubva kune vamwe vanogadzira (semuenzaniso vamiririri).
Mufananidzo 6: Unogona kuseta mamiriro echinhu uchishandisa Kafka mugadziri
Diki complication: Kafka ine akawanda partitions
Tevere, isu taida kugovera iyo yekugadzirisa mutoro uye nekuvandudza kuwanikwa nekupa sumbu rekugoverana-nyika microservices pamamiriro ezvinhu. Setup yaive mhepo: kana tangogadzirisa zviitiko zvese kuti zvifambe pasi peiyo yakafanana application ID (uye yakafanana bootstrap maseva), zvimwe zvese zvaiitwa otomatiki. Isu takatsanangurawo kuti chinyorwa chega chega chaizove nezvikamu zvakati wandei, kuitira kuti chiitiko chimwe nechimwe chigove chikamu chechikamu chezvikamu zvakadaro.
Ini zvakare ndichataura kuti zvakajairika kuita kopi yekuchengetedza yechitoro chehurumende kuitira kuti, semuenzaniso, kana ukapora mushure mekutadza, endesa iyi kopi kune imwe chiitiko. Kune yega yega chitoro chenyika muKafka Streams, yakadzokororwa musoro inogadzirwa nerogi yekuchinja (iyo inoteedzera zvigadziriso zvemuno). Saka, Kafka inogara ichitsigira chitoro chehurumende. Naizvozvo, kana paine kutadza kweimwe kana imwe Kafka Streams muenzaniso, chitoro chehurumende chinogona kukurumidza kudzoserwa pane imwe chiitiko, uko kunoenderana zvikamu zvichaenda. Miedzo yedu yakaratidza kuti izvi zvinoitwa mumasekondi mashoma, kunyangwe paine mamirioni emarekodhi muchitoro.
Kufamba kubva kune imwechete microservice ine yakagovaniswa nyika kuenda kuboka remicroservices, zvinova zvishoma diki kuita iyo Get State API. Mumamiriro ezvinhu matsva, chitoro chehurumende che microservice yega yega chinongova nechikamu chemufananidzo wakazara (izvo zvinhu zvine makiyi akaiswa mepu kune yakatarwa chikamu). Taifanira kuona kuti ndeipi muenzaniso yaive nemamiriro echinhu chataida, uye isu takaita izvi zvichibva pashinda metadata, sezvaratidzwa pazasi:
Mufananidzo 7: Tichishandisa metadata yerukova, tinosarudza kubva pamuenzaniso upi kuti tibvunze mamiriro echinhu chaunoda; nzira yakafanana yakashandiswa neGET ALL API
Main mhedziso
Zvitoro zvehurumende muKafka Streams zvinogona kushanda sede facto yakagoverwa dhatabhesi,
- inogara ichidzokororwa muKafka
- A CRUD API inogona kuvakwa nyore pamusoro peiyo system
- Kubata akawanda partitions kwakatonyanya kuoma
- Izvo zvakare zvinogoneka kuwedzera imwe kana zvimwe zvitoro zvehurumende kune yekutepfenyura topology kuchengetedza data rekubatsira. Iyi sarudzo inogona kushandiswa kune:
- Yenguva refu yekuchengetedza data inodiwa pakuverenga panguva yekuyerera kwerukova
- Kuchengetedzwa kwenguva refu kwedata iyo inogona kubatsira panguva inotevera apo muenzaniso wekutepfenyura wapihwa
- zvakawanda...
Izvi nezvimwe zvakanaka zvinoita kuti Kafka Streams inyatsokodzera kuchengetedza nyika yepasi rose muhurongwa hwakagoverwa sewedu. Kafka Streams yakaratidza kuve yakavimbika kwazvo mukugadzira (hatina kana meseji kurasikirwa kubva payaitumira), uye tine chivimbo chekuti kugona kwayo hakugumire ipapo!
Source: www.habr.com