Incwadi ethi “Kafka Streams in Action. Izicelo kanye nezinsizakalo ezincane zomsebenzi wesikhathi sangempela"

Sawubona, bahlali baseKhabro! Leli bhuku lifanele noma yimuphi unjiniyela ofuna ukuqonda ukucutshungulwa kochungechunge. Ukuqonda izinhlelo ezisabalalisiwe kuzokusiza uqonde kangcono i-Kafka ne-Kafka Streams. Kungaba kuhle ukwazi uhlaka lwe-Kafka ngokwalo, kodwa lokhu akudingekile: ngizokutshela konke okudingayo. Onjiniyela be-Kafka abanolwazi nabaqalayo bazofunda ukuthi bangakha kanjani izinhlelo zokusebenza ezithokozisayo zokucubungula ukusakaza kusetshenziswa ilabhulali ye-Kafka Streams kuleli bhuku. Onjiniyela be-Java abamaphakathi nabathuthukisiwe asebejwayelene nemiqondo efana nokwenza uchungechunge bazofunda ukusebenzisa amakhono abo ukuze bakhe izinhlelo zokusebenza ze-Kafka Streams. Ikhodi yomthombo wencwadi ibhalwe ku-Java 8 futhi isebenzisa kakhulu i-Java 8 lambda expression syntax, ngakho-ke ukwazi ukusebenza ngemisebenzi ye-lambda (ngisho nangolunye ulimi lokuhlela) kuzoba usizo.

Ingcaphuno. 5.3. Imisebenzi yokuhlanganisa nokufaka amafasitela

Kulesi sigaba, sizoqhubeka nokuhlola izingxenye ezithembisa kakhulu ze-Kafka Streams. Kuze kube manje sihlanganise izici ezilandelayo ze-Kafka Streams:

• ukudala i-topology yokucubungula;
• ukusebenzisa isimo ekusakazeni izinhlelo zokusebenza;
• ukwenza uxhumano lokusakazwa kwedatha;
• umehluko phakathi kokusakazwa komcimbi (KStream) nokuvuselela ukusakaza (KTable).

Ezibonelweni ezilandelayo sizohlanganisa zonke lezi zakhi. Uzofunda futhi mayelana nokwenza iwindi, esinye isici esihle sokusakaza izinhlelo zokusebenza. Isibonelo sethu sokuqala kuzoba ukuhlanganisa okulula.

5.3.1. Ukuhlanganiswa kokuthengiswa kwamasheya ngomkhakha wemboni

Ukuhlanganisa nokuqoqa kungamathuluzi abalulekile uma usebenza ngokusakaza idatha. Ukuhlolwa kwamarekhodi ngamanye njengoba amukelwa ngokuvamile akwanele. Ukukhipha ulwazi olwengeziwe kudatha, kuyadingeka ukuhlanganisa nokuhlanganisa.

Kulesi sibonelo, uzogqoka imvunulo yomthengisi wosuku odinga ukulandelela umthamo wokuthengisa wamasheya ezinkampani ezimbonini ezimbalwa. Ngokukhethekile, unentshisekelo ezinkampanini ezinhlanu ezinokuthengiswa okukhulu kwamasheya embonini ngayinye.

Ukuhlanganisa okunjalo kuzodinga izinyathelo ezimbalwa ezilandelayo ukuze uhumushe idatha efomini oyifunayo (ukukhuluma ngokujwayelekile).

1. Dala umthombo osuselwe esihlokweni oshicilela ulwazi lokuhweba ngamasheya. Kuzodingeka senze imephu into yohlobo lwe-StockTransaction entweni yohlobo lwe-ShareVolume. Iphuzu liwukuthi into ye-StockTransaction iqukethe imethadatha yokuthengisa, kodwa sidinga kuphela idatha mayelana nenani lamasheya athengiswayo.
2. Idatha ye-ShareVolume yeqembu ngophawu lwesitoko. Uma isiqoqwe ngophawu, ungagoqa le datha ibe ngamanani amancane amavolumu okuthengiswa kwesitoko. Kubalulekile ukuqaphela ukuthi indlela ye-KStream.groupBy ibuyisela isibonelo sohlobo lwe-KGroupedStream. Futhi ungathola isibonelo se-KTable ngokuqhubeka ngokushayela indlela ye-KGroupedStream.reduce.

Siyini isixhumi esibonakalayo se-KGroupedStream

Izindlela ze-KStream.groupBy kanye ne-KStream.groupByKey zibuyisela isibonelo se-KGroupedStream. I-KGroupedStream iwumfanekiso omaphakathi wochungechunge lwemicimbi ngemva kokuqoqwa ngokhiye. Akuhloselwe nhlobo ukusebenza ngqo nayo. Kunalokho, i-KGroupedStream isetshenziselwa imisebenzi yokuhlanganisa, ehlala iphumela ku-KTable. Futhi njengoba umphumela wokusebenza kokuhlanganisa kuyi-KTable futhi basebenzisa isitolo sikahulumeni, kungenzeka ukuthi akuzona zonke izibuyekezo ezithunyelwayo ngokuqhubekayo.

Indlela ye-KTable.groupBy ibuyisela i-KGroupedTable efanayo - ukumelwa okuphakathi kokusakazwa kwezibuyekezo, ezihlanganiswe kabusha ngokhiye.

Ake sithi ukuphumula kancane sibheke Fig. 5.9, ekhombisa lokho esikuzuzile. Le topology kufanele isivele yazi kakhulu kuwe.

Manje ake sibheke ikhodi yale topology (ingatholakala kufayela elithi src/main/java/bbejeck/chapter_5/AggregationsAndReducingExample.java) (Uhlu 5.2).

Ikhodi enikeziwe ihlukaniswa ubufushane bayo kanye nevolumu enkulu yezenzo ezenziwa emigqeni eminingana. Ungase uqaphele okuthile okusha kupharamitha yokuqala yendlela ye-builder.stream: inani lohlobo lwe-enum AutoOffsetReset.EARLIEST (kukhona neyakamuva), isethwe kusetshenziswa indlela ye-Consumed.withOffsetResetPolicy. Lolu hlobo lokubala lungasetshenziswa ukuze kucaciswe isu lokusetha kabusha le-KStream ngayinye noma i-KTable ngayinye futhi liza kuqala kunenketho yokusetha kabusha yokusetha kabusha kusukela ekucushweni.

I-GroupByKey kanye ne-GroupBy

I-interface ye-KStream inezindlela ezimbili zokuqoqa amarekhodi: I-GroupByKey ne-GroupBy. Zombili zibuyisela i-KGroupedTable, ngakho-ke ungahle uzibuze ukuthi uyini umehluko phakathi kwakho nokuthi kufanele uyisebenzise nini?

Indlela ye-GroupByKey isetshenziswa uma okhiye ku-KStream sebevele bengenalutho. Futhi okubaluleke kakhulu, ifulegi elithi "idinga ukuhlukaniswa kabusha" alizange lisethwe.

Indlela ye-GroupBy ithatha ngokuthi ushintshe okhiye bokuqoqa, ngakho ifulegi lokuhlukanisa lisethwe kuqiniso. Ukwenza ukujoyina, ukuhlanganisa, njll. ngemva kwendlela ye-GroupBy kuzoholela ekuhlukaniseni kabusha okuzenzakalelayo.
Isifinyezo: Noma nini lapho kunokwenzeka, kufanele usebenzise i-GroupByKey kune-GroupBy.

Kuyacaca ukuthi i-mapValues ​​nezindlela ze-groupBy zenzani, ngakho-ke ake sibheke isamba () indlela (etholakala ku-src/main/java/bbejeck/model/ShareVolume.java) (Uhlu 5.3).

Indlela ye-ShareVolume.sum ibuyisela inani elisebenzayo levolumu yokuthengiswa kwesitoko, futhi umphumela walo lonke uchungechunge lwezibalo uyinto ye-KTable. . Manje usuyayiqonda indima edlalwa yi-KTable. Lapho izinto ze-ShareVolume zifika, into ehambisanayo ye-KTable igcina isibuyekezo sakamuva. Kubalulekile ukukhumbula ukuthi zonke izibuyekezo ziboniswa ku-shareVolumeKTable yangaphambilini, kodwa akuzona zonke ezithunyelwa ngokuqhubekayo.

Sibe sesisebenzisa le KTable ukuze sihlanganise (ngenombolo yamasheya adayiswayo) ukuze sifike ezinkampanini ezinhlanu ezinenani eliphezulu kakhulu lamasheya ahwebayo embonini ngayinye. Izenzo zethu kuleli cala zizofana nalezo zokuhlanganiswa kokuqala.

1. Yenza elinye iqembuNgokusebenza ukuqoqa izinto ze-ShareVolume ngayinye ngomkhakha.
2. Qala ukufingqa izinto ze-ShareVolume. Kulokhu into yokuhlanganisa iwumugqa obalulekile wosayizi ongashintshi. Kulo mugqa wosayizi ongashintshi, yizinkampani ezinhlanu kuphela ezinamanani amakhulu wamasheya athengisiwe ezigcinwayo.
3. Imephu yolayini ukusuka esigabeni sangaphambilini iye kunani leyunithi yezinhlamvu bese ubuyisela izitoko ezinhlanu eziphezulu ezihweba kakhulu ngenombolo ngemboni.
4. Bhala imiphumela ngeyunithi yezinhlamvu esihlokweni.

Emfanekisweni. Umfanekiso 5.10 ubonisa igrafu yokugeleza kwedatha. Njengoba ubona, umjikelezo wesibili wokucubungula ulula.

Manje njengoba sesinokuqonda okucacile kwesakhiwo salo mzuliswano wesibili wokucubungula, singaphendukela kukhodi yawo yomthombo (uzoyithola kufayela elithi src/main/java/bbejeck/chapter_5/AggregationsAndReducingExample.java) (Uhlu 5.4) .

Lesi siqalisi siqukethe okuguquguqukayo koQueue. Lena into yangokwezifiso eyi-adaptha ye-java.util.TreeSet esetshenziselwa ukulandelela imiphumela ephezulu engu-N ngohlelo olwehlayo lwamasheya adayiswayo.

Usuvele ulibonile izingcingo zeqembuBy kanye nemephuValues, ngakho-ke ngeke singene kulawo (sibiza indlela ye-KTable.toStream ngoba indlela ye-KTable.print ihoxisiwe). Kodwa awukakayiboni inguqulo ye-KTable ye-aggregate() okwamanje, ngakho-ke sizochitha isikhathi esincane sixoxa ngalokho.

Njengoba ukhumbula, okwenza i-KTable ihluke ukuthi amarekhodi anokhiye abafanayo abhekwa njengezibuyekezo. I-KTable ithatha indawo yokufaka endala ngendawo entsha. Ukuhlanganisa kwenzeka ngendlela efanayo: amarekhodi akamuva anokhiye ofanayo ahlanganisiwe. Uma irekhodi lifika, liyengezwa esibonelweni sekilasi le-FixedSizePriorityQueue kusetshenziswa i-adder (ipharamitha yesibili ocingweni lwendlela yokuhlanganisa), kodwa uma elinye irekhodi selivele likhona nokhiye ofanayo, irekhodi elidala liyasuswa kusetshenziswa i-subtractor (ipharamitha yesithathu ikholi yendlela ehlanganisiwe).

Lokhu konke kusho ukuthi i-aggregator yethu, i-FixedSizePriorityQueue, ayihlanganisi wonke amanani ngokhiye owodwa, kodwa igcina isamba esinyakazayo senani lezinhlobo zezitoko ezihweba kakhulu ezingu-N. Okufakiwe ngakunye okungenayo kuqukethe isamba senani lamasheya athengisiwe kuze kube manje. I-KTable izokunikeza ulwazi mayelana nokuthi yimaphi amasheya ezinkampani athengiswa kakhulu njengamanje, ngaphandle kokudinga ukuqoqwa kokuhlanganiswa kwesibuyekezo ngasinye.

Sifunde ukwenza izinto ezimbili ezibalulekile:

• amanani eqembu ku-KTable ngokhiye ojwayelekile;
• yenza imisebenzi ewusizo efana nokuhlanganisa nokuhlanganisa kulawa manani aqoqwe.

Ukwazi ukwenza le misebenzi kubalulekile ukuze uqonde incazelo yedatha ehamba ngohlelo lokusebenza lwe-Kafka Streams nokuqonda ukuthi iphethe luphi ulwazi.

Siphinde sahlanganisa amanye amasu abalulekile okukhulunywe ngawo ekuqaleni kwale ncwadi. Esahlukweni 4, sixoxe ngokuthi ukubekezelela amaphutha, isimo sendawo kubaluleke kangakanani kuhlelo lokusebenza lokusakaza-bukhoma. Isibonelo sokuqala kulesi sahluko sibonise ukuthi kungani isifunda sendawo sibaluleke kangaka—sikunikeza ikhono lokulandelela ukuthi yiluphi ulwazi osuvele ulubonile. Ukufinyelela kwasendaweni kugwema ukubambezeleka kwenethiwekhi, okwenza uhlelo lokusebenza lusebenze kakhulu futhi lumelane namaphutha.

Uma wenza umsebenzi wokugoqa noma wokuhlanganisa, kufanele ucacise igama lesitolo sikahulumeni. Imisebenzi yokuqoqwa nokuhlanganisa ibuyisela isibonelo se-KTable, futhi i-KTable isebenzisa isitoreji sesimo ukuze imiselele imiphumela emidala ngemisha. Njengoba ubonile, akuzona zonke izibuyekezo ezithunyelwayo, futhi lokhu kubalulekile ngoba imisebenzi yokuhlanganisa iklanyelwe ukukhiqiza ulwazi olufingqiwe. Uma ungasisebenzisi isimo sendawo, i-KTable izodlulisela yonke imiphumela yokuhlanganisa neyokugoqwa.

Okulandelayo, sizobheka ukwenza imisebenzi efana nokuhlanganisa phakathi nesikhathi esithile - lokho okubizwa ngokuthi imisebenzi yokufaka amawindi.

5.3.2. Ukusebenza kwewindi

Esigabeni esidlule, sethule i-convolution yokuslayida nokuhlanganisa. Uhlelo lokusebenza lwenze ukuqoqwa kwevolumu yokuthengiswa kwesitoko okuqhubekayo, okulandelwa ukuhlanganiswa kwezitoko ezinhlanu ezihweba kakhulu emakethe.

Kwesinye isikhathi ukuhlanganiswa okunjalo okuqhubekayo kanye nokuqoqwa kwemiphumela kuyadingeka. Futhi ngezinye izikhathi udinga ukwenza imisebenzi kuphela ngesikhathi esithile. Isibonelo, bala ukuthi zingaki izinkokhelo ezenziwe ngamasheya enkampani ethile emizuzwini eyi-10 edlule. Noma bangaki abasebenzisi abachofoze isibhengezo esisha sokukhangisa emizuzwini engu-15 edlule. Uhlelo lokusebenza lungenza imisebenzi enjalo izikhathi eziningi, kodwa ngemiphumela esebenza ezikhathini ezithile kuphela (amawindi esikhathi).

Ibala okwenziwa ngumthengi

Esibonelweni esilandelayo, sizolandelela ukuthengiswa kwesitoko kubo bonke abadayisi abaningi—okungaba izinhlangano ezinkulu noma abaxhasi bezimali abahlakaniphile.

Kunezizathu ezimbili ezingaba khona zalokhu kulandelelwa. Esinye sazo isidingo sokwazi ukuthi abaholi bezimakethe bathenga/bathengisa ini. Uma laba badlali abakhulu nabatshalizimali abasezingeni eliphezulu bebona ithuba, kunengqondo ukulandela isu labo. Isizathu sesibili isifiso sokubona noma yiziphi izimpawu ezingaba khona zokuhweba kwangaphakathi okungekho emthethweni. Ukuze wenze lokhu, uzodinga ukuhlaziya ukuhlobana kwama-spikes amakhulu okuthengisa nokukhishwa kwezindaba ezibalulekile.

Ukulandelela okunjalo kuqukethe izinyathelo ezilandelayo:

• ukudala umfudlana wokufunda esihlokweni sokuthengiselana kwesitoko;
• ukuqoqa amarekhodi angenayo nge-ID yomthengi nophawu lwesitoko. Ukushayela iqembuNgendlela kubuyisela isibonelo sekilasi le-KGroupedStream;
• Indlela ye-KGroupedStream.windowedBy ibuyisela ukusakaza kwedatha okukhawulelwe efasiteleni lesikhathi, elivumela ukuhlanganisa okunefasitela. Kuye ngohlobo lwewindi, i-TimeWindowedKStream noma i-SessionWindowedKStream iyabuyiswa;
• inani lomsebenzi womsebenzi wokuhlanganisa. Ukugeleza kwedatha okufakwe ewindini kunquma ukuthi irekhodi elithile liyanakwa yini kulokhu kubalwa;
• ukubhala imiphumela esihlokweni noma ukuyikhiphela kukhonsoli ngesikhathi sokuthuthukiswa.

I-topology yalolu hlelo lokusebenza ilula, kodwa isithombe esicacile sayo singasiza. Ake sibheke i-Fig. 5.11.

Okulandelayo, sizobheka ukusebenza kokusebenza kwewindi kanye nekhodi ehambisanayo.

Izinhlobo zamawindi

Kunezinhlobo ezintathu zamawindi e-Kafka Streams:

• okwesikhathi;
• “ukuwa” (ukuwa);
• ngokushibilika/gxuma.

Iyiphi ongayikhetha incike ezidingweni zebhizinisi lakho. Amawindi okugingqika nokweqa akhawulelwe isikhathi, kuyilapho amawindi eseshini anqunyelwe umsebenzi womsebenzisi—ubude besikhathi se(ama)seshini bunqunywa kuphela ukuthi umsebenzisi usebenza kangakanani. Into esemqoka okufanele uyikhumbule ukuthi zonke izinhlobo zamawindi zisuselwa kuzitembu zedethi/isikhathi sokufakiwe, hhayi isikhathi sesistimu.

Okulandelayo, sisebenzisa i-topology yethu ngohlobo ngalunye lwewindi. Ikhodi ephelele izonikezwa kuphela esibonelweni sokuqala; kwezinye izinhlobo zamafasitela akukho okuzoshintsha ngaphandle kohlobo lokusebenza kwewindi.

Amawindi esikhathi

Amawindi esikhathi ahluke kakhulu kuzo zonke ezinye izinhlobo zamafasitela. Azikhawulelwanga kakhulu ngesikhathi njengomsebenzi womsebenzisi (noma umsebenzi webhizinisi ongathanda ukuwulandela). Amawindi esikhathi ahlukaniswa yizikhathi zokungasebenzi.

Umfanekiso 5.12 ubonisa umqondo wewindi leseshini. Iseshini encane izohlangana neseshini kwesokunxele sayo. Futhi iseshini engakwesokudla izohlukaniswa ngoba ilandela isikhathi eside sokungasebenzi. Amawindi esikhathi asekelwe kumsebenzi womsebenzisi, kodwa sebenzisa izitembu zedethi/isikhathi kokufakiwe ukuze unqume ukuthi iyiphi iseshini okufakwayo okungekayo.

Kusetshenziswa amawindi eseshini ukulandelela ukuthengiselana kwesitoko

Masisebenzise amawindi eseshini ukuze sithwebule ulwazi mayelana nokuhwebelana. Ukuqaliswa kwamawindi esikhathi kuboniswa kokuthi Uhlu 5.5 (okungatholakala kokuthi src/main/java/bbejeck/chapter_5/CountingWindowingAndKTableJoinExample.java).

Usuyibonile imisebenzi eminingi kule topology, ngakho-ke asikho isidingo sokuyibheka futhi lapha. Kodwa futhi kunezici ezimbalwa ezintsha lapha, esizoxoxa ngazo manje.

Noma iyiphi i-groupBy operation ngokuvamile yenza uhlobo oluthile lomsebenzi wokuhlanganisa (ukuhlanganiswa, ukugoqa, noma ukubala). Ungenza noma ukuhlanganisa okuqongelelwe ngenani elisebenzayo, noma ukuhlanganiswa kwewindi, okucabangela amarekhodi ngaphakathi kwewindi lesikhathi elishiwo.

Ikhodi ekuHlulweni 5.5 ibala inani lemisebenzi phakathi kwewindi leseshini. Emfanekisweni. 5.13 lezi zenzo zihlaziywa isinyathelo ngesinyathelo.

Ngokubiza i-windowedBy(SessionWindows.with(twentySeconds).kuze kube(imizuzu eyishumi nanhlanu)) sakha iwindi leseshini elinesikhawu sokungasebenzi esingamasekhondi angu-20 kanye nesikhawu sokuphikelela esiyimizuzu eyi-15. Isikhawu sokungenzi lutho samasekhondi angu-20 sisho ukuthi isicelo sizofaka noma yikuphi ukufakwa okufika phakathi kwamasekhondi angu-20 ekupheleni noma ekuqaleni kweseshini yamanje kuseshini yamanje (esebenzayo).

Okulandelayo, sicacisa ukuthi imuphi umsebenzi wokuhlanganisa okudingeka wenziwe efasiteleni leseshini - kulokhu, bala. Uma okungenayo kuwela ngaphandle kwewindi lokungasebenzi (noma yiluphi uhlangothi lwesitembu sedethi/isikhathi), uhlelo lokusebenza ludala iseshini entsha. Isikhathi sokugcinwa sisho ukugcinwa kweseshini isikhathi esithile futhi kuvumela idatha engemuva kwesikhathi edlulela ngalé kwesikhathi sokungasebenzi seseshini kodwa isenganamathiselwa. Ukwengeza, ukuqala nokuphela kweseshini entsha ewumphumela wokuhlanganiswa kuhambelana nesitembu sangaphambili nesakamuva sedethi/isikhathi.

Ake sibheke okufakiwe okumbalwa endleleni yokubala ukuze sibone ukuthi amaseshini asebenza kanjani (Ithebula 5.1).

Lapho amarekhodi efika, sibheka izikhathi ezikhona ngokhiye ofanayo, isikhathi sokuphela esingaphansi kosuku lwamanje/isitembu sesikhathi - isikhawu sokungasebenzi, nesikhathi sokuqala esikhulu kunedethi yamanje/isitembu sesikhathi + nesikhawu sokungasebenzi. Ukucabangela lokhu, okufakiwe okune okuvela etafuleni. 5.1 zihlanganiswa zibe iseshini eyodwa kanje.

1. Irekhodi elingu-1 lifika kuqala, ngakho-ke isikhathi sokuqala silingana nesikhathi sokuphela futhi ngu-00:00:00.

2. Okulandelayo, ukungena 2 kuyafika, futhi sibheka amaseshini angapheli ngaphambi kuka-23:59:55 futhi aqale kungakedluli u-00:00:35. Sithola irekhodi 1 futhi sihlanganise iseshini 1 no-2. Sithatha isikhathi sokuqala seseshini 1 (ngaphambilini) kanye nesikhathi sokuphela seseshini 2 (kamuva), ukuze iseshini yethu entsha iqale ngo-00:00:00 futhi iphele ngo-00: 00:15.

3. Irekhodi elingu-3 lifika, sibheka izikhathi phakathi kuka-00:00:30 no-00:01:10 futhi asitholi lutho. Engeza iseshini yesibili yokhiye 123-345-654,FFBE, eqala futhi ephele ngo-00:00:50.

4. Irekhodi elingu-4 liyafika futhi sifuna izikhathi phakathi kuka-23:59:45 no-00:00:25. Kulokhu kuyatholakala zombili izikhathi zoku-1 no-2. Zontathu izikhathi zihlanganiswa zibe yinto eyodwa, isikhathi sokuqala esingu-00:00:00 nesikhathi sokuphela esingu-00:00:15.

Kusukela kulokho okuchazwe kulesi sigaba, kufanelekile ukukhumbula ama-nuances alandelayo abalulekile:

• amaseshini awawona amawindi anosayizi ongashintshi. Ubude beseshini bunqunywa umsebenzi phakathi nenkathi ethile yesikhathi;
• Izitembu zedethi/isikhathi kudatha zinquma ukuthi umcimbi uwela phakathi nesikhathi esikhona noma phakathi nesikhathi sokungenzi lutho.

Okulandelayo sizoxoxa ngohlobo olulandelayo lwewindi - amafasitela "awayo".

Amafasitela "ama-Tumling".

Amawindi adilikayo athwebula imicimbi ewela phakathi nenkathi ethile yesikhathi. Cabanga ukuthi udinga ukuthwebula konke ukuthengiselana kwesitoko senkampani ethile njalo ngemizuzwana engama-20, ukuze uqoqe yonke imicimbi ngaleso sikhathi. Ekupheleni kwesikhawu samasekhondi angu-20, iwindi liyagingqika bese liya endaweni entsha yokubuka engamasekhondi angu-20. Umfanekiso 5.14 ukhombisa lesi simo.

Njengoba ubona, yonke imicimbi etholwe emizuzwaneni engu-20 edlule ifakiwe efasiteleni. Ekupheleni kwalesi sikhathi, iwindi elisha liyadalwa.

Uhlu 5.6 lubonisa ikhodi ebonisa ukusetshenziswa kwamafasitela awela phansi ukuze uthwebule ukuthengiswa kwesitoko njalo ngemizuzwana engama-20 (kutholakala kokuthi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java).

Ngalolu shintsho oluncane ocingweni lwendlela ye-TimeWindows.of, ungasebenzisa iwindi eliwayo. Lesi sibonelo asiyibizi indlela ethi until(), ngakho-ke kuzosetshenziswa isikhawu sokugcina esizenzakalelayo samahora angu-24.

Ekugcineni, sekuyisikhathi sokudlulela kokugcina kwezinketho zewindi - "ukugxuma" amafasitela.

Amawindi ashelelayo ("ukweqa).

Amawindi ashelelayo/agxumayo afana namafasitela adilikayo, kodwa anomehluko omncane. Amawindi okuslayida awalindi kuze kube sekupheleni kwesikhathi ngaphambi kokudala iwindi elisha lokucubungula imicimbi yakamuva. Baqala izibalo ezintsha ngemva kwesikhathi sokulinda esingaphansi kobude bewindi.

Ukukhombisa umehluko phakathi kokuwa nokugxuma amafasitela, ake sibuyele esibonelweni sokubala ukuthengiswa kwesitoko. Umgomo wethu kusewukubala inani lemisebenzi, kodwa asifuni ukulinda sonke isikhathi ngaphambi kokubuyekeza ikhawunta. Kunalokho, sizobuyekeza ikhawunta ngezikhathi ezimfushane. Isibonelo, sisazobala inani lemisebenzi njalo ngemizuzwana engama-20, kodwa sibuyekeze isibali njalo ngemizuzwana emi-5, njengoba kuboniswe ku-Fig. 5.15. Kulokhu, sigcina sinemiphumela emithathu yamawindi anedatha egqagqene.

Uhlu 5.7 lubonisa ikhodi yokuchaza amawindi ashelelayo (atholakala kokuthi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java).

Iwindi eliwayo lingaguqulwa libe yifasitela lokugxuma ngokungeza ikholi ku- advanceBy() indlela. Esibonelweni esibonisiwe, isikhawu sokulondoloza siyimizuzu eyi-15.

Ubonile kulesi sigaba ukuthi ungakhawulela kanjani imiphumela yokuhlanganisa ibe amawindi esikhathi. Ikakhulukazi, ngifuna ukhumbule izinto ezintathu ezilandelayo kulesi sigaba:

• usayizi wewindi leseshini awunqunyelwe isikhathi, kodwa ngomsebenzi womsebenzisi;
• amafasitela “awayo” anikeza umbono wezenzakalo phakathi nenkathi ethile yesikhathi;
• Ubude besikhathi sokugxuma amawindi bulungisiwe, kodwa abuyekezwa njalo futhi angaqukatha okufakiwe okugqagqene kuwo wonke amawindi.

Okulandelayo, sizofunda ukuguqula i-KTable ibuyele ku-KStream ukuze kuxhunywe.

5.3.3. Ixhuma izinto ze-KStream ne-KTable

ESahlukweni 4, sixoxe ngokuxhumanisa izinto ezimbili ze-KStream. Manje kufanele sifunde indlela yokuxhuma i-KTable ne-KStream. Lokhu kungase kudingeke ngenxa yesizathu esilandelayo esilula. I-KStream iwuchungechunge lwamarekhodi, futhi i-KTable iwuchungechunge lwezibuyekezo zamarekhodi, kodwa ngezinye izikhathi ungase ufune ukwengeza umongo owengeziwe ekusakazweni kwerekhodi usebenzisa izibuyekezo ezivela ku-KTable.

Ake sithathe idatha ngenani lokushintshisana kwesitoko futhi sikuhlanganise nezindaba zokushintshaniswa kwesitoko ezimbonini ezifanele. Nakhu okudingeka ukwenze ukuze uzuze lokhu uma ubheka ikhodi osuvele unayo.

1. Guqula into ye-KTable enedatha yenani lokuthengiswa kwesitoko ibe i-KStream, okulandelwa wukushintsha ukhiye ngokhiye obonisa umkhakha wemboni ohambisana nalolu phawu lwesitoko.
2. Dala into ye-KTable efunda idatha esihlokweni esinezindaba zokushintshisana kwesitoko. Le KTable entsha izohlukaniswa ngomkhakha wezimboni.
3. Xhuma izibuyekezo zezindaba ngolwazi olumayelana nenani lemisebenzi yokushintshisana ngesitoko ngomkhakha womkhakha.

Manje ake sibone ukuthi singalusebenzisa kanjani lolu hlelo lokusebenza.

Guqula i-KTable ibe yi-KStream

Ukuguqula i-KTable ibe yi-KStream udinga ukwenza okulandelayo.

1. Shayela indlela ye-KTable.toStream().
2. Ngokubiza indlela ye-KStream.map, shintsha ukhiye ufake igama lemboni, bese ubuyisela into ye-TransactionSummary kusibonelo se-Windodow.

Sizohlanganisa le misebenzi ngendlela elandelayo (ikhodi ingatholakala kufayela elithi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uhlu 5.8).

Ngoba senza umsebenzi we-KStream.map, isibonelo se-KStream esibuyisiwe sihlukaniswa kabusha ngokuzenzakalelayo uma sisetshenziswa ekuxhumekeni.

Sesiyiqedile inqubo yokuguqula, ngokulandelayo sidinga ukudala into ye-KTable yokufunda izindaba zesitoko.

Ukudalwa kwe-KTable yezindaba zesitoko

Ngenhlanhla, ukwakha into ye-KTable kuthatha umugqa owodwa nje wekhodi (ikhodi ingatholakala kokuthi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uhlu 5.9).

Kuyaphawuleka ukuthi azikho izinto ze-Serde ezidingekayo ukuze zicaciswe, njengoba i-string Serdes isetshenziswa kuzilungiselelo. Futhi, ngokusebenzisa ukubala KWAKUQALA, ithebula ligcwaliswa ngamarekhodi ekuqaleni.

Manje singadlulela esinyathelweni sokugcina - ukuxhumana.

Ixhuma izibuyekezo zezindaba ngedatha yokubala okwenziwe

Ukudala uxhumano akunzima. Sizosebenzisa ukujoyina kwesokunxele uma kwenzeka zingekho izindaba zesitoko zemboni efanelekile (ikhodi edingekayo ingatholakala kufayela elithi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uhlu 5.10).

Lo opharetha we-LeftJoin ulula kakhulu. Ngokungafani namajoyini eSahluko 4, indlela ye-JoinWindow ayisetshenziswa ngoba uma wenza ukujoyina kwe-KStream-KTable, kunokufakwa okukodwa kuphela ku-KTable kukhiye ngamunye. Uxhumano olunjalo alunqunyelwe ngesikhathi: irekhodi liku-KTable noma alikho. Isiphetho esikhulu: usebenzisa izinto ze-KTable ungacebisa i-KStream ngedatha eyinkomba ebuyekezwa kancane.

Manje sizobheka indlela esebenza kahle kakhulu yokucebisa imicimbi evela ku-KStream.

5.3.4. Izinto ze-GlobalKTable

Njengoba ubona, kunesidingo sokucebisa ukusakazwa kwemicimbi noma ukwengeza umongo kukho. ESahlukweni 4 ubone ukuxhumana phakathi kwezinto ezimbili ze-KStream, futhi esigabeni sangaphambilini ubone ukuxhumana phakathi kwe-KStream ne-KTable. Kuzo zonke lezi zimo, kuyadingeka ukuphinda uhlukanise ukusakaza kwedatha lapho udweba okhiye bohlobo olusha noma inani. Kwesinye isikhathi ukuhlukaniswa kabusha kwenziwa ngokusobala, futhi kwesinye isikhathi i-Kafka Streams ikwenza ngokuzenzakalelayo. Ukuhlukaniswa kabusha kuyadingeka ngoba izihluthulelo zishintshile futhi amarekhodi kufanele agcine ezigabeni ezintsha, ngaphandle kwalokho uxhumano ngeke lube yinto engenakwenzeka (lokhu kwaxoxwa ngakho eSahlukweni 4, esigabeni "Ukuhlukaniswa kabusha kwedatha" esigatshaneni 4.2.4).

Ukuhlukanisa kabusha kunezindleko

Ukuhlukanisa kabusha kudinga izindleko - izindleko ezengeziwe zensiza zokudala izihloko eziphakathi nendawo, ukugcina idatha eyimpinda kwesinye isihloko; kusho futhi ukubambezeleka okwandisiwe ngenxa yokubhala nokufunda kulesi sihloko. Ukwengeza, uma udinga ukujoyina esicini esingaphezu kwesisodwa noma ubukhulu, kufanele uhlanganise ukujoyina, ubeke amarekhodi ngokhiye abasha, bese uqhuba inqubo yokuhlukanisa kabusha futhi.

Ixhuma kumadathasethi amancane

Kwezinye izimo, umthamo wedatha yereferensi ezoxhunywa incane uma kuqhathaniswa, ngakho amakhophi ayo aphelele angangena kalula endaweni endaweni ngayinye. Ezimweni ezinjengalezi, i-Kafka Streams inikeza ikilasi le-GlobalKTable.

Izimo ze-GlobalKTable zihlukile ngoba uhlelo lokusebenza luphindaphinda yonke idatha ku-node ngayinye. Futhi njengoba yonke idatha ikhona endaweni ngayinye, asikho isidingo sokuhlukanisa ukusakazwa komcimbi ngokhiye wedatha yereferensi ukuze itholakale kuzo zonke izingxenye. Ungenza futhi ukujoyina okungenakhiye usebenzisa izinto ze-GlobalKTable. Ake sibuyele kwesinye sezibonelo ezedlule ukuze sibonise lesi sici.

Ixhuma izinto ze-KStream ezintweni ze-GlobalKTable

Esigatshaneni 5.3.2, senze ukuhlanganiswa kwewindi kokushintshisana ngabathengi. Imiphumela yalokhu kuhlanganiswa ibukeke kanje:

{customerId='074-09-3705', stockTicker='GUTM'}, 17
{customerId='037-34-5184', stockTicker='CORK'}, 16

Nakuba le miphumela ifeze injongo, bekungaba usizo kakhulu ukube igama lekhasimende negama eliphelele lenkampani nakho bekubonisiwe. Ukwengeza igama lekhasimende negama lenkampani, ungenza ukujoyina okuvamile, kodwa uzodinga ukwenza amamephu okhiye ababili nokuhlukanisa kabusha. Nge-GlobalKTable ungagwema izindleko zemisebenzi enjalo.

Ukuze senze lokhu, sizosebenzisa into ethi countStream esuka ku-Listing 5.11 (ikhodi ehambisanayo ingatholakala kokuthi src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) futhi siyixhume ezintweni ezimbili ze-GlobalKTable.

Sesike saxoxa ngalokhu ngaphambilini, ngakho-ke ngeke ngikuphinde. Kodwa ngiyaqaphela ukuthi ikhodi ku-toStream().umsebenzi wemephu ikhishiwe entweni yokusebenza esikhundleni senkulumo ye-lambda esemgqeni ukuze ifundeke.

Isinyathelo esilandelayo ukumemezela izimo ezimbili ze-GlobalKTable (ikhodi ebonisiwe ingatholakala kufayela elithi src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) (Uhlu 5.12).

Sicela uqaphele ukuthi amagama esihloko achazwa kusetshenziswa izinhlobo ezibaliwe.

Manje njengoba sezilungile zonke izingxenye, okusele ukubhala ikhodi yokuxhumana (engatholakala kufayela elithi src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) (Uhlu 5.13).

Nakuba kunamajoyinti amabili kule khodi, aboshwe ngamaketanga ngoba ayikho imiphumela yawo esetshenziswa ngokuhlukana. Imiphumela iboniswa ekupheleni kwawo wonke umsebenzi.

Uma uqalisa ukusebenza kokujoyina okungenhla, uzothola imiphumela enjengale:

{customer='Barney, Smith' company="Exxon", transactions= 17}

Ingqikithi ayikashintshi, kodwa le miphumela ibukeka icace kakhudlwana.

Uma ubala wehlele eSahlukweni 4, usuvele ubone izinhlobo ezimbalwa zokuxhuma zisebenza. Ahlelwe etafuleni. 5.2. Leli thebula libonisa amandla okuxhumana kusukela kunguqulo 1.0.0 ye-Kafka Streams; Okuthile kungase kushintshe ekukhishweni okuzayo.

Ukuze sisonge izinto, ake siphinde sifunde okuyisisekelo: ungaxhuma ukusakazwa kwemicimbi (KStream) futhi ubuyekeze ukusakaza (KTable) usebenzisa izwe lasendaweni. Kungenjalo, uma usayizi wedatha yereferensi ungemkhulu kakhulu, ungasebenzisa into ye-GlobalKTable. I-GlobalKTables iphindaphinda zonke izingxenye zenodi yohlelo lokusebenza lwe-Kafka Streams ngayinye, iqinisekisa ukuthi yonke idatha iyatholakala kungakhathaliseki ukuthi ukhiye uhambisana yiphi ingxenye.

Okulandelayo sizobona isici se-Kafka Streams, sibonga lapho singabona izinguquko zesimo ngaphandle kokusebenzisa idatha esihlokweni se-Kafka.

5.3.5. Isimo esibuzwayo

Sesivele senze imisebenzi eminingana ebandakanya izwe futhi sihlala sikhipha imiphumela kukhonsoli (ngezinjongo zokuthuthukisa) noma siyibhale esihlokweni (ngokwezinjongo zokukhiqiza). Uma ubhala imiphumela esihlokweni, kufanele usebenzise umthengi we-Kafka ukuze uyibuke.

Ukufunda idatha kusuka kulezi zihloko kungathathwa njengohlobo lokubuka okuphathekayo. Ngezinjongo zethu, singasebenzisa incazelo yombono owenziwe umuntu ovela ku-Wikipedia: “...into egciniwe ephathekayo equkethe imiphumela yombuzo. Isibonelo, kungaba ikhophi yendawo yedatha ekude, noma isethi engaphansi yemigqa kanye/noma amakholomu ethebula noma imiphumela yokuhlanganisa, noma ithebula lesifinyezo elitholwe ngokuhlanganisa” (https://en.wikipedia.org/wiki /Ukubuka_okubonakalayo).

I-Kafka Streams iphinde ikuvumela ukuthi uqhube imibuzo ezitolo zombuso, okukuvumela ukuthi ufunde ngokuqondile lokhu kubuka okwenziwe imizimba. Kubalulekile ukuqaphela ukuthi umbuzo oya esitolo sikahulumeni uwumsebenzi wokufunda kuphela. Lokhu kuqinisekisa ukuthi akudingeki ukhathazeke ngokwenza isimo singahambisani ngephutha ngenkathi uhlelo lwakho lokusebenza lucubungula idatha.

Ikhono lokubuza ngokuqondile izitolo zezwe libalulekile. Lokhu kusho ukuthi ungakwazi ukudala izinhlelo zokusebenza zedeshibhodi ngaphandle kokuthi uqale ulande idatha kumthengi we-Kafka. Iphinde ikhulise ukusebenza kahle kohlelo lokusebenza, ngenxa yokuthi asikho isidingo sokuphinda ubhale idatha:

• ngenxa yendawo yedatha, ingafinyeleleka ngokushesha;
• ukuphindwaphindwa kwedatha kuqediwe, njengoba kungabhalwanga kwisitoreji sangaphandle.

Into esemqoka engifuna uyikhumbule ukuthi ungabuza ngokuqondile isimo ngaphakathi kwesicelo sakho. Amathuba akunika wona lokhu awanakweqiwa. Esikhundleni sokudla idatha evela e-Kafka futhi ugcine amarekhodi kusizindalwazi sohlelo lokusebenza, ungabuza izitolo zezwe ngomphumela ofanayo. Imibuzo eqondile ezitolo zesifunda isho ikhodi encane (akukho mthengi) kanye nesofthiwe encane (asikho isidingo setafula lesizindalwazi ukuze ugcine imiphumela).

Sifunde kabanzi kulesi sahluko, ngakho-ke sizoyishiya ingxoxo yethu yemibuzo esebenzisanayo ngokumelene nezitolo zikahulumeni okwamanje. Kodwa ungakhathazeki: Esahlukweni 9, sizodala uhlelo lokusebenza olulula lwedeshibhodi olunemibuzo esebenzisanayo. Izosebenzisa ezinye zezibonelo zalesi sahluko kanye nezahluko ezedlule ukuze ibonise imibuzo yokusebenzisana nokuthi ungayengeza kanjani ezinhlelweni zokusebenza ze-Kafka Streams.

Isifingqo

• Izinto ze-KStream zimelela ukusakazwa kwemicimbi, uma kuqhathaniswa nokufakwa kusizindalwazi. Izinto ze-KTable zimelela ukusakazwa kokubuyekezwa, okufana nezibuyekezo kusizindalwazi. Usayizi wento ye-KTable awukhuli, amarekhodi amadala athathelwa indawo amasha.
• Izinto ze-KTable ziyadingeka ekusebenzeni kokuhlanganisa.
• Ngokusebenzisa imisebenzi yokufaka amawindi, ungahlukanisa idatha ehlanganisiwe ibe amabhakede esikhathi.
• Ngenxa yezinto ze-GlobalKTable, ungafinyelela idatha yereferensi noma kuphi kuhlelo lokusebenza, ngaphandle kokuhlukaniswa.
• Ukuxhumana phakathi kwezinto ze-KStream, KTable kanye ne-GlobalKTable kungenzeka.

Kuze kube manje, sigxile ekwakheni izinhlelo zokusebenza ze-Kafka Streams sisebenzisa i-KStream DSL yezinga eliphezulu. Nakuba indlela esezingeni eliphezulu ikuvumela ukuthi udale izinhlelo ezihlanzekile nezimfushane, ukuyisebenzisa kumelela ukuhwebelana. Ukusebenza nge-DSL KStream kusho ukukhulisa ubumfushane bekhodi yakho ngokunciphisa izinga lokulawula. Esahlukweni esilandelayo, sizobheka i-API yesibambi sezinga eliphansi bese sizame okunye ukuhwebelana. Izinhlelo zizoba zinde kunangaphambili, kodwa sizokwazi ukudala cishe noma iyiphi indawo yesibambi esingase siyidinge.

→ Imininingwane eyengeziwe mayelana nencwadi ingatholakala ku iwebhusayithi yomshicileli

→ Ngesaphulelo sika-Habrozhiteli esingu-25% usebenzisa isigqebhezana - Kafka Ukusakaza

→ Uma sekukhokhelwe uhlobo lwephepha lwencwadi, incwadi kagesi izothunyelwa nge-imeyili.

Source: www.habr.com