🥇Incwadi ethi "I-Kafka ihamba ngokusebenza. Usetyenziso kunye nee-microservices zomsebenzi wexesha lokwenyani"

Molweni, bahlali baseKhabro! Le ncwadi ifanelekile kuye nawuphi na umphuhlisi ofuna ukuqonda ukusetyenzwa komsonto. Ukuqonda inkqubo esasazwayo kuya kukunceda uqonde ngcono iKafka kunye neKafka Streams. Kuya kuba kuhle ukwazi isakhelo seKafka ngokwayo, kodwa oku akuyomfuneko: Ndiza kukuxelela yonke into oyifunayo. Abaphuhlisi be-Kafka abanamava kunye nabaqalayo ngokufanayo baya kufunda ukwenza usetyenziso olunomdla lokuhambisa umjelo usebenzisa ilayibrari yeKafka Streams kule ncwadi. Abaphuhlisi beJava abaphakathi nabaphambili asele beqhelene neengqikelelo ezinje ngothotho baya kufunda ukusebenzisa izakhono zabo ukwenza izicelo zeKafka Streams. Ikhowudi yomthombo wencwadi ibhalwe kwiJava 8 kwaye yenza ukusetyenziswa okubalulekileyo kwe-Java 8 lambda expression syntax, ngoko ukwazi indlela yokusebenza ngemisebenzi ye-lambda (kwanakolunye ulwimi lwenkqubo) kuya kuba luncedo.

Isicatshulwa. 5.3. Udibaniso kunye nokusebenza kweefestile

Kweli candelo, siya kuqhubela phambili sijonge ezona ndawo zithembisayo zeKafka Streams. Ukuza kuthi ga ngoku sigubungele le miba ilandelayo yeKafka Streams:

ukudala i-topology yokucubungula;
usebenzisa i-state kwi-apps yostrimisho;
ukwenza uqhagamshelwano lomjelo wedatha;
iyantlukwano phakathi kweemilambo zesiganeko (KStream) kunye nohlaziyo lwamanzi (KTable).

Kule mizekelo ilandelayo siza kudibanisa zonke ezi elementi. Uyakufunda kwakhona malunga nefestile, enye into entle yokusasaza usetyenziso. Umzekelo wethu wokuqala uya kuba yi-aggregation elula.

5.3.1. Ukudityaniswa kokuthengiswa kwempahla ngokwecandelo leshishini

Udibaniso kunye nokwahlulahlula zizixhobo ezibalulekileyo xa usebenza ngedatha yokusasaza. Ukuvavanywa kweerekhodi zomntu ngamnye njengoko zifunyenwe kudla ngokungonelanga. Ukukhupha ulwazi olongezelelweyo kwidatha, kuyimfuneko ukuhlanganisa kunye nokudibanisa.

Kulo mzekelo, uya kunxiba isinxibo somrhwebi wosuku ofuna ukulandelela umthamo wokuthengisa weempahla zeenkampani kumashishini amaninzi. Ngokukodwa, unomdla kwiinkampani ezintlanu ezineentengiso zesabelo esikhulu kwishishini ngalinye.

Ukuhlanganiswa okunjalo kuya kufuna amanyathelo amaninzi alandelayo ukuguqulela idatha kwifom efunwayo (ukuthetha ngokubanzi).

Yenza umthombo osekelwe kwisihloko opapasha ulwazi lokuthengisa isitokhwe ekrwada. Kuya kufuneka senze imephu into yohlobo lweStockTransaction kwinto yohlobo lweShareVolume. Ingongoma kukuba into ye-StockTransaction iqulethe i-metadata yokuthengisa, kodwa sifuna kuphela idatha malunga nenani lezabelo ezithengiswayo.
Iqela ShareVolume data ngokwesimboli stock. Nje ukuba udityaniswe ngesimboli, ungawa phantsi le datha kwiitotali zexabiso lentengiso yesitokhwe. Kubalulekile ukuqaphela ukuba indlela ye KStream.groupBy ibuyisela umzekelo wodidi lwe KGroupedStream. Kwaye ungafumana umzekelo weKTable ngokubiza ngokuqhubekayo i KGroupedStream.reduce indlela.

Yintoni ujongano lwe KGroupedStream

Iindlela ze KStream.groupBy kunye ne KStream.groupByKey zibuyisela umzekelo we KGroupedStream. I-KGroupedStream luphawu oluphakathi lothotho lweziganeko emva kokwahlulahlula ngezitshixo. Ayenzelwanga kwaphela ukusebenza ngokuthe ngqo kunye nayo. Endaweni yoko, iKGroupedStream isetyenziselwa udibaniso lwemisebenzi, ehlala ikhokelela kwiKTable. Kwaye njengoko isiphumo semisebenzi yodibaniso yiKTable kwaye basebenzisa ivenkile karhulumente, kunokwenzeka ukuba ayilulo lonke uhlaziyo njengesiphumo oluthunyelwayo phantsi kombhobho.

Indlela ye-KTable.groupBy ibuyisela i-KGroupedTable efanayo - umboniso ophakathi woluhlu lohlaziyo, oludityaniswe ngokutsha ngesitshixo.

Makhe sithathe ikhefu elifutshane kwaye sijonge kwiFig. 5.9, ebonisa into esiyiphumezileyo. Le topology kufuneka sele iqhelekile kuwe.

Ngoku makhe sijonge ikhowudi yale topology (inokufumaneka kwifayile src/main/java/bbejeck/chapter_5/AggregationsAndReducingExample.java) (Uluhlu 5.2).

Ikhowudi enikeziweyo iyahlukaniswa bubufutshane bayo kunye nomthamo omkhulu wezenzo ezenziwa kwimigca emininzi. Unokuqaphela into entsha kwiparameter yokuqala ye-builder.stream method: ixabiso lohlobo lwe-enum yohlobo lwe-AutoOffsetReset.EARLIEST (kukwakhona i-LATEST), sebenzisa i-Consumed.withOffsetResetPolicy indlela. Olu didi lobalo lunokusetyenziselwa ukukhankanya iqhinga lokusetha kwakhona kwi KStream nganye okanye iKTable nganye kwaye ithatha indawo yokuqala phezu kokusetha ngokutsha ukhetho kuqwalaselo.

GroupByKey kunye neQelaBy

Ujongano lwe KStream luneendlela ezimbini zokuhlanganisa iirekhodi: IQelaByKey kunye neQelaBy. Zombini zibuyisela i-KGroupedTable, ngoko unokuzibuza ukuba yintoni umahluko phakathi kwazo kwaye usebenzise eyiphi?

Indlela yeQelaByKey isetyenziswa xa izitshixo kwi KStream sele zingenanto. Kwaye okona kubaluleke kakhulu, iflegi "ifuna ukwahlulahlulwa kwakhona" ayizange imiselwe.

Indlela yeQela ngeQela ithatha ukuba utshintshe izitshixo zeqela, ngoko iflegi yolwabiwo isetelwe kwinyani. Ukwenza ukudibanisa, ukudityaniswa, njl.njl. emva kwendlela yeQela ngeQela kuya kukhokelela ekwahluleleni kwakhona okuzenzekelayo.
Ushwankathelo: Nanini na kusenzeka, kufuneka usebenzise iGroupByKey kuneGroupBy.

Kucacile ukuba yintoni imephuValues kunye neendlela zeqelaBy, ngoko ke makhe sijonge kwi-sum() indlela (efumaneka kwi-src/main/java/bbejeck/model/ShareVolume.java) (Uluhlu 5.3).

Indlela ye-ShareVolume.sum ibuyisela inani elisebenzayo lexabiso lentengiso yesitokhwe, kwaye isiphumo setsheyini yonke yokubala yinto yeKTable. . Ngoku uyayiqonda indima edlalwa yiKTable. Xa izinto zeShareVolume zifika, into ehambelana neKTable igcina uhlaziyo lwangoku. Kubalulekile ukukhumbula ukuba lonke uhlaziyo lubonakaliswe kwi-shareVolumeKTable yangaphambili, kodwa ayizizo zonke ezithunyelwa ngokubhekele phaya.

Emva koko sisebenzise le KTable ukuhlanganisa (ngenani lezabelo ezithengisiweyo) ukuze sifike kwiinkampani ezintlanu ezinomthamo ophezulu wezabelo ezithengiswayo kushishino ngalunye. Izenzo zethu kule meko ziya kufana nezo ze-aggregation yokuqala.

Yenza elinye iqelaNgokusebenza ukwenza iqela izinto zeShareVolume nganye ngoshishino.
Qala ukushwankathela izinto zeShareVolume. Ngeli xesha into yohlanganiso ngumgca obalulekileyo osisigxina. Kulo mgca wobungakanani obusisigxina, ziinkampani ezintlanu kuphela ezinezabelo ezinkulu zezabelo ezithengisiweyo ezigciniweyo.
Imephu yemigca ukusuka kumhlathi wangaphambili ukuya kwixabiso lomtya kwaye ubuyisele ezona zintlanu eziphezulu kakhulu ezithengiswayo ngenani ngeshishini.
Bhala iziphumo ngendlela yomtya kwisihloko.

KwiFig. Umzobo 5.10 ubonisa igrafu ye-data flow topology. Njengoko ubona, umjikelo wesibini wokusetyenzwa ulula kakhulu.

Ngoku ukuba sinokuqonda okucacileyo kwesakhiwo somjikelo wesibini wokucubungula, sinokujika kwikhowudi yomthombo (uya kuyifumana kwifayile src/main/java/bbejeck/chapter_5/AggregationsAndReducingExample.java) (Uluhlu 5.4) .

Esi siqalisi siqulathe iguquguquko esisigxina. Le yinto yesiko eyiadaptha ye-java.util.TreeSet esetyenziselwa ukulandelela iziphumo eziphezulu ze-N kulandelelwano oluhla lwezabelo ezirhwetywayo.

Sele ulibonile iqelaNge kunye nemephuAmaxabiso eminxeba, ngoko ke asizukungena kwezo (sibiza indlela yeKTable.toStream kuba indlela yeKTable.print ihoxisiwe). Kodwa awukaluboni uguqulelo lwe KTable ye aggregate() okwangoku, ke sizakuchitha ixesha elincinci sixoxa ngalonto.

Njengoko ukhumbula, yintoni eyenza i-KTable yahluke kukuba iirekhodi ezinezitshixo ezifanayo ziqwalaselwa ukuhlaziywa. I-KTable ibuyisela ungeno oludala ngentsha. Udibaniso lwenzeka ngendlela efanayo: iirekhodi zamva nje ezineqhosha elifanayo ziyadityaniswa. Xa irekhodi ifika, yongezwa kumzekelo weklasi ye-FixedSizePriorityQueue usebenzisa i-adder (iparameter yesibini kwindlela yokufowuna), kodwa ukuba enye irekhodi sele ikhona kunye nesitshixo esifanayo, ngoko irekhodi elidala liyasuswa ngokusebenzisa i-subtractor (iparameter yesithathu kwi umnxeba wendlela edityanisiweyo).

Konke oku kuthetha ukuba i-aggregator yethu, i-FixedSizePriorityQueue, ayiwahlanganisi onke amaxabiso ngesitshixo esinye, kodwa igcina isixa esishukumayo sobuninzi beentlobo ze-N ezithengiswa kakhulu zesitokhwe. Ingeniso nganye engenayo iqulethe inani elipheleleyo lezabelo ezithengisiweyo ukuza kuthi ga ngoku. I-KTable iya kukunika ulwazi malunga nokuba zeziphi izabelo zeenkampani ezithengiswa kakhulu ngoku, ngaphandle kokufuna ukudityaniswa kohlaziyo ngalunye.

Sifunde ukwenza izinto ezimbini ezibalulekileyo:

amaxabiso eqela kwi KTable ngesitshixo esiqhelekileyo;
yenza imisebenzi eluncedo njengokuqengqeleka kunye nokudibanisa kula maxabiso adityanisiweyo.

Ukwazi ukwenza le misebenzi kubalulekile ukuqonda intsingiselo yedatha ehamba ngesicelo seKafka Streams kunye nokuqonda ukuba yeyiphi ingcaciso ephetheyo.

Sikwadibanise amanye amabinzana aphambili axoxwe ngaphambili kule ncwadi. KwiSahluko sesi-4, sixoxe ngendlela yokunyamezela impazamo, imeko yendawo ibalulekile kwisicelo sokusasaza. Umzekelo wokuqala kwesi sahluko ubonise ukuba kutheni urhulumente wengingqi ubaluleke kangaka-ukunika ukukwazi ukugcina umkhondo woluphi ulwazi osele ulubonile. Ufikelelo lwasekuhlaleni lunqanda ulibaziseko lwenethiwekhi, lusenza isicelo sisebenze ngakumbi kwaye singanyangeki iimpazamo.

Xa usenza nayiphi na i-rollup okanye umsebenzi wokudibanisa, kufuneka uchaze igama lesitora sikarhulumente. Imisebenzi yokuqengqeleka kunye nokudibanisa ibuyisela umzekelo weKTable, kwaye iKTable isebenzisa ugcino lwelizwe ukubuyisela iziphumo ezindala ngezitsha. Njengoko ubonile, ayilulo lonke uhlaziyo oluthunyelwayo, kwaye oku kubalulekile kuba imisebenzi yokudibanisa iyilelwe ukuvelisa ulwazi olusisishwankathelo. Ukuba awusisebenzisi ilizwe lasekuhlaleni, i-KTable iyakudlulisa lonke udibaniso kunye neziphumo zokusongwa.

Okulandelayo, siza kujonga ukwenza imisebenzi efana nokudityaniswa ngexesha elithile - okubizwa ngokuba yimisebenzi yefestile.

5.3.2. Imisebenzi yefestile

Kwicandelo elidlulileyo, sazise i-sliding convolution kunye nokudibanisa. Isicelo senza i-roll-up eqhubekayo yomthamo wentengiso yesitokhwe, elandelwa kukudityaniswa kwezitokhwe ezintlanu ezithengiswa kakhulu kutshintshiselwano.

Ngamanye amaxesha udibaniso oluqhubekayo kunye nokuqengqeleka kweziphumo kuyafuneka. Kwaye ngamanye amaxesha kufuneka wenze imisebenzi kuphela ngexesha elithile. Umzekelo, bala ukuba zingaphi iintengiselwano ezenziweyo kunye nezabelo zenkampani ethile kwimizuzu eyi-10 yokugqibela. Okanye bangaphi abasebenzisi abacofe ibhena entsha yentengiso kwimizuzu eli-15 edlulileyo. Isicelo singenza imisebenzi enjalo amaxesha amaninzi, kodwa ngeziphumo ezisebenza kuphela kwixesha elichaziweyo (iifestile zexesha).

Ukubala utshintshiselwano ngomthengi

Kumzekelo olandelayo, siza kulandelela ukuthengiselana kwesitokhwe kubo bonke abarhwebi abaninzi-imibutho emikhulu okanye abaxhasi bezimali abakrelekrele.

Zimbini izizathu ezinokubakho zokulandelwa koku. Enye yazo yimfuneko yokwazi ukuba zeziphi iinkokeli zeemarike ezithengayo/zithengisayo. Ukuba aba badlali bakhulu kunye nabatyali-zimali abaphucukileyo babona ithuba, kuyavakala ukulandela isicwangciso sabo. Isizathu sesibini ngumnqweno wokubona naziphi na iimpawu ezinokubakho zorhwebo lwangaphakathi olungekho mthethweni. Ukwenza oku, kuya kufuneka uhlalutye ukulungelelaniswa kwee-spikes ezinkulu zokuthengisa kunye nokukhutshwa kweendaba ezibalulekileyo.

Ukulandelela okunjalo kubandakanya la manyathelo alandelayo:

ukudala umsinga wokufunda kwisihloko sentengiselwano yesitokhwe;
ukubeka ngokwamaqela iirekhodi ezingenayo nge-ID yomthengi kunye nesimboli sesitokhwe. Ukubiza iqelaNgendlela ibuyisela umzekelo wodidi lwe KGroupedStream;
Indlela ye KGroupedStream.windowedBy ibuyisela ujelo lwedatha olukhawulelwe kwifestile yexesha, evumela udibaniso lwefestile. Kuxhomekeke kuhlobo lwefestile, nokuba yi-TimeWindowedKStream okanye iSeshiniWindowedKStream iyabuyiswa;
inani lentengiselwano yomsebenzi wodibaniso. Ukuhamba kwedatha okufakwe efestileni kugqiba ukuba ngaba irekhodi elithile lithathelwe ingqalelo kolu balo;
ukubhala iziphumo kwisihloko okanye ukuzikhupha kwi console ngexesha lophuhliso.

I-topology yesi sicelo ilula, kodwa umfanekiso ocacileyo wayo unokuba luncedo. Makhe sijonge kwiFig. 5.11.

Emva koko, siza kujonga ukusebenza kwemisebenzi yefestile kunye nekhowudi ehambelanayo.

Iintlobo zefestile

Kukho iintlobo ezintathu zeefestile kwi-Kafka Streams:

okwexeshana
“ukuwa” (ukuwa);
ukutyibilika/ukutsibatsiba.

Yeyiphi onokuyikhetha ixhomekeke kwiimfuno zeshishini lakho. Ukudilika kunye nokutsiba iifestile zinexesha elilinganiselweyo, ngelixa iifestile zeseshoni zikhawulelwe ngumsebenzi wabasebenzisi-ixesha leseshoni (ii) limiselwa kuphela ukuba umsebenzi uyasebenza kangakanani. Into ephambili ekufuneka uyikhumbule kukuba zonke iintlobo zefestile zisekwe kumhla/ixesha izitampu zamangeno, hayi ixesha lenkqubo.

Emva koko, siphumeza i-topology yethu ngohlobo ngalunye lwefestile. Ikhowudi epheleleyo iya kunikwa kuphela kumzekelo wokuqala, kwezinye iindidi zeefestile akukho nto iya kutshintsha ngaphandle kohlobo lomsebenzi wefestile.

Iifestile zeseshoni

Iifestile zeseshoni zahluke kakhulu kuzo zonke ezinye iintlobo zeefestile. Zithintelwe hayi kakhulu ngexesha njengoko umsebenzi womsebenzisi (okanye umsebenzi wequmrhu ongathanda ukuwulandela). Iifestile zeseshoni zicandwa ngamaxesha okungasebenzi.

Umfanekiso 5.12 ubonisa ingqikelelo yeefestile zeseshoni. Iseshoni encinci iya kudibaniswa kunye neseshoni ngasekhohlo. Kwaye iseshoni ekunene iya kwahluka kuba ilandela ixesha elide lokungasebenzi. Iifestile zeseshoni zisekwe kumsebenzi wabasebenzisi, kodwa sebenzisa izitampu zomhla/ixesha kumangenelo ukumisela ukuba yeyiphi iseshoni ekungenwa kuyo.

Ukusebenzisa iifestile zeseshoni ukulandelela iintengiselwano zesitokhwe

Masisebenzise iifestile zeseshoni ukubamba ulwazi malunga neentengiselwano zotshintshiselwano. Ukuphunyezwa kweefestile zeseshoni kuboniswe kuLuhlu lwe-5.5 (enokufumaneka kwi-src/main/java/bbejeck/chapter_5/CountingWindowingAndKTableJoinExample.java).

Sele ulubonile uninzi lwemisebenzi kule topology, akukho mfuneko yokuba ujonge kwakhona apha. Kodwa kukho izinto ezininzi ezintsha apha, esiza kuzixoxa ngoku.

Naliphi na iqelaNgemisebenzi lenza uhlobo oluthile lomsebenzi wokudityaniswa (ukudibanisa, ukusonga, okanye ukubala). Unokwenza nokuba udibaniso oluqokelelweyo ngenani elisebenzayo, okanye udibaniso lwefestile, oluthathela ingqalelo iirekhodi ngaphakathi kwexesha elichaziweyo lefestile.

Ikhowudi kuLuhlu 5.5 ibala inani leentengiselwano ngaphakathi kweefestile zeseshoni. KwiFig. 5.13 la manyathelo acazululwa inyathelo nenyathelo.

Ngokufowunela windowsedBy(SessionWindows.with(twentySeconds).de(fifteenMinutes)) senza iseshoni yefestile enesithuba sokungasebenzi semizuzwana engama-20 kunye nesithuba esizingisileyo semizuzu eyi-15. Ithuba lokungenzi nto lemizuzwana engama-20 lithetha ukuba isicelo siya kubandakanya naluphi na ungeniso olufika kwimizuzwana engama-20 yokuphela okanye ukuqala kweseshoni yangoku kwiseshoni yangoku (esebenzayo).

Okulandelayo, sicacisa ukuba yeyiphi intsebenzo yokudibanisa ekufuneka yenziwe kwifestile yeseshoni - kulo mzekelo, bala. Ukuba ungeno olungenayo luwela ngaphandle kwefestile yokungasebenzi (nakweliphi na icala lomhla/isitampu sexesha), isicelo sidala iseshoni entsha. Ithuba lokugcinwa lithetha ukugcina iseshoni kangangexesha elithile kwaye ivumela idatha kade edlulela ngaphaya kwexesha lokungasebenzi kweseshoni kodwa inokuqhotyoshelwa. Ukongeza, ukuqala kunye nokuphela kweseshoni entsha evela ekudityanisweni kuhambelana nomhla wokuqala kunye nesitampu sexesha.

Makhe sijonge amangenelo ambalwa kwindlela yokubala ukubona ukuba iiseshini zisebenza njani (Uluhlu 5.1).

Xa iirekhodi zifika, sijonga iiseshoni ezikhoyo kunye nesitshixo esifanayo, ixesha lokugqibela elingaphantsi komhla wangoku / isitampu sexesha - ikhefu lokungasebenzi, kunye nexesha lokuqala elikhulu kunomhla wangoku / isitampu sexesha + lokungasebenzi. Ukuthathela ingqalelo oku, amangeniso amane avela kwitheyibhile. 5.1 adityaniswa abe yiseshoni enye ngolu hlobo lulandelayo.

1. Ingxelo i-1 ifika kuqala, ngoko ixesha lokuqala lilingana nexesha lokugqibela kwaye ngu-00:00:00.

2. Okulandelayo, ukungena kwe-2 kuyafika, kwaye sijonge iiseshini eziphela ngaphambi ko-23:59:55 kwaye ziqale kungadlulanga 00:00:35. Sifumana irekhodi ye-1 kwaye sidibanise iiseshoni ze-1 kunye ne-2. Sithatha ixesha lokuqala leseshoni 1 (ngaphambili) kunye nexesha lokuphela kweseshoni 2 (kamva), ukwenzela ukuba iseshoni yethu entsha iqale ngo-00:00:00 kwaye iphele ngo-00: 00:15.

3. Irekhodi ye-3 ifika, sijonge iiseshoni phakathi kuka-00:00:30 kunye no-00:01:10 kwaye asifumani nanye. Yongeza iseshoni yesibini yesitshixo 123-345-654,FFBE, eqala nephela ngo-00:00:50.

4. Ingxelo yesi-4 ifikile kwaye sijonge iiseshoni phakathi kwe-23:59:45 kunye ne-00:00:25. Ngeli xesha zifunyenwe zombini iiseshoni 1 no-2. Zontathu iiseshoni zidityaniswa zibe nye, kunye nexesha lokuqalisa ngo-00:00:00 kunye nexesha lokuphela elingu-00:00:15.

Ukusuka kwinto echazwe kweli candelo, kufanelekile ukukhumbula ezi zinto zilandelayo zibalulekileyo:

iiseshini azikho iifestile ezinobungakanani obusisigxina. Ubude beseshoni bumiselwa ngumsebenzi kwixesha elinikiweyo;
Izitampu zomhla / ixesha kwidatha zinquma ukuba isiganeko siwela kwiseshoni ekhoyo okanye ngexesha lokungenzi lutho.

Okulandelayo siza kuxubusha uhlobo olulandelayo lwefestile - "ukuwisa" iifestile.

"Ukudilika" iifestile

Iifestile eziwayo zibamba iziganeko eziwela kwixesha elithile. Khawucinge ukuba kufuneka ubambe zonke iintengiselwano zesitokhwe zenkampani ethile rhoqo ngemizuzwana ye-20, ngoko uqokelela zonke iziganeko ngelo xesha. Ekupheleni kwekhefu le-20-yesibini, ifestile iqengqeleka kwaye ihambela kwithuba elitsha lokujonga imizuzwana engama-20. Umfanekiso 5.14 ubonisa le meko.

Njengoko ubona, zonke iziganeko ezifunyenwe kwimizuzwana engama-20 yokugqibela zibandakanyiwe kwifestile. Ekupheleni kweli xesha lexesha, iwindow entsha yenziwa.

Uluhlu lwe-5.6 lubonisa ikhowudi ebonisa ukusetyenziswa kweefestile zokuwa ukubamba ukuthengiselana kwesitokhwe rhoqo ngemizuzwana ye-20 (efunyenwe kwi-src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java).

Ngolu tshintsho luncinci kwi-TimeWindows.yendlela yokufowuna, ungasebenzisa iwindow ewayo. Lo mzekelo awubizi indlela de() indlela, ngoko ke ugcino lwesithuba seeyure ezingama-24 luya kusetyenziswa.

Ekugqibeleni, lixesha lokuqhubela phambili kwiinketho zokugqibela zefestile - "i-hopping" windows.

Ukutyibilika ("ukutsiba") iifestile

Ukutyibilika/ukutsiba iifestile ziyafana neefestile eziwayo, kodwa zinomahluko omncinci. Ukutyibilika iifestile musa ukulinda de kuphele ixesha lokuphumla phambi kokudala ifestile entsha ukuqhubekekisa imicimbi yamva nje. Baqala izibalo ezintsha emva kwexesha lokulinda elingaphantsi kobude befestile.

Ukubonisa umahluko phakathi kokuwa kunye nokutsiba iifestile, masibuyele kumzekelo wokubala iintengiselwano zokutshintshiselana ngesitokhwe. Injongo yethu isekubala inani lentengiselwano, kodwa asifuni kulinda lonke ixesha phambi kokuhlaziya ikhawunta. Endaweni yoko, siya kuhlaziya ikhawuntara ngamaxesha amafutshane. Umzekelo, siya kuhlala sibala inani lentengiselwano rhoqo ngemizuzwana engama-20, kodwa sihlaziye ikhawunta rhoqo ngemizuzwana emi-5, njengoko kubonisiwe kumfanekiso. 5.15. Kule meko, sigqiba ngeefestile ezintathu zeziphumo ezinedatha ehambayo.

Uluhlu 5.7 lubonisa ikhowudi yokuchaza iifestile zesilayidi (efumaneka kwi-src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java).

Iwindow ewayo ingaguqulelwa kwifestile yokutsiba ngokudibanisa umnxeba kwi advanceBy() indlela. Kumzekelo obonisiweyo, ikhefu lokulondoloza yimizuzu eli-15.

Ubonile kweli candelo ukuba unganciphisa njani iziphumo zohlanganiso kwixesha leefestile. Ngokukodwa, ndifuna ukuba ukhumbule ezi zinto zintathu zilandelayo kweli candelo:

ubungakanani beefestile zeseshoni bukhawulelwe hayi ngexesha, kodwa ngomsebenzi wabasebenzisi;
Iifestile “ezidilikayo” zibonelela ngesishwankathelo seziganeko ngexesha elithile;
Ubude bexesha lokutsiba kweefestile lulungisiwe, kodwa luhlaziywa rhoqo kwaye lunokuqulatha amangeniso adlulanayo kuzo zonke iifestile.

Okulandelayo, sizakufunda indlela yokuguqula iKTable ibuyele kwi KStream yodibaniso.

5.3.3. Idibanisa i KStream kunye ne KTable izinto

KwiSahluko sesi-4, sixoxe ngokudibanisa izinto ezimbini ze-KStream. Ngoku kufuneka sifunde indlela yokudibanisa iKTable kunye neKStream. Oku kungafuneka ngenxa yesi sizathu silula silandelayo. I-KStream ngumjelo weerekhodi, kwaye i-KTable ngumjelo wohlaziyo lwerekhodi, kodwa ngamanye amaxesha unokufuna ukongeza umongo owongezelelweyo kumjelo werekhodi usebenzisa uhlaziyo olusuka kwiKTable.

Masithathe idatha kwinani leentengiselwano ze-stock exchange kwaye sidibanise neendaba zokutshintshiselana ngesitokhwe kumashishini afanelekileyo. Nantsi into ekufuneka uyenzile ukuze ufezekise oku kunikwe ikhowudi osele unayo.

Guqula into ye-KTable enedatha kwinani lentengiselwano yesitokhwe kwi-KStream, ilandelwa kukutshintsha isitshixo ngesitshixo esibonisa icandelo leshishini elihambelana nesimboli sesitokhwe.
Yenza into ye-KTable efunda idatha kwisihloko ngeendaba zokutshintshiselana ngesitokhwe. Le KTable intsha iza kuhlulwa ngokwecandelo loshishino.
Qhagamshela uhlaziyo lweendaba ngolwazi olumalunga nenani leentengiselwano zokutshintshiselana ngesitokhwe licandelo leshishini.

Ngoku makhe sibone indlela yokuphumeza esi sicwangciso somsebenzi.

Guqula iKTable ibe yiKStream

Ukuguqula iKTable kwi KStream kufuneka wenze oku kulandelayo.

Fowunela i-KTable.toStream() indlela.
Ngokubiza indlela ye-KStream.map, buyisela isitshixo ngegama loshishino, kwaye emva koko ufumane kwakhona into yeNguqulelo yesiShwankathelo kumzekelo we Windowed.

Siza kudibanisa le misebenzi ngokulandelayo (ikhowudi inokufumaneka kwifayile src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uluhlu 5.8).

Kuba sisenza umsebenzi we KStream.map, umzekelo we KStream obuyisiweyo wahlulwa kwakhona ngokuzenzekelayo xa isetyenziswa kuqhagamshelwano.

Siyigqibile inkqubo yokuguqula, ngokulandelayo kufuneka senze into yeKTable yokufunda iindaba zesitokhwe.

Ukudalwa kweKTable yeendaba zesitokhwe

Ngethamsanqa, ukwenza into yeKTable ithatha nje umgca omnye wekhowudi (ikhowudi inokufumaneka kwi src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uluhlu 5.9).

Kuyafaneleka ukuba uqaphele ukuba akukho zixhobo zeSerde ezifunekayo ukuba zichazwe, ekubeni umtya weSerdes usetyenziswa kwizicwangciso. Kwakhona, ngokusebenzisa UKUBALA KWAKUQALA, itheyibhile izaliswe ngamarekhodi kwasekuqaleni.

Ngoku sinokuqhubekela phambili kwinyathelo lokugqibela - uqhagamshelo.

Ukudibanisa uhlaziyo lweendaba ngedatha yokubala yentengiselwano

Ukudala uxhulumaniso akukho nzima. Siza kusebenzisa ukujoyina ngasekhohlo kwimeko yokuba akukho ndaba zesitokhwe kushishino olufanelekileyo (ikhowudi efunekayo inokufumaneka kwifayile src/main/java/bbejeck/chapter_5/CountingWindowingAndKtableJoinExample.java) (Uluhlu 5.10).

Le leftJoyina umsebenzisi ilula kakhulu. Ngokungafaniyo nezidibaniselo kwiSahluko 4, indlela yeJoyina Window ayisetyenziswanga kuba xa usenza udibano lwe-KStream-KTable, kukho ungeno olunye kuphela kwiKTable yesitshixo ngasinye. Uqhagamshelo olunjalo alukhawulelwanga ngexesha: irekhodi ikwi-KTable okanye ayikho. Esona sigqibo: usebenzisa izinto zeKTable ungatyebisa i KStream ngedatha yereferensi ehlaziywa rhoqo.

Ngoku siza kujonga indlela esebenzayo yokutyebisa imisitho evela kwi-KStream.

5.3.4. Izinto zeGlobalKTable

Njengoko ubona, kukho imfuneko yokutyebisa imijelo yomnyhadala okanye ukongeza umxholo kubo. KwiSahluko 4 ubone udibaniso phakathi kwezinto ezimbini ze KStream, kwaye kwicandelo elingaphambili ubone udibaniso phakathi kwe KStream ne KTable. Kuzo zonke ezi meko, kuyimfuneko ukwahlula kwakhona umjelo wedatha xa ubeka imephu yezitshixo kuhlobo olutsha okanye ixabiso. Ngamanye amaxesha ukwahlulahlulwa kwenziwa ngokucacileyo, kwaye ngamanye amaxesha iKafka Streams iyenza ngokuzenzekelayo. Ukwahlula kwakhona kuyimfuneko kuba izitshixo zitshintshile kwaye iirekhodi kufuneka ziphele kumacandelo amatsha, ngaphandle koko uxhulumaniso luya kuba lungenakwenzeka (oku kwaxutyushwa kwiSahluko 4, kwicandelo elithi "Ukwahlula kwakhona idatha" kwisiqendwana 4.2.4).

Ukwahlulahlula kwakhona kuneendleko

Ukwahlula kwakhona kufuna iindleko - iindleko ezongezelelweyo zezibonelelo zokudala izihloko eziphakathi, ukugcina idatha ephindwe kabini kwesinye isihloko; kukwathetha ukubambezeleka okwandisiweyo ngenxa yokubhala nokufunda kwesi sihloko. Ukongeza, ukuba ufuna ukujoyina ngaphezu kwenkalo enye okanye imilinganiselo, kufuneka udibanise amakhonkco, wenze imephu yeerekhodi ngezitshixo ezitsha, kwaye uqhube inkqubo yokwahlula kwakhona.

Iqhagamshela kwiiseti zedatha ezincinci

Kwezinye iimeko, umthamo wedatha yereferensi ekufuneka iqhagamshelwe incinci, ngoko ke iikopi zayo ezipheleleyo zinokungena lula kwindawo kwindawo nganye. Kwiimeko ezinje, iKafka Streams ibonelela ngeklasi yeGlobalKTable.

Iimeko zeGlobalKTable zizodwa kuba isicelo siphindaphinda yonke idatha kwindawo nganye. Kwaye ekubeni yonke idatha ikhona kwindawo nganye, akukho mfuneko yokwahlula umjelo wesiganeko ngeqhosha ledatha yereferensi ukuze ifumaneke kuzo zonke izahlulo. Unokwenza udibaniso olungenazitshixo usebenzisa izinto zeGlobalKTable. Masibuyele komnye wemizekelo yangaphambili ukubonisa eli nqaku.

Iqhagamshela izinto ze-KStream kwizinto ze-GlobalKTable

Kwicandelwana 5.3.2, senze iwindow aggregation yotshintshiselwano ngabathengi. Iziphumo zolu hlanganiso zijonge ngolu hlobo:

{customerId='074-09-3705', stockTicker='GUTM'}, 17
{customerId='037-34-5184', stockTicker='CORK'}, 16

Ngelixa ezi ziphumo bezisebenzela injongo, bekuya kuba luncedo ngakumbi ukuba igama lomthengi kunye negama elipheleleyo lenkampani bekubonisiwe. Ukongeza igama lomthengi kunye negama lenkampani, unokwenza ukudibanisa okuqhelekileyo, kodwa kuya kufuneka wenze iimephu ezimbini eziphambili kunye nokwahlulahlula kwakhona. Nge-GlobalKTable unokuphepha iindleko zemisebenzi enjalo.

Ukwenza oku, siya kusebenzisa into ye-countStream evela kuLuhlu lwe-5.11 (ikhowudi ehambelanayo inokufumaneka kwi-src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) kwaye uyidibanise kwizinto ezimbini ze-GlobalKTable.

Sele siyixoxile le nto ngaphambili, ngoko ke andiyi kuphinda. Kodwa ndiyaqaphela ukuba ikhowudi kuStream().umsebenzi wemephu ukhutshelwa kwinto esebenzayo endaweni yentetho yelambda engaphakathi ukwenzela ukufundeka.

Isinyathelo esilandelayo kukuvakalisa imizekelo emibini ye-GlobalKTable (ikhowudi ebonisiweyo inokufumaneka kwifayile src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) (Uluhlu 5.12).

Nceda uqaphele ukuba amagama ezihloko achazwa kusetyenziswa iindidi ezibaliweyo.

Ngoku ukuba sinawo onke amacandelo alungile, konke okuseleyo kukubhala ikhowudi yoxhulumaniso (enokufumaneka kwifayile src/main/java/bbejeck/chapter_5/GlobalKTableExample.java) (Uluhlu 5.13).

Nangona kukho amakhonkco amabini kule khowudi, aboshwe ngenxa yokuba akukho ziphumo zawo zisetyenziswa ngokwahlukeneyo. Iziphumo ziboniswa ekupheleni komsebenzi wonke.

Xa usenza lo msebenzi wokudibanisa ungentla, uya kufumana iziphumo ezinje:

{customer='Barney, Smith' company="Exxon", transactions= 17}

Undoqo awukatshintshi, kodwa ezi ziphumo zibonakala zicace ngakumbi.

Ukuba ubala ukuya kutsho kwiSahluko 4, sele uzibonile iindidi ezininzi zoqhagamshelo zisebenza. Adweliswe kwitheyibhile. 5.2. Le theyibhile ibonakalisa amandla okuqhagamsheleka njengenguqulelo 1.0.0 yeKafka Streams; Kukho into enokutshintsha kukhupho oluzayo.

Ukusonga izinto, makhe siphinde sibuyele kwiziseko: ungaqhagamshela imisinga yesiganeko (KStream) kwaye uhlaziye imisinga (KTable) usebenzisa urhulumente wasekhaya. Kungenjalo, ukuba ubungakanani bedatha yereferensi abukho bukhulu kakhulu, ungasebenzisa into yeGlobalKTable. I-GlobalKTables iphindaphinda zonke izahlulo kwindawo nganye yesicelo se-Kafka Streams, iqinisekisa ukuba yonke idatha iyafumaneka kungakhathaliseki ukuba yeyiphi isahlulelo esihambelana nayo.

Okulandelayo siza kubona i-Kafka Streams feature, enkosi apho sinokujonga utshintsho lwelizwe ngaphandle kokutya idatha kwisihloko se-Kafka.

5.3.5. Imeko ethandabuzekayo

Sele senze imisebenzi emininzi ebandakanya urhulumente kwaye sihlala sikhupha iziphumo kwikhonsoli (ngeenjongo zophuhliso) okanye sizibhale kwisihloko (ngeenjongo zemveliso). Xa ubhala iziphumo kwisihloko, kufuneka usebenzise umthengi we-Kafka ukuzijonga.

Ukufunda idatha kwezi zihloko kunokuqwalaselwa njengoluhlobo lweembono eziphathekayo. Ngeenjongo zethu, sinokusebenzisa inkcazo yembono ebonakalayo evela kwi-Wikipedia: "... into egciniweyo yesiseko equlethe iziphumo zombuzo. Umzekelo, inokuba yikopi yasekuhlaleni yedata ekude, okanye iseti engaphantsi yemigca kunye/okanye iikholamu zetafile okanye ukudibanisa iziphumo, okanye itheyibhile yesishwankathelo efunyenwe ngokudityaniswa” (https://en.wikipedia.org/wiki /Imbonakalo_yezinto).

I-Kafka Streams ikwavumela ukuba uqhube imibuzo edibeneyo kwiivenkile zikarhulumente, ikuvumela ukuba ufunde ngokuthe ngqo ezi mbono zenziweyo. Kubalulekile ukuqaphela ukuba umbuzo kwivenkile karhulumente ngumsebenzi wokufunda kuphela. Oku kuqinisekisa ukuba akufuneki ube nexhala malunga nokwenza imeko ingahambelani ngempazamo ngelixa isicelo sakho sisetyenzwa ngedatha.

Ukukwazi ukubuza ngokuthe ngqo kwiivenkile zikarhulumente kubalulekile. Oku kuthetha ukuba unokwenza usetyenziso lwedashboard ngaphandle kokuba uqale ulande idatha kumthengi weKafka. Kwakhona kwandisa ukusebenza kakuhle kwesicelo, ngenxa yokuba akukho mfuneko yokubhala idatha kwakhona:

ngenxa yendawo yedatha, banokufumaneka ngokukhawuleza;
ukuphinda-phindwa kwedatha kuphelisiwe, kuba ayibhalwanga kwisitoreji sangaphandle.

Into ephambili endifuna ukuba uyikhumbule kukuba ungabuza ngokuthe ngqo ingxelo ngaphakathi kwesicelo sakho. Amathuba okukunika oku akanakuwabaxa. Esikhundleni sokutya idatha esuka eKafka kunye nokugcina iirekhodi kwisiseko sedatha yesicelo, unokubuza iivenkile zombuso ngesiphumo esifanayo. Imibuzo ethe ngqo kwiivenkile zikarhulumente ithetha ikhowudi encinci (akukho mthengi) kunye nesoftware encinci (akukho mfuneko yetafile yedatha yokugcina iziphumo).

Sigubungele umhlaba omninzi kwesi sahluko, ke siza kuyishiya ingxoxo yethu yemibuzo esebenzisanayo ngokuchasene neevenkile zikarhulumente okwangoku. Kodwa ungakhathazeki: kwiSahluko se-9, siza kwenza isicelo esilula sedeshibhodi esinemibuzo esebenzayo. Iza kusebenzisa eminye yemizekelo evela kule kunye nezahluko ezidlulileyo ukubonisa imibuzo esebenzayo kunye nendlela onokuthi ubongeze ngayo kwizicelo zeKafka Streams.

Isishwankathelo

Izinto ze-KStream zimele imisinga yeziganeko, ezithelekiseka nokufakwa kuvimba wedatha. Izinto zeKTable zimele uhlaziyo lwamanzi, ngakumbi njengohlaziyo lwesiseko sedatha. Ubungakanani bento yeKTable ayikhuli, iirekhodi ezindala zitshintshwa ngamatsha.
Izinto zeKTable ziyafuneka kwimisebenzi yohlanganiso.
Usebenzisa imisebenzi yefestile, unokwahlula idatha edityanisiweyo kwiibhakethi zexesha.
Ndiyabulela kwizinto ze-GlobalKTable, unokufikelela kwidatha yereferensi naphi na kwisicelo, kungakhathaliseki ukuba ulwahlulo.
Uqhakamshelwano phakathi kwe KStream, KTable kunye ne GlobalKTable izinto ziyenzeka.

Ukuza kuthi ga ngoku, sigxile ekwakheni usetyenziso lweKafka Streams sisebenzisa inqanaba eliphezulu le-KStream DSL. Nangona indlela yomgangatho ophezulu ikuvumela ukuba wenze iinkqubo ezicocekileyo nezifutshane, ukuyisebenzisa kubonisa ukurhweba. Ukusebenza nge-DSL KStream kuthetha ukonyusa ubufutshane bekhowudi yakho ngokunciphisa inqanaba lolawulo. Kwisahluko esilandelayo, siza kujonga i-API yenqanaba elisezantsi lokuphatha kwaye sizame ezinye iindlela zorhwebo. Iinkqubo ziya kuba zinde kunokuba bezinjalo ngaphambili, kodwa siya kukwazi ukwenza phantse nayiphi na indawo yokuphatha esinokuthi siyifune.

→ Iinkcukacha ezithe vetshe malunga nencwadi zinokufumaneka apha iwebhusayithi yompapashi

→ Kuba Habrozhiteli 25% isaphulelo usebenzisa ikhuphoni - Kafka imijelo

→ Emva kwentlawulo yephepha lencwadi, incwadi ye-elektroniki iya kuthunyelwa nge-imeyile.

umthombo: www.habr.com

Incwadi ethi "I-Kafka ihamba ngentshukumo. Izicelo kunye nee-microservices zomsebenzi wexesha langempela"