Sawubona, Habr.
Muva nje I mayelana nokuthi yimaphi amapharamitha thina njengeqembu esivame ukuwasebenzisela Umkhiqizi we-Kafka kanye Nomthengi ukuze asondele ekulethweni okuqinisekisiwe. Kulesi sihloko ngifuna ukukutshela ukuthi sikuhlele kanjani ukucutshungulwa kabusha komcimbi owamukelwe kwa-Kafka ngenxa yokungatholakali kwesikhashana kwesistimu yangaphandle.
Izinhlelo zokusebenza zesimanje zisebenza endaweni eyinkimbinkimbi kakhulu. Ingqondo yebhizinisi esongwe ngesitaki sobuchwepheshe besimanje, esebenza ngesithombe se-Docker esiphethwe i-orchestrator efana ne-Kubernetes noma i-OpenShift, futhi ixhumana nezinye izinhlelo zokusebenza noma izixazululo zebhizinisi ngochungechunge lwamarutha aphathekayo nangokoqobo. Esimweni esinjalo, okuthile kungahlala kuphuka, ngakho-ke ukucubungula kabusha imicimbi uma enye yezinhlelo zangaphandle ingatholakali kuyingxenye ebalulekile yezinqubo zebhizinisi lethu.
Kwakunjani ngaphambi kweKafka
Phambilini kuphrojekthi sasebenzisa i-IBM MQ ukulethwa kwemiyalezo evumelanayo. Uma kwenzeka noma yiliphi iphutha ngesikhathi sokusebenza kwesevisi, umlayezo owamukelwe ungabekwa ku-dead-letter-queue (DLQ) ukuze kuhlaziywe okwengeziwe. I-DLQ idalwe eduze komugqa ongenayo, umlayezo udluliselwe ngaphakathi kwe-IBM MQ.
Uma iphutha bekungelesikhashana futhi besingakwazi ukulinquma (isibonelo, i-ResourceAccessException ocingweni lwe-HTTP noma i-MongoTimeoutException ngesicelo se-MongoDb), isu lokuzama kabusha lizosebenza. Ngaphandle kokucabanga kwegatsha lohlelo lokusebenza, umlayezo wangempela uhanjiswe kumugqa wesistimu ukuze kubambezeleke ukuthunyelwa, noma kuhlelo lokusebenza oluhlukile olwenziwa kudala ukuze kuphinde kuthunyelwe imilayezo. Lokhu kufaka inombolo yokuthunyelwa kabusha kunhlokweni yomlayezo, eboshelwe kusikhawu sokulibaziseka noma ekupheleni kwesu lezinga lohlelo lokusebenza. Uma sesifike esiphethweni sesu kodwa isistimu yangaphandle ingakatholakali, umlayezo uzofakwa ku-DLQ ukuze kuhlaziywe mathupha.
Ukuthola isixazululo
, ungathola okulandelayo . Ngamafuphi, kuhlongozwa ukudala isihloko sesikhathi sokulibaziseka ngasinye futhi kusetshenziswe izinhlelo zokusebenza zabathengi eceleni, okuzofunda imilayezo ngokulibaziseka okudingekayo.
Naphezu kwenani elikhulu lokubuyekezwa okuhle, kubonakala kimi kungaphumeleli ngokuphelele. Okokuqala, ngoba umthuthukisi, ngaphezu kokufaka izidingo zebhizinisi, kuzodingeka achithe isikhathi esiningi esebenzisa indlela echaziwe.
Ngaphezu kwalokho, uma ukulawula ukufinyelela kunikwe amandla kuqoqo le-Kafka, kuzodingeka uchithe isikhathi udala izihloko futhi unikeze ukufinyelela okudingekayo kuzo. Ngaphezu kwalokhu, uzodinga ukukhetha ipharamitha elungile ye-retention.ms esihlokweni ngasinye sokuphinda uzame ukuze imilayezo ibe nesikhathi sokuthunjwa futhi inganyamalali kuyo. Ukuqaliswa kanye nesicelo sokufinyelela kuyodingeka kuphindwe kusevisi ngayinye ekhona noma entsha.
Manje ake sibone ukuthi yiziphi izindlela ze-spring ngokuvamile kanye ne-spring-kafka ezisinikeza zona zokucutshungulwa kabusha komlayezo. I-Spring-kafka inokuncika okuguquguqukayo ekuzameni kabusha kwentwasahlobo, okunikeza izinto ezifingqiwe zokuphatha i-BackOffPolicies ehlukile. Leli ithuluzi elivumelana nezimo, kodwa i-drawback yalo ebalulekile ukugcina imilayezo ezothunyelwa kabusha kumemori yohlelo lokusebenza. Lokhu kusho ukuthi ukuqalisa kabusha uhlelo lokusebenza ngenxa yesibuyekezo noma iphutha lokusebenza kuzoholela ekulahlekeni kwayo yonke imilayezo esalindwe ukucutshungulwa kabusha. Njengoba leli phuzu libalulekile ohlelweni lwethu, asizange silicabangele ngokuqhubekayo.
i-spring-kafka ngokwayo inikeza ukusetshenziswa okuningana kwe-ContainerAwareErrorHandler, isibonelo , ongacubungula ngayo umlayezo kamuva ngaphandle kokugudluza i-offset uma kwenzeka iphutha. Ukuqala ngenguqulo ye-spring-kafka 2.3, kube nokwenzeka ukusetha i-BackOffPolicy.
Le ndlela ivumela imilayezo ecutshungulwe kabusha ukuthi isinde ekuqaliseni kabusha uhlelo lokusebenza, kodwa akukabikho indlela ye-DLQ. Sakhetha le nketho ekuqaleni kuka-2019, sikholelwa ngethemba lokuthi i-DLQ ngeke idingeke (sabe nenhlanhla futhi asizange siyidinge ngemva kwezinyanga ezimbalwa zokusebenza kohlelo lokusebenza ngohlelo olunjalo lokucutshungulwa kabusha). Amaphutha esikhashana abangele ukuthi i-SeekToCurrentErrorHandler iqhume. Amaphutha asele aphrintiwe kulogi, okuholele ekutheni kube ne-offset, futhi ukucubungula kwaqhubeka nomlayezo olandelayo.
Isinqumo sokugcina
Ukuqaliswa okusekelwe ku-SeekToCurrentErrorHandler kusishukumisele ukuba sakhe indlela yethu yokuthumela kabusha imilayezo.
Okokuqala nje, besifuna ukusebenzisa ulwazi olukhona futhi silunwebe kuye ngohlelo lokusebenza. Ngohlelo lokusebenza olunengqondo lomugqa, kungaba kuhle ukuyeka ukufunda imilayezo emisha isikhathi esifushane esishiwo isu lokuzama futhi. Kwezinye izinhlelo zokusebenza, bengifuna ukuba nephuzu elilodwa elizophoqelela isu lokuphinda uzame. Ngaphezu kwalokho, leli phuzu elilodwa kufanele libe nokusebenza kwe-DLQ kuzo zombili izindlela.
Isu lokuzama kabusha ngokwalo kufanele ligcinwe kuhlelo lokusebenza, elinesibopho sokubuyisa isikhawu esilandelayo lapho kwenzeka iphutha lesikhashana.
Ukumisa Umsebenzisi ngohlelo lokusebenza lwe-Linear Logic
Lapho usebenza nge-spring-kafka, ikhodi yokumisa uMthengi ingase ibukeke kanje:
public void pauseListenerContainer(MessageListenerContainer listenerContainer,
Instant retryAt) {
if (nonNull(retryAt) && listenerContainer.isRunning()) {
listenerContainer.stop();
taskScheduler.schedule(() -> listenerContainer.start(), retryAt);
return;
}
// to DLQ
}Esibonelweni, i-retryAt yisikhathi sokuqala kabusha i-MessageListenerContainer uma sisasebenza. Ukwethulwa kabusha kuzokwenzeka kuchungechunge oluhlukile olwethulwe ku-TaskScheduler, ukuqaliswa kwalo okuhlinzekwa ngentwasahlobo.
Sithola inani le-retryAt ngale ndlela elandelayo:
- Inani lekhawunta yokuphinda ushayele libhekwe phezulu.
- Ngokusekelwe kwinani eliphikisayo, isikhawu samanje sokulibaziseka kuqhinga lokuzama futhi siyaseshwa. Isu limenyezelwe kuhlelo lokusebenza ngokwalo; sikhethe ifomethi ye-JSON ukuze siyigcine.
- Isikhawu esitholakala kumalungu afanayo e-JSON siqukethe inani lamasekhondi ngemva kwalokho ukucubungula kuzodingeka kuphindwe. Le nombolo yamasekhondi yengezwa esikhathini samanje ukuze kwakheke inani lokuzama kabusha.
- Uma isikhawu singatholakali, inani le-retryAt liyize futhi umlayezo uzothunyelwa ku-DLQ ukuze kuhlaziywe mathupha.
Ngale ndlela, okusele ukugcina inani lezingcingo eziphindaphindiwe zomlayezo ngamunye osacutshungulwayo njengamanje, isibonelo kwimemori yohlelo lokusebenza. Ukugcina isibalo sokuzama futhi enkumbulweni akubalulekile kule ndlela, njengoba uhlelo lokusebenza olunengqondo lomugqa alukwazi ukuphatha ukucubungula sekukonke. Ngokungafani nokuzama kabusha kwentwasahlobo, ukuqalisa kabusha uhlelo lokusebenza ngeke kubangele ukuthi yonke imilayezo ilahleke ukuthi icutshungulwe kabusha, kodwa kuzovele kuqalise kabusha isu.
Le ndlela isiza ukususa umthwalo ohlelweni lwangaphandle, okungenzeka ungatholakali ngenxa yomthwalo onzima kakhulu. Ngamanye amazwi, ngaphezu kokucubungula kabusha, sizuze ukuqaliswa kwephethini .
Esimweni sethu, umkhawulo wephutha u-1 kuphela, futhi ukuze unciphise isikhathi sokuphumula sesistimu ngenxa yokucisha kwenethiwekhi kwesikhashana, sisebenzisa isu lokuzama kabusha elingenalutho elinezikhawu ezincane zokubambezeleka. Lokhu kungase kungazifanele zonke izinhlelo zokusebenza zeqembu, ngakho ubudlelwano phakathi komkhawulo wephutha kanye nenani lesikhawu kufanele kukhethwe ngokusekelwe kuzici zesistimu.
Uhlelo lokusebenza oluhlukile lokucubungula imilayezo kusuka kuzinhlelo zokusebenza ezinomqondo onganqunyelwe
Nasi isibonelo sekhodi ethumela umlayezo kuhlelo lokusebenza olunjalo (Phinda uzame), ozophinde uthunyelwe esihlokweni se-DESTINATION uma isikhathi se-RETRY_AT sesifinyelelwe:
public <K, V> void retry(ConsumerRecord<K, V> record, String retryToTopic,
Instant retryAt, String counter, String groupId, Exception e) {
Headers headers = ofNullable(record.headers()).orElse(new RecordHeaders());
List<Header> arrayOfHeaders =
new ArrayList<>(Arrays.asList(headers.toArray()));
updateHeader(arrayOfHeaders, GROUP_ID, groupId::getBytes);
updateHeader(arrayOfHeaders, DESTINATION, retryToTopic::getBytes);
updateHeader(arrayOfHeaders, ORIGINAL_PARTITION,
() -> Integer.toString(record.partition()).getBytes());
if (nonNull(retryAt)) {
updateHeader(arrayOfHeaders, COUNTER, counter::getBytes);
updateHeader(arrayOfHeaders, SEND_TO, "retry"::getBytes);
updateHeader(arrayOfHeaders, RETRY_AT, retryAt.toString()::getBytes);
} else {
updateHeader(arrayOfHeaders, REASON,
ExceptionUtils.getStackTrace(e)::getBytes);
updateHeader(arrayOfHeaders, SEND_TO, "backout"::getBytes);
}
ProducerRecord<K, V> messageToSend =
new ProducerRecord<>(retryTopic, null, null, record.key(), record.value(), arrayOfHeaders);
kafkaTemplate.send(messageToSend);
}Isibonelo sibonisa ukuthi ulwazi oluningi luhanjiswa ngezihloko. Inani le-RETRY_AT litholakala ngendlela efanayo nendlela yokuzama kabusha ngokuma Komthengi. Ngaphezu kwe-DESTINATION ne-RETRY_AT siyadlula:
- I-GROUP_ID, esiqoqa ngayo imilayezo ukuze ihlaziywe mathupha kanye nokusesha okwenziwa lula.
- ORIGINAL_PARTITION ukuze uzame ukugcina Umsebenzisi ofanayo ukuze acutshungulwe kabusha. Le pharamitha ingaba yize, lapho ukwahlukanisa kuzotholwa kusetshenziswa ukhiye othi record.key() womlayezo wangempela.
- Kubuyekezwe inani COUNTER ukuze kulandele isu lokuzama futhi.
- I-SEND_TO ikhombisa njalo ukuthi umlayezo uthunyelwe ukuze ucutshungulwe kabusha lapho ufika ku-RETRY_AT noma ubekwe ku-DLQ.
- ISIZATHU - isizathu sokuthi kungani ukucutshungulwa komlayezo kuphazanyiswe.
Ozama futhi ugcina imilayezo yokuphinda ithunyelwe kanye nokwahlukanisa mathupha ku-PostgreSQL. Isibali sikhathi siqala umsebenzi othola imilayezo nge-RETRY_AT futhi siyibuyisela engxenyeni engu-ORIGINAL_PARTITION yesihloko se-DESTINATION ngokhiye othi record.key().
Uma isithunyelwe, imilayezo iyasuswa ku-PostgreSQL. Ukudluliswa mathupha kwemilayezo kwenzeka ku-UI elula ehlanganyela ne-Retryer nge-REST API. Izici zayo eziyinhloko ukuthumela kabusha noma ukususa imilayezo ku-DLQ, ukubuka ulwazi lwamaphutha nokusesha imiyalezo, isibonelo ngegama lephutha.
Njengoba ukulawula ukufinyelela kunikwe amandla kumaqoqo ethu, kuyadingeka ukuthi ucele ngokungeziwe ukufinyelela esihlokweni i-Retryer esilalelayo, futhi uvumele u-Retryer ukuthi abhalele esihlokweni se-DESTINATION. Lokhu akulungile, kodwa, ngokungafani nendlela yesihloko sekhefu, sine-DLQ egcwele kanye ne-UI yokuyiphatha.
Kunezimo lapho isihloko esingenayo sifundwa amaqembu amaningana ahlukene wabathengi, izinhlelo zabo zokusebenza zisebenzisa ukuqonda okuhlukile. Ukucubungula kabusha umlayezo Nge-Retryer eyodwa yalezi zinhlelo zokusebenza kuzoholela ekuphindlini kolunye. Ukuvikela kulokhu, sakha isihloko esihlukile sokucubungula kabusha. Izihloko ezingenayo nezizama kabusha zingafundwa nguMthengi ofanayo ngaphandle kwemikhawulo.
Ngokuzenzakalelayo le ndlela ayinikezi ukusebenza kwe-circuit breaker, nokho ingangezwa kuhlelo lokusebenza kusetshenziswa noma entsha , ukugoqa izindawo lapho izinsiza zangaphandle zibizelwa ezintweni ezifanele. Ngaphezu kwalokho, kuba nokwenzeka ukukhetha isu for iphethini, okungase futhi kube usizo. Isibonelo, ku-spring-cloud-netflix lokhu kungaba i-thread pool noma i-semaphore.
isiphetho
Njengomphumela, sinohlelo lokusebenza oluhlukile olusivumela ukuthi siphinde ukucubungula umlayezo uma noma iyiphi isistimu yangaphandle ingatholakali okwesikhashana.
Enye yezinzuzo eziyinhloko zohlelo lokusebenza ukuthi ingasetshenziswa izinhlelo zangaphandle ezisebenza kuqoqo elifanayo le-Kafka, ngaphandle kwezinguquko ezibalulekile ohlangothini lwazo! Uhlelo lokusebenza olunjalo luzodinga kuphela ukufinyelela esihlokweni sokuzama kabusha, ugcwalise izihloko ezimbalwa ze-Kafka bese uthumela umlayezo ku-Retryer. Asikho isidingo sokukhulisa noma iyiphi ingqalasizinda eyengeziwe. Futhi ukuze kwehliswe inani lemilayezo edluliswayo isuka kuhlelo lokusebenza iye ku-Retryer futhi ibuye, sihlonze izinhlelo zokusebenza ezinomqondo oqondile futhi sazicubungula kabusha ngokumisa Komthengi.
Source: www.habr.com