Kukonzanso zochitika zolandiridwa kuchokera ku Kafka

Pa Habr.

Posachedwapa ine anafotokoza zimene zinamuchitikira za magawo omwe ife monga gulu timagwiritsa ntchito nthawi zambiri kuti Kafka Producer ndi Consumer ayandikire kubweretsa kotsimikizika. M'nkhaniyi ndikufuna ndikuuzeni momwe tidakonzera kukonzanso kwa chochitika chomwe chinalandiridwa kuchokera ku Kafka chifukwa cha kusapezeka kwa kachitidwe kakunja kwakanthawi.

Mapulogalamu amakono amagwira ntchito m'malo ovuta kwambiri. Malingaliro abizinesi atakulungidwa muukadaulo wamakono, akuthamanga mu chithunzi cha Docker choyendetsedwa ndi oimba ngati Kubernetes kapena OpenShift, ndikulumikizana ndi mapulogalamu ena kapena mayankho abizinesi kudzera pagulu la ma routers akuthupi ndi enieni. M'malo oterowo, china chake chimatha kusweka nthawi zonse, kotero kukonzanso zochitika ngati imodzi mwazinthu zakunja sizikupezeka ndi gawo lofunikira pazantchito zathu.

Zinali bwanji pamaso pa Kafka

M'mbuyomu pulojekitiyi tidagwiritsa ntchito IBM MQ potumiza uthenga mosagwirizana. Ngati cholakwika chilichonse chachitika panthawi yogwira ntchito, uthenga womwe walandilidwa ukhoza kuyikidwa mumzere wakufa-letter (DLQ) kuti muwunikenso pamanja. DLQ idapangidwa pafupi ndi mzere womwe ukubwera, uthengawo unasamutsidwa mkati mwa IBM MQ.

Ngati cholakwikacho chinali chakanthawi ndipo titha kudziwa (mwachitsanzo, ResourceAccessException pa foni ya HTTP kapena MongoTimeoutException pa pempho la MongoDb), ndiye kuti njira yoyeseranso iyamba kugwira ntchito. Mosasamala kanthu za tsatanetsatane wa ntchitoyo, uthenga woyambirira udasunthidwa kupita pamzere wamakina kuti achedwe kutumiza, kapena ku pulogalamu ina yomwe idapangidwa kalekale kuti itumizenso mauthenga. Izi zikuphatikiza nambala yotumizanso pamutu wa uthenga, womwe umalumikizidwa ndi nthawi yochedwa kapena kumapeto kwa njira yogwiritsira ntchito. Ngati tafika kumapeto kwa ndondomekoyi koma dongosolo lakunja silinapezeke, ndiye kuti uthengawo udzayikidwa mu DLQ kuti tiwunikire pamanja.

Kupeza yankho

Kusaka pa intaneti, mungapeze zotsatirazi chisankho. Mwachidule, akufuna kupanga mutu wa nthawi iliyonse yochedwa ndikugwiritsa ntchito mapulogalamu a Consumer pambali, yomwe idzawerenge mauthenga ndi kuchedwa kofunikira.

Ngakhale kuchuluka kwa ndemanga zabwino, zikuwoneka kwa ine sizopambana kwathunthu. Choyamba, chifukwa wopanga mapulogalamuwo, kuwonjezera pakukwaniritsa zofunikira zamabizinesi, adzayenera kuthera nthawi yambiri akukwaniritsa zomwe zafotokozedwazo.

Kuphatikiza apo, ngati kuwongolera kofikira kumayatsidwa pagulu la Kafka, mudzayenera kukhala ndi nthawi yopanga mitu ndikupereka mwayi wofunikira kwa iwo. Kuphatikiza pa izi, muyenera kusankha retention.ms parameter yoyenera pa mutu uliwonse woyesanso kuti mauthenga akhale ndi nthawi yoti atumizidwenso ndipo asasowepo. Kukhazikitsa ndi pempho la mwayi wofikira kuyenera kubwerezedwa pa ntchito iliyonse yomwe ilipo kapena yatsopano.

Tiyeni tiwone zomwe zimayambira nthawi zambiri komanso kasupe-kafka makamaka zimatipatsa ife kuti tikonzenso uthenga. Spring-kafka ili ndi kudalira kosinthika pakuyesanso kasupe, komwe kumapereka zotsalira pakuwongolera BackOffPolicies zosiyanasiyana. Ichi ndi chida chosinthika, koma chovuta chake chachikulu ndikusunga mauthenga kuti atumizidwenso mu kukumbukira kwa pulogalamu. Izi zikutanthauza kuti kuyambitsanso pulogalamuyo chifukwa chakusintha kapena cholakwika chantchito kupangitsa kuti mauthenga onse atayike podikirira kukonzedwanso. Popeza mfundo iyi ndi yofunika kwambiri pa dongosolo lathu, sitinaganizirenso.

spring-kafka palokha imapereka machitidwe angapo a ContainerAwareErrorHandler, mwachitsanzo SeekToCurrentErrorHandler, yomwe mutha kusinthira uthengawo pambuyo pake osasintha kusintha pakachitika cholakwika. Kuyambira ndi mtundu wa kasupe-kafka 2.3, zidakhala zotheka kukhazikitsa BackOffPolicy.

Njirayi imalola mauthenga osinthidwanso kuti apulumuke kuyambiranso ntchito, koma palibe njira ya DLQ. Tidasankha izi kumayambiriro kwa chaka cha 2019, tikukhulupirira kuti DLQ sidzafunika (tinali ndi mwayi ndipo sitinayifune patatha miyezi ingapo tikugwiritsa ntchito pulogalamuyo). Zolakwika kwakanthawi zidapangitsa kuti SeeToCurrentErrorHandler iyake. Zolakwa zotsalazo zidasindikizidwa mu chipikacho, zomwe zidapangitsa kuti pakhale kusintha, ndipo kukonza kumapitilira ndi uthenga wotsatira.

Chosankha chomaliza

Kukhazikitsa kutengera SeekToCurrentErrorHandler kudatipangitsa kupanga njira yathu yotumiziranso mauthenga.

Choyamba, tinkafuna kugwiritsa ntchito zomwe zilipo ndikuzikulitsa kutengera malingaliro akugwiritsa ntchito. Kuti mugwiritse ntchito liniyani, zingakhale bwino kusiya kuwerenga mauthenga atsopano kwa nthawi yochepa yotchulidwa ndi njira yoyeseranso. Pazinthu zina, ndimafuna kukhala ndi mfundo imodzi yomwe ingakamize njira yoyeseranso. Kuphatikiza apo, mfundo imodziyi iyenera kukhala ndi magwiridwe antchito a DLQ panjira zonse ziwiri.

Njira yoyeseranso iyenera kusungidwa mu pulogalamuyo, yomwe ili ndi udindo wopezanso nthawi ina pakavuta kwakanthawi.

Kuyimitsa Wogula pa Linear Logic Application

Mukamagwira ntchito ndi spring-kafka, code yoyimitsa Consumer ikhoza kuwoneka motere:

public void pauseListenerContainer(MessageListenerContainer listenerContainer, 
                                   Instant retryAt) {
        if (nonNull(retryAt) && listenerContainer.isRunning()) {
            listenerContainer.stop();
            taskScheduler.schedule(() -> listenerContainer.start(), retryAt);
            return;
        }
        // to DLQ
    }

Mu chitsanzo, retryAt ndi nthawi yoti muyambitsenso MessageListenerContainer ngati ikugwirabe ntchito. Kukhazikitsanso kudzachitika mu ulusi wosiyana womwe wakhazikitsidwa mu TaskScheduler, kukhazikitsidwa kwake kumaperekedwanso ndi masika.

Timapeza mtengo wa retryAt motere:

1. Mtengo wa kauntala woyitaniranso umayang'aniridwa.
2. Kutengera kuchuluka kwa mtengo, nthawi yomwe ikuchedwa munjira yoyeseranso imafufuzidwa. Njirayi imalengezedwa mu pulogalamuyo; tidasankha mtundu wa JSON kuti tiwusunge.
3. Nthawi yopezeka mugulu la JSON ili ndi masekondi angapo pambuyo pake kukonza kuyenera kubwerezedwa. Nambala iyi ya masekondi awonjezedwa kunthawi yapano kuti apange mtengo wa retryAt.
4. Ngati nthawiyo sinapezeke, ndiye kuti mtengo wa retryAt ndi wopanda pake ndipo uthengawo utumizidwa ku DLQ kuti muwunikire pamanja.

Ndi njirayi, zomwe zatsala ndikusunga manambala obwerezabwereza pa uthenga uliwonse womwe ukukonzedwa, mwachitsanzo pokumbukira ntchito. Kusunga kuwerengeranso kukumbukira sikofunikira panjira iyi, chifukwa kugwiritsa ntchito mzere wotsatira sikungagwire ntchito yonse. Mosiyana ndi kuyesanso kasupe, kuyambitsanso ntchito sikungapangitse kuti mauthenga onse atayike, koma kungoyambitsanso njirayo.

Njirayi imathandiza kuchotsa katundu kunja kwa dongosolo lakunja, lomwe silingakhalepo chifukwa cha katundu wolemera kwambiri. M'mawu ena, kuwonjezera pa reprocessing, ife anakwaniritsa kukhazikitsidwa kwa chitsanzo wophulika.

Kwa ife, malire olakwika ndi 1 okha, ndipo kuti tichepetse nthawi yochepetsera makina chifukwa cha kuzimitsidwa kwakanthawi kwamanetiweki, timagwiritsa ntchito njira yoyesereranso pang'ono yokhala ndi nthawi yaying'ono ya latency. Izi sizingakhale zoyenera pamagulu onse amagulu, kotero mgwirizano pakati pa zolakwikazo ndi mtengo wapakati uyenera kusankhidwa malinga ndi mawonekedwe a dongosolo.

Ntchito ina yosinthira mauthenga kuchokera kuzinthu zomwe zili ndi malingaliro osatsimikiza

Nachi chitsanzo cha khodi yomwe imatumiza uthenga ku pulogalamu yotere (Retryer), yomwe idzatumizanso kumutu wa DESTINATION nthawi ya RETRY_AT ikafika:

public <K, V> void retry(ConsumerRecord<K, V> record, String retryToTopic, 
                         Instant retryAt, String counter, String groupId, Exception e) {
        Headers headers = ofNullable(record.headers()).orElse(new RecordHeaders());
        List<Header> arrayOfHeaders = 
            new ArrayList<>(Arrays.asList(headers.toArray()));
        updateHeader(arrayOfHeaders, GROUP_ID, groupId::getBytes);
        updateHeader(arrayOfHeaders, DESTINATION, retryToTopic::getBytes);
        updateHeader(arrayOfHeaders, ORIGINAL_PARTITION, 
                     () -> Integer.toString(record.partition()).getBytes());
        if (nonNull(retryAt)) {
            updateHeader(arrayOfHeaders, COUNTER, counter::getBytes);
            updateHeader(arrayOfHeaders, SEND_TO, "retry"::getBytes);
            updateHeader(arrayOfHeaders, RETRY_AT, retryAt.toString()::getBytes);
        } else {
            updateHeader(arrayOfHeaders, REASON, 
                         ExceptionUtils.getStackTrace(e)::getBytes);
            updateHeader(arrayOfHeaders, SEND_TO, "backout"::getBytes);
        }
        ProducerRecord<K, V> messageToSend =
            new ProducerRecord<>(retryTopic, null, null, record.key(), record.value(), arrayOfHeaders);
        kafkaTemplate.send(messageToSend);
    }

Chitsanzo chikuwonetsa kuti zambiri zimaperekedwa pamitu. Mtengo wa RETRY_AT umapezeka mofanana ndi momwe mungayeserenso kudzera pa Consumer stop. Kuphatikiza pa DESTINATION ndi RETRY_AT timadutsa:

• GROUP_ID, momwe timayikamo mauthenga kuti tiwunike pamanja ndikusaka kosavuta.
• ORIGINAL_PARTITION kuyesa kusunga Consumer yemweyo kuti mukonzenso. Izi zitha kukhala zopanda pake, pomwe gawo latsopanoli lipezeka pogwiritsa ntchito kiyi ya record.key () ya uthenga woyambirira.
• Zasinthidwa COUNTER mtengo kutsatira njira yoyeseranso.
• SEND_TO ndizomwe zikuwonetsa nthawi zonse ngati uthengawo watumizidwa kuti uwonjezedwenso ukafika RETRY_AT kapena kuyikidwa mu DLQ.
• CHIFUKWA - chifukwa chomwe kusinthidwa kwa uthenga kudasokonezedwa.

Oyesanso amasunga mauthenga oti atumizenso ndi kugawa pamanja mu PostgreSQL. Chowunikira nthawi chimayamba ntchito yopeza mauthenga ndi RETRY_AT ndikuwatumizanso kugawo la ORIGINAL_PARTITION la mutu wa DESTINATION ndi key record.key().

Akatumizidwa, mauthenga amachotsedwa ku PostgreSQL. Kugawa mauthenga pamanja kumachitika mu UI yosavuta yomwe imalumikizana ndi Retryer kudzera pa REST API. Zina zake zazikulu ndikutumiza kapena kuchotsa mauthenga kuchokera ku DLQ, kuyang'ana zolakwika ndikufufuza mauthenga, mwachitsanzo ndi dzina lolakwika.

Popeza ulamuliro wofikira ndiwoyatsidwa pamagulu athu, ndikofunikira kupemphanso mwayi wofikira kumutu womwe Retryer akumvera, ndikulola Retryer kulembera mutu wa DESTINATION. Izi ndizosasangalatsa, koma, mosiyana ndi mutu wanthawi yayitali, tili ndi DLQ ndi UI yokwanira kuti tiyendetse.

Pali zochitika pamene mutu womwe ukubwera ukuwerengedwa ndi magulu angapo ogula, omwe ntchito zawo zimagwiritsa ntchito malingaliro osiyanasiyana. Kukonzanso uthenga kudzera pa Retryer pa imodzi mwamapulogalamuwa kupangitsa kuti mubwerezenso winayo. Kuti titeteze ku izi, timapanga mutu wosiyana kuti tikonzenso. Mitu yomwe ikubwera ndikuyesanso ikhoza kuwerengedwa ndi Consumer yemweyo popanda zoletsa zilizonse.

Mwachikhazikitso njira iyi siimapereka magwiridwe antchito a circuit breaker, komabe imatha kuwonjezeredwa ku pulogalamuyo pogwiritsa ntchito Spring-cloud-netflix kapena watsopano spring cloud circuit breaker, kukulunga malo omwe mautumiki akunja amatchulidwa muzowonjezera zoyenera. Komanso, zimakhala zotheka kusankha njira chochuluka chitsanzo, chomwe chingakhalenso chothandiza. Mwachitsanzo, mu kasupe-mtambo-netflix izi zikhoza kukhala dziwe la ulusi kapena semaphore.

Pomaliza

Zotsatira zake, tili ndi pulogalamu yosiyana yomwe imatilola kubwereza kukonzanso uthenga ngati dongosolo lililonse lakunja silikupezeka kwakanthawi.

Chimodzi mwazabwino kwambiri pakugwiritsa ntchito ndikuti chitha kugwiritsidwa ntchito ndi machitidwe akunja omwe akuyenda pagulu limodzi la Kafka, popanda kusintha kwakukulu kumbali yawo! Ntchito yotereyi idzangofunika kupeza mutu woyeseranso, lembani mitu yochepa ya Kafka ndikutumiza uthenga kwa Retryer. Palibe chifukwa chokweza zowonjezera zowonjezera. Ndipo kuti tichepetse kuchuluka kwa mauthenga omwe amasamutsidwa kuchoka ku pulogalamu kupita ku Retryer ndi kubwereranso, tidazindikira mapulogalamu omwe ali ndi mizera ya mzere ndikuwongoleranso kudzera pa Consumer stop.

Source: www.habr.com