Ndi chaka cha 2019, ndipo tilibe njira yokhazikika pakuphatikiza logi ku Kubernetes. M'nkhaniyi, tikufuna, pogwiritsa ntchito zitsanzo zochokera ku zochitika zenizeni, kugawana zomwe tafufuza, mavuto omwe timakumana nawo ndi mayankho awo.
Komabe, choyamba, ndikusungitsa kuti makasitomala osiyanasiyana amvetsetse zinthu zosiyana kwambiri posonkhanitsa zipika:
- wina akufuna kuwona zipika zachitetezo ndi zowerengera;
- wina - kudula mitengo pakati pa zomangamanga zonse;
- ndipo kwa ena, ndikwanira kusonkhanitsa zipika zogwiritsira ntchito, osaphatikizapo, mwachitsanzo, owerengera.
Pansipa pali mdulidwe womwe uli pansipa momwe tidakhazikitsira "mndandanda wazofuna" zosiyanasiyana komanso zovuta zomwe tidakumana nazo.
Lingaliro: za zida zodula mitengo
Mbiri pazigawo za dongosolo lodula mitengo
Kudula mitengo kwafika patali, chifukwa cha njira zosonkhanitsira ndi kusanthula zipika zapangidwa, zomwe ndizomwe timagwiritsa ntchito masiku ano. Kalelo m'ma 1950s, Fortran adayambitsa njira yofananira yolowera / zotulutsa, zomwe zidathandizira wopanga mapulogalamuwo kuti athetse vuto lake. Izi zinali zolemba zoyambirira zamakompyuta zomwe zidapangitsa moyo kukhala wosavuta kwa opanga mapulogalamu anthawi imeneyo. Lero tikuwona mwa iwo chigawo choyamba cha ndondomeko yodula mitengo - gwero kapena "wopanga" wa zipika.
Sayansi yamakompyuta sinayime pompo: maukonde apakompyuta adawonekera, masango oyamba ... Machitidwe ovuta okhala ndi makompyuta angapo adayamba kugwira ntchito. Tsopano oyang'anira machitidwe adakakamizika kusonkhanitsa zipika kuchokera kumakina angapo, ndipo mwapadera amatha kuwonjezera mauthenga a OS kernel ngati angafunikire kufufuza kulephera kwadongosolo. Kufotokozera zapakati zosonkhanitsira zipika, kumayambiriro kwa zaka za m'ma 2000 zidasindikizidwa , zomwe zimayimira remote_syslog. Umu ndi momwe chigawo china chofunikira chinawonekera: wokhometsa chipika ndi kusunga kwawo.
Chifukwa cha kuchuluka kwa zipika komanso kufalikira kwa matekinoloje a pa intaneti, funso lidabuka la zomwe mitengoyo iyenera kuwonetsedwa mosavuta kwa ogwiritsa ntchito. Zida zosavuta za console (awk/sed/grep) zasinthidwa ndi zapamwamba kwambiri log viewers - chigawo chachitatu.
Chifukwa cha kuchuluka kwa zipika, china chinadziwika bwino: matabwa amafunikira, koma osati onse. Ndipo zipika zosiyanasiyana zimafunikira kusungidwa kosiyanasiyana: zina zimatha kutayika tsiku limodzi, pomwe zina zimafunika kusungidwa kwa zaka zisanu. Chifukwa chake, gawo losefera ndikusintha mayendedwe a data idawonjezedwa pamakina odula mitengo - tiyeni tiyitchule. fyuluta.
Kusungirako kwathandiziranso kwambiri: kuchokera pamafayilo okhazikika kupita kumalo osungira maubale, kenako mpaka kusungitsa zolemba (mwachitsanzo, Elasticsearch). Kotero zosungirako zidalekanitsidwa ndi wosonkhanitsa.
Pamapeto pake, lingaliro lomwelo la chipika lakula kukhala mtundu wa zochitika zosawerengeka zomwe tikufuna kusunga mbiri yakale. Kapena m'malo mwake, ngati mungafunike kuchita kafukufuku kapena kupanga lipoti la analytical ...
Zotsatira zake, m'kanthawi kochepa, kusonkhanitsa zipika kwasintha kukhala gawo lofunikira, lomwe limatha kutchedwa kuti ndi limodzi mwamagawo a Big Data.
Ngati zosindikiza wamba zikadakhala zokwanira "kudula mitengo," tsopano zinthu zasintha kwambiri.
Kubernetes ndi zipika
Kubernetes atabwera ku zomangamanga, vuto lomwe linalipo kale lotolera zipika silinadutsenso. Mwanjira zina, zidakhala zowawa kwambiri: kuyang'anira nsanja ya zomangamanga sikunangokhala kosavuta, komanso kunali kovuta nthawi yomweyo. Ntchito zambiri zakale zayamba kusamukira ku ma microservices. Pankhani ya zipika, izi zikuwonekera mu kuchuluka kwa magwero a logi, kusintha kwawo kwapadera kwa moyo, komanso kufunikira kotsata maubwenzi a zigawo zonse zamakina kudzera muzolemba ...
Kuyang'ana m'tsogolo, nditha kunena kuti tsopano, mwatsoka, palibe njira yodula mitengo ya Kubernetes yomwe ingafanane bwino ndi ena onse. Njira zodziwika kwambiri m'derali ndi izi:
- wina amavundukula phulusa Mtengo wa EFK (Elasticsearch, Fluentd, Kibana);
- wina akuyesera zomwe zangotulutsidwa kumene kapena kugwiritsa ntchito ;
- Π½Π°Ρ (ndipo mwina osati ife tokha? ..) Ndine wokhutitsidwa kwambiri ndi chitukuko changa - ...
Monga lamulo, timagwiritsa ntchito mitolo yotsatirayi m'magulu a K8s (pazothetsera tokha):
- ;
- .
Komabe, sindikhala pa malangizo a kukhazikitsa ndi kasinthidwe awo. M'malo mwake, ndidzayang'ana pa zofooka zawo ndi mfundo zambiri zapadziko lonse za momwe zinthu zilili ndi zipika zambiri.
Yesetsani ndi zipika mu K8s
"Zolemba zatsiku ndi tsiku", ndi angati a inu? ..
Kutolera zipika zapakati kuchokera kumalo okulirapo kumafuna chuma chochuluka, chomwe chidzagwiritsidwa ntchito kusonkhanitsa, kusunga ndi kukonza zipika. Panthawi yogwira ntchito zosiyanasiyana, tidakumana ndi zofunikira zosiyanasiyana komanso zovuta zogwirira ntchito zomwe zidachokera kwa iwo.
Tiyeni tiyese ClickHouse
Tiyeni tiwone zosungira zapakati pa projekiti yokhala ndi pulogalamu yomwe imapanga zipika mwachangu: mizere yopitilira 5000 pamphindikati. Tiyeni tiyambe kugwira ntchito ndi zipika zake, ndikuziwonjezera ku ClickHouse.
Mukangofunika nthawi yeniyeni, seva ya 4-core ndi ClickHouse idzakhala itadzaza kale pa disk subsystem:
Kutsitsa kwamtunduwu kumachitika chifukwa chakuti tikuyesera kulemba mu ClickHouse mwachangu momwe tingathere. Ndipo nkhokwe imakhudzidwa ndi izi ndi kuchuluka kwa disk katundu, zomwe zingayambitse zolakwika zotsatirazi:
DB::Exception: Too many parts (300). Merges are processing significantly slower than inserts
Point ndi kuti mu ClickHouse (ali ndi logi) ali ndi zovuta zawo panthawi yolemba. Zomwe zimayikidwamo zimapanga magawo osakhalitsa, omwe amaphatikizidwa ndi tebulo lalikulu. Zotsatira zake, kujambula kumakhala kovuta kwambiri pa diski, ndipo kumakhalanso ndi malire omwe tidalandirapo: palibe magawo opitilira 1 omwe angaphatikizidwe mu sekondi imodzi (inde, izi ndizoyika 300). pa sekondi).
Kupewa khalidweli, m'zidutswa zazikulu momwe ndingathere ndipo osapitilira nthawi imodzi masekondi awiri aliwonse. Komabe, kulemba mophulika kwakukulu kumasonyeza kuti tiyenera kulemba kawirikawiri mu ClickHouse. Izi, nazonso, zitha kupangitsa kuti buffer kusefukira ndi kutayika kwa zipika. Yankho ndikuwonjezera buffer ya Fluentd, koma kugwiritsa ntchito kukumbukira kudzawonjezekanso.
ndemanga: Gawo lina lovuta la yankho lathu ndi ClickHouse linali logwirizana ndi mfundo yakuti kugawa kwa ife (loghouse) kumayendetsedwa ndi matebulo akunja olumikizidwa. . Izi zimatsogolera ku mfundo yoti mukamayesa nthawi yayikulu, RAM yochulukirapo imafunika, popeza ma metatable amatha kugawa magawo onse - ngakhale omwe mwachidziwikire alibe zofunikira. Komabe, tsopano njira iyi ikhoza kunenedwa kuti sinagwiritsidwe ntchito pamitundu yaposachedwa ya ClickHouse (c ).
Chotsatira chake, zikuwonekeratu kuti si polojekiti iliyonse yomwe ili ndi zothandizira zokwanira zosonkhanitsa zipika mu nthawi yeniyeni mu ClickHouse (mochuluka, kugawa kwawo sikungakhale koyenera). Komanso, muyenera kugwiritsa ntchito Π°ΠΊΠΊΡΠΌΡΠ»ΡΡΠΎΡ, kumene tidzabwereranso pambuyo pake. Mlandu womwe wafotokozedwa pamwambapa ndi weniweni. Ndipo panthawiyo sitinathe kupereka yankho lodalirika komanso lokhazikika lomwe lingagwirizane ndi kasitomala ndikutilola kuti titolere zipika ndi kuchedwa kochepa ...
Nanga bwanji Elasticsearch?
Elasticsearch imadziwika kuti imagwira ntchito zolemetsa. Tiyeni tiyese mu polojekiti yomweyo. Tsopano katunduyo akuwoneka motere:
Elasticsearch idakwanitsa kukumba mayendedwe a data, komabe, kulemba ma voliyumu otere kumagwiritsa ntchito kwambiri CPU. Izi zimaganiziridwa pokonza gulu. Mwaukadaulo, ili si vuto, koma zikuwonekeratu kuti kungogwiritsa ntchito makina osonkhanitsira zipika timagwiritsa ntchito kale ma cores 8 ndikukhala ndi gawo lina lodzaza kwambiri mudongosolo ...
Mfundo yofunika kwambiri: njirayi ikhoza kulungamitsidwa, koma pokhapokha ngati polojekitiyo ndi yaikulu ndipo oyang'anira ake ali okonzeka kugwiritsa ntchito ndalama zambiri pa ndondomeko yodula mitengo yapakati.
Ndiye funso lachilengedwe limabuka:
Ndi mitengo iti yomwe ikufunika kwenikweni?
Tiyeni tiyese kusintha njira yokhayokha: zipika ziyenera kukhala zodziwitsa nthawi imodzi osati zophimba aliyense chochitika mu ndondomeko.
Tiyerekeze kuti tili ndi malo ogulitsira pa intaneti ochita bwino. Ndi mitengo iti yomwe ili yofunika? Kusonkhanitsa zambiri momwe mungathere, mwachitsanzo, kuchokera pachipata cha malipiro, ndilo lingaliro lalikulu. Koma si zipika zonse zochokera ku ntchito yodula zithunzi zomwe zili m'gulu lazinthu zomwe zili zofunika kwa ife: zolakwika zokha ndi kuyang'anira kwapamwamba ndizokwanira (mwachitsanzo, kuchuluka kwa zolakwika 500 zomwe chigawochi chimapanga).
Kotero ife tafika potsimikiza kuti Kudula mitengo pakati sikuyenera nthawi zonse. Nthawi zambiri kasitomala amafuna kusonkhanitsa zipika zonse pamalo amodzi, ngakhale kwenikweni, kuchokera pachipika chonsecho, 5% yokha ya mauthenga omwe ndi ofunikira pabizinesi amafunikira:
- Nthawi zina kumakhala kokwanira kukonza, kunena, kukula kokha kwa chipika cha chidebe ndi chosonkhanitsa zolakwika (mwachitsanzo, Sentry).
- Chidziwitso cholakwa ndi chipika chachikulu chomwe nthawi zambiri chimakhala chokwanira kufufuza zochitika.
- Tinali ndi mapulojekiti omwe amangoyesa kuyesa ndi kusonkhanitsa zolakwika. Wopangayo samafunikira zipika monga choncho - adawona chilichonse kuchokera pazolakwika.
Chitsanzo cha moyo
Nkhani ina ingakhale chitsanzo chabwino. Tidalandira pempho kuchokera ku gulu lachitetezo la m'modzi mwamakasitomala athu omwe anali kugwiritsa ntchito kale njira yamalonda yomwe idapangidwa kale Kubernetes asanakhazikitsidwe.
Zinali zofunikira "kupanga abwenzi" apakati otolera zipika ndi makina ozindikira vuto lamakampani - QRadar. Dongosololi limatha kulandira zipika kudzera pa syslog protocol ndikuzipeza kuchokera ku FTP. Komabe, sikunali kotheka kuti muphatikizepo ndi pulogalamu yowonjezera ya remote_syslog kuti ikhale yodziwika bwino (monga momwe zidakhalira, ). Mavuto pakukhazikitsa QRadar adakhala kumbali ya gulu lachitetezo cha kasitomala.
Chotsatira chake, gawo la zipika zamalonda-zovuta zamalonda zidakwezedwa ku FTP QRadar, ndipo gawo lina linatumizidwa kudzera pa syslog yakutali molunjika kuchokera ku node. Pakuti ife ngakhale analemba - mwinamwake zingathandize wina kuthetsa vuto lofanana ... Chifukwa cha chiwembu chotsatira, kasitomala mwiniwakeyo adalandira ndikusanthula zipika zovuta (pogwiritsa ntchito zida zomwe amakonda), ndipo tinatha kuchepetsa mtengo wa ndondomeko yodula mitengo, kupulumutsa kokha mwezi watha.
Chitsanzo china ndi chosonyeza zimene sitiyenera kuchita. Mmodzi wa makasitomala athu pokonza aliyense zochitika zochokera kwa wogwiritsa ntchito, zopangidwa ndi multiline zotuluka zosakonzedwa chidziwitso mu log. Monga momwe mungaganizire, mitengo yotereyi inali yovuta kwambiri kuti muwerenge ndi kusunga.
Zofunikira pazolemba
Zitsanzo zoterezi zimatsimikizira kuti kuwonjezera pa kusankha ndondomeko yosonkhanitsa zipika, muyenera kutero komanso kupanga matabwa okha! Kodi zofunika pano ndi zotani?
- Malogi akuyenera kukhala owerengeka ndi makina (mwachitsanzo, JSON).
- Mitengo iyenera kukhala yaying'ono komanso yokhoza kusintha kuchuluka kwa mitengo kuti ithetse mavuto. Nthawi yomweyo, m'malo opanga muyenera kuyendetsa makina okhala ndi mitengo yodula ngati chenjezo kapena cholakwa.
- Mitengo iyenera kukhala yokhazikika, ndiye kuti, mu chipika, mizere yonse iyenera kukhala yofanana.
Mitengo yosasinthika imatha kubweretsa zovuta pakukweza zipika muzosungirako ndikuyimitsa kwathunthu pakukonza kwawo. Monga fanizo, nachi chitsanzo chokhala ndi cholakwika 400, chomwe ambiri adakumana nacho muzolemba zomveka bwino:
2019-10-29 13:10:43 +0000 [warn]: dump an error event: error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch"
Cholakwikacho chikutanthauza kuti mukutumiza gawo lomwe mtundu wake ndi wosakhazikika ku index yokhala ndi mapu okonzeka. Chitsanzo chosavuta kwambiri ndi gawo mu chipika cha nginx chokhala ndi zosinthika $upstream_status. Itha kukhala ndi nambala kapena chingwe. Mwachitsanzo:
{ "ip": "1.2.3.4", "http_user": "-", "request_id": "17ee8a579e833b5ab9843a0aca10b941", "time": "29/Oct/2019:16:18:57 +0300", "method": "GET", "uri": "/staffs/265.png", "protocol": "HTTP/1.1", "status": "200", "body_size": "906", "referrer": "https://example.com/staff", "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36", "request_time": "0.001", "cache_status": "-", "upstream_response_time": "0.001, 0.007", "upstream_addr": "127.0.0.1:9000", "upstream_status": "200", "upstream_response_length": "906", "location": "staff"}
{ "ip": "1.2.3.4", "http_user": "-", "request_id": "47fe42807f2a7d8d5467511d7d553a1b", "time": "29/Oct/2019:16:18:57 +0300", "method": "GET", "uri": "/staff", "protocol": "HTTP/1.1", "status": "200", "body_size": "2984", "referrer": "-", "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36", "request_time": "0.010", "cache_status": "-", "upstream_response_time": "0.001, 0.007", "upstream_addr": "10.100.0.10:9000, 10.100.0.11:9000", "upstream_status": "404, 200", "upstream_response_length": "0, 2984", "location": "staff"}
Zolemba zikuwonetsa kuti seva 10.100.0.10 idayankha ndi cholakwika cha 404 ndipo pempholo lidatumizidwa kumalo ena osungira zinthu. Chifukwa chake, mtengo wa zipika udakhala motere:
"upstream_response_time": "0.001, 0.007"
Mkhalidwe umenewu ndi wofala kwambiri moti uyeneranso kudzipatula .
Nanga bwanji kudalirika?
Pali nthawi zina zomwe zipika zonse popanda kupatula zimakhala zofunika. Ndipo ndi izi, njira zosonkhanitsira zipika za K8 zomwe zafotokozedwa / zomwe takambirana pamwambapa zimakhala ndi zovuta.
Mwachitsanzo, odziwa bwino sangathe kutolera zipika kuchokera m'mitsuko yaifupi. Mu imodzi mwama projekiti athu, chidebe chosamukira ku database chidakhala kwa masekondi ochepera 4 kenako ndikuchotsedwa - molingana ndi mawu ofotokozera:
"helm.sh/hook-delete-policy": hook-succeeded
Chifukwa cha ichi, chipika choyendetsera kusamuka sichinaphatikizidwe mu yosungirako. Ndale zingathandize pamenepa. before-hook-creation.
Chitsanzo china ndi kuzungulira kwa chipika cha Docker. Tiyerekeze kuti pali pulogalamu yomwe imalemba mwachangu ku zipika. M'mikhalidwe yabwino, timatha kukonza zipika zonse, koma vuto likangowoneka - mwachitsanzo, monga tafotokozera pamwambapa ndi mawonekedwe olakwika - kuyimitsa kuyimitsa, ndipo Docker atembenuza fayilo. Chotsatira chake ndi chakuti zipika zofunika kwambiri zamabizinesi zitha kutayika.
Ndicho chifukwa chake ndikofunikira kulekanitsa mitsinje ya chipika, embedding kutumiza zofunika kwambiri mwachindunji mu ntchito kuonetsetsa chitetezo chawo. Kuphatikiza apo, sizingakhale zosayenera kupanga zina "accumulator" ya zipika, yomwe imatha kupulumuka pakasungidwe kwakanthawi kochepa posunga mauthenga ovuta.
Pomaliza, tisaiwale zimenezo Ndikofunika kuyang'anira kachitidwe kalikonse moyenera. Kupanda kutero, ndikosavuta kuthamangira mumkhalidwe womwe wolankhula bwino ali m'boma CrashLoopBackOff ndipo sichitumiza chilichonse, ndipo izi zimalonjeza kutayika kwa chidziwitso chofunikira.
anapezazo
M'nkhaniyi, sitikuyang'ana njira za SaaS monga Datadog. Mavuto ambiri omwe akufotokozedwa pano adathetsedwa kale mwa njira imodzi ndi makampani amalonda odziwa kusonkhanitsa zipika, koma si aliyense amene angagwiritse ntchito SaaS pazifukwa zosiyanasiyana. (zambiri ndi mtengo ndi kutsatira 152-FZ).
Kusonkhanitsa zipika zapakati poyamba kumawoneka ngati ntchito yosavuta, koma sichoncho. Ndikofunika kukumbukira kuti:
- Zigawo zofunikira zokha ziyenera kulowetsedwa mwatsatanetsatane, pamene kuyang'anira ndi kusonkhanitsa zolakwika kungakonzedwe ku machitidwe ena.
- Zolemba zopanga ziyenera kukhala zochepa kuti musawonjezere katundu wosafunika.
- Zolemba ziyenera kukhala zowerengeka ndi makina, zokhazikika, komanso kukhala ndi mawonekedwe okhwima.
- Zolemba zenizeni zenizeni ziyenera kutumizidwa mumtsinje wosiyana, womwe uyenera kulekanitsidwa ndi zazikuluzo.
- Ndikoyenera kulingalira za chipika accumulator, chomwe chingakupulumutseni kuphulika kwa katundu wambiri ndikupanga katundu wosungirako yunifolomu.
Malamulo osavutawa, ngati agwiritsidwa ntchito paliponse, angalole kuti mabwalo omwe tawafotokozera pamwambapa agwire ntchito - ngakhale akusowa zigawo zofunika (batire). Ngati simutsatira mfundo zotere, ntchitoyi idzakutsogolerani inu ndi zomangamanga mosavuta ku gawo lina lodzaza kwambiri (ndipo nthawi yomweyo losathandiza) la dongosolo.
PS
Werenganinso pa blog yathu:
- Β«";
- Β«";
- Β«".
Source: www.habr.com