Iilogi ziyinxalenye ebalulekileyo yenkqubo, ekuvumela ukuba uqonde ukuba iyasebenza (okanye ayisebenzi) njengoko kulindelekile. Ngaphantsi kweemeko ze-microservice architecture, ukusebenza kunye nezigodo kuba yinto eyahlukileyo ye-Olympiad ekhethekileyo. Kukho imiba emininzi ekufuneka iqwalaselwe ngoku:
- indlela yokubhala iilogi ukusuka kwisicelo;
- indawo yokubhala iilog;
- indlela yokuhambisa iilog ukuze zigcinwe kwaye zilungiswe;
- indlela yokuqhuba nokugcina iinkuni.
Ukusetyenziswa kwetekhnoloji ethandwayo yokufaka isikhongozeli songeza isanti ngaphezulu kweharika kwindawo yokhetho lokusombulula iingxaki.
Kanye malunga nalo mbhalo wengxelo kaYuri Bushmelev "Imephu yeraki kwintsimi yokuqokelela kunye nokuhambisa izingodo"
Ngubani okhathalayo, nceda phantsi kwekati.
Igama lam nguYuri Bushmelev. Ndisebenzela uLazada. Namhlanje ndiza kuthetha ngendlela esazenza ngayo iinkuni zethu, indlela esiziqokelele ngayo, kunye noko sikubhalayo apho.
Sivela phi? Singoobani? I-Lazada ngumthengisi we-intanethi we-#1 kumazwe amathandathu akuMzantsi-mpuma we-Asia. Onke la mazwe asasazwa phakathi kwamaziko edatha. Ngoku kukho amaziko edatha e-4 ewonke. Kutheni oku kubalulekile? Kaloku ezinye izigqibo bezibangelwa kukuba kukho unxulumano olubuthathaka kakhulu phakathi kwamaziko. Sine-architecture ye-microservice. Ndothuswa kukufumanisa ukuba sele sinee-microservices ezingama-80. Ukuqala kwam umsebenzi ngezigodo, zazingama-20 kuphela.Kwaye kukho isiqwenga esikhulu selifa le-PHP, ekufuneka ndiphile nalo kwaye ndinyamezele. Konke oku kusenzela thina okwangoku ngaphezu kwezigidi ezi-6 zemiyalezo ngomzuzu kwisistim xa iyonke. Ngapha koko ndiza kubonisa indlela esizama ngayo ukuhlala nale nto, kwaye kutheni oku kunjalo.
Kuya kufuneka uphile nale miyalezo eyi-6 yezigidi ngandlela thile. Yintoni esimele siyenze ngabo? 6 yezigidi zemiyalezo efunekayo:
- thumela kwi-app
- vuma ukuhanjiswa
- ukuhambisa uhlalutyo kunye nokugcinwa.
- hlalutya
- gcina ngandlela thile.
Xa kukho izigidi ezithathu zemiyalezo, ndandinenkangeleko efanayo. Kaloku siqale ngeepeni. Kucacile ukuba iilogi zesicelo zibhalwe apho. Umzekelo, ayikwazanga ukuqhagamshela kwisiseko sedatha, inokuqhagamshela kwisiseko sedatha, kodwa ayikwazanga ukufunda okuthile. Kodwa ngaphandle koku, nganye yee-microservices zethu ikwabhala i-log yokufikelela. Isicelo ngasinye esifika kwi-microservice siwela kwilog. Kutheni sisenza oku? Abaphuhlisi bafuna ukukwazi ukulandelela. Ilog nganye yofikelelo iqulathe ibala letraceid, ngokubhekiselele kuyo ujongano olukhethekileyo emva koko lukhulule lonke ikhonkco kwaye lubonise kakuhle umkhondo. Umkhondo ubonisa ukuba isicelo sihambe njani, kwaye oku kunceda abaphuhlisi bethu bajongane nayo nayiphi na inkunkuma engaziwayo ngokukhawuleza.
Indlela yokuphila nayo? Ngoku ndiza kuchaza ngokufutshane intsimi yeenketho - indlela le ngxaki isonjululwe ngayo ngokubanzi. Indlela yokusombulula ingxaki yokuqokelela, ukudlulisa nokugcina iingodo.
Ubhalwa njani kwisicelo? Kucacile ukuba kukho iindlela ezahlukeneyo. Ngokukodwa, kukho eyona nkqubo ilungileyo, njengoko amaqabane asefashonini esixelela. Zimbini iindidi zesikolo esidala, njengoko ootatomkhulu besitsho. Kukho ezinye iindlela.
Ngokuqokelelwa kwezigodo, imeko iphantse ifane. Azikho iindlela ezininzi zokusombulula le ndawo. Zininzi zazo, kodwa azikabi ninzi.
Kodwa ngokuhanjiswa kunye nohlalutyo olulandelayo, inani leenguqu liqala ukuqhuma. Andizukuchaza ukhetho ngalunye ngoku. Ndicinga ukuba ukhetho oluphambili luyaziwa kuwo wonke umntu owayenomdla kwisihloko.
Ndiza kukubonisa ukuba siyenze njani eLazada nokuba yaqala njani.
Kunyaka ophelileyo, ndafika eLazada kwaye ndathunyelwa kwiprojekthi yelogi. Kwakunje apho. Ilog esuka kwisicelo yayibhalelwe stdout kunye stderr. Yonke into yenziwa ngendlela esefashonini. Kodwa ke abaphuhlisi bayiphose ngaphandle kwemijelo eqhelekileyo, kwaye ke iingcali zeziseko zophuhliso ziya kuyifumanisa ngandlela thile. Phakathi kweengcali zeziseko ezingundoqo kunye nabaphuhlisi, kukho nabakhululi abathi: "uh ... kulungile, masibasonge nje kwifayile eneqokobhe, kwaye yiloo nto." Kwaye ekubeni yonke le nto ikwisikhongozelo, bayisongele kanye kwisikhongozeli ngokwaso, imephu yolawulo ngaphakathi kwaye bayibeke apho. Ndicinga ukuba icace gca kuye wonke umntu okwenzekileyo.
Makhe sijonge phambili kancinci. Sazihambisa njani ezi logs. Umntu ukhethe i-td-agent, eneneni etyibilikayo kodwa engakwazi kakuhle. Andikaluqondi unxibelelwano lwezi projekthi zimbini, kodwa zibonakala ngathi ziyinto enye. Kwaye oku kutyibilika, kubhalwe kwiRuby, funda iifayile zelog, zahlulahlulwe zibe yi-JSON usebenzisa iintetho eziqhelekileyo. Emva koko bathunyelwa eKafka. Ngaphezu koko, eKafka, sinezihloko ezi-4 ezahlukeneyo kwi-API nganye. Kutheni 4? Ngenxa yokuba kukho ubomi, kukho isiteji, kwaye ngenxa yokuba kukho i-stdout kunye ne-stderr. Abaphuhlisi baziveza, kwaye abasebenzi beziseko zophuhliso kufuneka babenze eKafka. Ngaphezu koko, iKafka yayilawulwa lelinye isebe. Ke ngoko, bekuyimfuneko ukwenza itikiti ukuze benze izihloko ezi-4 apho kwi-api nganye. Wonke umntu walibala ngayo. Ngokubanzi, yayiyinkunkuma kunye nenkunkuma.
Senze ntoni emva koko? Sayithumela e kafka. Ukusuka eKafka, isiqingatha sezigodo sabhabha saya eLogstash. Esinye isiqingatha sezigodo zabelwana ngazo. Abanye babhabhele kwenye iGreylog, abanye baya kwenye iGreylog. Ngenxa yoko, konke oku kwabhabhela kwiqela elinye le-Elasticsearch. Oko kukuthi, yonke le mess wawela ekugqibeleni apho. Akunyanzelekanga ukuba uyenze loo nto!
Le yindlela ekhangeleka ngayo xa ijongwe phezulu. Akunyanzelekanga ukuba uyenze loo nto! Apha, iindawo zengxaki ziphawulwa ngokukhawuleza ngamanani. Zininzi, kodwa ezi-6 ziyingxaki ngokwenene, ekufuneka yenziwe ngazo. Ndiza kubalisa ngokwahlukeneyo ngoku.
Apha (1,2,3) sibhala iifayile kwaye, ngokufanelekileyo, kukho iiraki ezintathu apha kanye.
Eyokuqala (1) kukuba kufuneka sizibhale ndaweni ithile. Akusoloko kunqweneleka ukunika i-API ukukwazi ukubhala ngokuthe ngqo kwifayile. Kunqweneleka ukuba i-API ibe yodwa kwisikhongozeli, kwaye nokuba ngcono, ukuba ifundwe kuphela. Ndingumlawuli wenkqubo, ngoko ke ndinembono eyenye kancinane yezi zinto.
Inqaku lesibini (2,3) kukuba sinezicelo ezininzi eziza kwi-API. I-API ibhala idatha eninzi kwifayile. Iifayile ziyakhula. Kufuneka sizijikelezise. Kuba kungenjalo awuzukukwazi ukugcina naziphi na iidiski apho. Ukuzijikelezisa kubi kuba ziphinda zithunyelwe ngeqokobhe kulawulo. Akukho ndlela sinokuyijikeleza. Awukwazi ukuxelela isicelo ukuba siphinde sivule iziphatho. Kuba abaphuhlisi baya kukujonga njengesidenge: “Ziziphi izichazi? Sibhalela ngokubanzi kwi-stdout. Izikhokelo zenze i-copytruncate ibe yi-logrotate, eyenza nje ikopi yefayile kunye nemiqobo yemvelaphi. Ngokufanelekileyo, phakathi kwezi nkqubo zokukhuphela, indawo yedisk ihlala ibaleka.
(4) Sasinefomathi ezahlukeneyo kwii-APIs ezahlukeneyo. Zazahluke kancinane, kodwa i-regexp kwafuneka ibhalwe ngokwahlukileyo. Ekubeni yonke yayilawulwa nguPuppet, kwakukho iqela elikhulu leeklasi ezinamaphela azo. Ngaphezu koko, i-td-agent ixesha elininzi inokutya inkumbulo, ibe sisidenge, inokwenza ngathi yayisebenza kwaye ingenzi nto. Ngaphandle, kwakungenakwenzeka ukuqonda ukuba wayengenzi nto. Okona kulungileyo, uya kuwa, yaye ubani uya kumthabatha kamva. Ngokuchanekileyo, isilumkiso siya kubhabha, kwaye umntu uya kuhamba asiphakamise ngezandla zakhe.
(6) Kwaye eyona nkunkuma kunye nenkunkuma - yayiyi-elasticsearch. Kuba yayiyinguqulelo endala. Kungenxa yokuba ngelo xesha sasingenazo iinkosi ezizinikeleyo. Sasinamaplanga ahlukahlukeneyo namasimi anokudibana. Iilogi ezahlukeneyo zezicelo ezahlukeneyo zinokubhalwa ngamagama afanayo entsimi, kodwa kwangaxeshanye kunokubakho idatha eyahlukileyo ngaphakathi. Oko kukuthi, ilogi enye iza ne-Integer kwintsimi, umzekelo, inqanaba. Enye ilogi iza kunye noMtya kwinqanaba lenqanaba. Ukungabikho kwemephu engatshintshiyo, into emangalisayo enjalo iyajika. Ukuba, emva kokujikeleza kwesalathisi, umyalezo onomtya ufike kuqala kwi-elasticsearch, ngoko siphila ngokuqhelekileyo. Kwaye ukuba eyokuqala yafika ne-Integer, ke yonke imiyalezo elandelayo efike nge-String ilahliwe. Ngenxa yokuba uhlobo lwebala aluhambelani.
Saqalisa ukubuza le mibuzo. Sagqiba ekubeni singabajongi abanetyala.
Kodwa kukho into ekufuneka yenziwe! Into ecacileyo kukuba kufuneka simisele imigangatho. Sasisele sinemilinganiselo ethile. Ezinye sizise emva kwexeshana. Ngethamsanqa, ifomathi yelogi enye yazo zonke ii-APIs yayisele ivunyiwe ngelo xesha. Ibhalwe ngokuthe ngqo kwimigangatho yokusebenzisana kwenkonzo. Ngokufanelekileyo, abo bafuna ukufumana iilog kufuneka bazibhale ngolu hlobo. Ukuba umntu akabhali izingodo kule fomati, ngoko asiqinisekisi nto.
Ngapha koko, ndingathanda ukuba nomgangatho omnye weendlela zokurekhoda, ukuhambisa kunye nokuqokelela izingodo. Ngokwenyani, zibhalwe phi, kwaye zihanjiswa njani. Imeko efanelekileyo kuxa iiprojekthi zisebenzisa ithala leencwadi elifanayo. Kukho ithala leencwadi elahlukileyo le-Go, kukho ithala leencwadi elahlukileyo le-PHP. Wonke umntu esinaye, wonke umntu makawasebenzise. Okwangoku, ndingathi siphumelela ngama-80 ekhulwini. Kodwa abanye bayaqhubeka besitya i-cacti.
Kwaye apho (kwisilayidi) "i-SLA yokuhanjiswa kwelogi" iqala ukubonakala. Ayikabikho, kodwa sisebenza kuyo. Kuba kulula kakhulu xa i-infra isithi ukuba ubhala ngendlela enjalo kunye nefomathi kwindawo enjalo kwaye ungabi ngaphezu kwemiyalezo engu-N ngomzuzwana, ngoko siyakuyisa apho. Isusa iintloko ezininzi. Ukuba kukho i-SLA, ke ilungile!
Saqala njani ukucombulula le ngxaki? Eyona rakhi yayine-td-arhente. Kwakungacacanga ukuba iinkuni zethu ziya phi. Ngaba ziyaziswa? Ngaba bayahamba? Baphi phofu? Ke ngoko, kwagqitywa ekubeni kutshintshelwe i-td-arhente ngento yokuqala. Iinketho zento onokuyibuyisela endaweni yayo, ndichaze ngokufutshane apha.
Ukwazi kakuhle. Okokuqala, ndadibana naye emsebenzini wangaphambili, kwaye naye wayewela apho. Okwesibini, oku kuyafana, kuphela kwiprofayili.
ukubetha kwefayile. Kwaba kuhle njani kuthi? Inyani yokuba ukuGo, kwaye sinobuchule obukhulu kwi-Go. Ngokufanelekileyo, ukuba kukho nantoni na, sinokuyongeza kuthi ngandlel’ ithile. Yiyo loo nto singayithathanga. Ukuze kungabikho nasiphi na isilingo sokuqala ukuzibhala kwakhona.
Isisombululo esicacileyo se-sysadmin zizo zonke iintlobo ze-syslogs kulo buninzi (syslog-ng/rsyslog/nxlog).
Okanye bhala into eyakho, kodwa siyilahlile, kunye nefayile yefayile. Ukuba ubhala into, ke kungcono ukubhala into eluncedo kwishishini. Ukuhambisa iingodo, kungcono ukuthatha into esele yenziwe.
Ke ngoko, ukhetho lwehla kukhetho phakathi kwe-syslog-ng kunye ne-rsyslog. Ndiye ndoyama ngakwi-rsyslog kuba besele sineeklasi ze-rsyslog kwiPuppet, kwaye andizange ndifumane mahluko ucacileyo phakathi kwazo. Yintoni i-syslog, yintoni i-syslog. Ewe, amanye amaxwebhu ambi, amanye angcono. Uyazi ngale ndlela, kwaye wenza ngokwahlukileyo.
Kwaye kancinci malunga ne-rsyslog. Okokuqala, ipholile kuba ineemodyuli ezininzi. Ine-RainerScript efundeka ngabantu (ulwimi loqwalaselo lwangoku). Ibhonasi eyoyikekayo kukuba sinokuxelisa indlela yokuziphatha ye-td-arhente ngezixhobo zayo eziqhelekileyo, kwaye akukho nto itshintshileyo kwizicelo. Oko kukuthi, sitshintsha i-td-arhente ukuya kwi-rsyslog, kwaye ungabambi yonke enye into okwangoku. Kwaye ngokukhawuleza sifumana ukuhanjiswa okusebenzayo. Okulandelayo, mmnormalize yinto epholileyo malunga ne-rsyslog. Ikuvumela ukuba ucazulule iilog, kodwa hayi ngeGrok kunye ne-regexp. Yenza umthi we-syntax ongabonakaliyo. Icalula iilog ngendlela efanayo naleyo athi umqokeleli acazulule ngayo ikhowudi yomthombo. Oku kukuvumela ukuba usebenze ngokukhawuleza, utye i-CPU encinci, kwaye, ngokubanzi, yinto nje epholileyo. Kukho iqela lezinye iibhonasi. andiyi kuhlala phezu kwabo.
I-rsyslog inezinto ezininzi ezingalunganga. Ziphantse zifana neebhonasi. Iingxaki eziphambili kukuba kufuneka ukwazi ukuyipheka, kwaye kufuneka ukhethe inguqulo.
Saye sagqiba kwelokuba siza kubhala iingodo kwi-unix socket. Kwaye hayi kwi /dev/log, kuba apho sinobugxwayiba benkqubo yelogi, kukho ijenali kulo mbhobho. Ngoko ke masibhalele kwisokethi yesiko. Siza kuyiqhoboshela kwiseti yomthetho eyahlukileyo. Masingaphazamisi nantoni na. Yonke into iya kucaca kwaye iyaqondakala. Ngoko ngokwenene senza. Uluhlu olunezi sokethi lusemgangathweni kwaye lugqithiselwe kuzo zonke izikhongozeli. Izikhongozeli zinokubona isiseko ezifunekayo, zivule kwaye zibhalele kuyo.
Kutheni kungeyofayile? Kuba wonke umntu ufundile , ezame ukuhambisa ifayile kwidocker, kwaye yafumanisa ukuba emva kokuqalisa kwakhona i-rsyslog, inkcazo yefayile iyatshintsha, kwaye idocker ilahlekelwa yile fayile. Ugcina evule enye into, kodwa hayi kwaloo sokethi inye apho babhala khona. Sigqibe kwelokuba siyakuyigqitha le ngxaki, kwaye, kwangaxeshanye, siyidlule ingxaki yokuvala.
I-Rsyslog yenza iintshukumo ezibonisiweyo kwisilayidi kwaye ithumela iilog nokuba yirelay okanye iKafka. I-Kafka ilandela indlela endala. Rayleigh - Ndizamile ukusebenzisa i-rsyslog ecocekileyo ukuhambisa izingodo. Ngaphandle komgca woMyalezo, usebenzisa izixhobo ezisemgangathweni ze-rsyslog. Ngokwenene, iyasebenza.
Kodwa kukho ama-nuances ngendlela yokuwafaka kamva kule ndawo (Logstash/Graylog/ES). Eli candelo (rsyslog-rsyslog) lisetyenziswa phakathi kwamaziko edatha. Nantsi ikhonkco ye-tcp ecinezelweyo, ekuvumela ukuba ugcine i-bandwidth kwaye, ngokufanelekileyo, ngandlela-thile ukwandisa amathuba okuba siya kufumana ezinye iilogi ezivela kwelinye iziko ledatha xa ishaneli igcwele. Ngenxa yokuba sine-Indonesia, apho yonke into imbi. Kulapho ingxaki eqhubekayo ilele.
Sicinge malunga nendlela esibeka iliso ngayo ngokwenene, ngawaphi amathuba okuba iilog esizirekhode kwisicelo zifikelela esiphelweni? Sagqiba ekubeni siqale iimethrikhi. I-Rsyslog inemodyuli yayo yokuqokelela izibalo, enohlobo oluthile lwezinto zokubala. Umzekelo, inokukubonisa ubungakanani bomgca, okanye mingaphi imiyalezo engenileyo malunga nesenzo esinje. Usenokuthatha into kubo. Ngaphezu koko, inezinto zokubala zesiko onokuthi uziqwalasele, kwaye iya kukubonisa, umzekelo, inani lemiyalezo eyenziwa yi-API ethile. Emva koko, ndabhala i-rsyslog_exporter kwiPython, kwaye sayithumela yonke kwi-Prometheus kwaye siceba. Besizifuna ngokwenene iimetriki zeGreylog, kodwa ukuza kuthi ga ngoku asikabi nalo ixesha lokuziseka.
Ziziphi iingxaki? Ingxaki yavela ngento yokuba sifumene (NGOKO!) Ukuba i-Live APIs yethu ibhala imiyalezo ye-50k ngesekhondi. Oku kuphela Live API ngaphandle kweqonga. Kwaye iGraylog isibonisa kuphela i-12 lamawaka emiyalezo ngomzuzwana. Kwaye kwaphakama umbuzo osengqiqweni, iphi intsalela? Ukusuka apho sagqiba kwelokuba iGreylog ayinakukwazi ukumelana nayo. Sijongile, kwaye, eneneni, iGreylog ene-Elasticsearch ayizange iyibambe kakuhle le ndlela.
Okulandelayo, ezinye izinto esiye sazifumana endleleni.
Ubhala kwisokethi kuvaliwe. Ingaba yenzeke njani? Xa ndisebenzisa i-rsyslog yokuhanjiswa, ngaxa lithile saphula umjelo phakathi kwamaziko edatha. Ukuhanjiswa kwavuka kwindawo enye, ukuhanjiswa kwavuka kwenye indawo. Konke oku kwehle kumatshini one-APIs ebhala kwi-rsyslog socket. Kwakukho umgca. Emva koko umgca wokubhalela kwi-socket ye-unix ugcwaliswe, ethi ngokungagqibekanga iipakethi ze-128. Kwaye okulandelayo bhala () kwiibhloko zesicelo. Xa sijonga kwithala leencwadi esilisebenzisa kwii-Go applications, kwabhalwa apho ukuba ukubhala kwi-socket kwenzeka kwimodi yokungathinteli. Sasiqinisekile ukuba akukho nto ivaliweyo. Kuba sifundile ngubani owabhala ngayo. Kodwa kukho umzuzwana. Kwakukho ne-loop engapheliyo malunga nalo mnxeba, apho kwakukho iinzame eziqhubekayo zokutyhala umyalezo kwi-socket. Khange simqaphele. Kwafuneka ndilibhale kwakhona ithala leencwadi. Ukusukela ngoko, iye yatshintsha amaxesha amaninzi, kodwa ngoku siye sazilahla izitshixo kuzo zonke iinkqubo ezingaphantsi. Ke ngoko, unokumisa i-rsyslog kwaye akukho nto iya kuwa.
Kuyimfuneko ukubeka iliso ubungakanani bemigca, okunceda ukuba unganyatheli kule raki. Okokuqala, sinokubeka iliso xa siqala ukuphulukana nemiyalezo. Okwesibini, sinokubeka iliso ukuba sineengxaki ngokuhambisa.
Kwaye omnye umzuzu ongathandekiyo - ukukhulisa ngamaxesha e-10 kwi-architecture ye-microservice kulula kakhulu. Asinazo izicelo ezininzi ezingenayo, kodwa ngenxa yegrafu apho le miyalezo iqhubela phambili, ngenxa yeelogi zokufikelela, ngokwenene sonyusa umthwalo kwiilogi malunga namaxesha alishumi. Ngelishwa, bendingenalo ixesha lokubala amanani achanekileyo, kodwa ii-microservices yile nto ziyiyo. Oku kufuneka kugcinwe engqondweni. Kuyavela ukuba okwangoku i-log yokuqokelela i-subsystem yeyona ilayishwe kakhulu eLazada.
Uyisombulula njani ingxaki ye-elasticsearch? Ukuba ufuna ukukhawuleza ufumane izingodo kwindawo enye, ukuze ungabaleki kuwo wonke oomatshini kwaye uqokelele apho, sebenzisa ukugcinwa kweefayile. Oku kuqinisekisiwe ukusebenza. Yenziwa nakuwuphi na umncedisi. Kufuneka nje uncamathele iidiski apho kwaye ubeke isyslog. Emva koko, uqinisekisiwe ukuba zonke iilogi kwindawo enye. Emva koko kuyakwenzeka ukucwangcisa kancinci kancinci elasticsearch, greylog, okanye enye into. Kodwa uya kuba sele unazo zonke iilogi, kwaye, ngaphezu koko, ungazigcina, ngokubhekiselele ekubeni kukho uluhlu olwaneleyo lwediski.
Ngexesha lengxelo yam, iskimu saqala ukubonakala ngolu hlobo. Sayeka ukubhalela loo fayile. Ngoku, kunokwenzeka ukuba, siya kucima iintsalela. Kumatshini wendawo osebenzisa i-API, siya kuyeka ukubhala kwiifayile. Okokuqala, kukho ukugcinwa kwefayile, esebenza kakuhle kakhulu. Okwesibini, aba matshini bahlala bephelelwa yindawo, kufuneka uyibeke iliso rhoqo.
Le nxalenye kunye neLogstash kunye neGreylog, iyenyuka ngokwenene. Ngoko ke, kufuneka uyisuse. Kufuneka ukhethe ibe nye.
Sagqiba kwelokuba siyilahle iLogstash neKibana. Kaloku sinesebe lezokhuseleko. Yintoni unxibelelwano? Uxhulumaniso kukuba i-Kibana ngaphandle kwe-X-Pack kwaye ngaphandle kweShield ayikuvumeli ukuba uhlukanise amalungelo okufikelela kwiilogi. Ngoko ke, bathatha Graylog. Inayo yonke into. Andiyithandi, kodwa iyasebenza. Sithenge i-hardware entsha, safaka iGreylog entsha apho, kwaye sahambisa zonke iilog ezinefomathi engqongqo kwiGreylog eyahlukileyo. Siyisombulule ingxaki ngeentlobo ezahlukeneyo zemimandla efanayo ngombutho.
Yintoni kanye efakwe kwiGreylog entsha. Sibhale nje yonke into edocker. Sathatha iqela labancedisi, sakhupha iimeko ezintathu ze-Kafka, i-7 ye-Graylog servers version 2.3 (kuba ndifuna i-Elasticsearch version 5). Konke oku kwaphakanyiswa kuhlaselo oluvela kwi-HDD. Sibone ireyithi yezalathiso ukuya kutsho kwi-100 lamawaka emiyalezo ngesekhondi. Siye sabona inani le-140 terabytes yedatha ngeveki.
Kwaye kwakhona iraki! Sinentengiso ezimbini ezizayo. Siye ngaphaya kwe-6 yezigidi zezithuba. Thina iGreylog asinaxesha lokuhlafuna. Ngandlela ithile kufuneka usinde.
Le yindlela esisinda ngayo. Kongezwe iiseva ezimbalwa kunye nee-SSD. Okwangoku siphila ngoluhlobo. Ngoku sele sihlafuna imiyalezo eyi-160k ngomzuzwana. Asikafiki kumda okwangoku, ke akukacaci ukuba singakanani na esinokuphuma kuyo ngokwenyani.
Ezi zizicwangciso zethu zekamva. Kwezi, eneneni, eyona nto ibalulekileyo mhlawumbi kukufumaneka okuphezulu. Asikabi nayo. Iimoto ezininzi zisekwe ngendlela efanayo, kodwa ukuza kuthi ga ngoku yonke into ihamba ngemoto enye. Kuyimfuneko ukuchitha ixesha ukuseta i-faillover phakathi kwabo.
Qokelela iimetriki kwiGreylog.
Yenza umda wesantya ukuze sibe ne-API enye ephambeneyo engasibulali i-bandwidth kunye nayo yonke enye into.
Kwaye okokugqibela, sayina uhlobo oluthile lwe-SLA kunye nabaphuhlisi ukuze sikwazi ukukhonza kakhulu. Ukuba ubhala ngakumbi, ngoko uxolo.
Kwaye ubhale amaxwebhu.
Ngokufutshane, iziphumo zayo yonke into esiyifumeneyo. Okokuqala, imigangatho. Okwesibini, i-syslog yikhekhe. Okwesithathu, i-rsyslog isebenza kanye njengoko ibhaliwe kwisilayidi. Kwaye makhe sifike kwimibuzo.
Imibuzo yakho.
Umbuzo wakho: Kwakutheni ukuze bathathe isigqibo sokungayithathi ... (filebeat?)
Phendula: Kufuneka ubhalele kwifayile. Ndandingafuni nyani. Xa i-API yakho ibhala amawaka emiyalezo ngomzuzwana, nokuba ujikeleza kube kanye ngeyure, oku akuseyiyo inketho. Ungabhalela kumbhobho. Apho abaphuhlisi bandibuze: "Kuza kwenzeka ntoni ukuba inkqubo esibhala kuyo iwe phantsi"? Andizange ndifumane ukuba ndibaphendule ndithini, kwaye ndathi: "Kulungile, ok, masingayenzi loo nto."
Umbuzo wakho: Kutheni ungabhali nje iilog kwiHDFS?
PhendulaA: Eli linyathelo elilandelayo. Sayicinga kwasekuqaleni, kodwa kuba kungekho zixhobo zokujongana nayo okwangoku, ijinga kwisisombululo sethu sexesha elide.
Umbuzo wakho: Ifomathi yekholamu iya kufaneleka ngakumbi.
Phendula: Ndiyaqonda. Thina "se" ngezandla zozibini.
Umbuzo wakho: Ubhalela rsyslog. Zombini i-TCP kunye ne-UDP ziyafumaneka apho. Kodwa ukuba i-UDP, ke uqinisekisa njani ukuhanjiswa?
PhendulaA: Kukho amanqaku amabini. Okokuqala, ngokukhawuleza ndixelela wonke umntu ukuba asikuqinisekisi ukuhanjiswa kweelogi. Kuba xa abaphuhlisi beza bathi: "Masiqale ukubhala idatha yezemali apho, kwaye uya kusibeka kwindawo ethile xa kwenzeka into eyenzekayo," sibaphendula, "Mkhulu! Masiqale ukubhloka ekubhaleni kwisokethi, kwaye siyenze kwintengiselwano, ukuze uqinisekiswe ukuba uyibeke kwisokethi yethu kwaye uqiniseke ukuba siyifumene kwelinye icala. Kwaye ngalo mzuzu, wonke umntu ngokukhawuleza uba ngokungeyomfuneko. Kwaye ukuba akunjalo, ngoko yeyiphi imibuzo esinayo? Ukuba awufuni kuqinisekisa ngokubhalela kwisokethi, kutheni ke siqinisekisa ukuhanjiswa? Senza owona mgudu ubalaseleyo. Sizama ngokwenene ukuhambisa kangangoko sinakho kwaye kangangoko sinakho, kodwa asiniki siqinisekiso se-100%. Ke ngoko, akukho mfuneko yokuba ubhale idatha yezemali apho. Kukho idatabase yentengiselwano yale nto.
Umbuzo wakho: Xa i-API ivelisa umyalezo othile kwilogi kwaye idlulisela ulawulo kwiinkonzo ezincinci, ngaba ukhe wadibana nengxaki yokuba imiyalezo evela kwiinkonzo ezincinci ezahlukeneyo ifika ngendlela engalunganga? Ngenxa yoko, kuvela ukudideka.
PhendulaIMPENDULO: Yinto eqhelekileyo ukuba ziza ngendlela eyahlukileyo. Kufuneka ulungele oku. Ngenxa yokuba naluphi na ukuhanjiswa kwenethiwekhi akuqinisekisi iodolo kuwe, okanye kufuneka uchithe izixhobo ezikhethekileyo kule nto. Ukuba sithatha ukugcinwa kweefayile, ngoko i-API nganye igcina iilogi kwifayile yayo. Endaweni yoko, i-rsyslog ibolisa kubalawuli apho. I-API nganye inezigodo zayo apho, apho unokuya khona kwaye ujonge, kwaye emva koko ungazithelekisa usebenzisa isitampu sexesha kule log. Ukuba baya kujonga kwiGreylog, ke apho baya kuhlelwa ngesitampu sexesha. Yonke into izakulunga apho.
Umbuzo wakho: Isitampu sexesha singohluka ngeemilliseconds.
Phendula: Isitampu sexesha senziwa yi-API ngokwayo. Oku, enyanisweni, yiyo yonke ingongoma. Sine-NTP. I-API yenza isitampu sexesha esele sikumyalezo ngokwawo. Ayidityaniswanga ngu rsyslog.
Umbuzo wakho: Ukusebenzisana phakathi kwamaziko edatha akucaci kakuhle. Ngaphakathi kwesakhelo seziko ledatha, kucacile ukuba iilogi zaqokelelwa kwaye zicutshungulwa njani. Injani intsebenziswano phakathi kwamaziko edatha? Okanye ngaba iziko ledatha ngalinye liphila ubomi balo?
Phendula: Phantse. Sinelizwe ngalinye elibekwe kwiziko ledatha enye. Okwangoku asinakusasaza, ukuze ilizwe elinye libekwe kumaziko ahlukeneyo edatha. Ngoko ke, akukho mfuneko yokudibanisa. Ngaphakathi kwiziko ngalinye kukho i-Log Relay. Lo ngumncedisi we Rsyslog. Enyanisweni, oomatshini ababini bolawulo. Zimiswe ngendlela efanayo. Kodwa okwangoku, itrafikhi idlula nje enye yazo. Ufaka yonke into ngokudibeneyo. Inomgca wedisk xa kunokwenzeka. Ucinezela iingodo kwaye uzithumele kwiziko ledatha eliphakathi (eSingapore), apho sele sele etyhefwe eGreylog. Kwaye iziko ngalinye ledatha linefayile yalo yokugcina ifayile. Kwimeko apho siphulukene nonxibelelwano, sinazo zonke iilog apho. Baya kuhlala apho. Ziya kugcinwa khona.
Umbuzo wakho: Ngaba ufumana iilogi ukusuka apho ngexesha leemeko ezingaqhelekanga?
Phendula: Ungaya apho (kwindawo yokugcina ifayile) kwaye ujonge.
Umbuzo wakho: Ujonga njani ukuba awulahleki zigodo?
Phendula: Ngokwenene siyaphulukana nabo, kwaye siyibeka esweni. Ukubeka iliso kuqale kwinyanga ephelileyo. Ithala leencwadi elisetyenziswa yiGo APIs lineemetriki. Uyakwazi ukubala ukuba mangaphi amatyeli awasilelayo ukubhala kwisokethi. Kukho okwangoku kukho i-heuristic ekhohlisayo. Kukho isithinteli apho. Izama ukubhala umyalezo ukusuka kuyo ukuya kwisokethi. Ukuba isithinteli siyaphuphuma, siqalisa ukusiwisa. Yaye ubala ukuba zininzi kangakanani na aziwisayo. Ukuba izixhobo zokubala ziqala ukuphuphuma apho, siya kukwazi ngayo. Ngoku nabo beza kwi-prometheus, kwaye unokuzibona iigrafu eGrafana. Unokuseta izivuseleli. Kodwa akukacaci ukuba mazithunyelwe kubani.
Umbuzo wakho: Kwi-elasticsearch, ugcina izingodo ngokuphindaphinda. Zingaphi iikopi onazo?
Phendula: Ikopi enye.
Umbuzo wakho: Ngaba ngumgca omnye nje?
Phendula: Le yinkosi kunye ne-replica. Idatha igcinwe ngokuphindwe kabini.
Umbuzo wakho: Ngaba uye walungisa ubungakanani be-rsyslog buffer ngandlela ithile?
Phendula: Sibhala iidatagrams kwi-socket ye-unix yesiko. Oku kubeka ngokukhawuleza umda we-128 kilobytes kuthi. Asikwazi ukubhala ngaphezulu kuyo. Sibhale oku kumgangatho. Ngubani ofuna ukungena kwisitoreji, babhala i-128 kilobytes. Amathala eencwadi, ngaphezu koko, anqunyulwe, abeke iflegi ukuba umyalezo unqunyulwe. Sinentsimi ekhethekileyo kumgangatho womyalezo ngokwawo, obonisa ukuba unqunyulwe ngexesha lokurekhoda okanye cha. Ke sinethuba lokulandelela lo mzuzu.
Umbuzo: Ngaba ubhala i-JSON eyaphukileyo?
Phendula: I-JSON eyaphukileyo iya kulahlwa nokuba ngexesha lokudluliselwa kuba ipakethi inkulu kakhulu. Okanye iGreylog iya kuwiswa, kuba ayizukwazi ukwahlula i-JSON. Kodwa kukho iinuances apha ekufuneka zilungiswe, kwaye ikakhulu zibotshelelwe kwi-rsyslog. Sele ndigcwalise imiba embalwa apho, nesafuna ukusetyenzwa.
Umbuzo: Kutheni Kafka? Ngaba ukhe wazama iRabbitMQ? I-Greylog ayidibanisi phantsi kwemithwalo enjalo?
Phendula: Ayisebenzi ngeGreylog. Kwaye iGreylog iyakhula. Yingxaki ngokwenene kuye. Uluhlobo lwento. Kwaye, enyanisweni, ayidingeki. Ndingathanda ukubhala ukusuka kwi-rsyslog ngqo ukuya kwi-elasticsearch kwaye emva koko ndibukele i-Kibana. Kodwa kufuneka siwulungise umcimbi noonogada. Oku kukwahluka okunokwenzeka kophuhliso lwethu xa silahla iGreylog kwaye sisebenzisa iKibana. I-Logstash ayiyi kuba nengqiqo. Kuba ndingenza okufanayo nge rsyslog. Kwaye inemodyuli yokubhalela elasticsearch. NgeGraylog sizama ukuphila ngandlela ithile. Siye sayilungisa kancinci. Kodwa kusekho indawo yokuphucula.
Malunga neKafka. Yenzeka ngolo hlobo ngokwembali. Ndifike sele ekhona, sele kubhalwe nezigodo. Siphakamise nje iqela lethu kwaye sahambisa iingodo kuyo. Siyamlawula, siyayazi indlela aziva ngayo. Ngokuphathelele uMvundlaMQ... sinengxaki noRabbitMQ. Kwaye iRabbitMQ iyasiphuhlisa. Sinayo kwimveliso, kwaye bekukho iingxaki ngayo. Ngoku, ngaphambi kokuthengisa, wayeza kuhlanjululwa, kwaye wayeza kuqala ukusebenza ngokuqhelekileyo. Kodwa ngaphambi koko, andizange ndilungele ukuyikhulula kwimveliso. Kukho enye ingongoma. I-Greylog inokufunda i-AMQP 0.9 version kunye ne-rsyslog ingabhala i-AMQP 1.0 version. Kwaye akukho sisombululo esinye esinokwenza zombini phakathi. Kukho nokuba enye okanye enye. Ke ngoku kuphela iKafka. Kodwa kukho kwakhona ama-nuances. Ngenxa yokuba i-omkafka yoguqulelo lwe-rsyslog esiyisebenzisayo inokuphulukana nesithinteli somyalezo siphela esichongwe kwi-rsyslog. Okoko sinyamezele.
Umbuzo: Usebenzisa iKafka kuba ubunayo? Ayisetyenziselwa nayiphi na enye injongo?
Phendula: I-Kafka, eyayisetyenziswa liqela leSayensi yeDatha. Le yiprojekthi eyahlukileyo ngokupheleleyo, malunga nayo, ngelishwa, andinakuyithetha nantoni na. Andazi. Wayeqhutywa liqela leNzululwazi yeDatha. Xa iinkuni ziqala, bagqiba ekubeni basebenzise, ukuze bangazibeki ezabo. Ngoku sihlaziye iGreylog, kwaye silahlekelwe ukuhambelana, kuba kukho inguqulo endala yeKafka. Kwafuneka sizenzele eyethu. Kwangaxeshanye, siye salahla ezi zihloko ezine kwi-API nganye. Senze umphezulu obanzi omnye kubo bonke abaphilayo, umphezulu obanzi obanzi kuzo zonke iqonga kwaye sidubula yonke into apho. I-Greylog ikhupha yonke le nto ngokuhambelanayo.
Umbuzo: Kutheni sifuna le shamanism eneesokethi? Ukhe wazama ukusebenzisa umqhubi welog yesyslog kwizikhongozeli.
Phendula: Ngela xesha sasibuza lo mbuzo, sasinobudlelwane obuqinileyo kunye nomthengisi. Yayiyi-docker 1.0 okanye i-0.9. IDocker ngokwayo yayingaqhelekanga. Okwesibini, ukuba nawe utyhale iinkuni kuyo ... ndinesikrokro esingangqinisiswanga sokuba idlula zonke iilog ngokwayo, nge-docker daemon. Ukuba sine-API enye iyaphambana, ke ezinye ii-APIs ziya kungena kwinto yokuba azikwazi ukuthumela i-stdout kunye ne-stderr. Andazi ukuba oku kuya kukhokelela phi. Ndinesikrokro kwinqanaba lokuziva ukuba akuyomfuneko ukusebenzisa umqhubi we-docker syslog kule ndawo. Isebe lethu lokuvavanya elisebenzayo lineqela lalo leGreylog elinezigodo. Basebenzisa abaqhubi belog ye-docker kwaye yonke into ibonakala ilungile apho. Kodwa ngokukhawuleza babhala i-GELF kuGreylog. Okwangoku xa sasiqala yonke le nto, sasiyidinga ukuba sisebenze nje. Mhlawumbi kamva, xa umntu esiza esithi ibisebenza ngokuqhelekileyo kangangeminyaka elikhulu, siya kuzama.
Umbuzo: Uhambisa phakathi kwamaziko edatha usebenzisa i-rsyslog. Kutheni ungayi eKafka?
Phendula: Senza oku, kwaye le yindlela ekuyiyo ngokwenene. Ngenxa yezizathu ezibini. Ukuba umjelo ufile ngokupheleleyo, ke zonke iingodo zethu, nakwifom ecinezelweyo, aziyi kukhwela kuyo. Kwaye i-kafka ivumela ukuba baphulukene ngokulula kwinkqubo. Ngale ndlela, sisusa ukuncamathela kwezi zigodo. Sisebenzisa nje iKafka kule meko ngokuthe ngqo. Ukuba sinetshaneli elungileyo kwaye sifuna ukuyikhulula, sisebenzisa i-rsyslog yabo. Kodwa eneneni, ungayiseta ukuze iwise into engakhange ifikelele kuyo. Okwangoku sisebenzisa ukuhanjiswa kwe-rsyslog ngqo kwindawo ethile, kwindawo ethile eKafka.
umthombo: www.habr.com