{"id":93294,"date":"2020-09-05T07:42:58","date_gmt":"2020-09-05T05:42:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi"},"modified":"2020-09-05T07:42:58","modified_gmt":"2020-09-05T05:42:58","slug":"logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","title":{"rendered":"Logimi n\u00eb Kubernetes: si t\u00eb mbledhim, ruajm\u00eb, analizojm\u00eb dhe x\u1eednajm\u00eb log\u00ebt","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Do t\u00eb shqyrtojm\u00eb bazat e logimit n\u00eb Docker dhe Kubernetes, dhe pastaj do t\u00eb shqyrtojm\u00eb dy instrumente q\u00eb mund t\u2019i p\u00ebrdorim me besim n\u00eb prodhim: Grafana Loki dhe staku EFK (Elasticsearch + Fluent Bit + Kibana).<\/p>\n

<\/p>\n

Materiali i artikullit \u00ebsht\u00eb nj\u00eb p\u00ebrmbledhje nga lectura e hapur e shkoll\u00ebs \u201cSlyrm\u201d<\/a><\/noindex>. N\u00ebse keni d\u00ebshir\u00eb dhe ve\u00e7an\u00ebrisht nevoj\u00eb prodhuese, mund t\u00eb kaloni nj\u00eb trajnim t\u00eb plot\u00eb \u2014 regjistrohuni p\u00ebr kursin mbi Monitorimin dhe logimin e infrastruktur\u00ebs n\u00eb Kubernetes<\/a><\/noindex>.<\/em><\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/a><\/noindex><\/p>\n

Logimi n\u00eb Docker<\/h2>\n

<\/p>\n

N\u00eb nivelin Kubernetes, aplikacionet kryhen n\u00eb kontejner\u00eb, por n\u00eb nivelin m\u00eb t\u00eb ul\u00ebt ato zakonisht funksionojn\u00eb n\u00eb Docker. Prandaj, duhet t\u00eb konfigurojm\u00eb logimin n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb mbledhim loget nga kontejner\u00ebt. Kontejner\u00ebt nisen nga Docker \u2014 k\u00ebshtu q\u00eb duhet t\u00eb kuptojm\u00eb se si funksionon logimi n\u00eb nivelin Docker.<\/p>\n

<\/p>\n

Shpresoj q\u00eb \u00e7do lexues ta dij\u00eb: loget e aplikacioneve duhet t\u00eb shkruhen n\u00eb stdout\/stderr, dhe jo brenda kontejnerit. Loget mblidhen nga Docker Daemon, dhe ai punon sakt\u00ebsisht me ato loge q\u00eb d\u00ebrgohen n\u00eb stdout\/stderr. P\u00ebr m\u00eb tep\u00ebr, regjistrimi i logeve brenda kontejnerit sjell probleme: kontejneri fryhet nga rritja e logut (sepse zakonisht nuk ka Logrotate n\u00eb kontejner), dhe Docker Daemon nuk \u00ebsht\u00eb n\u00eb dijeni p\u00ebr k\u00ebt\u00eb log.<\/p>\n

<\/p>\n

Docker ka disa shofer\u00eb logu ose plugin p\u00ebr mbledhjen e logeve nga kontejner\u00ebt. N\u00eb versionin falas Docker Community Edition (CE), ka m\u00eb pak shofer\u00eb logu se n\u00eb Docker Enterprise Edition (EE) komercial.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Nuk kam p\u00ebrdorur kurr\u00eb Docker EE n\u00eb praktik\u00eb: n\u00eb Southbridge p\u00ebrpiqemi t\u00eb respektojm\u00eb zgjidhjet Open Source, dhe p\u00ebr klient\u00ebt shumica e mund\u00ebsive shtes\u00eb t\u00eb Docker EE nuk jan\u00eb t\u00eb nevojshme. <\/p>\n

<\/p>\n

Shofer\u00ebt e logeve n\u00eb Docker CE:<\/p>\n

<\/p>\n

local<\/strong> \u2014 regjistrimi i logeve n\u00eb skedar\u00ebt e brendsh\u00ebm t\u00eb Docker Daemon;
\njson-file<\/strong> \u2014 krijimi i json-log n\u00eb dosjen e \u00e7do kontejneri;
\njournald<\/strong> \u2014 d\u00ebrgimi i logeve n\u00eb journald.<\/p>\n

<\/p>\n

Caktimet e logimit n\u00eb Docker ndodhen n\u00eb skedarin daemon.json. <\/p>\n

<\/p>\n

N\u00eb fush\u00ebn \u201clog-driver\u201d tregohet plugin, dhe n\u00eb fush\u00ebn \u201clog-opts\u201d jan\u00eb parametrat e tij. N\u00eb shembullin m\u00eb sip\u00ebr, \u00ebsht\u00eb tregohet plugin \u201cjson-file\u201d, kufizimi i madh\u00ebsis\u00eb s\u00eb logut \u00ebsht\u00eb \u201cmax-size\u201d: \u201c10m\u201d; kufizimi i numrit t\u00eb skedar\u00ebve (caktimet e rotacionit) \u00ebsht\u00eb \u201cmax-file\u201d: \u201c3\u201d; si dhe vlerat q\u00eb do t\u00eb lidhen me loget.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Disa caktime t\u00eb shoferit t\u00eb logut mund t\u00eb p\u00ebrcaktohen p\u00ebrmes utilitarit komandor. Kjo \u00ebsht\u00eb e p\u00ebrshtatshme, n\u00ebse nj\u00eb kontejner i ve\u00e7ant\u00eb duhet t\u00eb fillohet me nj\u00eb shofer logu tjet\u00ebr. <\/p>\n

<\/p>\n

Ja si duket skema e logimit n\u00eb Docker:<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Si p\u00ebr funksionon skema: nj\u00eb log-driver, si p.sh. json-file, krijon skedar\u00eb. Mblodh\u00ebsit e logve (Rsyslog, Fluentd, Logagent dhe t\u00eb tjer\u00eb) mbledhin k\u00ebta skedar\u00eb dhe i d\u00ebrgojn\u00eb p\u00ebr ruajtje n\u00eb Elastic, Sematext ose depove t\u00eb tjera.<\/p>\n

<\/p>\n

Ve\u00e7orit\u00eb e regjistrimit n\u00eb Kubernetes<\/h2>\n

<\/p>\n

Thjesht\u00ebsisht, skema e regjistrimit n\u00eb Kubernetes duket k\u00ebshtu: ka nj\u00eb pod, brenda t\u00eb cilit \u00ebsht\u00eb nj\u00eb kontejner, ky kontejner d\u00ebrgon loget n\u00eb stdout\/stderr. Pastaj Docker krijon nj\u00eb skedar dhe regjistron loget, t\u00eb cilat mund t\u00eb rrotullohen m\u00eb pas.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Le t\u00eb shqyrtojm\u00eb ve\u00e7orit\u00eb e regjistrimit n\u00eb Kubernetes.<\/p>\n

<\/p>\n

T\u00eb ruash loget mes p\u00ebrdit\u00ebsimeve<\/strong>. \u042d\u0442\u043e \u043e\u0431\u044f\u0437\u0430\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0443\u0441\u043b\u043e\u0432\u0438\u0435 \u0434\u043b\u044f \u043a\u043e\u0440\u0440\u0435\u043a\u0442\u043d\u043e\u0439 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0438 \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f. \u0415\u0441\u043b\u0438 \u043d\u0435 \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u0442\u044c \u043b\u043e\u0433\u0438 \u043c\u0435\u0436\u0434\u0443 \u0434\u0435\u043f\u043b\u043e\u044f\u043c\u0438, \u0442\u043e \u043f\u0440\u0438 \u0432\u044b\u0445\u043e\u0434\u0435 \u043d\u043e\u0432\u043e\u0439 \u0432\u0435\u0440\u0441\u0438\u0438 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u043b\u043e\u0433\u0438 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u0439 \u0431\u0443\u0434\u0443\u0442 \u0437\u0430\u0442\u0438\u0440\u0430\u0442\u044c\u0441\u044f, \u043f\u0435\u0440\u0435\u0437\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043a\u043e\u043d\u0442\u0435\u0439\u043d\u0435\u0440\u0430 \u0442\u0430\u043a\u0436\u0435 \u0431\u0443\u0434\u0435\u0442 \u0447\u0440\u0435\u0432\u0430\u0442\u0430 \u043f\u043e\u0442\u0435\u0440\u0435\u0439 \u043b\u043e\u0433\u043e\u0432. \u0423 Kubernetes \u0435\u0441\u0442\u044c \u043a\u043b\u044e\u0447\u0438\u043a —previous, \u043e\u043d \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u043f\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u0442\u044c \u043b\u043e\u0433\u0438 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0434\u043e \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0433\u043e \u0440\u0435\u0441\u0442\u0430\u0440\u0442\u0430 Pod, \u043d\u043e \u043d\u0435 \u0433\u043b\u0443\u0431\u0436\u0435.<\/p>\n

<\/p>\n

T\u00eb agregosh loget nga t\u00eb gjitha instancat<\/strong>. N\u00ebse mikrosh\u00ebrbimet strehohen n\u00eb mjete cloud, at\u00ebher\u00eb p\u00ebr mbik\u00ebqyrjen e sistemit p\u00ebrgjigjet ofruesi i cloud. N\u00ebse mikrosh\u00ebrbimet jan\u00eb n\u00eb pajisjet e veta, at\u00ebher\u00eb p\u00ebrve\u00e7 logeve nga kontenier\u00ebt duhet t\u00eb mblidhen edhe loget e sistemit.<\/p>\n

<\/p>\n

Deri m\u00eb par\u00eb nuk kishte mjete t\u00eb p\u00ebrshtatshme p\u00ebr mbledhjen e logeve si nga sistemi, ashtu edhe nga mikrosh\u00ebrbimet. Zakonisht nj\u00eb mjet mbante loget e sistemit (p.sh. Rsyslog), dhe nj\u00eb tjet\u00ebr - loget nga Docker (p.sh. journal-bit me konfigurimin e log-driver-it t\u00eb Docker p\u00ebr journald). U p\u00ebrpoq\u00ebm t\u00eb p\u00ebrdornim journal-bit - p\u00ebr t\u00eb mbledhur loget edhe nga kontenier\u00ebt (n\u00eb log-driver-in e Docker duhet t\u00eb spesifikosh se duhet t\u00eb shkruhen loget n\u00eb journald), dhe nga sistemi (n\u00eb CentOS 7 tashm\u00eb ka systemd dhe journald). Zgjidhja funksionuese, por jo ideale. N\u00ebse ka shum\u00eb loge, journal-bit fillon t\u00eb ngadal\u00ebsohet, mesazhet humbasin.<\/p>\n

<\/p>\n

Eksperimentet vazhduan - dhe u gjet nj\u00eb m\u00ebnyr\u00eb tjet\u00ebr. N\u00eb CentOS 7 loget kryesore sistemore (messages, audit, secure) p\u00ebrs\u00ebriten n\u00eb var-log si skedar\u00eb. Edhe n\u00eb Docker mund t\u00eb konfigurohet ruajtja e logeve n\u00eb skedar\u00eb json. P\u00ebrkat\u00ebsisht, k\u00ebta skedar\u00eb nga CentOS 7 dhe Docker mund t\u00eb mblidhen s\u00eb bashku.<\/p>\n

<\/p>\n

Me kalimin e koh\u00ebs zgjidhja ELK Stack u b\u00eb e njohur. Kjo \u00ebsht\u00eb nj\u00eb kombinim i disa veglave: Elasticsearch, Logstash dhe Kibana.<\/p>\n

<\/p>\n

Elasticsearch ruan loget nga kontenier\u00ebt, Logstash mbledh loget nga instancat, Kibana lejon p\u00ebrpunimin e logeve t\u00eb marra dhe ndihmon n\u00eb nd\u00ebrtimin e grafimeve p\u00ebr to. P\u00ebr nj\u00eb koh\u00eb, ELK Stack u p\u00ebrdor aktivisht, por, sipas mendimit tim, koha e tij po kalon. M\u00eb von\u00eb do t'ju tregoj pse.<\/p>\n

<\/p>\n

Shto metadatat<\/strong>. Podet, aplikacionet, kontejner\u00ebt mund t\u00eb ekzekutohen kudo. M\u00eb shum\u00eb se kaq, nj\u00eb aplikacion mund t\u00eb ket\u00eb disa instanca. Loget e regjistruara jan\u00eb n\u00eb nj\u00eb format t\u00eb vet\u00ebm, dhe ne duhet t\u00eb kuptojm\u00eb se cila sakt\u00ebsisht \u00ebsht\u00eb replicimi, cili Pod e shkruan at\u00eb, n\u00eb cilin namespace ndodhet. Kjo \u00ebsht\u00eb arsyeja pse logeve u nevojiten metadat\u00eb.<\/p>\n

<\/p>\n

Parsing loget<\/strong>. \u00ebsht\u00eb interesant, por shpenzimet p\u00ebr mb\u00ebshtetje t\u00eb sistemit t\u00eb regjistrimit dhe monitorimit mund t\u00eb tejkalojn\u00eb kostot e aplikacionit kryesor. Kur qarkullojn\u00eb dhjet\u00ebra dhe qindra mij\u00ebra loge n\u00eb sekond\u00eb, kjo duket e arsyeshme, por prap\u00ebseprap\u00eb duhet ta dini kufirin. Nj\u00eb nga m\u00ebnyrat p\u00ebr ta gjetur k\u00ebt\u00eb kufi \u00ebsht\u00eb parsing logeve. <\/p>\n

<\/p>\n

Si zakonisht, nuk \u00ebsht\u00eb e nevojshme t\u00eb mblidhni dhe ruani t\u00eb gjitha loget, duhet t\u00eb d\u00ebrgoni p\u00ebr ruajtje vet\u00ebm nj\u00eb pjes\u00eb \u2014 p\u00ebr shembull, loget me status 'warning' ose 'error'. N\u00ebse flasim p\u00ebr loget nginx ose ingress-controller, at\u00ebher\u00eb mund t\u00eb d\u00ebrgoni p\u00ebr ruajtje vet\u00ebm ata q\u00eb kan\u00eb nj\u00eb status t\u00eb ndrysh\u00ebm nga 200. Por kjo nuk \u00ebsht\u00eb nj\u00eb k\u00ebshill\u00eb universale: n\u00ebse po nd\u00ebrtoni ndonj\u00eb lloj analize t\u00eb logeve t\u00eb Nginx, at\u00ebher\u00eb \u00ebsht\u00eb e qart\u00eb se duhet t'i mbani ato. <\/p>\n

<\/p>\n

P\u00ebr t\u00eb filtruar loget pa menduar nuk rekomandohet, sepse t\u00eb dh\u00ebnat e filtruar mund t\u00eb mos mjaftojn\u00eb p\u00ebr nj\u00eb analiz\u00eb t\u00eb duhur. Nga ana tjet\u00ebr, ndoshta analitika duhet t\u00eb kryhet jo n\u00eb nivelin e regjistrimit, por n\u00eb nivelin e mbledhjes s\u00eb metrikave. At\u00ebher\u00eb nuk do t\u00eb duhet t\u00eb ruani qindra mij\u00ebra rreshta me kod 200. Nj\u00eb nga qasjet \u00ebsht\u00eb t\u00eb merrni informacion p\u00ebr trafikun dhe gabimet nga metrikat e ingress-controller\u00ebve. <\/p>\n

<\/p>\n

N\u00eb p\u00ebrgjith\u00ebsi, k\u00ebtu duhet t\u00eb mendoni mir\u00eb: \u00e7far\u00eb d\u00ebshironi t\u00eb ruani dhe p\u00ebr sa koh\u00eb, sepse n\u00eb t\u00eb kund\u00ebrt do t\u00eb ket\u00eb nj\u00eb situat\u00eb ku sistemi i regjistrimit do t'ju marr\u00eb m\u00eb shum\u00eb burime se projekti kryesor.<\/p>\n

<\/p>\n

P\u00ebr momentin nuk ka nj\u00eb zgjidhje standarde p\u00ebr regjistrimin<\/strong>. Ndryshe nga monitorimi, ku ka nj\u00eb zgjidhje m\u00eb t\u00eb p\u00ebrhapur si Prometheus, n\u00eb regjistrim nuk ka standard. <\/p>\n

<\/p>\n

N\u00eb kuad\u00ebr t\u00eb k\u00ebsaj leksione ne do t\u00eb shqyrtojm\u00eb dy instrumente: nj\u00eb t\u00eb njohur dhe nj\u00eb tjet\u00ebr n\u00eb rritje popullariteti. P\u00ebrve\u00e7 k\u00ebtyre ka edhe t\u00eb tjera, por n\u00eb k\u00ebt\u00eb artikull ne nuk do t'i trajtojm\u00eb.<\/p>\n

<\/p>\n

Duke marr\u00eb parasysh t\u00eb gjitha k\u00ebto karakteristika t\u00eb diskutuar m\u00eb lart, regjistrimi n\u00eb Kubernetes tani mund t\u00eb paraqitet me k\u00ebt\u00eb skem\u00eb:<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Mbete logu i kontejnerit, rotacioni, por shfaqet nj\u00eb agjent-mbledh\u00ebs, i cili grumbullon loget dhe i d\u00ebrgon p\u00ebr ruajtje (n\u00eb skem\u00eb \u2014 n\u00eb Logging Backend). Agjenti punon n\u00eb \u00e7do nod dhe, si zakonisht, \u00ebsht\u00eb nisur n\u00eb Kubernetes.<\/p>\n

<\/p>\n

Tani t\u00eb shqyrtojm\u00eb mjetet p\u00ebr regjistrimin e logeve.<\/p>\n

<\/p>\n

Grafana Loki<\/h2>\n

<\/p>\n

Grafana Loki<\/a><\/noindex> ka dal\u00eb s\u00eb fundmi, por tashm\u00eb ka fituar popullaritet. Avantazhet e tij: instalohet leht\u00eb, konsumon pak burime, nuk k\u00ebrkon instalimin e Elasticsearch, pasi mban t\u00eb dh\u00ebnat n\u00eb TSDB (baza e t\u00eb dh\u00ebnave me seri kohore). N\u00eb artikullin e kaluar kam shkruar se n\u00eb nj\u00eb baz\u00eb t\u00eb till\u00eb mban t\u00eb dh\u00ebnat Prometheus, dhe kjo \u00ebsht\u00eb nj\u00eb nga shum\u00eb ngjashm\u00ebrit\u00eb midis dy produkteve. Zhvilluesit madje deklarojn\u00eb se Loki \u00ebsht\u00eb \"Prometheus p\u00ebr bot\u00ebn e logeve\".<\/p>\n

<\/p>\n

Nj\u00eb shpjegim i vog\u00ebl n\u00eb lidhje me TSDB p\u00ebr ata q\u00eb nuk lexuan artikullin e kaluar<\/a><\/noindex>: TSDB p\u00ebrshtatet shum\u00eb mir\u00eb me detyr\u00ebn p\u00ebr t\u00eb ruajtur nj\u00eb sasi t\u00eb madhe t\u00eb dh\u00ebnash me seri kohore, por nuk \u00ebsht\u00eb e destinuar p\u00ebr ruajtje t\u00eb gjat\u00eb. N\u00ebse p\u00ebr ndonj\u00eb arsye keni nevoj\u00eb t\u00eb ruani loget m\u00eb shum\u00eb se dy jav\u00eb, \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb konfiguroni transmetimin e tyre n\u00eb nj\u00eb tjet\u00ebr DB.<\/p>\n

<\/p>\n

Nj\u00eb tjet\u00ebr avantazh i Loki \u00ebsht\u00eb se p\u00ebr vizualizimin e t\u00eb dh\u00ebnave p\u00ebrdoret Grafana. Shum\u00eb e p\u00ebrshtatshme: n\u00eb Grafana shikojm\u00eb t\u00eb dh\u00ebnat p\u00ebr monitorimin dhe po aty, duke e lidhur Loki, shohim loget. Nga loget mund t\u00eb nd\u00ebrtojm\u00eb grafik\u00eb.<\/p>\n

<\/p>\n

Arkitektura e Loki duket p\u00ebraf\u00ebrsisht k\u00ebshtu: <\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Me ndihm\u00ebn e DaemonSet, n\u00eb t\u00eb gjitha serverat e klasterit shp\u00ebrndahet agjenti \u2014 Promtail ose Fluent Bit. Agjenti mbledh loget. Loki i merr dhe i ruan n\u00eb TSDB. Logeve i shtohen menj\u00ebher\u00eb t\u00eb dh\u00ebna shtes\u00eb, q\u00eb \u00ebsht\u00eb e p\u00ebrshtatshme: mund t\u00eb filtrohet sipas Pods, namespaces, emrave t\u00eb kontejner\u00ebve dhe madje edhe etiketave.<\/p>\n

<\/p>\n

Udh\u00ebzimi p\u00ebr instalimin e Loki<\/a><\/noindex><\/p>\n

<\/p>\n

Loki punon n\u00eb nj\u00eb nd\u00ebrfaqe t\u00eb njohur Grafana. Loki ka edhe nj\u00eb gjuh\u00eb t\u00eb vet t\u00eb pyetjeve, e cila quhet LogQL \u2014 emri dhe sintaksa e saj i ngjajn\u00eb PromQL n\u00eb Prometheus. N\u00eb nd\u00ebrfaqen e Loki ka sugjerime p\u00ebr pyetjet, prandaj nuk \u00ebsht\u00eb e nevojshme t'i dini ato p\u00ebrmendsh.<\/p>\n

<\/p>\n

Dokumentacioni p\u00ebr gjuh\u00ebn LogQL<\/a><\/noindex><\/p>\n

<\/p>\n

\"Logimi
\nLoki n\u00eb nd\u00ebrfaqen Grafana<\/em><\/p>\n

<\/p>\n

Duke p\u00ebrdorur filtre, n\u00eb Loki mund t\u00eb gjeni kodet (\"400\", \"404\" dhe \u00e7do tjet\u00ebr); shikoni loget nga i gjith\u00eb nodi; filtrojini t\u00eb gjitha loget q\u00eb kan\u00eb fjal\u00ebn \"error\". N\u00ebse klikoni mbi logun, do t\u00eb shfaqet nj\u00eb kart\u00eb me t\u00eb gjitha informacionet p\u00ebr ngjarjen.<\/p>\n

<\/p>\n

N\u00eb Loki ka mjaft mjete q\u00eb lejojn\u00eb nxjerrjen e logeve t\u00eb nevojshme, megjithat\u00eb sinqerisht, teknikisht mund t\u00eb ishin edhe m\u00eb shum\u00eb. Tani Loki \u00ebsht\u00eb duke u zhvilluar aktivisht dhe po fiton popullaritet. <\/p>\n

<\/p>\n

Elastic + Fluent Bit + Kibana (EFK Stack)<\/h2>\n

<\/p>\n

Staku EFK \u00ebsht\u00eb nj\u00eb mjet m\u00eb klasik dhe gjithashtu po aq popullor p\u00ebr regjistrimin e logeve. <\/p>\n

<\/p>\n

N\u00eb fillim t\u00eb artikullit u p\u00ebrmend ELK (Elasticsearch + Logstash + Kibana), por ky stak \u00ebsht\u00eb vjetruar p\u00ebr shkak t\u00eb Logstash-it t\u00eb pandjesh\u00ebm dhe t\u00eb p\u00ebrdorur shum\u00eb burime. N\u00eb vend t\u00eb tij \u00ebsht\u00eb p\u00ebrdorur nj\u00eb agent m\u00eb i leht\u00eb dhe m\u00eb efikas, Fluentd, dhe pas nj\u00eb kohe i \u00ebsht\u00eb bashkuar Fluent Bit<\/a><\/noindex> \u2014 nj\u00eb agjent mbledh\u00ebs edhe m\u00eb i leht\u00eb dhe edhe m\u00eb efikas. <\/p>\n

<\/p>\n

N\u00ebse i besojm\u00eb zhvilluesve, at\u00ebher\u00eb Fluent Bit \u00ebsht\u00eb m\u00eb shum\u00eb se 100 her\u00eb m\u00eb i shpejt\u00eb se Fluentd: \u00abaty ku Fluentd konsumon 20 MB RAM, Fluent Bit do t\u00eb konsumoj\u00eb 150 KB\u00bb \u2014 nj\u00eb citim direkt nga dokumentacioni. Duke e marr\u00eb parasysh k\u00ebt\u00eb, Fluent Bit \u00ebsht\u00eb p\u00ebrdorur m\u00eb shpesh.<\/p>\n

<\/p>\n

Fluent Bit ka m\u00eb pak mund\u00ebsi se Fluentd, por plot\u00ebson nevojat kryesore, prandaj ne kryesisht p\u00ebrdorim Fluent Bit.<\/p>\n

<\/p>\n

Skema e funksionimit t\u00eb stakut EFK: agjenti mbledh log-et nga t\u00eb gjitha pod-et (zakonisht, ky \u00ebsht\u00eb nj\u00eb DaemonSet i aktivizuar n\u00eb t\u00eb gjitha server\u00ebt e klustrit) dhe i d\u00ebrgon n\u00eb depo (Elasticsearch, PostgreSQL ose Kafka). Kibana lidhet me depotin dhe nxjerr t\u00eb gjitha informacionet e nevojshme nga aty.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Kibana<\/a><\/noindex> paraqet informacionin n\u00eb nj\u00eb nd\u00ebrfaqe t\u00eb rehatshme n\u00eb web. Ka grafik\u00eb, filtra dhe shum\u00eb gj\u00ebra t\u00eb tjera.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Nga log-et mund t\u00eb krijohen dashboard-e t\u00eb plota.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Mund\u00ebsit\u00eb e Fluent Bit<\/h2>\n

<\/p>\n

Duke qen\u00eb se p\u00ebr Fluent Bit, zakonisht, \u00ebsht\u00eb d\u00ebgjuar m\u00eb pak se p\u00ebr Logstash, le t\u00eb e shqyrtojm\u00eb at\u00eb pak m\u00eb n\u00eb detaje. Fluent Bit mund t\u00eb ndahet logjikisht n\u00eb 6 module, disa nga t\u00eb cilat mund t\u00eb ken\u00eb plugin-e q\u00eb zgjasin mund\u00ebsit\u00eb e Fluent Bit.<\/p>\n

<\/p>\n

\"Logimi<\/p>\n

<\/p>\n

Moduli Input<\/strong> mbledh log-et nga skedar\u00ebt, sh\u00ebrbimet systemd dhe madje edhe nga tcp-socket (duhet vet\u00ebm t\u00eb tregoni endpoint-in, dhe Fluent Bit do t\u00eb filloj\u00eb t\u00eb shkoj\u00eb atje). K\u00ebto mund\u00ebsi jan\u00eb t\u00eb mjaftueshme p\u00ebr t\u00eb mbledhur log-et si nga sistemi ashtu edhe nga konteiner\u00ebt.<\/p>\n

<\/p>\n

N\u00eb prodhim, ne zakonisht p\u00ebrdorim plugin-et tail<\/a><\/noindex> (mund ta drejtoni mbi nj\u00eb dosje me log-e) dhe systemd<\/a><\/noindex> (mund t\u2019i thoni se nga cilat sh\u00ebrbime t\u00eb mbledh log-et).<\/p>\n

<\/p>\n

Moduli Parser<\/strong> e b\u00ebn log-un n\u00eb nj\u00eb form\u00eb t\u00eb nj\u00ebjt\u00eb. N\u00eb m\u00ebnyr\u00eb q\u00eb log-et Nginx, sipas t\u00eb dh\u00ebnave t\u00eb paracaktuara, p\u00ebrb\u00ebjn\u00eb nj\u00eb varg. Me ndihm\u00ebn e plugin-it, ky varg mund t\u00eb konvertohet n\u00eb JSON: mund t\u00eb p\u00ebrcaktoni fushat dhe vlerat e tyre. Duke punuar me JSON \u00ebsht\u00eb shum\u00eb m\u00eb e leht\u00eb se me log-un me varg, sepse ka mund\u00ebsi m\u00eb fleksibile p\u00ebr renditjen.<\/p>\n

<\/p>\n

Moduli Filter<\/strong>. N\u00eb k\u00ebt\u00eb nivel, filtrohen log-et e panevojshme. P\u00ebr shembull, p\u00ebr t\u00eb ruajtur, d\u00ebrgohen vet\u00ebm log-et q\u00eb kan\u00eb vler\u00ebn \u201cwarning\u201d ose me etiketa t\u00eb caktuara. Log-et e zgjedhura kalojn\u00eb n\u00eb buffer.<\/p>\n

<\/p>\n

Moduli Buffer<\/strong>. Fluent Bit ka dy tipo buffer: buferri i memories dhe buferri n\u00eb disk. Bufferi \u00ebsht\u00eb nj\u00eb ruajt\u00ebs temporar i logeve, e nevojshme n\u00eb rast gabimesh ose defekteve. T\u00eb gjith\u00eb duan t\u00eb kursejn\u00eb n\u00eb RAM, prandaj zakonisht zgjidhet bufferi n\u00eb disk. Mir\u00ebpo, duhet t\u00eb merret parasysh se para se t\u00eb transferohen n\u00eb disk, loget s\u00ebrish shkarkohen n\u00eb memories.<\/p>\n

<\/p>\n

Moduli Routing\/Output<\/strong> p\u00ebrmban rregullat dhe adresat p\u00ebr d\u00ebrgimin e logeve. Si\u00e7 u tha m\u00eb par\u00eb, loget mund t\u00eb d\u00ebrgohen n\u00eb Elasticsearch, PostgreSQL ose, p\u00ebr shembull, Kafka. <\/p>\n

<\/p>\n

E interesant \u00ebsht\u00eb se nga Fluent Bit loget mund t\u00eb d\u00ebrgohen n\u00eb Fluentd. Duke qen\u00eb se i pari \u00ebsht\u00eb m\u00eb i leht\u00eb dhe m\u00eb pak funksional, p\u00ebrmes tij mund t\u00eb mblidhen loget dhe t\u00eb d\u00ebrgohen n\u00eb Fluentd, dhe atje, me ndihm\u00ebn e plugins shtes\u00eb, t\u00eb p\u00ebrpunohen dhe d\u00ebrgohen n\u00eb ruajtje.<\/p>\n

<\/p>\n

N\u00ebse planifikoni t\u00eb p\u00ebrdorni Elasticsearch\u2026<\/p>\n

N\u00eb fund, dy k\u00ebshilla p\u00ebr ata q\u00eb planifikojn\u00eb t\u00eb p\u00ebrdorin Elasticsearch n\u00eb prodhim si ruajtje logesh. <\/p>\n

    \n
  1. Konfiguroni njoftimet me an\u00eb t\u00eb ElastAlert<\/a><\/noindex>. Ky program nxjerr nga rrjedha e p\u00ebrgjithshme e logeve mesazhe t\u00eb r\u00ebnd\u00ebsishme dhe b\u00ebn alarme p\u00ebr to n\u00eb email ose n\u00eb nj\u00eb kanal tjet\u00ebr. Megjithat\u00eb, jo shum\u00eb koh\u00eb m\u00eb par\u00eb doli nj\u00eb lajm i trishtuar se projekti mund t\u00eb ndaloj\u00eb s\u00eb ekzistuari s\u00eb shpejti<\/a><\/noindex><\/em>.<\/li>\n
  2. Rrotulloni loget me ndihm\u00ebn e aplikacionit Curator<\/a><\/noindex> apo duke iu drejtuar API-t t\u00eb Elasticsearch. Vet\u00eb Elastic, n\u00eb fakt, tani po b\u00ebn hapa t\u00eb r\u00ebnd\u00ebsish\u00ebm n\u00eb menaxhimin e jet\u00ebs s\u00eb indekseve pa p\u00ebrdorimin e mjeteve t\u00eb jashtme. N\u00eb p\u00ebrgjith\u00ebsi, nuk ka asnj\u00eb kuptim t\u00eb ruhet loget p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb: \u00ebsht\u00eb e v\u00ebshtir\u00eb q\u00eb ndonj\u00eb log t\u00eb nevojitet pas dy jav\u00ebsh - n\u00ebse realmente \u00ebsht\u00eb kritik, brenda dy jav\u00ebsh do t\u00eb jet\u00eb sigurisht p\u00ebrpunuar. N\u00eb rastet m\u00eb t\u00eb skajshme, loget e vjetra mund t\u00eb arkivohen dhe d\u00ebrgohen diku p\u00ebr ruajtje afatgjat\u00eb. Kam d\u00ebgjuar p\u00ebr loge t\u00eb ve\u00e7anta, q\u00eb sipas ligjit duhet t\u00eb ruhen deri n\u00eb 5 vjet. Personalish, nuk kam hasur n\u00eb k\u00ebt\u00eb, por nuk do ta barazoja k\u00ebt\u00eb informacion me loget e zakonshme, dhe ndoshta do t'i ruaja ato ve\u00e7mas.<\/li>\n<\/ol>\n

    \n<\/p><\/blockquote>\n

    \u041f\u0440\u043e\u0434\u043e\u043b\u0436\u0435\u043d\u0438\u0435 \u0441\u043b\u0435\u0434\u0443\u0435\u0442…<\/p>\n

    <\/p>\n

    Autor: Marsel Ibraev, administrator i certifikuar Kubernetes, inxhinier praktik n\u00eb kompanin\u00eb Southbridge<\/a><\/noindex>, fol\u00ebs dhe zhvillues kursesh Slurm<\/a><\/noindex>.<\/em><\/p>\n

    Burimi: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

    \u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana). \u041c\u0430\u0442\u0435\u0440\u0438\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u0432\u044b\u0436\u0438\u043c\u043a\u0430 \u0438\u0437 \u043e\u0442\u043a\u0440\u044b\u0442\u043e\u0439 \u043b\u0435\u043a\u0446\u0438\u0438 \u0448\u043a\u043e\u043b\u044b \u00ab\u0421\u043b\u0451\u0440\u043c\u00bb. \u0415\u0441\u043b\u0438 \u0435\u0441\u0442\u044c \u0436\u0435\u043b\u0430\u043d\u0438\u0435 \u0438 \u0442\u0435\u043c \u0431\u043e\u043b\u0435\u0435 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u0430\u044f \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0439\u0442\u0438 \u043f\u043e\u043b\u043d\u043e\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u2014 \u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u0439\u0442\u0435\u0441\u044c \u043d\u0430 \u043a\u0443\u0440\u0441 \u043f\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":93295,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-93294","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47 Logimi n\u00eb Kubernetes: si t\u00eb mbledhim, ruajm\u00eb, analizojm\u00eb dhe p\u00ebrpunojm\u00eb loget | ProHoster","description":"Do t\u00eb shqyrtojm\u00eb bazat e logimit n\u00eb Docker dhe Kubernetes, dhe m\u00eb pas do t\u00eb shqyrtojm\u00eb dy mjete q\u00eb mund t\u00eb p\u00ebrdoren qet\u00ebsisht n\u00eb prodhim: Grafana Loki dhe becin EFK (Elasticsearch + Fluent Bit + Kibana).","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0432 Kubernetes: \u043a\u0430\u043a \u0441\u043e\u0431\u0438\u0440\u0430\u0442\u044c, \u0445\u0440\u0430\u043d\u0438\u0442\u044c, \u043f\u0430\u0440\u0441\u0438\u0442\u044c \u0438 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043b\u043e\u0433\u0438 | ProHoster","og:description":"\u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana)..","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-05T05:42:58+00:00","article:modified_time":"2020-09-05T05:42:58+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"93294","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:50:29","updated":"2022-09-27 18:43:48","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/93294","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=93294"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/93294\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/93295"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=93294"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=93294"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=93294"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}