{"id":93294,"date":"2020-09-05T07:42:58","date_gmt":"2020-09-05T05:42:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi"},"modified":"2020-09-05T07:42:58","modified_gmt":"2020-09-05T05:42:58","slug":"logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","title":{"rendered":"Regjistrimi n\u00eb Kubernetes: si t\u00eb mbledhim, ruajm\u00eb, analizojm\u00eb dhe p\u00ebrpunojm\u00eb logje","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Le t\u00eb shqyrtojm\u00eb bazat e regjistrimit n\u00eb Docker dhe Kubernetes, dhe pastaj t\u00eb shqyrtojm\u00eb dy mjete q\u00eb mund t'i p\u00ebrdorim me besim n\u00eb prodhim: Grafana Loki dhe staku EFK (Elasticsearch + Fluent Bit + Kibana).<\/p>\n

<\/p>\n

Materiali i k\u00ebtij artikulli \u00ebsht\u00eb nj\u00eb p\u00ebrmbledhje nga lecturesh t\u00eb hapura t\u00eb shkoll\u00ebs 'Slyorm'<\/a><\/noindex>. N\u00ebse ka d\u00ebshir\u00eb, dhe p\u00ebr m\u00eb tep\u00ebr, nj\u00eb nevoj\u00eb prodhuese, mund t\u00eb ndiqni nj\u00eb trajnim t\u00eb plot\u00eb \u2014 regjistrohuni n\u00eb kursin p\u00ebr Monitorimin dhe regjistrimin e infrastruktur\u00ebs n\u00eb Kubernetes<\/a><\/noindex>.<\/em><\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/a><\/noindex><\/p>\n

Regjistrimi n\u00eb Docker<\/h2>\n

<\/p>\n

N\u00eb nivelin e Kubernetes, aplikacionet jan\u00eb t\u00eb nisura n\u00eb pod-et, por n\u00eb nivelin m\u00eb t\u00eb ul\u00ebt, ato funksionojn\u00eb zakonisht n\u00eb Docker. Prandaj, duhet t\u00eb konfigurojm\u00eb regjistrimin n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb mbledhim log-e nga kontejner\u00ebt. Kontejner\u00ebt nisen nga Docker \u2014 k\u00ebshtu q\u00eb duhet t\u00eb kuptojm\u00eb si \u00ebsht\u00eb e organizuar regjistrimi n\u00eb nivelin e Docker.<\/p>\n

<\/p>\n

Shpresoj se \u00e7do lexues e di: log-et e aplikacionit duhet t\u00eb shkruhen n\u00eb stdout\/stderr, dhe jo brenda kontejnerit. Log-et agregohen nga Docker Daemon, dhe ai punon me ato log-e q\u00eb d\u00ebrgohen n\u00eb stdout\/stderr. Po ashtu, regjistrimi i log-eve brenda kontejnerit sjell probleme: kontejneri fryhet nga rritja e log-ut (pasi ndoshta nuk ka Logrotate brenda tij), dhe Docker Daemon nuk \u00ebsht\u00eb i vet\u00ebdijsh\u00ebm p\u00ebr k\u00ebt\u00eb log.<\/p>\n

<\/p>\n

Docker ka disa shofer\u00eb log-u ose plugins p\u00ebr mbledhjen e log-eve t\u00eb kontejner\u00ebve. N\u00eb versionin falas t\u00eb Docker Community Edition (CE) ka m\u00eb pak shofer\u00eb log-u se n\u00eb Docker Enterprise Edition (EE) komercial.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Nuk e kam p\u00ebrdorur kurr\u00eb Docker EE n\u00eb praktik\u00eb: n\u00eb Southbridge p\u00ebrpiqemi t\u00eb q\u00ebndrojm\u00eb te zgjidhjet Open Source, dhe shumica e mund\u00ebsive shtes\u00eb t\u00eb Docker EE nuk jan\u00eb t\u00eb nevojshme p\u00ebr klient\u00ebt. <\/p>\n

<\/p>\n

Shofer\u00ebt e log-ut n\u00eb Docker CE:<\/p>\n

<\/p>\n

local<\/strong> \u2014 regjistrimi i log-eve n\u00eb skedar\u00ebt e brendsh\u00ebm t\u00eb Docker Daemon;
\njson-file<\/strong> \u2014 krijimi i log-eve json n\u00eb dosjen e \u00e7do konteneri;
\njournald<\/strong> \u2014 d\u00ebrgimi i log-eve n\u00eb journald.<\/p>\n

<\/p>\n

Caktimet e logimit n\u00eb Docker ndodhen n\u00eb skedarin daemon.json. <\/p>\n

<\/p>\n

N\u00eb fush\u00ebn \u201clog-driver\u201d tregohet plugin-i, dhe n\u00eb fush\u00ebn \u201clog-opts\u201d kjo jan\u00eb caktimet e tij. N\u00eb shembullin e m\u00ebsip\u00ebrm tregohet plugin-i \u201cjson-file\u201d, kufizimi n\u00eb madh\u00ebsin\u00eb e log-ut \u2014 \u201cmax-size\u201d: \u201c10m\u201d; kufizimi n\u00eb numrin e skedar\u00ebve (caktimet p\u00ebr rotacion) \u2014 \u201cmax-file\u201d: \u201c3\u201d; si dhe vlerat q\u00eb do t\u00eb lidhen me log-et.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Disa caktime t\u00eb shoferit t\u00eb log-ut mund t\u00eb caktohen p\u00ebrmes utilitarit komandor. Kjo \u00ebsht\u00eb e p\u00ebrshtatshme n\u00ebse nj\u00eb kontener i ve\u00e7ant\u00eb duhet t\u00eb startohet me nj\u00eb shofer log-u tjet\u00ebr. <\/p>\n

<\/p>\n

K\u00ebshtu duket skema e logimit n\u00eb Docker:<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Si sihemi funksionon: drejtp\u00ebrdrejt\u00ebsi log, p\u00ebr shembull json-file, krijon skedar\u00eb. Mbledh\u00ebsit e logut (Rsyslog, Fluentd, Logagent dhe t\u00eb tjer\u00eb) i mbledhin k\u00ebta skedar\u00eb dhe i d\u00ebrgojn\u00eb p\u00ebr ruajtje n\u00eb Elastic, Sematext ose depo t\u00eb tjera.<\/p>\n

<\/p>\n

Ve\u00e7orit\u00eb e regjistrimit t\u00eb logeve n\u00eb Kubernetes<\/h2>\n

<\/p>\n

N\u00eb m\u00ebnyr\u00eb t\u00eb thjesht\u00eb, skema e regjistrimit t\u00eb logeve n\u00eb Kubernetes duket k\u00ebshtu: ka nj\u00eb pod, n\u00eb t\u00eb cilin ekzekutohet nj\u00eb kontejner, kontejneri d\u00ebrgon loget n\u00eb stdout\/stderr. M\u00eb pas, Docker krijon nj\u00eb skedar dhe shkruan loget, t\u00eb cilat mund t\u00eb rrotullohen.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Le t\u00eb shqyrtojm\u00eb ve\u00e7orit\u00eb e regjistrimit t\u00eb logeve n\u00eb Kubernetes.<\/p>\n

<\/p>\n

T\u00eb ruani loget midis publikimeve<\/strong>. Kjo \u00ebsht\u00eb nj\u00eb k\u00ebrkes\u00eb e domosdoshme p\u00ebr konfigurimin e sakt\u00eb t\u00eb regjistrimeve. N\u00ebse nuk ruani regjistrimet nd\u00ebrmjet t\u00eb deplojeve, ato nga versioni i m\u00ebparsh\u00ebm t\u00eb aplikacionit do t\u00eb fshihen kur t\u00eb dal\u00eb nj\u00eb version i ri, dhe rinegocimi i kontejnerit gjithashtu do t\u00eb \u00e7oj\u00eb n\u00eb humbje t\u00eb regjistrimeve. Kubernetes ka nj\u00eb opsion \u2014previous, q\u00eb lejon shikimin e regjistrimeve t\u00eb aplikacionit p\u00ebrpara rinovimit t\u00eb fundit t\u00eb Pod-it, por jo m\u00eb thell\u00eb.<\/p>\n

<\/p>\n

T\u00eb agregoni loget nga t\u00eb gjitha instancat<\/strong>. N\u00ebse mikrosh\u00ebrbimet hostohen n\u00eb re, at\u00ebher\u00eb kontrolli i sistemit i n\u00ebnshtrohet ofruesit t\u00eb sh\u00ebrbimit t\u00eb cloud. N\u00ebse mikrosh\u00ebrbimet jan\u00eb n\u00eb pajisje t\u00eb veta, at\u00ebher\u00eb p\u00ebrve\u00e7 logeve nga kontejner\u00ebt, duhet t\u00eb mblidhen edhe loget e sistemit.<\/p>\n

<\/p>\n

M\u00eb par\u00eb nuk kishte mjete t\u00eb p\u00ebrshtatshme p\u00ebr mbledhjen e log-eve si nga sistemi ashtu edhe nga mikrosh\u00ebrbimet. Zakonisht nj\u00eb mjet mbledh log-et sistemore (p.sh., Rsyslog), kurse nj\u00eb tjet\u00ebr log-et nga Docker (p.sh., journal-bit me konfigurimin e log-drajverit t\u00eb Dockerit mbi journald). Provova t\u00eb p\u00ebrdor journal-bit p\u00ebr t\u00eb mbledhur log-et edhe nga kontejner\u00ebt (n\u00eb log-drajverin e Dockerit t\u00eb tregohet q\u00eb duhet t\u00eb shkruhen log-e n\u00eb journald), dhe nga sistemi (n\u00eb CentOS 7 tashm\u00eb ka systemd dhe journald). Zgjidhja funksionon, por nuk \u00ebsht\u00eb ideale. N\u00ebse ka shum\u00eb log-e, journal-bit fillon t\u00eb vonoj\u00eb, mesazhet humbasin.<\/p>\n

<\/p>\n

Eksperimentet vazhduan \u2014 dhe u gjet nj\u00eb m\u00ebnyr\u00eb tjet\u00ebr. N\u00eb CentOS 7 log-et kryesore sistemore (messages, audit, secure) duplohen n\u00eb var-log si skedar\u00eb. Edhe n\u00eb Docker mund t\u00eb konfigurohet ruajtja e log-eve n\u00eb skedar\u00eb json. P\u00ebrkat\u00ebsisht, k\u00ebta skedar\u00eb nga CentOS 7 dhe Docker mund t\u00eb mblidhen s\u00eb bashku.<\/p>\n

<\/p>\n

Me kalimin e koh\u00ebs, zgjidhja ELK Stack u b\u00eb e njohur. Kjo \u00ebsht\u00eb nj\u00eb kombinim i disa mjeteve: Elasticsearch, Logstash dhe Kibana.<\/p>\n

<\/p>\n

Elasticsearch ruan log-et nga kontejner\u00ebt, Logstash mbledh log-et nga instancat, Kibana lejon p\u00ebrpunimin e log-eve t\u00eb marra, nd\u00ebrtimin e grafik\u00ebve mbi to. P\u00ebr disa koh\u00eb, ELK Stack u p\u00ebrdor aktivisht, por, sipas mendimit tim, koha e tij po kalon. M\u00eb von\u00eb do t\u00eb tregoj pse.<\/p>\n

<\/p>\n

Shto metadata<\/strong>. Pods, aplikacionet, kontejner\u00ebt mund t\u00eb ekzekutohen kudo. P\u00ebr m\u00eb tep\u00ebr, nj\u00eb aplikacion mund t\u00eb ket\u00eb disa instanca. Log-et regjistrohen n\u00eb nj\u00eb format t\u00eb caktuar, dhe ne duhet t\u00eb kuptojm\u00eb se cila \u00ebsht\u00eb sakt\u00ebsisht kjo kopje, cili Pod po e shkruan, n\u00eb cilin namespace ndodhet. Pik\u00ebrisht p\u00ebr k\u00ebt\u00eb arsye, log-et duhet t\u00eb ken\u00eb metadata t\u00eb shtuar.<\/p>\n

<\/p>\n

T\u00eb analizosh log-et<\/strong>. \u00cbsht\u00eb e \u00e7uditshme, por shpenzimet p\u00ebr mb\u00ebshtetje t\u00eb sistemit t\u00eb regjistrimit dhe monitorimit mund t\u00eb tejkalojn\u00eb kostot p\u00ebr aplikacionin kryesor. Kur keni dhjet\u00ebra dhe qindra mij\u00ebra log-e n\u00eb sekond\u00eb, kjo duket e natyrshme, por megjithat\u00eb duhet t\u00eb dini kufirin. Nj\u00eb nga m\u00ebnyrat p\u00ebr ta gjetur k\u00ebt\u00eb kufi \u00ebsht\u00eb analiza e log-eve. <\/p>\n

<\/p>\n

Si zakonisht, nuk \u00ebsht\u00eb e nevojshme t\u00eb mbani dhe ruani t\u00eb gjith\u00eb log-et, duhet t\u00eb d\u00ebrgoni p\u00ebr ruajtje vet\u00ebm nj\u00eb pjes\u00eb \u2014 p\u00ebr shembull, log-et me statusin 'warning' ose 'error'. N\u00ebse flasim p\u00ebr log-et e nginx ose kontrollor\u00ebve t\u00eb ingress-it, at\u00ebher\u00eb mund t\u00eb d\u00ebrgoni p\u00ebr ruajtje vet\u00ebm ato, statusi i t\u00eb cilave \u00ebsht\u00eb ndryshe nga 200. Por kjo nuk \u00ebsht\u00eb nj\u00eb k\u00ebshill\u00eb universale: n\u00ebse jeni duke nd\u00ebrtuar ndonj\u00eb m\u00ebnyr\u00eb analitik\u00eb mbi log-et e Nginx, at\u00ebher\u00eb \u00ebsht\u00eb e qart\u00eb se ato duhet t\u00eb mblidhen. <\/p>\n

<\/p>\n

Grumbullisht nuk rekomandohet filtrimi i logeve, sepse t\u00eb dh\u00ebnat e filtruar mund t\u00eb mos jen\u00eb t\u00eb mjaftueshme p\u00ebr nj\u00eb analiz\u00eb normale. Nga ana tjet\u00ebr, ndoshta analitika duhet t\u00eb b\u00ebhet jo n\u00eb nivelin e regjistrimit, por n\u00eb nivelin e mbledhjes s\u00eb metrikeve. K\u00ebshtu nuk do t\u00eb duhet t\u00eb ruajm\u00eb qindram mij\u00ebra rreshta me kod 200. Nj\u00eb nga qasjet \u00ebsht\u00eb t\u00eb marrim informacion rreth trafik\u00ebve dhe gabimeve nga metrikat e kontrollor\u00ebve t\u00eb ingress. <\/p>\n

<\/p>\n

N\u00eb p\u00ebrgjith\u00ebsi, k\u00ebtu duhet t\u00eb mendoni mir\u00eb: \u00e7far\u00eb d\u00ebshironi t\u00eb ruani dhe p\u00ebr sa koh\u00eb, sepse ndryshe do t\u00eb krijohet nj\u00eb situat\u00eb kur sistemi i regjistrimit do t\u00eb marr\u00eb m\u00eb shum\u00eb burime se projekti kryesor.<\/p>\n

<\/p>\n

Aktualisht nuk ka nj\u00eb zgjidhje standarde p\u00ebr regjistrimin<\/strong>. Ndryshe nga monitorimi, ku ka nj\u00eb zgjidhje t\u00eb vetme m\u00eb t\u00eb njohur si Prometheus, n\u00eb regjistrim nuk ka nj\u00eb standard. <\/p>\n

<\/p>\n

N\u00eb kuad\u00ebr t\u00eb k\u00ebsaj leksioni ne do t\u00eb shqyrtojm\u00eb dy mjete: nj\u00eb t\u00eb njohur dhe nj\u00eb tjet\u00ebr q\u00eb po merr popullaritet. P\u00ebrve\u00e7 k\u00ebtyre, ka edhe t\u00eb tjera, por n\u00eb k\u00ebt\u00eb artikull nuk do t\u2019i prekim.<\/p>\n

<\/p>\n

Duke marr\u00eb parasysh gjith\u00eb karakteristikat e p\u00ebrmendura m\u00eb sip\u00ebr, regjistrimi n\u00eb Kubernetes tani mund t\u00eb paraqitet n\u00eb k\u00ebt\u00eb skem\u00eb:<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Mbajtja e log-ut t\u00eb kontejnerit, rrotullimi vazhdon, por shfaqet nj\u00eb agjent-mbledh\u00ebs q\u00eb merr loget dhe i d\u00ebrgon p\u00ebr ruajtje (n\u00eb skem\u00eb \u2014 n\u00eb Logging Backend). Agjenti punon n\u00eb \u00e7do nod dhe, zakonisht, aktivizohet n\u00eb Kubernetes.<\/p>\n

<\/p>\n

Tani le t\u00eb shqyrtojm\u00eb mjetet p\u00ebr regjistrimin e logeve.<\/p>\n

<\/p>\n

Grafana Loki<\/h2>\n

<\/p>\n

Grafana Loki<\/a><\/noindex> u shfaq koh\u00ebt e fundit, por tashm\u00eb ka fituar njohje t\u00eb konsiderueshme. Avantazhet e tij jan\u00eb: instalohet leht\u00ebsisht, konsumon pak burime, nuk k\u00ebrkon instalimin e Elasticsearch, pasi ruan t\u00eb dh\u00ebnat n\u00eb TSDB (baza e t\u00eb dh\u00ebnave t\u00eb serive t\u00eb koh\u00ebs). N\u00eb artikullin e kaluar kam shkruar se nj\u00eb baz\u00eb e till\u00eb ruan t\u00eb dh\u00ebnat e Prometheus, dhe kjo \u00ebsht\u00eb nj\u00eb nga shum\u00eb ngjashm\u00ebrit\u00eb mes dy produkteve. Zhvilluesit madje deklarojn\u00eb se Loki \u00ebsht\u00eb \"Prometheus-i p\u00ebr bot\u00ebn e logeve.\"<\/p>\n

<\/p>\n

Nj\u00eb p\u00ebrjashtim i vog\u00ebl p\u00ebr TSDB p\u00ebr ata q\u00eb nuk e kan\u00eb lexuar artikulli i m\u00ebparsh\u00ebm<\/a><\/noindex>: TSDB e menaxhon shk\u00eblqyer ruajtjen e shumic\u00ebs s\u00eb t\u00eb dh\u00ebnave, t\u00eb serive t\u00eb koh\u00ebs, por nuk \u00ebsht\u00eb e dizajnuar p\u00ebr ruajtje afatgjat\u00eb. N\u00ebse p\u00ebr ndonj\u00eb arsye ju nevojitet t\u00eb ruani loget m\u00eb gjat\u00eb se dy jav\u00eb, \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb konfiguroni d\u00ebrgimin e tyre n\u00eb nj\u00eb DB tjet\u00ebr.<\/p>\n

<\/p>\n

Nj\u00eb p\u00ebrfitim tjet\u00ebr i Loki \u00ebsht\u00eb se p\u00ebr vizualizimin e t\u00eb dh\u00ebnave p\u00ebrdoret Grafana. Mjaft e p\u00ebrshtatshme: n\u00eb Grafana shohim t\u00eb dh\u00ebnat p\u00ebr monitorimin dhe gjithashtu, duke lidhur Loki, shohim log\u00ebt. Nga log\u00ebt mund t\u00eb nd\u00ebrtojm\u00eb grafika.<\/p>\n

<\/p>\n

Arkitektura e Loki duket k\u00ebshtu: <\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Me an\u00eb t\u00eb DaemonSet n\u00eb t\u00eb gjith\u00eb server\u00ebt e klasterit, zhvillohet nj\u00eb agjent \u2014 Promtail ose Fluent Bit. Agjenti mbledh log\u00ebt. Loki i merr dhe i ruan n\u00eb TSDB. I gjith\u00eb log\u00ebve u shtohen metadatat menj\u00ebher\u00eb, gj\u00eb q\u00eb \u00ebsht\u00eb e p\u00ebrshtatshme: mund t\u00eb filtrosh sipas Pods, namespaces, emrave t\u00eb kontejner\u00ebve dhe madje edhe etiketave.<\/p>\n

<\/p>\n

Udh\u00ebzimi p\u00ebr instalimin e Loki<\/a><\/noindex><\/p>\n

<\/p>\n

Loki funksionon n\u00eb nj\u00eb nd\u00ebrfaqe t\u00eb njohur Grafana. Loki ka madje nj\u00eb gjuh\u00eb t\u00eb vet, e cila quhet LogQL \u2014 duke u frym\u00ebzuar nga emri dhe sintaksa q\u00eb i ngjan PromQL n\u00eb Prometheus. N\u00eb nd\u00ebrfaqen e Loki ka sugjerime me k\u00ebrkesa, k\u00ebshtu q\u00eb nuk \u00ebsht\u00eb e nevojshme t'i dish p\u00ebrmendsh.<\/p>\n

<\/p>\n

Dokumentacioni p\u00ebr gjuh\u00ebn LogQL<\/a><\/noindex><\/p>\n

<\/p>\n

\"Regjistrimi
\nLoki n\u00eb nd\u00ebrfaqen Grafana<\/em><\/p>\n

<\/p>\n

Duke p\u00ebrdorur filtrat, n\u00eb Loki mund t\u00eb gjesh kode (\u201c400\u201d, \u201c404\u201d dhe \u00e7do tjet\u00ebr); t\u00eb shikosh log\u00ebt nga e gjith\u00eb noda; t\u00eb filtroni t\u00eb gjith\u00eb log\u00ebt q\u00eb p\u00ebrmbajn\u00eb fjal\u00ebn \u201cerror\u201d. N\u00ebse klikoni mbi logun, do t\u00eb hapet nj\u00eb kartel\u00eb me t\u00eb gjith\u00eb informacionin p\u00ebr ngjarjen.<\/p>\n

<\/p>\n

N\u00eb Loki ka mjaft mjete t\u00eb cilat lejojn\u00eb nxjerrjen e log-eve t\u00eb nevojshme, megjith\u00ebse sinqerisht, mund t\u00eb kishin qen\u00eb edhe m\u00eb shum\u00eb teknikisht. Tani Loki po zhvillohet aktivisht dhe po fiton popullaritet. <\/p>\n

<\/p>\n

Elastic + Fluent Bit + Kibana (EFK Stack)<\/h2>\n

<\/p>\n

Staku EFK \u00ebsht\u00eb nj\u00eb mjet m\u00eb klasik dhe gjithashtu mjaft popullor p\u00ebr regjistrimin e log-eve. <\/p>\n

<\/p>\n

N\u00eb fillim t\u00eb artikullit u p\u00ebrmend ELK (Elasticsearch + Logstash + Kibana), por ky stak \u00ebsht\u00eb b\u00ebr\u00eb i vjetruar p\u00ebr shkak t\u00eb Logstash-it, i cili nuk \u00ebsht\u00eb shum\u00eb efektiv dhe \u00ebsht\u00eb gjithashtu k\u00ebrkesor p\u00ebr burime. N\u00eb vend t\u00eb tij, filluan t\u00eb p\u00ebrdorin nj\u00eb zgjidhje m\u00eb t\u00eb leht\u00eb dhe m\u00eb efikase si Fluentd, dhe pas pak kohe iu shtua Fluent Bit<\/a><\/noindex> \u2014 nj\u00eb agjent mbledh\u00ebs edhe m\u00eb i leht\u00eb dhe edhe m\u00eb efikas. <\/p>\n

<\/p>\n

N\u00ebse i besohet zhvilluesve, allora Fluent Bit \u00ebsht\u00eb m\u00eb shum\u00eb se 100 her\u00eb m\u00eb efikas se Fluentd: \u00abaty ku Fluentd konsumon 20 MB RAM, Fluent Bit do t\u00eb konsumoj\u00eb 150 KB\u00bb \u2014 nj\u00eb citat direkt nga dokumentacioni. Duke e par\u00eb k\u00ebt\u00eb, Fluent Bit \u00ebsht\u00eb p\u00ebrdorur m\u00eb shpesh.<\/p>\n

<\/p>\n

Fluent Bit ka m\u00eb pak mund\u00ebsi sesa Fluentd, por p\u00ebrmbush nevojat thelb\u00ebsore, prandaj ne kryesisht p\u00ebrdorim Fluent Bit.<\/p>\n

<\/p>\n

Skema e funksionimit t\u00eb stack-ut EFK: agjenti mbledh log-et nga t\u00eb gjith\u00eb pod-\u00ebt (zakonisht, kjo \u00ebsht\u00eb nj\u00eb DaemonSet q\u00eb \u00ebsht\u00eb e aktivizuar n\u00eb t\u00eb gjitha server\u00ebt e klasterit) dhe i d\u00ebrgon n\u00eb ruajtje (Elasticsearch, PostgreSQL ose Kafka). Kibana lidhet me ruajtjen dhe nxjerr t\u00eb gjith\u00eb informacionin e nevojsh\u00ebm nga aty.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Kibana<\/a><\/noindex> paraqet informacionin n\u00eb nj\u00eb nd\u00ebrfaqe web t\u00eb p\u00ebrshtatshme. Ka grafik\u00eb, filtra dhe shum\u00eb m\u00eb tep\u00ebr.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Nga log-et mund t\u00eb krijohen t\u00eb t\u00ebra dashboard-e.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Aft\u00ebsit\u00eb e Fluent Bit<\/h2>\n

<\/p>\n

Duke qen\u00eb se p\u00ebr Fluent Bit, zakonisht, \u00ebsht\u00eb d\u00ebgjuar m\u00eb pak se p\u00ebr Logstash, le t\u00eb shqyrtojm\u00eb at\u00eb pak m\u00eb n\u00eb thell\u00ebsi. Fluent Bit mund t\u00eb ndahet n\u00eb m\u00ebnyr\u00eb logjike n\u00eb 6 module, disa nga t\u00eb cilat mund t\u00eb ken\u00eb shtesa q\u00eb zgjasin kapacitetet e Fluent Bit.<\/p>\n

<\/p>\n

\"Regjistrimi<\/p>\n

<\/p>\n

Moduli Input<\/strong> mbledh log-et nga skedar\u00ebt, sh\u00ebrbimet systemd dhe madje edhe nga tcp-socket (thjesht duhet t\u00eb specifikoni endpoint-in dhe Fluent Bit do t\u00eb filloj\u00eb ta kontrolloj\u00eb at\u00eb). K\u00ebto mund\u00ebsi jan\u00eb t\u00eb mjaftueshme p\u00ebr t\u00eb mbledhur log-et si nga sistemi ashtu edhe nga kontejner\u00ebt.<\/p>\n

<\/p>\n

N\u00eb prodhim zakonisht p\u00ebrdorim shtesat tail<\/a><\/noindex> (mund ta drejtoni at\u00eb drejt dosjes me log-e) dhe systemd<\/a><\/noindex> (mund t'i thoni asaj se nga cilat sh\u00ebrbime t\u00eb mbledh\u00eb log-et).<\/p>\n

<\/p>\n

Moduli Parser<\/strong> shnd\u00ebrron log\u00ebt n\u00eb nj\u00eb format t\u00eb p\u00ebrbashk\u00ebt. Sipas parazgjedhjes, log\u00ebt e Nginx-it jan\u00eb nj\u00eb varg. Me ndihm\u00ebn e k\u00ebtij plugini, ky varg mund t\u00eb konvertohet n\u00eb JSON: t\u00eb caktohen fushat dhe vlerat e tyre. Me JSON \u00ebsht\u00eb m\u00eb e leht\u00eb t\u00eb punosh sesa me logun n\u00eb format vargu, sepse ofron mund\u00ebsi m\u00eb fleksib\u00ebl p\u00ebr renditje.<\/p>\n

<\/p>\n

Moduli Filter<\/strong>. N\u00eb k\u00ebt\u00eb nivel filtrohen log\u00ebt e pad\u00ebshiruar. P\u00ebr shembull, vet\u00ebm log\u00ebt me vler\u00ebn \u201cwarning\u201d apo me etiketa t\u00eb caktuara d\u00ebrgohen p\u00ebr ruajtje. Log\u00ebt e p\u00ebrzgjedhur kalojn\u00eb n\u00eb bufer.<\/p>\n

<\/p>\n

Moduli Buffer<\/strong>. Fluent Bit ka dy lloje buferash: bufer memorie dhe bufer n\u00eb disk. Buferi \u00ebsht\u00eb nj\u00eb ruajtje temporale e log\u00ebve, e nevojshme p\u00ebr rastet e gabimeve ose prishjeve. T\u00eb gjith\u00eb duan t\u00eb kursejn\u00eb n\u00eb RAM, prandaj zakonisht zgjidhet buferi n\u00eb disk. Por duhet t\u00eb mbani parasysh se log\u00ebt ende shkarkohen n\u00eb memory para se t\u00eb shkojn\u00eb n\u00eb disk.<\/p>\n

<\/p>\n

Moduli Routing\/Output<\/strong> p\u00ebrmban rregulla dhe adresa p\u00ebr d\u00ebrgimin e log\u00ebve. Si\u00e7 u tha m\u00eb par\u00eb, log\u00ebt mund t\u00eb d\u00ebrgohen n\u00eb Elasticsearch, PostgreSQL ose, p\u00ebr shembull, Kafka. <\/p>\n

<\/p>\n

\u00ebsht\u00eb interesante se si log-et nga Fluent Bit mund t\u00eb d\u00ebrgohen n\u00eb Fluentd. Duke qen\u00eb se e para \u00ebsht\u00eb m\u00eb e leht\u00eb dhe m\u00eb pak funksionale, mund t\u00eb grumbullohen log-et dhe t\u00eb d\u00ebrgohen n\u00eb Fluentd, ku mund t\u00eb p\u00ebrpunohen m\u00eb tej me ndihm\u00ebn e plugins shtes\u00eb dhe t\u00eb d\u00ebrgohen n\u00eb depo.<\/p>\n

<\/p>\n

N\u00ebse planifikoni t\u00eb p\u00ebrdorni Elasticsearch...<\/p>\n

N\u00eb fund, dy k\u00ebshilla p\u00ebr ata q\u00eb planifikojn\u00eb t\u00eb p\u00ebrdorin Elasticsearch n\u00eb prodhim si nj\u00eb depo log-esh. <\/p>\n

    \n
  1. Konfiguroni njoftimet me an\u00eb t\u00eb ElastAlert<\/a><\/noindex>. Ky program nxjerr nga fluksi i p\u00ebrgjithsh\u00ebm i log-eve mesazhet e r\u00ebnd\u00ebsishme dhe krijon alerte p\u00ebr to n\u00eb email ose kanale t\u00eb tjera. Megjithat\u00eb, jo shum\u00eb koh\u00eb m\u00eb par\u00eb dol\u00ebn lajme trishtuese se projekti mund t\u00eb ndaloj\u00eb s\u00eb ekzistuari s\u00eb shpejti<\/a><\/noindex><\/em>.<\/li>\n
  2. Rrotulloni log-et me ndihm\u00ebn e aplikacionit Curator<\/a><\/noindex> ose dhe thirrjeve ndaj API Elasticsearch. Elastic aktualisht po b\u00ebn hapa t\u00eb r\u00ebnd\u00ebsish\u00ebm n\u00eb menaxhimin e jet\u00ebs s\u00eb indekseve pa p\u00ebrdorimin e mjeteve t\u00eb tjera. N\u00eb p\u00ebrgjith\u00ebsi, nuk ka ndonj\u00eb kuptim t\u00eb ruash log\u00ebt p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb: \u00ebsht\u00eb e pamundur q\u00eb nj\u00eb log t\u00eb nevojitet pas dy jav\u00ebsh \u2014 n\u00ebse \u00ebsht\u00eb me t\u00eb v\u00ebrtet\u00eb kritik, brenda dy jav\u00ebsh do t\u00eb jet\u00eb trajtuar patjet\u00ebr. N\u00eb rastin m\u00eb t\u00eb keq, log\u00ebt e vjetra mund t\u00eb arkivohen dhe d\u00ebrgohen diku p\u00ebr ruajtje afatgjat\u00eb. Kam d\u00ebgjuar p\u00ebr log\u00eb t\u00eb ve\u00e7ant\u00eb q\u00eb ligji k\u00ebrkon t\u00eb ruhet deri n\u00eb 5 vjet. Personalish nuk kam hasur n\u00eb k\u00ebt\u00eb, por nuk do ta barazoja k\u00ebt\u00eb informacion me log\u00eb t\u00eb zakonsh\u00ebm, dhe ndoshta do t\u2019i ruaja ato ve\u00e7mas.<\/li>\n<\/ol>\n

    \n<\/p><\/blockquote>\n

    Vazhdohet...<\/p>\n

    <\/p>\n

    Autori: Marsel Ibraev, administrator i \u00e7ertifikuar i Kubernetes, inxhinier praktik n\u00eb kompanin\u00eb Southbridge<\/a><\/noindex>, fol\u00ebs dhe zhvillues kurse Slyorm<\/a><\/noindex>.<\/em><\/p>\n

    Burimi: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

    \u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana). \u041c\u0430\u0442\u0435\u0440\u0438\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u0432\u044b\u0436\u0438\u043c\u043a\u0430 \u0438\u0437 \u043e\u0442\u043a\u0440\u044b\u0442\u043e\u0439 \u043b\u0435\u043a\u0446\u0438\u0438 \u0448\u043a\u043e\u043b\u044b \u00ab\u0421\u043b\u0451\u0440\u043c\u00bb. \u0415\u0441\u043b\u0438 \u0435\u0441\u0442\u044c \u0436\u0435\u043b\u0430\u043d\u0438\u0435 \u0438 \u0442\u0435\u043c \u0431\u043e\u043b\u0435\u0435 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u0430\u044f \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0439\u0442\u0438 \u043f\u043e\u043b\u043d\u043e\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u2014 \u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u0439\u0442\u0435\u0441\u044c \u043d\u0430 \u043a\u0443\u0440\u0441 \u043f\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":93295,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-93294","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Logimi n\u00eb Kubernetes: si t\u00eb mbledhim, ruajm\u00eb, parse dhe p\u00ebrpunojm\u00eb log\u00ebt | ProHoster","description":"Do t\u00eb shqyrtojm\u00eb bazat e logimit n\u00eb Docker dhe Kubernetes, dhe m\u00eb pas do t\u00eb shqyrtojm\u00eb dy mjete q\u00eb mund t'i p\u00ebrdorim n\u00eb prodhim: Grafana Loki dhe staku EFK (Elasticsearch + Fluent Bit + Kibana). Materiali i artikullit \u00ebsht\u00eb nj\u00eb p\u00ebrmbledhje e nj\u00eb ligj\u00ebrate t\u00eb hapur n\u00eb shkoll\u00ebn 'Slyerm'. N\u00ebse keni d\u00ebshir\u00eb dhe sidomos nevoj\u00eb p\u00ebr prodhim, mund t\u00eb ndiqni trajnimin e plot\u00eb \u2014 regjistrohuni n\u00eb kursin p\u00ebr","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0432 Kubernetes: \u043a\u0430\u043a \u0441\u043e\u0431\u0438\u0440\u0430\u0442\u044c, \u0445\u0440\u0430\u043d\u0438\u0442\u044c, \u043f\u0430\u0440\u0441\u0438\u0442\u044c \u0438 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043b\u043e\u0433\u0438 | ProHoster","og:description":"\u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana). \u041c\u0430\u0442\u0435\u0440\u0438\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u0432\u044b\u0436\u0438\u043c\u043a\u0430 \u0438\u0437 \u043e\u0442\u043a\u0440\u044b\u0442\u043e\u0439 \u043b\u0435\u043a\u0446\u0438\u0438 \u0448\u043a\u043e\u043b\u044b \u00ab\u0421\u043b\u0451\u0440\u043c\u00bb. \u0415\u0441\u043b\u0438 \u0435\u0441\u0442\u044c \u0436\u0435\u043b\u0430\u043d\u0438\u0435 \u0438 \u0442\u0435\u043c \u0431\u043e\u043b\u0435\u0435 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u0430\u044f \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0439\u0442\u0438 \u043f\u043e\u043b\u043d\u043e\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u2014 \u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u0439\u0442\u0435\u0441\u044c \u043d\u0430 \u043a\u0443\u0440\u0441 \u043f\u043e","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-05T05:42:58+00:00","article:modified_time":"2020-09-05T05:42:58+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"93294","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:50:29","updated":"2022-09-27 18:43:48"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/93294","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=93294"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/93294\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/93295"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=93294"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=93294"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=93294"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}