{"id":93294,"date":"2020-09-05T07:42:58","date_gmt":"2020-09-05T05:42:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi"},"modified":"2020-09-05T07:42:58","modified_gmt":"2020-09-05T05:42:58","slug":"logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","title":{"rendered":"Protokolov\u00e1n\u00ed v Kubernetes: jak shroma\u017e\u010fovat, uchov\u00e1vat, analyzovat a zpracov\u00e1vat logy","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Pod\u00edv\u00e1me se na z\u00e1klady logov\u00e1n\u00ed v Dockeru a Kubernetes a pot\u00e9 si p\u0159edstav\u00edme dva n\u00e1stroje, kter\u00e9 lze s klidem pou\u017e\u00edvat v produkci: Grafana Loki a EFK stack (Elasticsearch + Fluent Bit + Kibana).<\/p>\n

<\/p>\n

Materi\u00e1l \u010dl\u00e1nku \u2014 shrnut\u00ed z otev\u0159en\u00e9 p\u0159edn\u00e1\u0161ky \u0161koly \u201eSljerm\u201c<\/a><\/noindex>. Pokud m\u00e1te z\u00e1jem a nav\u00edc v\u00fdrobn\u00ed pot\u0159ebu, m\u016f\u017eete absolvovat kompletn\u00ed \u0161kolen\u00ed \u2014 p\u0159ihlaste se na kurz o monitorov\u00e1n\u00ed a logov\u00e1n\u00ed infrastruktury v Kubernetes<\/a><\/noindex>.<\/em><\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/a><\/noindex><\/p>\n

Logov\u00e1n\u00ed v Dockeru<\/h2>\n

<\/p>\n

Na \u00farovni Kubernetes jsou aplikace spu\u0161t\u011bny v podech, ale na m\u011bs\u00ed\u010dn\u00ed \u00farovni obvykle funguj\u00ed v Dockeru. Proto je t\u0159eba nastavit logov\u00e1n\u00ed tak, aby se shrom\u00e1\u017edily logy z kontejner\u016f. Kontejnery spou\u0161t\u00ed Docker \u2014 tak\u017ee je t\u0159eba zjistit, jak je logov\u00e1n\u00ed nastaveno na \u00farovni Dockeru.<\/p>\n

<\/p>\n

Douf\u00e1m, \u017ee ka\u017ed\u00fd \u010dten\u00e1\u0159 v\u00ed: logy aplikace by m\u011bly b\u00fdt ps\u00e1ny na stdout\/stderr, a ne dovnit\u0159 kontejneru. Logy agreguje Docker Daemon a ten pracuje p\u0159esn\u011b s t\u011bmi logy, kter\u00e9 jsou odes\u00edl\u00e1ny na stdout\/stderr. Krom\u011b toho psan\u00ed log\u016f dovnit\u0159 kontejneru p\u0159in\u00e1\u0161\u00ed probl\u00e9my: kontejner se nafukuje d\u00edky rostouc\u00edm log\u016fm (proto\u017ee v kontejneru pravd\u011bpodobn\u011b nen\u00ed \u017e\u00e1dn\u00fd Logrotate) a Docker Daemon o t\u011bchto log\u00e1ch ani nev\u00ed.<\/p>\n

<\/p>\n

Docker m\u00e1 n\u011bkolik logovac\u00edch ovlada\u010d\u016f nebo plugin\u016f pro shroma\u017e\u010fov\u00e1n\u00ed log\u016f kontejner\u016f. V bezplatn\u00e9 verzi Docker Community Edition (CE) je m\u00e9n\u011b logovac\u00edch ovlada\u010d\u016f ne\u017e v komer\u010dn\u00ed Docker Enterprise Edition (EE).<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Docker EE jsem nikdy v praxi nepou\u017eil: ve Southbridge se sna\u017e\u00edme dr\u017eet otev\u0159en\u00fdch zdrojov\u00fdch \u0159e\u0161en\u00ed a na\u0161im klient\u016fm v\u011bt\u0161ina dodate\u010dn\u00fdch funkc\u00ed Docker EE nen\u00ed pot\u0159ebn\u00e1. <\/p>\n

<\/p>\n

Logovac\u00ed ovlada\u010de v Docker CE:<\/p>\n

<\/p>\n

local<\/strong> \u2014 z\u00e1znam log\u016f do intern\u00edch soubor\u016f Docker Daemon;
\njson-file<\/strong> \u2014 vytv\u00e1\u0159en\u00ed json-logu v ka\u017ed\u00e9m kontejneru;
\njournald<\/strong> \u2014 odes\u00edl\u00e1n\u00ed log\u016f do journald.<\/p>\n

<\/p>\n

Nastaven\u00ed logov\u00e1n\u00ed v Dockeru se nach\u00e1z\u00ed v souboru daemon.json. <\/p>\n

<\/p>\n

V poli \u201elog-driver\u201c je uveden plugin a v poli \u201elog-opts\u201c \u2014 jeho nastaven\u00ed. V p\u0159\u00edkladu v\u00fd\u0161e byl uveden plugin \u201ejson-file\u201c, limit velikosti logu \u2014 \u201emax-size\u201c: \u201e10m\u201c; limit po\u010dtu soubor\u016f (nastaven\u00ed rotace) \u2014 \u201emax-file\u201c: \u201e3\u201c; a tak\u00e9 hodnoty, kter\u00e9 budou p\u0159ipojeny k log\u016fm.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

N\u011bkter\u00e1 nastaven\u00ed logovac\u00edho ovlada\u010de lze definovat pomoc\u00ed p\u0159\u00edkazov\u00e9 utility. To je v\u00fdhodn\u00e9, pokud je pot\u0159eba spustit konkr\u00e9tn\u00ed kontejner s jin\u00fdm logovac\u00edm ovlada\u010dem. <\/p>\n

<\/p>\n

Takto vypad\u00e1 sch\u00e9ma logov\u00e1n\u00ed v Dockeru:<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Jak sch\u00e9ma funguje: log-drivre, nap\u0159\u00edklad json-file, vytv\u00e1\u0159\u00ed soubory. Shroma\u017e\u010fova\u010de log\u016f (Rsyslog, Fluentd, Logagent a dal\u0161\u00ed) tyto soubory shroma\u017e\u010fuj\u00ed a odes\u00edlaj\u00ed na ukl\u00e1d\u00e1n\u00ed do Elastic, Sematext nebo jin\u00fdch \u00falo\u017ei\u0161\u0165.<\/p>\n

<\/p>\n

Vlastnosti logov\u00e1n\u00ed v Kubernetes<\/h2>\n

<\/p>\n

Zjednodu\u0161en\u011b sch\u00e9ma logov\u00e1n\u00ed v Kubernetes vypad\u00e1 takto: je zde pod, uvnit\u0159 je spu\u0161t\u011bn\u00fd kontejner, kontejner pos\u00edl\u00e1 logy do stdout\/stderr. Pot\u00e9 Docker vytv\u00e1\u0159\u00ed soubor a zapisuje logy, kter\u00e9 n\u00e1sledn\u011b m\u016f\u017ee rozt\u00e1\u010det.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Poj\u010fme se pod\u00edvat na vlastnosti logov\u00e1n\u00ed v Kubernetes.<\/p>\n

<\/p>\n

Udr\u017eovat logy mezi nasazen\u00edmi<\/strong>. To je nezbytn\u00e1 podm\u00ednka pro spr\u00e1vn\u00e9 nastaven\u00ed logov\u00e1n\u00ed. Pokud se logy mezi nasazen\u00edmi nebudou uchov\u00e1vat, p\u0159i vyd\u00e1n\u00ed nov\u00e9 verze aplikace dojde k p\u0159eps\u00e1n\u00ed log\u016f p\u0159edchoz\u00ed verze a restartov\u00e1n\u00ed kontejneru bude tak\u00e9 spojen\u00e9 se ztr\u00e1tou log\u016f. Kubernetes m\u00e1 p\u0159ep\u00edna\u010d \u2014previous, kter\u00fd umo\u017e\u0148uje prohl\u00ed\u017eet logy aplikace p\u0159ed posledn\u00edm restartem Podu, ale ne hloub\u011bji.<\/p>\n

<\/p>\n

Agregovat logy ze v\u0161ech instanc\u00ed<\/strong>. Pokud jsou mikroslu\u017eby hostov\u00e1ny v cloudech, za kontrolu syst\u00e9mu je zodpov\u011bdn\u00fd cloudov\u00fd poskytovatel. Pokud jsou mikroslu\u017eby na vlastn\u00edm hardwaru, krom\u011b log\u016f z kontejner\u016f je t\u0159eba shroma\u017e\u010fovat i logy syst\u00e9mu.<\/p>\n

<\/p>\n

D\u0159\u00edve nebyly k dispozici pohodln\u00e9 n\u00e1stroje pro shroma\u017e\u010fov\u00e1n\u00ed log\u016f jak ze syst\u00e9mu, tak z mikroslu\u017eeb. Obvykle jeden n\u00e1stroj shroma\u017e\u010foval syst\u00e9mov\u00e9 logy (nap\u0159\u00edklad Rsyslog), druh\u00fd \u2014 logy z Dockeru (nap\u0159\u00edklad journal-bit s nastaven\u00edm log-drivre Docker na journald). Zkou\u0161eli jsme pou\u017e\u00edvat journal-bit \u2014 shroma\u017e\u010fovat logy i z kontejner\u016f (v log-drivre Docker uv\u00e9st, \u017ee je pot\u0159eba zapisovat logy do journald), a i ze syst\u00e9mu (v CentOS 7 ji\u017e existuje systemd a journald). \u0158e\u0161en\u00ed je funk\u010dn\u00ed, ale ne ide\u00e1ln\u00ed. Pokud je log\u016f hodn\u011b, journal-bit za\u010d\u00edn\u00e1 lagovat, zpr\u00e1vy se ztr\u00e1cej\u00ed.<\/p>\n

<\/p>\n

Experimenty pokra\u010dovaly \u2014 a na\u0161el se jin\u00fd zp\u016fsob. V CentOS 7 se hlavn\u00ed syst\u00e9mov\u00e9 logy (messages, audit, secure) duplikuj\u00ed do var-log ve form\u011b soubor\u016f. V Dockeru tak\u00e9 lze nastavit ukl\u00e1d\u00e1n\u00ed log\u016f do soubor\u016f json. Odpov\u00eddaj\u00edc\u00edm zp\u016fsobem lze tyto soubory z CentOS 7 a Dockeru shroma\u017e\u010fovat dohromady.<\/p>\n

<\/p>\n

S postupem \u010dasu se stalo popul\u00e1rn\u00edm \u0159e\u0161en\u00ed ELK Stack. To je kombinace n\u011bkolika n\u00e1stroj\u016f: Elasticsearch, Logstash a Kibana.<\/p>\n

<\/p>\n

Elasticsearch ukl\u00e1d\u00e1 logy z kontejner\u016f, Logstash shroma\u017e\u010fuje logy z instanc\u00ed, Kibana umo\u017e\u0148uje zpracov\u00e1vat z\u00edskan\u00e9 logy a vytv\u00e1\u0159et podle nich grafy. N\u011bjakou dobu byl ELK Stack aktivn\u011b pou\u017e\u00edv\u00e1n, ale podle m\u00e9ho n\u00e1zoru jeho doba kon\u010d\u00ed. Pozd\u011bji vysv\u011btl\u00edm pro\u010d.<\/p>\n

<\/p>\n

P\u0159id\u00e1vat metadata<\/strong>Pody, aplikace a kontejnery mohou b\u00fdt spu\u0161t\u011bny kdekoli. Nav\u00edc m\u016f\u017ee m\u00edt jedna aplikace n\u011bkolik instanc\u00ed. Logy jsou zaznamen\u00e1ny ve stejn\u00e9m form\u00e1tu, ale mus\u00edme pochopit, kter\u00e1 replika je to, kter\u00fd Pod to p\u00ed\u0161e a v jak\u00e9m namespace se nach\u00e1z\u00ed. Proto je nutn\u00e9 log\u016fm p\u0159id\u00e1vat metadata.<\/p>\n

<\/p>\n

Parselog\u016f<\/strong>Je zaj\u00edmav\u00e9, \u017ee n\u00e1klady na podporu syst\u00e9mu logov\u00e1n\u00ed a monitorov\u00e1n\u00ed mohou p\u0159ekro\u010dit v\u00fddaje na hlavn\u00ed aplikaci. Kdy\u017e se v\u00e1m generuj\u00ed des\u00edtky a stovky tis\u00edc log\u016f za sekundu, to se zd\u00e1 b\u00fdt logick\u00e9, ale p\u0159esto je t\u0159eba zn\u00e1t hranici. Jedn\u00edm ze zp\u016fsob\u016f, jak tuto hranu naj\u00edt, je parsov\u00e1n\u00ed log\u016f. <\/p>\n

<\/p>\n

Obvykle nen\u00ed nutn\u00e9 shroma\u017e\u010fovat a uchov\u00e1vat v\u0161echny logy, m\u011bli bychom ukl\u00e1dat pouze \u010d\u00e1st \u2014 nap\u0159\u00edklad logy se stavem \u201ewarning\u201c nebo \u201eerror\u201c. Pokud jde o logy nginx nebo ingress kontrol\u00e9r\u016f, lze ukl\u00e1dat jen ty, jejich\u017e stav se li\u0161\u00ed od 200. Ale to nen\u00ed univerz\u00e1ln\u00ed rada: pokud n\u011bjak\u00fdm zp\u016fsobem budujete analytiku podle log\u016f Nginx, pak je samoz\u0159ejm\u011b stoj\u00ed za to shroma\u017e\u010fovat. <\/p>\n

<\/p>\n

Nedoporu\u010duje se bezmy\u0161lenkovit\u011b filtrovat logy, proto\u017ee m\u016f\u017ee b\u00fdt nedostatek odfiltrovan\u00fdch dat pro norm\u00e1ln\u00ed analytiku. Na druhou stranu, mo\u017en\u00e1 bychom m\u011bli prov\u00e1d\u011bt anal\u00fdzu nikoli na \u00farovni logov\u00e1n\u00ed, ale na \u00farovni sb\u011bru metrik. Pak nebude nutn\u00e9 uchov\u00e1vat stovky tis\u00edc \u0159\u00e1dk\u016f s k\u00f3dem 200. Jedn\u00edm z p\u0159\u00edstup\u016f je z\u00edskat informace o provozu a chyb\u00e1ch z metrik ingress kontrol\u00e9r\u016f. <\/p>\n

<\/p>\n

Obecn\u011b byste se m\u011bli dob\u0159e zamyslet nad t\u00edm, co chcete uchov\u00e1vat a jak dlouho, proto\u017ee jinak m\u016f\u017ee vzniknout situace, kdy syst\u00e9m logov\u00e1n\u00ed bude odeb\u00edrat v\u00edce zdroj\u016f, ne\u017e hlavn\u00ed projekt.<\/p>\n

<\/p>\n

Zat\u00edm neexistuje standardn\u00ed \u0159e\u0161en\u00ed pro logov\u00e1n\u00ed<\/strong>. Na rozd\u00edl od monitorov\u00e1n\u00ed, kde je jedno nejroz\u0161\u00ed\u0159en\u011bj\u0161\u00ed \u0159e\u0161en\u00ed Prometheus, v logov\u00e1n\u00ed neexistuje standard. <\/p>\n

<\/p>\n

V r\u00e1mci t\u00e9to p\u0159edn\u00e1\u0161ky se pod\u00edv\u00e1me na dva n\u00e1stroje: jeden popul\u00e1rn\u00ed a druh\u00fd \u2013 st\u00e1le z\u00edsk\u00e1vaj\u00edc\u00ed na oblib\u011b. Krom\u011b nich existuj\u00ed i dal\u0161\u00ed, ale v tomto \u010dl\u00e1nku se jimi nebudeme zab\u00fdvat.<\/p>\n

<\/p>\n

S ohledem na v\u0161echny v\u00fd\u0161e uveden\u00e9 vlastnosti lze nyn\u00ed logov\u00e1n\u00ed v Kubernetes p\u0159edstavit na n\u00e1sleduj\u00edc\u00ed sch\u00e9matu:<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Z\u016fst\u00e1v\u00e1 log kontejneru, rotace, ale zjevn\u011b se objevuje agent-zb\u00edra\u010de, kter\u00fd shroma\u017e\u010fuje logy a odes\u00edl\u00e1 je na \u00falo\u017ei\u0161t\u011b (na sch\u00e9matu \u2014 do Logging Backend). Agent b\u011b\u017e\u00ed na ka\u017ed\u00e9m uzlu a obvykle je spu\u0161t\u011bn v Kubernetes.<\/p>\n

<\/p>\n

Nyn\u00ed se pod\u00edv\u00e1me na n\u00e1stroje pro logov\u00e1n\u00ed.<\/p>\n

<\/p>\n

Grafana Loki<\/h2>\n

<\/p>\n

Grafana Loki<\/a><\/noindex> se objevil ned\u00e1vno, ale ji\u017e si z\u00edskal zna\u010dnou popularitu. Jeho v\u00fdhody: snadn\u00e1 instalace, n\u00edzk\u00e1 spot\u0159eba zdroj\u016f, nevy\u017eaduje instalaci Elasticsearch, proto\u017ee ukl\u00e1d\u00e1 data v TSDB (datab\u00e1ze \u010dasov\u00fdch \u0159ad). V p\u0159edchoz\u00edm \u010dl\u00e1nku jsem psal, \u017ee takovou datab\u00e1zi pou\u017e\u00edv\u00e1 Prometheus, a to je jedno z mnoha podobnost\u00ed obou produkt\u016f. V\u00fdvoj\u00e1\u0159i dokonce tvrd\u00ed, \u017ee Loki je \"Prometheus pro sv\u011bt logov\u00e1n\u00ed.\"<\/p>\n

<\/p>\n

Mal\u00e9 odbo\u010den\u00ed k TSDB pro ty, kte\u0159\u00ed ne\u010detli p\u0159edchoz\u00ed \u010dl\u00e1nek<\/a><\/noindex>: TSDB skv\u011ble zvl\u00e1d\u00e1 \u00falohu uchov\u00e1v\u00e1n\u00ed velk\u00e9ho mno\u017estv\u00ed dat, \u010dasov\u00fdch \u0159ad, ale nen\u00ed ur\u010dena pro dlouhodob\u00e9 ulo\u017een\u00ed. Pokud z n\u011bjak\u00e9ho d\u016fvodu pot\u0159ebujete uchov\u00e1vat logy d\u00e9le ne\u017e dva t\u00fddny, je lep\u0161\u00ed nastavit jejich p\u0159enos do jin\u00e9 datab\u00e1ze.<\/p>\n

<\/p>\n

Dal\u0161\u00ed v\u00fdhodou Loki je, \u017ee pro vizualizaci dat se pou\u017e\u00edv\u00e1 Grafana. Velmi pohodln\u00e9: v Grafan\u011b sledujeme data pro monitorov\u00e1n\u00ed a tam tak\u00e9, p\u0159ipojen\u00edm Loki, sledujeme logy. Podle log\u016f lze vytv\u00e1\u0159et grafy.<\/p>\n

<\/p>\n

Architektura Loki vypad\u00e1 p\u0159ibli\u017en\u011b takto: <\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Pomoc\u00ed DaemonSet se na v\u0161ech serverech v clusteru nasazuje agent \u2014 Promtail nebo Fluent Bit. Agent sb\u00edr\u00e1 logy. Loki je shroma\u017e\u010fuje a uchov\u00e1v\u00e1 ve sv\u00e9 TSDB. K log\u016fm jsou hned p\u0159id\u00e1ny metadata, co\u017e je pohodln\u00e9: lze filtrovat podle Pods, namespaces, n\u00e1zv\u016f kontejner\u016f a dokonce i \u0161t\u00edtk\u016f.<\/p>\n

<\/p>\n

N\u00e1vod na instalaci Loki<\/a><\/noindex><\/p>\n

<\/p>\n

Loki funguje v zn\u00e1m\u00e9m rozhran\u00ed Grafana. Loki m\u00e1 dokonce sv\u016fj vlastn\u00ed dotazovac\u00ed jazyk, naz\u00fdv\u00e1 se LogQL \u2014 podle n\u00e1zvu a syntaxe p\u0159ipom\u00edn\u00e1 PromQL v Prometheus. V rozhran\u00ed Loki jsou n\u00e1pov\u011bdy s dotazy, tak\u017ee nen\u00ed nutn\u00e9 je zn\u00e1t zpam\u011bti.<\/p>\n

<\/p>\n

Dokumentace k jazyku LogQL<\/a><\/noindex><\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed
\nLoki v rozhran\u00ed Grafana<\/em><\/p>\n

<\/p>\n

Pomoc\u00ed filtr\u016f m\u016f\u017eete v Loki vyhled\u00e1vat k\u00f3dy (\"400\", \"404\" a jak\u00e9koli jin\u00e9); zobrazit logy z cel\u00e9 nody; filtrovat v\u0161echny logy, kde je slovo \"error\". Pokud kliknete na log, otev\u0159e se karta se v\u0161emi informacemi o ud\u00e1losti.<\/p>\n

<\/p>\n

V Loki je dostatek n\u00e1stroj\u016f, kter\u00e9 umo\u017e\u0148uj\u00ed z\u00edskat pot\u0159ebn\u00e9 logy, a\u010dkoliv up\u0159\u00edmn\u011b \u0159e\u010deno, technicky jich mohlo b\u00fdt i v\u00edce. Nyn\u00ed se Loki aktivn\u011b vyv\u00edj\u00ed a z\u00edsk\u00e1v\u00e1 popularitu. <\/p>\n

<\/p>\n

Elastic + Fluent Bit + Kibana (EFK Stack)<\/h2>\n

<\/p>\n

EFK stack je klasick\u00fdm a z\u00e1rove\u0148 velmi popul\u00e1rn\u00edm n\u00e1strojem pro logov\u00e1n\u00ed. <\/p>\n

<\/p>\n

Na za\u010d\u00e1tku \u010dl\u00e1nku bylo zm\u00edn\u011bno ELK (Elasticsearch + Logstash + Kibana), ale tento stack zastaral kv\u016fli ne p\u0159\u00edli\u0161 v\u00fdkonn\u00e9mu a z\u00e1rove\u0148 vysoce n\u00e1ro\u010dn\u00e9mu Logstashi. M\u00edsto n\u011bj se za\u010dal pou\u017e\u00edvat leh\u010d\u00ed a v\u00fdkonn\u011bj\u0161\u00ed Fluentd, a po n\u011bjak\u00e9 dob\u011b mu p\u0159i\u0161el na pomoc Fluent Bit<\/a><\/noindex> \u2014 je\u0161t\u011b leh\u010d\u00ed a je\u0161t\u011b v\u00fdkonn\u011bj\u0161\u00ed sb\u011bra\u010d log\u016f. <\/p>\n

<\/p>\n

Pokud v\u011b\u0159\u00edme v\u00fdvoj\u00e1\u0159\u016fm, Fluent Bit je v\u00edce ne\u017e 100kr\u00e1t v\u00fdkonn\u011bj\u0161\u00ed ne\u017e Fluentd: \"tam, kde Fluentd spot\u0159ebov\u00e1v\u00e1 20 MB RAM, Fluent Bit bude spot\u0159ebov\u00e1vat 150 kB\" \u2014 p\u0159\u00edm\u00fd cit\u00e1t z dokumentace. D\u00edky tomu se Fluent Bit za\u010dal pou\u017e\u00edvat \u010dast\u011bji.<\/p>\n

<\/p>\n

Fluent Bit m\u00e1 m\u00e9n\u011b funkc\u00ed ne\u017e Fluentd, ale pokr\u00fdv\u00e1 z\u00e1kladn\u00ed pot\u0159eby, tak\u017ee v\u011bt\u0161inou pou\u017e\u00edv\u00e1me Fluent Bit.<\/p>\n

<\/p>\n

Sch\u00e9ma fungov\u00e1n\u00ed stacku EFK: agent sb\u00edr\u00e1 logy ze v\u0161ech pod\u016f (obvykle to je DaemonSet spu\u0161t\u011bn\u00fd na v\u0161ech serverech clusteru) a pos\u00edl\u00e1 je do \u00falo\u017ei\u0161t\u011b (Elasticsearch, PostgreSQL nebo Kafka). Kibana se p\u0159ipojuje k \u00falo\u017ei\u0161ti a z\u00edsk\u00e1v\u00e1 odtud v\u0161echny pot\u0159ebn\u00e9 informace.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Kibana<\/a><\/noindex> prezentuje informace v p\u0159ehledn\u00e9m webov\u00e9m rozhran\u00ed. Jsou tam grafy, filtry a mnoh\u00e9 dal\u0161\u00ed.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Z log\u016f je mo\u017en\u00e9 vytv\u00e1\u0159et cel\u00e9 dashboardy.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Mo\u017enosti Fluent Bit<\/h2>\n

<\/p>\n

Jeliko\u017e o Fluent Bit se obvykle sly\u0161\u00ed m\u00e9n\u011b ne\u017e o Logstashi, pod\u00edvejme se na n\u011bj trochu podrobn\u011bji. Fluent Bit m\u016f\u017eeme logicky rozd\u011blit na 6 modul\u016f, na n\u011bkter\u00e9 moduly je mo\u017en\u00e9 p\u0159ipojit pluginy, kter\u00e9 roz\u0161i\u0159uj\u00ed mo\u017enosti Fluent Bit.<\/p>\n

<\/p>\n

\"Protokolov\u00e1n\u00ed<\/p>\n

<\/p>\n

Modul Input<\/strong> sb\u00edr\u00e1 logy z soubor\u016f, slu\u017eeb systemd a dokonce i z tcp-socket (sta\u010d\u00ed zadat endpoint a Fluent Bit za\u010dne tam chodit). T\u011bchto mo\u017enost\u00ed je dostatek k tomu, aby sb\u00edral logy jak ze syst\u00e9mu, tak z kontejner\u016f.<\/p>\n

<\/p>\n

V produkci nej\u010dast\u011bji pou\u017e\u00edv\u00e1me pluginy tail<\/a><\/noindex> (m\u016f\u017eete ho nasm\u011brovat na slo\u017eku s logy) a systemd<\/a><\/noindex> (m\u016f\u017eete mu \u0159\u00edct, z jak\u00fdch slu\u017eeb sb\u00edrat logy).<\/p>\n

<\/p>\n

Modul Parser<\/strong> p\u0159etv\u00e1\u0159\u00ed logy do jednotn\u00e9ho form\u00e1tu. Ve v\u00fdchoz\u00edm nastaven\u00ed logy Nginx jsou \u0159et\u011bzcem. S pomoc\u00ed pluginu lze tento \u0159et\u011bzec p\u0159etvo\u0159it na JSON: nastavit pole a jejich hodnoty. S JSON se mnohem l\u00e9pe pracuje ne\u017e s textov\u00fdmi logy, proto\u017ee nab\u00edz\u00ed flexibiln\u011bj\u0161\u00ed mo\u017enosti t\u0159\u00edd\u011bn\u00ed.<\/p>\n

<\/p>\n

Modul Filter<\/strong>. Na t\u00e9to \u00farovni se odfiltruj\u00ed nepot\u0159ebn\u00e9 logy. Nap\u0159\u00edklad do \u00falo\u017ei\u0161t\u011b se pos\u00edlaj\u00ed pouze logy se stavem \"warning\" nebo s ur\u010dit\u00fdmi \u0161t\u00edtky. Vyselektovan\u00e9 logy se dost\u00e1vaj\u00ed do vyrovn\u00e1vac\u00ed pam\u011bti.<\/p>\n

<\/p>\n

Modul Buffer<\/strong>. Fluent Bit m\u00e1 dva typy vyrovn\u00e1vac\u00edch pam\u011bt\u00ed: pam\u011b\u0165ovou a diskovou. Vyrovn\u00e1vac\u00ed pam\u011b\u0165 je do\u010dasn\u00e9 \u00falo\u017ei\u0161t\u011b log\u016f, kter\u00e9 je pot\u0159ebn\u00e9 v p\u0159\u00edpad\u011b chyb nebo selh\u00e1n\u00ed. V\u0161ichni cht\u011bj\u00ed \u0161et\u0159it na RAM, a proto se obvykle vol\u00ed diskov\u00e1 vyrovn\u00e1vac\u00ed pam\u011b\u0165. Je v\u0161ak t\u0159eba m\u00edt na pam\u011bti, \u017ee p\u0159ed ulo\u017een\u00edm na disk se logy stejn\u011b ukl\u00e1daj\u00ed do pam\u011bti.<\/p>\n

<\/p>\n

Modul Routing\/Output<\/strong> obsahuje pravidla a adresy pro odes\u00edl\u00e1n\u00ed log\u016f. Jak ji\u017e bylo zm\u00edn\u011bno, logy lze odes\u00edlat do Elasticsearch, PostgreSQL nebo nap\u0159\u00edklad do Kafky. <\/p>\n

<\/p>\n

Zaj\u00edmav\u00e9 je, \u017ee z Fluent Bit lze logy odes\u00edlat do Fluentd. Jeliko\u017e je prvn\u00ed leh\u010d\u00ed a m\u00e9n\u011b funk\u010dn\u00ed, lze p\u0159es n\u011bj sb\u00edrat logy a odes\u00edlat je do Fluentd, kde je mo\u017en\u00e9 je pomoc\u00ed dal\u0161\u00edch plugin\u016f d\u00e1le zpracov\u00e1vat a odes\u00edlat do \u00falo\u017ei\u0161\u0165.<\/p>\n

<\/p>\n

Pokud pl\u00e1nujete pou\u017e\u00edvat Elasticsearch\u2026<\/p>\n

Na z\u00e1v\u011br dva tipy pro ty, kte\u0159\u00ed pl\u00e1nuj\u00ed pou\u017e\u00edvat Elasticsearch jako \u00falo\u017ei\u0161t\u011b log\u016f v produk\u010dn\u00edm prost\u0159ed\u00ed. <\/p>\n

    \n
  1. Nastavte upozorn\u011bn\u00ed pomoc\u00ed ElastAlert<\/a><\/noindex>. Tento program vy\u010dle\u0148uje z celkov\u00e9ho toku log\u016f d\u016fle\u017eit\u00e9 zpr\u00e1vy a vytv\u00e1\u0159\u00ed na jejich z\u00e1klad\u011b upozorn\u011bn\u00ed do e-mailu nebo jin\u00e9ho kan\u00e1lu. Bohu\u017eel, ned\u00e1vno se objevila smutn\u00e1 zpr\u00e1va o tom, \u017ee projekt m\u016f\u017ee brzy p\u0159estat existovat<\/a><\/noindex><\/em>.<\/li>\n
  2. Rotujte logy pomoc\u00ed aplikace Curator<\/a><\/noindex> nebo vol\u00e1n\u00edm API Elasticsearch. Elastic v sou\u010dasnosti d\u011bl\u00e1 v\u00fdznamn\u00e9 kroky k \u0159\u00edzen\u00ed \u017eivotn\u00edho cyklu index\u016f bez pou\u017eit\u00ed extern\u00edch n\u00e1stroj\u016f. Obecn\u011b nen\u00ed smysl uchov\u00e1vat logy dlouho: m\u00e1lokter\u00fd log bude pot\u0159ebn\u00fd po dvou t\u00fddnech \u2014 pokud je opravdu kritick\u00fd, za dva t\u00fddny bude ur\u010dit\u011b vy\u0159e\u0161en. V krajn\u00edm p\u0159\u00edpad\u011b lze star\u00e9 logy archivovat a odes\u00edlat n\u011bkam na dlouhodob\u00e9 uchov\u00e1n\u00ed. Sly\u0161el jsem o zvl\u00e1\u0161tn\u00edch log\u00e1ch, kter\u00e9 je dle z\u00e1kona t\u0159eba uchov\u00e1vat a\u017e 5 let. Osobn\u011b jsem se s nimi nesetkal, ale takov\u00e9 informace bych neprov\u00e1zel s b\u011b\u017en\u00fdmi logy a mo\u017en\u00e1 bych je dokonce uchov\u00e1val odd\u011blen\u011b.<\/li>\n<\/ol>\n

    \n<\/p><\/blockquote>\n

    Pokra\u010dov\u00e1n\u00ed p\u0159\u00ed\u0161t\u011b\u2026<\/p>\n

    <\/p>\n

    Autor: Marcel Ibrajev, certifikovan\u00fd administr\u00e1tor Kubernetes, praktikuj\u00edc\u00ed in\u017een\u00fdr ve spole\u010dnosti Southbridge<\/a><\/noindex>, \u0159e\u010dn\u00edk a tv\u016frce kurz\u016f Sl\u0117rm<\/a><\/noindex>.<\/em><\/p>\n

    Zdroj: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

    \u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana). \u041c\u0430\u0442\u0435\u0440\u0438\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u0432\u044b\u0436\u0438\u043c\u043a\u0430 \u0438\u0437 \u043e\u0442\u043a\u0440\u044b\u0442\u043e\u0439 \u043b\u0435\u043a\u0446\u0438\u0438 \u0448\u043a\u043e\u043b\u044b \u00ab\u0421\u043b\u0451\u0440\u043c\u00bb. \u0415\u0441\u043b\u0438 \u0435\u0441\u0442\u044c \u0436\u0435\u043b\u0430\u043d\u0438\u0435 \u0438 \u0442\u0435\u043c \u0431\u043e\u043b\u0435\u0435 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u0430\u044f \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0439\u0442\u0438 \u043f\u043e\u043b\u043d\u043e\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u2014 \u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u0439\u0442\u0435\u0441\u044c \u043d\u0430 \u043a\u0443\u0440\u0441 \u043f\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":93295,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-93294","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Logov\u00e1n\u00ed v Kubernetes: jak sb\u00edrat, uchov\u00e1vat, analyzovat a zpracov\u00e1vat logy | ProHoster","description":"Probereme z\u00e1klady logov\u00e1n\u00ed v Dockera a Kubernetes a pot\u00e9 se pod\u00edv\u00e1me na dva n\u00e1stroje, kter\u00e9 lze s klidem pou\u017e\u00edvat v produk\u010dn\u00edm prost\u0159ed\u00ed: Grafana Loki a EFK stack (Elasticsearch + Fluent Bit + Kibana)..","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0432 Kubernetes: \u043a\u0430\u043a \u0441\u043e\u0431\u0438\u0440\u0430\u0442\u044c, \u0445\u0440\u0430\u043d\u0438\u0442\u044c, \u043f\u0430\u0440\u0441\u0438\u0442\u044c \u0438 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043b\u043e\u0433\u0438 | ProHoster","og:description":"\u0420\u0430\u0437\u0431\u0435\u0440\u0451\u043c \u043e\u0441\u043d\u043e\u0432\u044b \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432 Docker \u0438 Kubernetes, \u0430 \u0437\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0434\u0432\u0430 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0436\u043d\u043e \u0441\u043c\u0435\u043b\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u0435\u043d\u0435: Grafana Loki \u0438 \u0441\u0442\u0435\u043a EFK (Elasticsearch + Fluent Bit + Kibana)..","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/logirovanie-v-kubernetes-kak-sobirat-hranit-parsit-i-obrabatyvat-logi","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-05T05:42:58+00:00","article:modified_time":"2020-09-05T05:42:58+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"93294","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:50:29","updated":"2022-09-27 18:43:48","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/93294","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=93294"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/93294\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/93295"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=93294"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=93294"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=93294"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}