{"id":53590,"date":"2019-12-05T00:00:00","date_gmt":"2019-12-04T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/kak-my-v-tsian-ukroshhali-terabajty-logov"},"modified":"2020-02-18T14:01:30","modified_gmt":"2020-02-18T11:01:30","slug":"kak-my-v-tsian-ukroshhali-terabajty-logov","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-my-v-tsian-ukroshhali-terabajty-logov","title":{"rendered":"Jak jsme v CIAN zkrocovali terabajty log\u016f","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/4efc58ba81fcaa7c8481bbeaa6bb083e.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAhoj v\u0161ichni, jmenuji se Alexander, pracuji v CIAN jako in\u017een\u00fdr a zab\u00fdv\u00e1m se syst\u00e9mov\u00fdm administrativou a automatizac\u00ed infrastrukturn\u00edch proces\u016f. V koment\u00e1\u0159\u00edch k jednomu z p\u0159edchoz\u00edch \u010dl\u00e1nk\u016f n\u00e1s po\u017e\u00e1dali, abychom se pod\u011blili o to, odkud bereme 4 TB log\u016f denn\u011b a co s nimi d\u011bl\u00e1me. Ano, log\u016f m\u00e1me hodn\u011b, a pro jejich zpracov\u00e1n\u00ed byl vytvo\u0159en samostatn\u00fd infrastrukturn\u00ed klastr, kter\u00fd n\u00e1m umo\u017e\u0148uje rychle \u0159e\u0161it probl\u00e9my. V tomto \u010dl\u00e1nku se pod\u011bl\u00edm o to, jak jsme za rok p\u0159izp\u016fsobili jeho provoz pro st\u00e1l\u00e9 rostouc\u00ed mno\u017estv\u00ed dat.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>Jak jsme za\u010d\u00ednali<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/9b0919df70114d4ebb559c93ec012a4d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nV posledn\u00edch n\u011bkolika letech rostla z\u00e1t\u011b\u017e na cian.ru velmi rychle, a ke t\u0159et\u00edmu \u010dtvrtlet\u00ed 2018 dos\u00e1hla n\u00e1v\u0161t\u011bvnost webu 11,2 milionu unik\u00e1tn\u00edch u\u017eivatel\u016f m\u011bs\u00ed\u010dn\u011b. V t\u00e9 dob\u011b jsme v kritick\u00fdch okam\u017eic\u00edch ztr\u00e1celi a\u017e 40% log\u016f, co\u017e n\u00e1m br\u00e1nilo rychle \u0159e\u0161it incidenty a str\u00e1vili jsme tak hodn\u011b \u010dasu a sil jejich \u0159e\u0161en\u00edm. Tak\u00e9 jsme \u010dasto nemohli naj\u00edt p\u0159\u00ed\u010dinu probl\u00e9mu a ta se opakovala po n\u011bjak\u00e9m \u010dase. To byl chaos, se kter\u00fdm bylo t\u0159eba n\u011bco ud\u011blat.<\/p>\n<p>V t\u00e9 dob\u011b jsme pro ukl\u00e1d\u00e1n\u00ed log\u016f pou\u017e\u00edvali klastr deseti datov\u00fdch uzl\u016f s ElasticSearch verz\u00ed 5.5.2 s typick\u00fdmi nastaven\u00edmi index\u016f. Bylo to nasazeno p\u0159ed v\u00edce ne\u017e rokem jako popul\u00e1rn\u00ed a dostupn\u00e9 \u0159e\u0161en\u00ed: tehdy byl tok log\u016f je\u0161t\u011b relativn\u011b mal\u00fd, nem\u011blo smysl vym\u00fd\u0161let nestandardn\u00ed konfigurace.\u00a0<\/p>\n<p>Zpracov\u00e1n\u00ed p\u0159ich\u00e1zej\u00edc\u00edch log\u016f zaji\u0161\u0165oval Logstash na r\u016fzn\u00fdch portech na p\u011bti koordin\u00e1torech ElasticSearch. Jeden index, bez ohledu na velikost, se skl\u00e1dal z p\u011bti shard\u016f. Byla zavedena hodinov\u00e1 a denn\u00ed rotace, tak\u017ee ka\u017ed\u00fd hodinu se v klastru objevovalo p\u0159ibli\u017en\u011b 100 nov\u00fdch shard\u016f. Dokud log\u016f nebylo p\u0159\u00edli\u0161 mnoho, klastr to zvl\u00e1dal a nikdo se na jeho nastaven\u00ed v\u00edce nezam\u011b\u0159oval.\u00a0<\/p>\n<h3>Probl\u00e9my rychl\u00e9ho r\u016fstu<\/h3>\n<p>\nObjem generovan\u00fdch log\u016f rostl velmi rychle, proto\u017ee se na sebe p\u0159ekr\u00fdvaly dva procesy. Na jedn\u00e9 stran\u011b p\u0159ib\u00fdvalo u\u017eivatel\u016f slu\u017eby. Na druh\u00e9 stran\u011b jsme za\u010dali aktivn\u011b p\u0159ech\u00e1zet na mikroslu\u017ebovou architekturu, rozd\u011bluj\u00edc\u00ed na\u0161e star\u00e9 monolity na C# a Python. N\u011bkolik des\u00edtek nov\u00fdch mikroslu\u017eeb, nahrazuj\u00edc\u00edch \u010d\u00e1sti monolitu, generovalo v\u00fdrazn\u011b v\u00edce log\u016f pro infrastrukturn\u00ed klastr.\u00a0<\/p>\n<p>Pr\u00e1v\u011b \u0161k\u00e1lov\u00e1n\u00ed n\u00e1s dovedlo k tomu, \u017ee se cluster stal prakticky ne\u0159\u00edditeln\u00fdm. Kdy\u017e za\u010daly logy p\u0159ich\u00e1zet rychlost\u00ed 20 tis\u00edc zpr\u00e1v za sekundu, \u010dast\u00e1 zbyte\u010dn\u00e1 rotace zv\u00fd\u0161ila po\u010det shard\u016f na 6 tis\u00edc, a na jeden uzel p\u0159ipadalo v\u00edce ne\u017e 600 shard\u016f.\u00a0<\/p>\n<p>To vedlo k probl\u00e9m\u016fm s alokac\u00ed opera\u010dn\u00ed pam\u011bti, a p\u0159i p\u00e1du uzlu za\u010dal sou\u010dasn\u00fd p\u0159esun v\u0161ech shard\u016f, co\u017e zn\u00e1sobilo provoz a zat\u011b\u017eovalo ostatn\u00ed uzly, co\u017e prakticky znemo\u017enilo zapisov\u00e1n\u00ed dat do clusteru. A v t\u00e9to dob\u011b jsme z\u016fstali bez log\u016f. A p\u0159i probl\u00e9mu s <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/cs\/server\/dts-prohoster\/\"   title=\"serverem\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"2986\">serverem<\/a> jsme v podstat\u011b ztr\u00e1celi 1\/10 clusteru. Velk\u00e9 mno\u017estv\u00ed mal\u00fdch index\u016f situaci d\u00e1le komplikovalo.<\/p>\n<p>Bez log\u016f jsme nerozum\u011bli d\u016fvod\u016fm incidentu a mohli jsme se d\u0159\u00edve \u010di pozd\u011bji dostat do stejn\u00fdch probl\u00e9m\u016f znovu, co\u017e bylo pro na\u0161i t\u00fdmovou filozofii nep\u0159ijateln\u00e9, proto\u017ee v\u0161echny mechanismy na\u0161\u00ed pr\u00e1ce jsou zam\u011b\u0159en\u00e9 p\u0159esn\u011b na opak \u2013 nikdy neopakovat stejn\u00e9 probl\u00e9my. K tomu jsme pot\u0159ebovali pln\u00fd objem log\u016f a jejich praktickou dod\u00e1vku v re\u00e1ln\u00e9m \u010dase, proto\u017ee t\u00fdm dozoruj\u00edc\u00edch in\u017een\u00fdr\u016f sledoval upozorn\u011bn\u00ed nejen z metrik, ale i z log\u016f. Pro pochopen\u00ed rozsahu probl\u00e9mu \u2013 v t\u00e9 dob\u011b byl celkov\u00fd objem log\u016f p\u0159ibli\u017en\u011b 2 TB za den.\u00a0<\/p>\n<p>Stanovili jsme si \u00fakol \u2013 zcela vylou\u010dit ztr\u00e1tu log\u016f a zkr\u00e1tit dobu jejich dod\u00e1vky do clusteru ELK na maxim\u00e1ln\u011b 15 minut b\u011bhem nouzov\u00fdch situac\u00ed (na tento \u00fadaj jsme pozd\u011bji spol\u00e9hali jako na intern\u00ed KPI).<\/p>\n<h3>Nov\u00fd mechanismus rotace a hot-warm uzly<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/aca8d792d8b2f002475554d05e8d5451.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTransformaci clusteru jsme za\u010dali aktualizac\u00ed verze ElasticSearch z 5.5.2 na 6.4.3. Op\u011bt n\u00e1m selhal cluster 5. verze, a rozhodli jsme se ho vypnout a zcela aktualizovat \u2013 logy stejn\u011b nejsou. Tak\u017ee tento p\u0159echod jsme uskute\u010dnili za pouh\u00e9 p\u00e1r hodin.<\/p>\n<p>Nejv\u011bt\u0161\u00ed zm\u011bnou v t\u00e9to f\u00e1zi bylo zaveden\u00ed Apache Kafka na t\u0159ech nodech s koordin\u00e1torem jako mezistupn\u011bm. Zpr\u00e1vov\u00fd broker n\u00e1s zbavil ztr\u00e1ty log\u016f b\u011bhem probl\u00e9m\u016f s ElasticSearch. Z\u00e1rove\u0148 jsme p\u0159idali 2 nody do klastru a p\u0159e\u0161li na hot-warm architekturu se t\u0159emi \u201ehork\u00fdmi\u201c nody, kter\u00e9 byly um\u00edst\u011bny v r\u016fzn\u00fdch sk\u0159\u00edn\u00edch v datov\u00e9m centru. Na n\u011b jsme podle masky p\u0159esm\u011brovali logy, kter\u00e9 nelze za \u017e\u00e1dn\u00fdch okolnost\u00ed ztratit \u2014 nginx a tak\u00e9 logy chyb aplikac\u00ed. Na ostatn\u00ed nody odch\u00e1zely minoritn\u00ed logy \u2014 debug, warning atd., a tak\u00e9 po 24 hodin\u00e1ch byly p\u0159esouv\u00e1ny \u201ed\u016fle\u017eit\u00e9\u201c logy z \u201ehork\u00fdch\u201c nod\u016f.<\/p>\n<p>Abychom nezvy\u0161ovali po\u010det mal\u00fdch index\u016f, p\u0159e\u0161li jsme z rotace podle \u010dasu na mechanismus rollover. Na f\u00f3rech bylo mnoho informac\u00ed o tom, \u017ee rotace podle velikosti indexu je velmi nespolehliv\u00e1, proto jsme se rozhodli pou\u017e\u00edt rotaci podle po\u010dtu dokument\u016f v indexu. Analyzovali jsme ka\u017ed\u00fd index a zaznamenali jsme po\u010det dokument\u016f, po jeho\u017e dosa\u017een\u00ed by m\u011bla rotace nastat. T\u00edmto zp\u016fsobem jsme dos\u00e1hli optim\u00e1ln\u00ed velikosti shardu \u2014 maxim\u00e1ln\u011b 50 GB.\u00a0<\/p>\n<h3>Optimalizace klastru<\/h3>\n<p>\n<img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/89503fc8aa92060e750bdc9cdd63a7e3.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNicm\u00e9n\u011b jsme se zcela nezbavili probl\u00e9m\u016f. Bohu\u017eel se po\u0159\u00e1d objevovaly mal\u00e9 indexy: nedosahovaly stanoven\u00e9ho objemu, nebyly rotov\u00e1ny a byly odstra\u0148ov\u00e1ny glob\u00e1ln\u00edm vy\u010di\u0161t\u011bn\u00edm index\u016f star\u0161\u00edch t\u0159\u00ed dn\u016f, proto\u017ee jsme odstranili rotaci podle data. To vedlo ke ztr\u00e1t\u011b dat, proto\u017ee index z klastru zcela zmizel, a pokus o z\u00e1pis do neexistuj\u00edc\u00edho indexu naru\u0161oval logiku curator, kterou jsme pou\u017e\u00edvali pro spr\u00e1vu. Alias pro z\u00e1pis se transformoval na index a naru\u0161oval logiku rollover, co\u017e vedlo k nekontrolovan\u00e9mu r\u016fstu n\u011bkter\u00fdch index\u016f a\u017e na 600 GB.\u00a0<\/p>\n<p>Nap\u0159\u00edklad pro konfigura\u010dn\u00ed soubor rotace:<\/p>\n<pre><code class=\"plaintext\">curator-elk-rollover.yaml\n\n---\nactions:\n\u00a0\u00a01:\n\u00a0\u00a0\u00a0\u00a0action: rollover\n\u00a0\u00a0\u00a0\u00a0options:\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0name: \"nginx_write\"\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0conditions:\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0max_docs: 100000000\n\u00a0\u00a02:\n\u00a0\u00a0\u00a0\u00a0action: rollover\n\u00a0\u00a0\u00a0\u00a0options:\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0name: \"python_error_write\"\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0conditions:\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0max_docs: 10000000\n<\/code><\/pre>\n<p>P\u0159i absenci rollover alias se objevila chyba:<\/p>\n<pre><code class=\"plaintext\">ERROR \u00a0 \u00a0 alias \"nginx_write\" not found.\nERROR \u00a0 \u00a0 Failed to complete action: rollover.\u00a0 : Unable to perform index rollover with alias \"nginx_write\".\n<\/code><\/pre>\n<p>\u0158e\u0161en\u00ed tohoto probl\u00e9mu jsme ponechali na dal\u0161\u00ed iteraci a zab\u00fdvali se jinou ot\u00e1zkou: p\u0159e\u0161li jsme na pull logiku pr\u00e1ce Logstash, kter\u00fd se zab\u00fdv\u00e1 zpracov\u00e1n\u00edm p\u0159\u00edchoz\u00edch log\u016f (odstra\u0148ov\u00e1n\u00edm nadbyte\u010dn\u00fdch informac\u00ed a obohacov\u00e1n\u00edm). Um\u00edstili jsme jej do dockera, kter\u00fd spou\u0161t\u00edme p\u0159es docker-compose, a tam tak\u00e9 um\u00edstili logstash-exporter, kter\u00fd poskytuje metriky do Prometheusu pro aktu\u00e1ln\u00ed monitoring toku log\u016f. T\u00edm jsme si umo\u017enili plynule m\u011bnit po\u010det instanc\u00ed logstash, kter\u00e9 zpracov\u00e1valy jednotliv\u00e9 typy log\u016f.<\/p>\n<p>Zat\u00edmco jsme zdokonalovali cluster, n\u00e1v\u0161t\u011bvnost cian.ru vzrostla na 12,8 milionu unik\u00e1tn\u00edch u\u017eivatel\u016f za m\u011bs\u00edc. V d\u016fsledku toho se uk\u00e1zalo, \u017ee na\u0161e transformace m\u00edrn\u011b nezaost\u00e1valy za zm\u011bnami na produkci, a my jsme se setkali s t\u00edm, \u017ee 'tepl\u00e9' uzly nezvl\u00e1daly z\u00e1t\u011b\u017e a zpomalovaly celou dod\u00e1vku log\u016f. 'Hork\u00e1' data jsme z\u00edsk\u00e1vali bez probl\u00e9m\u016f, ale do dod\u00e1vky ostatn\u00edch bylo nutn\u00e9 zasahovat a prov\u00e1d\u011bt ru\u010dn\u00ed rollover, aby se rovnom\u011brn\u011b rozd\u011blily indexy.\u00a0<\/p>\n<p>P\u0159itom \u0161k\u00e1lov\u00e1n\u00ed a zm\u011bna nastaven\u00ed instanc\u00ed logstash v clusteru byla komplikov\u00e1na t\u00edm, \u017ee to byl lok\u00e1ln\u00ed docker-compose, a v\u0161echny akce se prov\u00e1d\u011bly ru\u010dn\u011b (pro p\u0159id\u00e1n\u00ed nov\u00fdch konc\u016f bylo nutn\u00e9 ru\u010dn\u011b proj\u00edt v\u0161echny servery a v\u0161ude prov\u00e9st docker-compose up -d).<\/p>\n<h3>P\u0159eorganizov\u00e1n\u00ed log\u016f<\/h3>\n<p>\nV z\u00e1\u0159\u00ed tohoto roku jsme st\u00e1le pokra\u010dovali v d\u011blen\u00ed monolitu, z\u00e1t\u011b\u017e na cluster rostla a tok log\u016f se bl\u00ed\u017eil k 30 tis\u00edc\u016fm zpr\u00e1v za sekundu.\u00a0<\/p>\n<p><img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/00f00af4ad5fa0080a19f36d0ac4b19e.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDal\u0161\u00ed iteraci jsme za\u010dali aktualizac\u00ed hardwaru. Z p\u011bti koordin\u00e1tor\u016f jsme p\u0159e\u0161li na t\u0159i, vym\u011bnili jsme datov\u00e9 uzly a z\u00edskali jsme v\u00fdhodu jak finan\u010dn\u011b, tak co do objemu \u00falo\u017ei\u0161t\u011b. Pro uzly pou\u017e\u00edv\u00e1me dv\u011b konfigurace:\u00a0<\/p>\n<ul>\n<li>Pro 'hork\u00e9' uzly: E3-1270 v6 \/ 960Gb SSD \/ 32 Gb x 3 x 2 (3 pro Hot1 a 3 pro Hot2).\n<\/li>\n<li>Pro 'tepl\u00e9' uzly: E3-1230 v6 \/ 4Tb SSD \/ 32 Gb x 4.\n<\/li>\n<\/ul>\n<p>\nNa t\u00e9to iteraci jsme p\u0159evedli index s p\u0159\u00edstupov\u00fdmi logy mikroservis\u016f, kter\u00fd zab\u00edr\u00e1 stejn\u00e9 m\u00edsto jako logy frontov\u00fdch nginx, do druh\u00e9 skupiny ze t\u0159\u00ed 'hork\u00fdch' uzl\u016f. Data na 'hork\u00fdch' uzlech nyn\u00ed uchov\u00e1v\u00e1me 20 hodin a pot\u00e9 je p\u0159ev\u00e1d\u00edme na 'tepl\u00e9' k ostatn\u00edm log\u016fm.\u00a0<\/p>\n<p>Probl\u00e9m s miz\u00edc\u00edmi mal\u00fdmi indexy jsme vy\u0159e\u0161ili p\u0159enastaven\u00edm jejich rotace. Nyn\u00ed jsou indexy rotov\u00e1ny ka\u017ed\u00fdch 23 hodin, i kdy\u017e obsahuj\u00ed m\u00e1lo dat. To m\u00edrn\u011b zv\u00fd\u0161ilo po\u010det shard\u016f (je jich asi 800), ale z pohledu v\u00fdkonu clusteru je to p\u0159ijateln\u00e9.\u00a0<\/p>\n<p>V d\u016fsledku toho m\u00e1 cluster \u0161est \"\u017ehav\u00fdch\" a pouze \u010dty\u0159i \"tepl\u00e9\" uzly. To zp\u016fsobuje m\u00edrn\u00e9 zpo\u017ed\u011bn\u00ed p\u0159i dotazech na dlouh\u00e9 \u010dasov\u00e9 intervaly, ale zv\u00fd\u0161en\u00ed po\u010dtu uzl\u016f v budoucnu tento probl\u00e9m vy\u0159e\u0161\u00ed.<\/p>\n<p>V t\u00e9to iteraci jsme tak\u00e9 opravili probl\u00e9m s absenc\u00ed polovi\u010dn\u011b automatick\u00e9ho \u0161k\u00e1lov\u00e1n\u00ed. K tomu jsme nasadili infrastrukturn\u00ed cluster Nomad \u2013 obdobn\u00fd tomu, kter\u00fd ji\u017e m\u00e1me na produkci. Zat\u00edm se po\u010det Logstash automaticky nem\u011bn\u00ed podle z\u00e1t\u011b\u017ee, ale k tomu se tak\u00e9 dostaneme.<\/p>\n<p><img decoding=\"async\" alt=\"Jak jsme v CIAN zkrocovali terabajty log\u016f\" src=\"\/wp-content\/uploads\/2019\/12\/c06905f266238990d989bf2d7c84be3d.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Pl\u00e1ny do budoucna<\/h3>\n<p>\nRealizovan\u00e1 konfigurace se skv\u011ble \u0161k\u00e1luje a nyn\u00ed uchov\u00e1v\u00e1me 13,3 TB dat \u2013 v\u0161echny logy za 4 dny, co\u017e je nezbytn\u00e9 pro okam\u017eit\u00e9 anal\u00fdzy alert\u016f. \u010c\u00e1st log\u016f transformujeme na metriky, kter\u00e9 ukl\u00e1d\u00e1me do Graphite. Abychom usnadnili pr\u00e1ci in\u017een\u00fdr\u016fm, m\u00e1me metriky pro infrastrukturn\u00ed cluster a skripty pro polovi\u010dn\u00ed automatick\u00e9 opravy typick\u00fdch probl\u00e9m\u016f. Po pl\u00e1novan\u00e9m zv\u00fd\u0161en\u00ed po\u010dtu datov\u00fdch uzl\u016f v p\u0159\u00ed\u0161t\u00edm roce p\u0159ejdeme na ukl\u00e1d\u00e1n\u00ed dat z 4 na 7 dn\u00ed. To bude dostate\u010dn\u00e9 pro operativn\u00ed pr\u00e1ci, proto\u017ee se v\u017edy sna\u017e\u00edme vy\u0161et\u0159ovat incidenty co nejd\u0159\u00edve, a pro dlouhodob\u00e1 vy\u0161et\u0159ov\u00e1n\u00ed jsou k dispozici telemetry.\u00a0<\/p>\n<p>V \u0159\u00edjnu 2019 vzrostla n\u00e1v\u0161t\u011bvnost cian.ru ji\u017e na 15,3 milionu unik\u00e1tn\u00edch u\u017eivatel\u016f za m\u011bs\u00edc. To byla v\u00e1\u017en\u00e1 zkou\u0161ka architektonick\u00e9ho \u0159e\u0161en\u00ed dod\u00e1vky log\u016f.\u00a0<\/p>\n<p>Nyn\u00ed se p\u0159ipravujeme na aktualizaci ElasticSearch na verzi 7. Pravda, pro to budeme muset aktualizovat mapping mnoha index\u016f v ElasticSearch, proto\u017ee p\u0159e\u0161li z verze 5.5 a byly vyhl\u00e1\u0161eny jako zastaral\u00e9 ve verzi 6 (ve verzi 7 ji\u017e neexistuj\u00ed). A to znamen\u00e1, \u017ee b\u011bhem aktualizace bude jist\u011b n\u011bjak\u00fd mimo\u0159\u00e1dn\u00fd p\u0159\u00edpad, kter\u00fd n\u00e1s na \u010das ponech\u00e1 bez log\u016f. Z verze 7 nejv\u00edce o\u010dek\u00e1v\u00e1me Kibanu s vylep\u0161en\u00fdm rozhran\u00edm a nov\u00fdmi filtry.\u00a0<\/p>\n<p>Hlavn\u00edho c\u00edle jsme dos\u00e1hli: p\u0159estali jsme ztr\u00e1cet logy a zkr\u00e1tili jsme dobu v\u00fdpadku infrastruktury z 2-3 p\u00e1d\u016f t\u00fddn\u011b na p\u00e1r hodin servisn\u00edch prac\u00ed m\u011bs\u00ed\u010dn\u011b. Cel\u00e1 tato pr\u00e1ce na produkci je t\u00e9m\u011b\u0159 neviditeln\u00e1. Nyn\u00ed v\u0161ak m\u016f\u017eeme p\u0159esn\u011b ur\u010dit, co se d\u011bje s na\u0161\u00edm servisem, m\u016f\u017eeme to rychle prov\u00e1d\u011bt v klidov\u00e9m re\u017eimu a nemus\u00edme se ob\u00e1vat, \u017ee logy zmiz\u00ed. Obecn\u011b jsme spokojeni, \u0161\u0165astn\u00ed a p\u0159ipravujeme se na nov\u00e9 \u00fasp\u011bchy, o kter\u00fdch se brzy pod\u011bl\u00edme.<br \/>\n<br \/>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/cian\/blog\/478564\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0432 \u0426\u0418\u0410\u041d \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043e\u043c \u0438 \u0437\u0430\u043d\u0438\u043c\u0430\u044e\u0441\u044c \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u043c \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0438 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0435\u0439 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0445 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0432. \u0412 \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u044f\u0445 \u043a \u043e\u0434\u043d\u043e\u0439 \u0438\u0437 \u043f\u0440\u043e\u0448\u043b\u044b\u0445 \u0441\u0442\u0430\u0442\u0435\u0439 \u043d\u0430\u0441 \u043f\u043e\u043f\u0440\u043e\u0441\u0438\u043b\u0438 \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c, \u043e\u0442\u043a\u0443\u0434\u0430 \u043c\u044b \u0431\u0435\u0440\u0435\u043c 4 \u0422\u0411 \u043b\u043e\u0433\u043e\u0432 \u0432 \u0434\u0435\u043d\u044c \u0438 \u0447\u0442\u043e \u0441 \u043d\u0438\u043c\u0438 \u0434\u0435\u043b\u0430\u0435\u043c. \u0414\u0430, \u043b\u043e\u0433\u043e\u0432 \u0443 \u043d\u0430\u0441 \u043c\u043d\u043e\u0433\u043e, \u0438 \u0434\u043b\u044f \u0438\u0445 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0441\u043e\u0437\u0434\u0430\u043d \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u044b\u0439 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0439 \u043a\u043b\u0430\u0441\u0442\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-53590","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0432 \u0426\u0418\u0410\u041d.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-my-v-tsian-ukroshhali-terabajty-logov\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"cs_CZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u0432 \u0426\u0418\u0410\u041d \u0443\u043a\u0440\u043e\u0449\u0430\u043b\u0438 \u0442\u0435\u0440\u0430\u0431\u0430\u0439\u0442\u044b \u043b\u043e\u0433\u043e\u0432 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0432 \u0426\u0418\u0410\u041d.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-my-v-tsian-ukroshhali-terabajty-logov\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-12-04T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T11:01:30+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Jak jsme v CIAN zkrotili terabajty log\u016f | ProHoster","description":"Ahoj v\u0161ichni, jmenuji se Alexander a pracuji v CIAN.","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-my-v-tsian-ukroshhali-terabajty-logov","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u0432 \u0426\u0418\u0410\u041d \u0443\u043a\u0440\u043e\u0449\u0430\u043b\u0438 \u0442\u0435\u0440\u0430\u0431\u0430\u0439\u0442\u044b \u043b\u043e\u0433\u043e\u0432 | ProHoster","og:description":"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0432 \u0426\u0418\u0410\u041d.","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-my-v-tsian-ukroshhali-terabajty-logov","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-04T21:00:00+00:00","article:modified_time":"2020-02-18T11:01:30+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"53590","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-02-09 22:17:34","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:22:49","updated":"2026-02-09 22:17:34","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/53590","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=53590"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/53590\/revisions"}],"predecessor-version":[{"id":160267,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/53590\/revisions\/160267"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=53590"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=53590"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=53590"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}