{"id":31139,"date":"2019-10-31T21:39:40","date_gmt":"2019-10-31T18:39:40","guid":{"rendered":"https:\/\/prohoster.info\/blog\/monitorim-resursy-klasterov-kubernetes\/"},"modified":"2019-10-31T21:39:40","modified_gmt":"2019-10-31T18:39:40","slug":"monitorim-resursy-klasterov-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitorim-resursy-klasterov-kubernetes","title":{"rendered":"Pratimo resurse Kubernetes klastera","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Pratimo<\/a><\/noindex><\/p>\n

<\/p>\n

Stvorio sam Kube Eagle - Prometheus izvoznik. Ispostavilo se da je to sjajan alat koji poma\u017ee bolje razumjeti resurse malih i srednjih klastera. Na kraju sam u\u0161tedio nekoliko stotina dolara jer sam odabrao prave vrste ma\u0161ina i postavio ograni\u010denja resursa aplikacija prema radnim optere\u0107enjima.<\/p>\n

<\/p>\n

Ispri\u010dat \u0107u o prednostima Kube Eagle<\/a><\/noindex>, ali prvo \u0107u objasniti zbog \u010dega je do\u0161lo do problema i za\u0161to je potrebna kvalitetna monitoringa.<\/p>\n

<\/a><\/noindex><\/p>\n

Upravljao sam nekoliko klastera s 4\u201350 nodova. U svakom klasteru - do 200 mikroservisa i aplikacija. Da bismo u\u010dinkovitije koristili dostupno hardver, ve\u0107ina implementacija je bila postavljena s burstable RAM-om i resursima CPU-a. Tako podovi mogu uzeti dostupne resurse kada je potrebno, a da pritom ne ometaju druge aplikacije na toj nodi. Zar to nije sjajno?<\/p>\n

<\/p>\n

Iako je klaster relativno malo tro\u0161io CPU (8%) i RAM (40%), stalno smo se suo\u010davali s problemima vezanima za izbacivanje podova kada su poku\u0161avali dodijeliti vi\u0161e memorije nego \u0161to je dostupno na nodi. Tada smo imali samo jednu plo\u010du za pra\u0107enje resursa Kubernetes-a. Ovako je izgledala:<\/p>\n

<\/p>\n

\"Pratimo<\/a><\/noindex>
\nGrafana plo\u010da samo s metrikama cAdvisor<\/em><\/p>\n

<\/p>\n

S ovom plo\u010dom, nodovi koji tro\u0161e mnogo memorije i CPU-a, jasno se vide. Problem je razumjeti uzrok. Da bi podovi ostali na svom mjestu, moglo bi se naravno postaviti garantirane resurse na svim podovima (zahtijevani resursi su jednaki limitima). No, to nije najsmar\u0161nije iskori\u0161tavanje hardvera. U klasteru je bilo nekoliko stotina gigabajta memorije, a neki nodovi su bili po\u0161te\u0111eni, dok su drugi imali u rezervi po 4\u201310 GB.<\/p>\n

<\/p>\n

Dakle, Kubernetes planer je nerazmjerno raspodijelio radna optere\u0107enja izme\u0111u dostupnih resursa. Kubernetes planer uzima u obzir razli\u010dite konfiguracije: pravila affinity, taints i tolerations, selektore nodova, koji mogu ograni\u010diti dostupne nodove. No, u mom slu\u010daju nije bilo ni\u0161ta od toga, a podovi su se planirali na temelju zahtijevanih resursa na svakoj nodi.<\/p>\n

<\/p>\n

Za pod se odabire noda koja ima najvi\u0161e slobodnih resursa i koja ispunjava uvjete zahtjeva. Imali smo situaciju da zahtijevani resursi na nodovima ne odgovaraju stvarnoj upotrebi, i tu je Kube Eagle do\u0161ao u pomo\u0107 sa svojim mogu\u0107nostima pra\u0107enja resursa.<\/p>\n

<\/p>\n

Gotovo svi moji Kubernetes klasteri bili su pra\u0107eni samo s Node exporter<\/a><\/noindex> i Kube State Metrics<\/a><\/noindex>. Node Exporter pru\u017ea statistiku o ulazu-izlazu i upotrebi diskova, CPU-a i RAM-a, dok Kube State Metrics prikazuje metrike objekata Kubernetes, poput zahtjeva i ograni\u010denja za CPU i memoriju.<\/p>\n

<\/p>\n

Moramo kombinirati metrike kori\u0161tenja s metrima zahtjeva i ograni\u010denja u Grafani, a tada \u0107emo dobiti sve informacije o problemu. Zvuci jednostavno, ali u ovim dvjema alatima oznake se nazivaju razli\u010dito, a neke metrike uop\u0107e nemaju oznake metapodataka. Kube Eagle sve to radi automatski, a panel izgleda ovako:<\/p>\n

<\/p>\n

\"Pratimo<\/a><\/noindex><\/p>\n

<\/p>\n

\"Pratimo<\/a><\/noindex>
\nKube Eagle nadzorna plo\u010da<\/a><\/noindex><\/em><\/p>\n

<\/p>\n

Uspjeli smo rije\u0161iti mnoge probleme s resursima i sa\u010duvati opremu:<\/p>\n

<\/p>\n

    \n
  1. Neki programeri nisu znali koliko je resursa potrebno mikroservisima (ili jednostavno nisu obra\u0107ali pa\u017enju). Nismo imali na\u010din da prona\u0111emo neispravne zahtjeve za resursima - za to je potrebno znati potro\u0161nju plus zahtjeve i ograni\u010denja. Sada vide metrike Prometheus-a, prate stvarnu upotrebu i prilago\u0111avaju zahtjeve i ograni\u010denja.<\/li>\n
  2. JVM aplikacije uzimaju koliko god RAM-a mogu. Sakuplja\u010d sme\u0107a osloba\u0111a memoriju samo ako je iskori\u0161teno vi\u0161e od 75%. Budu\u0107i da je ve\u0107ini servisa memorija burstable, JVM ju je uvijek zauzimao. Stoga su svi ti Java servisi tro\u0161ili mnogo vi\u0161e RAM-a nego \u0161to se o\u010dekivalo.<\/li>\n
  3. Neke aplikacije su tra\u017eile previ\u0161e memorije, a Kubernetes planer nije dopu\u0161tao ovim nodovima da budu dostupni drugim aplikacijama, iako su u stvarnosti bili slobodniji od drugih nodova. Jedan programer je slu\u010dajno dodao suvi\u0161nu cifru u zahtjevu i zauzeo veliki dio RAM-a: 20 GB umjesto 2. Nitko to nije primijetio. Aplikacija je imala 3 replike, tako da su pogo\u0111ena \u010dak 3 noda.<\/li>\n
  4. Postavili smo ograni\u010denja na resurse, reorganizirali podove s pravilnim zahtjevima i postigli savr\u0161enu ravnote\u017eu kori\u0161tenja hardvera izme\u0111u svih nodova. Nekoliko nodova se \u010dak moglo zatvoriti. A onda smo primijetili da imamo pogre\u0161ne ma\u0161ine (usmjerene na CPU, a ne na memoriju). Promijenili smo tip i uklonili jo\u0161 nekoliko nodova.<\/li>\n<\/ol>\n

    <\/p>\n

    Zaklju\u010dci<\/h3>\n

    <\/p>\n

    S burstable resursima u klasteru efikasnije koristite dostupni hardver, ali Kubernetes planer raspore\u0111uje podove prema zahtjevima za resursima, \u0161to mo\u017ee izazvati probleme. Da bismo ubili dvije muhe jednim udarcem: izbjegli probleme i u potpunosti iskoristili resurse, potrebna je dobra nadzorna plo\u010da. Zato je to od pomo\u0107i. Kube Eagle<\/a><\/noindex> (eksporter Prometheus i kontrolna plo\u010da Grafana).<\/p>\n

    Izvor: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

    \u042f \u0441\u043e\u0437\u0434\u0430\u043b Kube Eagle\u00a0\u2014 \u044d\u043a\u0441\u043f\u043e\u0440\u0442\u0435\u0440 Prometheus. \u041e\u043a\u0430\u0437\u0430\u043b\u043e\u0441\u044c, \u043a\u0440\u0443\u0442\u0430\u044f \u0448\u0442\u0443\u043a\u0430, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u043e\u043c\u043e\u0433\u0430\u0435\u0442 \u043b\u0443\u0447\u0448\u0435 \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0442\u044c\u0441\u044f \u0432 \u0440\u0435\u0441\u0443\u0440\u0441\u0430\u0445 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0438 \u0441\u0440\u0435\u0434\u043d\u0438\u0445 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u043e\u0432. \u0412 \u0438\u0442\u043e\u0433\u0435 \u044f \u0441\u044d\u043a\u043e\u043d\u043e\u043c\u0438\u043b \u043d\u0435 \u043e\u0434\u043d\u0443 \u0441\u043e\u0442\u043d\u044e \u0434\u043e\u043b\u043b\u0430\u0440\u043e\u0432, \u043f\u043e\u0442\u043e\u043c\u0443 \u0447\u0442\u043e \u043f\u043e\u0434\u0431\u0438\u0440\u0430\u043b \u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0435 \u0442\u0438\u043f\u044b \u043c\u0430\u0448\u0438\u043d \u0438 \u043d\u0430\u0441\u0442\u0440\u0430\u0438\u0432\u0430\u043b \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u0438\u044f \u0440\u0435\u0441\u0443\u0440\u0441\u043e\u0432 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439 \u043f\u043e\u0434 \u0440\u0430\u0431\u043e\u0447\u0438\u0435 \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0438. \u042f \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0443 \u043e \u043f\u0440\u0435\u0438\u043c\u0443\u0449\u0435\u0441\u0442\u0432\u0430\u0445 Kube Eagle, \u043d\u043e \u0441\u043d\u0430\u0447\u0430\u043b\u0430 \u043e\u0431\u044a\u044f\u0441\u043d\u044e, \u0438\u0437-\u0437\u0430 \u0447\u0435\u0433\u043e \u0432\u044b\u0448\u0435\u043b \u0441\u044b\u0440-\u0431\u043e\u0440 \u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":23107,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-31139","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Pratimo resurse Kubernetes klastera | ProHoster","description":"","canonical_url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitorim-resursy-klasterov-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hr_HR","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043c \u0440\u0435\u0441\u0443\u0440\u0441\u044b \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u043e\u0432 Kubernetes | ProHoster","og:url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitorim-resursy-klasterov-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:39:40+00:00","article:modified_time":"2019-10-31T18:39:40+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"31139","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 04:42:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 03:22:34","updated":"2026-01-21 04:42:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/31139","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/comments?post=31139"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/31139\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media\/23107"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media?parent=31139"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/categories?post=31139"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/tags?post=31139"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}