E noʻonoʻo i ka manaʻo o ka nānā ʻana o Kubernetes, e kamaʻāina me ka mea hana Prometheus, a kamaʻilio e pili ana i ka makaʻala.
He voluminous ke kumuhana o ka nānā ʻana, ʻaʻole hiki ke wehe ʻia i hoʻokahi ʻatikala. ʻO ke kumu o kēia kikokikona ka hāʻawi ʻana i kahi ʻike o nā mea hana, nā manaʻo a me nā ala.
ʻO ka mea o ka ʻatikala he ʻomi mai . Inā makemake ʻoe e hoʻomaʻamaʻa piha, e kākau inoa no kahi papa ma .
ʻO ka mea i nānā ʻia ma kahi pūʻulu Kubernetes
nā kikowaena kino. Inā kau ʻia kahi pūʻulu Kubernetes ma kāna mau kikowaena ponoʻī, pono ʻoe e nānā i ko lākou olakino. Hana ʻo Zabbix i kēia hana; inā ʻoe e hana pū me ia, ʻaʻole pono e hōʻole, ʻaʻohe hakakā. ʻO Zabbix ka mea nāna e nānā i ke kūlana o kā mākou mau kikowaena.
E neʻe kākou i ka nānā ʻana ma ka pae hui.
Nā ʻāpana o ka Plane Control: API, Scheduler a me nā mea ʻē aʻe. Ma ka liʻiliʻi loa, pono ʻoe e hōʻoia i ka API o nā kikowaena a i ʻole etcd ʻoi aku ka nui ma mua o 0. Hiki i ka Etcd ke hoʻihoʻi i nā metric he nui: ma nā disks kahi e wili ai, e ke olakino o kāna hui etcd, a me nā mea ʻē aʻe.
Docker ua ʻike ʻia i ka wā lōʻihi a ua ʻike maopopo nā mea a pau i kona mau pilikia: nui nā pahu e hoʻāla ai i nā pilikia a me nā pilikia ʻē aʻe. No laila, ʻo Docker ponoʻī, ma ke ʻano he ʻōnaehana, pono e hoʻomalu ʻia, ma ka liʻiliʻi loa no ka loaʻa.
Dns. Inā hāʻule ka DNS i loko o ka pūpū, a laila e hāʻule ka lawelawe Discovery holoʻokoʻa ma hope o ia, e pau nā kelepona mai nā pods i nā pods. I kaʻu hana, ʻaʻohe pilikia like, akā ʻaʻole ia he manaʻo ʻaʻole pono ke nānā ʻia ka mokuʻāina o ka DNS. Hiki ke nānā ʻia ka latency a me kekahi mau ana ʻē aʻe ma CoreDNS.
Hoʻokomo. Pono e hoʻomalu i ka loaʻa ʻana o nā mea komo (me ka Ingress Controller) ma ke ʻano he komo i ka papahana.
Ua hoʻokaʻawale ʻia nā mea nui o ka puʻupuʻu - i kēia manawa e iho kākou i ka pae o nā abstractions.
Me he mea lā e holo ana nā noi i loko o nā pods, ʻo ia hoʻi, pono lākou e hoʻomalu ʻia, akā ʻoiaʻiʻo ʻaʻole. He ephemeral nā Pods: i kēia lā e holo lākou ma kahi kikowaena, ʻapōpō ma kekahi; i kēia lā he 10 o lākou, ʻapōpō 2. No laila, ʻaʻohe mea nāna e nānā i nā pods. I loko o kahi hoʻolālā microservice, ʻoi aku ka nui o ka mālama ʻana i ka loaʻa o ka noi holoʻokoʻa. ʻO ka mea kūikawā, e nānā i ka loaʻa ʻana o nā wahi hope o ka lawelawe: hana kekahi mea? Inā loaʻa ka palapala noi, a laila he aha ka mea e hana ai ma hope o ia, ehia mau replicas i kēia manawa - ʻo ia nā nīnau o ka papa ʻelua. ʻAʻohe pono e nānā i nā hihia pākahi.
I ka pae hope, pono ʻoe e hoʻomalu i ka hana o ka noi ponoʻī, e lawe i nā metric ʻoihana: ka helu o nā kauoha, nā ʻano mea hoʻohana, a pēlā aku.
IHILA
ʻO ka ʻōnaehana maikaʻi loa no ka nānā ʻana i kahi hui . ʻAʻole au i ʻike i kekahi mea hana hiki ke hoʻohālikelike iā Prometheus ma ke ʻano o ka maikaʻi a me ka maʻalahi o ka hoʻohana. He mea maikaʻi loa ia no nā ʻōnaehana maʻalahi, no laila ke ʻōlelo lākou "Ka nānā ʻana o Kubernetes", ʻo ia ka mea maʻamau ʻo Prometheus.
Aia kekahi mau koho no ka hoʻomaka ʻana me Prometheus: me ka hoʻohana ʻana iā Helm, hiki iā ʻoe ke hoʻokomo i ka Prometheus maʻamau a i ʻole Prometheus Operator.
- Prometheus mau. Maikaʻi nā mea a pau iā ia, akā pono ʻoe e hoʻonohonoho i ka ConfigMap - ʻoiaʻiʻo, e kākau i nā faila hoʻonohonoho kikokikona, e like me kā mākou i hana ai ma mua, ma mua o ka hoʻolālā microservice.
- ʻO ka Prometheus Operator kahi pālahalaha liʻiliʻi, ʻoi aku ka paʻakikī e pili ana i ka loiloi kūloko, akā ʻoi aku ka maʻalahi o ka hana ʻana me ia: aia nā mea ʻokoʻa, hoʻohui ʻia nā abstractions i ka hui, no laila ʻoi aku ka maʻalahi o ka hoʻokele a hoʻonohonoho.
No ka hoʻomaopopo ʻana i ka huahana, manaʻo wau e kau mua iā Prometheus maʻamau. Pono ʻoe e hoʻonohonoho i nā mea āpau ma o ka config, akā e pōmaikaʻi kēia: e hoʻomaopopo ʻoe i ka mea a me ke ʻano o ka hoʻonohonoho ʻana. Ma Prometheus Operator, e piʻi koke ʻoe i kahi abstraction kiʻekiʻe, ʻoiai inā makemake ʻoe, hiki iā ʻoe ke komo i loko o ka hohonu.
Hoʻohui maikaʻi ʻia ʻo Prometheus me Kubernetes: hiki iā ia ke komo a launa pū me ka API Server.
Ua kaulana ʻo Prometheus, ʻo ia ke kumu e kākoʻo ai ka nui o nā noi a me nā ʻōlelo papahana. Pono ke kākoʻo, no ka mea, loaʻa iā Prometheus kona ʻano metric ponoʻī, a no ka hoʻoili ʻana, pono ʻoe i kahi waihona i loko o ka noi a i ʻole kahi mea hoʻopuka i mākaukau. A he kakaikahi wale no ka poe lawe aku. No ka laʻana, aia ʻo PostgreSQL Exporter: lawe ia i ka ʻikepili mai PostgreSQL a hoʻololi iā ia i ka format Prometheus i hiki iā Prometheus ke hana pū me ia.
Hoʻolālā Prometheus
Kahua Prometheus ʻo ia ke kua, ka lolo o Prometheus. Mālama ʻia nā metric ma ʻaneʻi.
Mālama ʻia nā metric i loko o ka waihona manawa manawa (TSDB). ʻAʻole ʻo TSDB kahi waihona ʻokoʻa, akā he pūʻolo ma ka ʻōlelo Go i hoʻokomo ʻia ma Prometheus. ʻO ka'ōlelo maʻamau, aia nā mea a pau i hoʻokahi binary.
Mai mālama i ka ʻikepili ma TSDB no ka manawa lōʻihi
ʻAʻole kūpono ka ʻōnaehana Prometheus no ka mālama lōʻihi o nā metric. ʻO ka manawa mālama paʻa he 15 mau lā. Hiki iā ʻoe ke ʻoi aku i kēia palena, akā e hoʻomanaʻo: ʻoi aku ka nui o ka ʻikepili āu e mālama ai ma TSDB a ʻoi aku ka lōʻihi o kāu hana ʻana, ʻoi aku ka nui o nā kumuwaiwai e hoʻopau ai. ʻO ka mālama ʻana i ka ʻikepili mōʻaukala ma Prometheus i manaʻo ʻia he hana ʻino.
Inā loaʻa iā ʻoe nā kaʻa nui, ʻo ka nui o nā metric i loko o nā haneli haneli i kēlā me kēia kekona, a laila ʻoi aku ka maikaʻi o ka kaupalena ʻana i kā lākou waiho ʻana e ka disk space a i ʻole ka lōʻihi. ʻO ka maʻamau, mālama ʻo TSDB i nā "ʻikepili wela", nā metric maoli no kekahi mau hola. No ka mālama ʻana i ka wā lōʻihi, hoʻohana ʻia ka waihona waho i kēlā mau waihona i kūpono maoli no kēia, no ka laʻana InfluxDB, ClickHouse, a pēlā aku. Ua ʻike au i nā loiloi maikaʻi hou aʻe e pili ana iā ClickHouse.
Hana ʻo Prometheus Server e like me ke kumu hoʻohālike huki: hele ʻo ia no nā metric i nā hopena a mākou i hāʻawi ai iā ia. 'Ōlelo lākou: "hele i ka API Server," a hele i kēlā me kēia helu nth o kekona a lawe i nā metric.
No nā mea me ka pōkole o ke ola (hana a i ʻole cron job) hiki ke ʻike ʻia ma waena o nā wā ʻōpala, aia kahi ʻāpana Pushgateway. Hoʻokomo ʻia nā metric mai nā mea pōkole i loko: ua piʻi ka hana, hana i kahi hana, hoʻouna i nā metric i Pushgateway a hoʻopau. Ma hope o kekahi manawa, e iho mai ʻo Prometheus i kona ʻano ponoʻī a lawe i kēia mau metric mai Pushgateway.
No ka hoʻonohonoho ʻana i nā leka hoʻomaopopo ma Prometheus aia kahi ʻāpana ʻokoʻa - Luna kiaʻi. A me nā lula makaʻala. No ka laʻana, pono ʻoe e hana i kahi mākaʻikaʻi inā ʻo 0 ke kikowaena API. Ke ahi ka hanana, hāʻawi ʻia ka mākaʻikaʻi i ka luna makaʻala no ka hoʻouna hou aku. Hiki ke hoʻouna ʻia kekahi pūʻulu makaʻala i ke kamaʻilio telegram a nā luna, ʻo kekahi i ke kamaʻilio o nā mea hoʻomohala, a ʻo ke kolu i ke kamaʻilio ʻana o nā limahana hana. Hiki ke hoʻouna ʻia nā leka i Slack, Telegram, leka uila, a me nā ala ʻē aʻe.
A ʻo ka hope, e haʻi wau iā ʻoe e pili ana i ka hiʻohiʻona pepehi kanaka o Prometheus - ike mai ai ia. Ke hana pū me Prometheus, ʻaʻole pono ʻoe e kuhikuhi i nā kikoʻī kikoʻī o nā mea no ka nānā ʻana, ua lawa ia e hoʻonohonoho i kā lākou ʻano. ʻO ia hoʻi, ʻaʻole pono ʻoe e kākau "eia ka IP address, eia ka port - monitor", akā, pono ʻoe e hoʻoholo i nā kumu e loaʻa ai kēia mau mea (nā polokalamu - pahu hopu). ʻO Prometheus ponoʻī, e pili ana i nā mea e hana nei i kēia manawa, huki i nā mea e pono ai a hoʻohui iā lākou i ka nānā ʻana.
Ua kūpono kēia ala me ka ʻōnaehana Kubernetes, kahi e lana ai nā mea a pau: i kēia lā aia nā kikowaena 10, ʻapōpō 3. I ʻole e wehewehe i ka IP address o ka kikowaena i kēlā me kēia manawa, kākau lākou i hoʻokahi manawa pehea e loaʻa ai - a ʻo Discovering e hana ia. .
Kapa ʻia ka ʻōlelo Prometheus PromQL. Ke hoʻohana nei i kēia ʻōlelo, hiki iā ʻoe ke kiʻi i nā waiwai o nā metric kikoʻī a laila hoʻololi iā lākou a kūkulu i nā helu analytical e pili ana iā lākou.
https://prometheus.io/docs/prometheus/latest/querying/basics/
Простой запрос
container_memory_usage_bytes
Математические операции
container_memory_usage_bytes / 1024 / 1024
Встроенные функции
sum(container_memory_usage_bytes) / 1024 / 1024
Уточнение запроса
100 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m]) * 100)ʻO Prometheus pūnaewele pūnaewele
Loaʻa iā Prometheus kāna kikowaena pūnaewele ponoʻī minimalistic. He kūpono wale no ka debug a hōʻikeʻike.
Ma ka laina Expression, hiki iā ʻoe ke kākau i kahi nīnau ma ka ʻōlelo PromQL.
Aia nā lula makaʻala ma ka ʻaoʻao ʻĀlapa, a ʻekolu kā lākou kūlana:
- hana ʻole - inā ʻaʻole ikaika ka makaʻala i kēia manawa, ʻo ia hoʻi, ua maikaʻi nā mea āpau me ia, ʻaʻole i hana;
- ke kali nei - inā ua hana ka makaʻala, akā ʻaʻole i hala ka hoʻouna ʻana. Ua hoʻonohonoho ʻia ka lohi e uku i ka uila uila: inā ua piʻi ka lawelawe i ʻōlelo ʻia i loko o hoʻokahi minuke, a laila ʻaʻole pono ke kani ʻia ke kani ʻana;
- ʻO ke kī ʻana ke kolu o ke kūlana, ke ʻā ka makaʻala a hoʻouna i nā leka.
Ma ka menu Status e ʻike ʻoe i ka ʻike e pili ana i ka Prometheus. Aia kekahi hoʻololi i nā pahuhopu (targets), a mākou i kamaʻilio ai ma luna.
No ka ʻike kikoʻī hou aku o ka interface Prometheus, e ʻike .
Hoʻohui me Grafana
Ma ka pūnaewele pūnaewele Prometheus, ʻaʻole ʻoe e ʻike i nā kiʻi nani a hoʻomaopopo ʻia e hiki ai iā ʻoe ke huki i ka hopena e pili ana i ke kūlana o ka hui. No ke kūkulu ʻana iā lākou, ua hoʻohui ʻia ʻo Prometheus me Grafana. Loaʻa iā mākou kēlā mau dashboards.
ʻAʻole paʻakikī loa ka hoʻonohonoho ʻana i ka Prometheus a me Grafana, hiki iā ʻoe ke ʻike i nā ʻōlelo aʻoaʻo ma ka palapala: ʻAe, e hoʻopau wau me kēia.
Ma nā ʻatikala aʻe, e hoʻomau mākou i ke kumuhana o ka nānā ʻana: e kamaʻilio mākou e pili ana i ka hōʻiliʻili ʻana a me ka nānā ʻana i nā lāʻau me ka hoʻohana ʻana iā Grafana Loki a me nā mea hana ʻē aʻe.
Mea kākau: ʻO Marcel Ibraev, luna hoʻomalu Kubernetes i hōʻoia ʻia, hoʻomaʻamaʻa ʻenekinia ma ka hui , mea ha'i'ōlelo a mea haku papa Slurm.
Source: www.habr.com
