ʻOi aku ka paʻakikī o ka ʻōnaehana, ʻoi aku ka nui o ka ulu ʻana me nā ʻano makaʻala āpau. A pono e pane i kēia mau makaʻala like, e hōʻuluʻulu iā lākou a nānā iā lākou. Manaʻo wau he kūlana kēia i kamaʻāina i nā mea he nui a hiki i ka makaʻu.
ʻO ka hopena e kūkākūkā ʻia ʻaʻole ia ka mea i manaʻo ʻole ʻia, akā ʻaʻole i hoʻihoʻi ka ʻimi i kahi ʻatikala piha e pili ana i kēia kumuhana.
No laila, ua hoʻoholo wau e kaʻana like i ka ʻike o FunCorp a kamaʻilio e pili ana i ke ʻano o ke kaʻina hana dute, ʻo wai ke kāhea, no ke aha a pehea ʻoe e nānā ai i nā mea āpau.
He aha ka PagerDuty?
No laila, e hoʻoponopono i kēia mau pilikia āpau, hoʻomaka mākou e ʻimi i kahi mea hana maʻalahi. Ma hope o ka huli ʻana, ua koho mākou iā PagerDuty. Ua manaʻo ʻia ʻo PD he hopena maikaʻi loa a pōkole me ka nui o nā hoʻohui a me nā hoʻonohonoho. He aha kona ʻano?
I ka pōkole, ʻo PagerDuty kahi kahua hoʻoponopono hanana e hiki ai ke hoʻoponopono i nā hanana e hiki mai ana ma o nā hoʻohui like ʻole, hoʻonohonoho i nā kauoha dute a laila makaʻala i ka ʻenekinia ma ka hana ma muli o ke kiʻekiʻe o ka hanana (ma kahi kiʻekiʻe - kahi kelepona, ma kahi haʻahaʻa - kahi pana mai ka noi / SMS) .
ʻO wai ka luna hana?
ʻO kēia paha ka wahi mua e hoʻomaka ai e hoʻonohonoho i ka PD.
Ma FunCorp, e like me nā hui ʻē aʻe, aia kahi kūlana hanohano o ka luna hana. Hoʻouna ʻia ia mai ka ʻenekinia i ka ʻenekinia hoʻokahi manawa i ka lā. Aia kahi laina mua a me ka lua o ka pane i kahi makaʻala mai PagerDuty. Inā hiki mai kahi makaʻala kiʻekiʻe, a inā he 10 mau minuke ma hope o ke kāhea ʻana i ka luna dute mai ka laina mua ʻaʻohe pane iā ia (ʻo ia hoʻi, ʻaʻole i hoʻololi ʻia i ke kūlana ʻae a hoʻoholo ʻia), hele ke kelepona i ka lua. ʻenekini hana. Hoʻonohonoho ʻia kēia ma PagerDuty ponoʻī ma o ka Escalation Policies.
Inā ʻaʻole pane ka luna hana ʻelua, e hoʻi ka leka hoʻomaopopo i nui i ka luna hana.
No laila, ʻaʻole hiki ke hoʻopaʻa ʻole ʻia nā makaʻala kiʻekiʻe kiʻekiʻe.
I kēia manawa e ʻike kākou i hea mai nā hanana.
He aha nā hoʻohui a mākou e hoʻohana ai?
Loaʻa iā PD nā hanana like ʻole mai nā lawelawe like ʻole. Loaʻa iā mākou ma kahi o 25 ia mau lawelawe, a no ka hoʻoponopono ʻana iā lākou, hoʻohana mākou i kekahi mau hoʻohui i mākaukau.
- IHILA
ʻO ka ʻōnaehana hōʻiliʻili metric nui ʻo Prometheus. Ua kākau ʻia he nui e pili ana iā ia ma Habré, e ʻōlelo wale wau he nui kā mākou o lākou no nā kaiapuni like ʻole: ʻohi kekahi i nā metric mai nā mīkini virtual a me nā dockers, ʻo kekahi mai nā lawelawe Amazon, ʻo ke kolu mai nā mīkini mīkini. Hoʻohana nui ʻia ʻo Telegraf ma ke ʻano he metric exporter.
- enamel
Eia kekahi, ke manaʻo nei au, ua maopopo nā mea a pau mai ke poʻo. Hoʻohana ʻia kēia hoʻohui no ka hoʻouna ʻana i nā leka mai kekahi mau palapala i hoʻokō ʻia e cron. Hāʻawi ʻo PD iā ʻoe i kahi helu wahi āu e hoʻouna ai i nā leka. I ka hana ʻana i kahi lawelawe me ia hoʻohui, hiki iā ʻoe ke hoʻonohonoho i nā mea nui, i ke ʻano o ka hana ʻana i nā hanana e hiki mai ana, pehea e hana ai i kahi makaʻala (no kēlā me kēia leka e hiki mai ana, no kahi leka e hiki mai ana + kekahi lula, etc.).
- Slack
I koʻu manaʻo, he hoʻohui hoihoi loa. Aia kekahi mau manawa ke hiki mai kekahi mea akā ʻaʻole i uhi ʻia e nā hanana. No laila, ua hoʻohui mākou i ka hoʻohui ʻana mai Slack e hana i kahi hanana. ʻO ia, hiki iā ʻoe ke kākau iā Slack hui /callofduty ua lohi nā mea a pau a e haki koke a na ka PD e hoʻoponopono a hoʻouna aku i ka hana i ka ʻenekini hana.
Hana mākou:
ʻIke mākou:
- API
Hoʻohui HTTP. ʻO kaʻoiaʻiʻo, ʻaʻohe mea hoihoi loa ma aneʻi, kahi noi POST wale nō me kahi kino ma JSON format. No ka laʻana, he mea hoihoi: hoʻohana mākou no ka nānā ʻana i waho . Ke nānā nei kēia lawelawe i ka hiki ʻana o kā mākou pūnaewele mai nā wahi like ʻole o ka honua. Inā loaʻa iā mākou kahi code pane ʻae ʻole (no ka laʻana, 502), hana ʻia kahi hanana a laila hahai nā mea āpau i ke kaulahao i hōʻike ʻia ma luna. Hiki iā StatusCake ke nānā i nā URL o loko, ka palapala SSL a i ʻole ka pau ʻana o ka domain.
- LibreNMS
He ʻōnaehana nānā ʻē aʻe kēia, hiki iā ʻoe ke heluhelu hou e pili ana iā ia ma kā lākou pūnaewele . Me kāna kōkua, nānā mākou i nā kikowaena pūnaewele a me iDRAC mai nā kikowaena.
Aia kekahi mau hoʻohui e like me Datadog, CloudWatch. Hiki iā ʻoe ke ʻike hou aku e pili ana i ka mea i loaʻa iā lākou .
Hōʻikeʻike
ʻO ka ʻōnaehana hōʻike hōʻike nui ʻo Slack. Ua kākau ʻia nā hanana āpau e hele mai ana i PD i kahi kamaʻilio kūikawā, a inā hoʻololi ko lākou kūlana, hōʻike ʻia kēia ma ke kamaʻilio.
I ka hiki ʻana mai o ka manawa kūpono e hōʻike i ka ʻikepili kūpono ma nā pale o nā mea nānā e kau ana mai ke kaupaku, ua ʻike koke mākou ʻaʻohe o mākou (ma ka ʻoihana devops) e hōʻike iā lākou. Aia kahi Grafana maikaʻi loa, akā ʻaʻole ia e uhi i nā mea āpau, a pane nā limahana i nā mākaʻikaʻi, ʻaʻole nā palapala.
Ma hope o ka ʻimi piha ʻana akā ʻaʻole i kūleʻa ma GitHub no kahi "papa" pōkole a me ka ʻike no PD, ua hoʻoholo mākou e kākau i kā mākou iho - me ka mea pono wale nō. ʻOiai i ka wā mua aia kahi manaʻo e hōʻike i ka interface PD ponoʻī, ʻoi aku ka paʻakikī.
No ke kākau ʻana, pono ʻoe e kiʻi i kahi kī mai kahi PD me nā kuleana heluhelu wale nō.
A eia ka mea i loaa ia makou:
Hōʻike ka pale i nā hanana i wehe ʻia i kēia manawa, ka inoa o ka ʻenekinia o kēia manawa e hana ana mai ka papa kuhikuhi i koho ʻia, a me ka manawa me ka ʻole o kahi hanana koʻikoʻi kiʻekiʻe (e hōʻike ʻia ka papa me kahi hanana koʻikoʻi i ka ʻulaʻula).
.
ʻO ka hopena, ua loaʻa iā mākou kahi dashboard kūpono no ka nānā ʻana i kā mākou mau hanana āpau. E hauʻoli wau inā ʻike kekahi o ʻoukou i kā mākou ʻike he pono.
Source: www.habr.com