{"id":39133,"date":"2019-10-31T22:28:01","date_gmt":"2019-10-31T19:28:01","guid":{"rendered":"https:\/\/prohoster.info\/blog\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev\/"},"modified":"2019-10-31T22:28:01","modified_gmt":"2019-10-31T19:28:01","slug":"monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","title":{"rendered":"J\u00e4lgimine + koormustestimine = prognoosimine ja t\u00f5rgete v\u00e4ltimine","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

VTB IT-department has faced emergency situations in system operations several times, where the load on them increased multiple times. Hence, there was a need to develop and test a model that would predict peak loads on critical systems. To achieve this, the bank's IT specialists set up monitoring, analyzed data, and learned to automate forecasts. We will discuss what tools helped to predict the load and whether they managed to optimize operations in a brief article.<\/p>\n

\"J\u00e4lgimine
\n<\/a><\/noindex>
\nIssues with high-load services arise in almost all industries, but they are critical in the financial sector. At the moment of truth, all combat units must be on alert, and it was necessary to know in advance what could happen, even determining the day when the load would surge and which systems would encounter it. Problems need to be fought and prevented, so the need to implement a predictive analytics system was never in question. It was necessary to modernize systems based on monitoring data.<\/p>\n

Analytics on the fly<\/h2>\n

\nThe salary project is one of the most sensitive in case of a failure. It is also the most predictable, so we decided to start with it. Due to high interconnectivity, other subsystems, including remote banking services (RBS), could also experience issues during peak load moments. For instance, clients, thrilled with SMS notifications about incoming funds, would start using them actively. In such cases, the load could surge more than tenfold.\u00a0<\/p>\n

The first forecasting model was created manually. We took the data for the last year and calculated which days would expect the highest peaks: for example, the 1st, 15th, and 25th of each month, as well as the last days of the month. This model required significant labor intensity and did not provide accurate predictions. Nevertheless, it identified bottlenecks where additional 'hardware' was needed and allowed us to optimize the money transfer process by negotiating with anchor clients: instead of disbursing salaries all at once, transactions from different regions were staggered over time. Now we process them in parts that the bank's IT infrastructure can handle without failures.<\/p>\n

Saades esimene positiivne tulemus, l\u00e4ksime prognoosimise automatiseerimise juurde. Ootas veel k\u00fcmme kriitilist ala.<\/p>\n

Kasutusmugav l\u00e4henemine<\/h2>\n

\nVTB-s rakendati MicroFocusi firma j\u00e4lgimiss\u00fcsteemi. Sealt saime andme kogumise prognoosimiseks, salvestuss\u00fcsteemi ja raportite genereerimise s\u00fcsteemi. Eess\u00f5nas oli j\u00e4lgimine juba olemas, j\u00e4i vaid lisada meetrikad, ennustamismoodul ja luua uusi raporteid. Selle lahenduse toetab v\u00e4line allt\u00f6\u00f6v\u00f5tja 'Tehnoserv', seega peamine t\u00f6\u00f6 projekti elluviimisel lasus nende spetsialistidel, kuid mudeli koostasime ise. Prognoosimisse s\u00fcsteemi tegime Prophet'i baasil \u2014 see avatud toode on loodud Facebookis. See on lihtne kasutada ja integreerub h\u00f5lpsasti meie olemasolevate integreeritud j\u00e4lgimist\u00f6\u00f6riistade ja Verticaga. \u00dcldiselt anal\u00fc\u00fcsib s\u00fcsteem koormuse graafikut ja teeb Fourier' seeriate p\u00f5hjal selle ekstrapoleerimist. Samuti on v\u00f5imalik lisada teatud koefitsiente p\u00e4evade kaupa, mis on saadud meie mudelist. Meetriikaid v\u00f5etakse ilma inimese sekkumiseta, kord n\u00e4dalas prognoosi arvutatakse automaatselt \u00fcmber ja uued raportid saadetakse adressaatidele.\u00a0<\/p>\n

See l\u00e4henemine tuvastab peamised ts\u00fcklilisused, n\u00e4iteks aastased, kuukesised, kvartali- ja n\u00e4dalats\u00fcklid. Palga ja ettemaksete maksmine, puhkuste ja p\u00fchade ajad ning allahindlused - k\u00f5ik see m\u00f5jutab s\u00fcsteemide seadmist. Selgus, et n\u00e4iteks m\u00f5ned ts\u00fcklid kattuvad \u00fcksteisega ning p\u00f5hikoormus (75%) s\u00fcsteemidele tuleb Keskf\u00f6deraalset \u00dchendusest. Juriidilised ja f\u00fc\u00fcsilised isikud k\u00e4ituvad erinevalt. Kui 'f\u00fc\u00fcsikute' koormus on n\u00e4dalap\u00e4evade vahel suhteliselt \u00fchtlaselt jaotatud (see on v\u00e4ga palju v\u00e4ikseid tehinguid), siis ettev\u00f5tete puhul langeb 99,9% t\u00f6\u00f6ajale, kusjuures tehingud v\u00f5ivad olla l\u00fchikesed, kuid neid v\u00f5ib t\u00f6\u00f6delda ka mitu minutit v\u00f5i isegi tundi.<\/p>\n

\"J\u00e4lgimine
\n
\nSaadud andmete p\u00f5hjal m\u00e4\u00e4ratakse pikaajalised trendid. Uus s\u00fcsteem on n\u00e4idanud, et inimesed liiguvad massiliselt DBO-sse. See on k\u00f5igile teada, kuid sellist ulatust me ei oodanud ja alguses ei uskunud me sellesse: panga kontorite k\u00fclastuste arv v\u00e4heneb \u00e4\u00e4rmiselt kiiresti, samas kui kaugtehingute arv kasvab sama kiiresti. Seega kasvab ka s\u00fcsteemide koormus ja see kasvab tulevikus veelgi. Praegu prognoosime koormust kuni 2020. aasta veebruarini. Tavalisi p\u00e4evi suudame prognoosida 3% veaulatusega, kuid tipptunde 10% veaulatusega. See on hea tulemus.<\/p>\n

Pahupooled<\/h2>\n

\nIlma raskusteta ei p\u00e4\u00e4senud. Fourier' jadadega ekstrapolatsiooni mehhanism ei \u00fcleta nulli h\u00e4sti \u2014 me teame, et n\u00e4dalavahetustel genereerivad juriidilised isikud v\u00e4he tehinguid, kuid prognoosimoodul annab kaugel nullist olevad v\u00e4\u00e4rtused. Me oleksime saanud need kunstlikult parandada, kuid meie meetod pole tugineda v\u00e4ikestele lahendustele. Lisaks tuli lahendada probleem s\u00fcsteemide allikast andmete valutuks kogumiseks. Regulaarne teabe kogumine n\u00f5uab t\u00f5siseid arvutusressursse, seega ehitasime kiireid vahem\u00e4lu lahendusi, kasutades replikatsiooni ja saame \u00e4rilisi andmeid juba replikate kaudu. Lisakoormuse puudumine peamise s\u00fcsteemi puhul sellistes olukordades on takistus.<\/p>\n

Uued v\u00e4ljakutsed<\/h2>\n

\nOtseselt seatud \u00fclesanne tipukoormuse prognoosimiseks on lahendatud: panga \u00fclekoormusega seotud t\u00f5rkeid ei ole olnud alates maist k\u00e4esolevast aastast, ja uus prognoosimisse s\u00fcsteem on m\u00e4nginud selles suurt rolli. Jah, seda on osutunud ebapiisavaks, ja n\u00fc\u00fcd soovib pank m\u00f5ista, kui ohtlikud on tipphetked. Me vajame prognoose koormustest, kasutades koormustestimise m\u00f5\u00f5dikuid, ja see t\u00f6\u00f6tab juba umbes 30% kriitiliste s\u00fcsteemide jaoks, teised on prognooside saamise protsessis. J\u00e4rgmises etapis plaanime prognoosida s\u00fcsteemide koormust mitte \u00e4ritehingutes, vaid IT-infrastruktuuri tasemel, st liigume madalamale kihile. Lisaks peame t\u00e4ielikult automatiseerima m\u00f5\u00f5dikute kogumise ja nende p\u00f5hjal prognooside koostamise, et mitte tegeleda andmete eksportimisega. Selles ei ole midagi eriskummalist \u2014 me lihtsalt ristame monitooringu ja koormustestimise vastavalt parimatele rahvusvahelistele praktikatele.
\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

IT-\u0434\u0435\u043f\u0430\u0440\u0442\u0430\u043c\u0435\u043d\u0442\u0443 \u0412\u0422\u0411 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437 \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u043b\u043e\u0441\u044c \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0442\u044c\u0441\u044f \u0441 \u043d\u0435\u0448\u0442\u0430\u0442\u043d\u044b\u043c\u0438 \u0441\u0438\u0442\u0443\u0430\u0446\u0438\u044f\u043c\u0438 \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441\u0438\u0441\u0442\u0435\u043c, \u043a\u043e\u0433\u0434\u0430 \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043d\u0430 \u043d\u0438\u0445 \u043c\u043d\u043e\u0433\u043e\u043a\u0440\u0430\u0442\u043d\u043e \u0432\u043e\u0437\u0440\u0430\u0441\u0442\u0430\u043b\u0430. \u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u043e\u044f\u0432\u0438\u043b\u0430\u0441\u044c \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u0442\u044c \u0438 \u043e\u043f\u0440\u043e\u0431\u043e\u0432\u0430\u0442\u044c \u043c\u043e\u0434\u0435\u043b\u044c, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u0440\u0435\u0434\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u043b\u0430 \u0431\u044b \u043f\u0438\u043a\u043e\u0432\u0443\u044e \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0443 \u043d\u0430 \u043a\u0440\u0438\u0442\u0438\u0447\u043d\u044b\u0435 \u0441\u0438\u0441\u0442\u0435\u043c\u044b. \u0414\u043b\u044f \u044d\u0442\u043e\u0433\u043e IT-\u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0441\u0442\u044b \u0431\u0430\u043d\u043a\u0430 \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u043b\u0438 \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433, \u043f\u0440\u043e\u0430\u043d\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b\u0438 \u0434\u0430\u043d\u043d\u044b\u0435 \u0438 \u043d\u0430\u0443\u0447\u0438\u043b\u0438\u0441\u044c \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u044b. \u041a\u0430\u043a\u0438\u0435 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u043f\u043e\u043c\u043e\u0433\u043b\u0438 \u0441\u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0443 \u0438 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c \u043b\u0438 \u0441 \u0438\u0445 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29368,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-39133","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47J\u00e4lgimine + koormustest = prognoosimine ja t\u00f5rgete puudumine | ProHoster","description":"VTB IT-osakonnal on mitu korda olnud probleemse olukorra loomine s\u00fcsteemide t\u00f6\u00f6s, kui nende koormus on mitmekordselt suurenemas.","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433 + \u043d\u0430\u0433\u0440\u0443\u0437\u043e\u0447\u043d\u043e\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 = \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0438 \u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432\u0438\u0435 \u0441\u0431\u043e\u0435\u0432 | ProHoster","og:description":"IT-\u0434\u0435\u043f\u0430\u0440\u0442\u0430\u043c\u0435\u043d\u0442\u0443 \u0412\u0422\u0411 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437 \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u043b\u043e\u0441\u044c \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0442\u044c\u0441\u044f \u0441 \u043d\u0435\u0448\u0442\u0430\u0442\u043d\u044b\u043c\u0438 \u0441\u0438\u0442\u0443\u0430\u0446\u0438\u044f\u043c\u0438 \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441\u0438\u0441\u0442\u0435\u043c, \u043a\u043e\u0433\u0434\u0430 \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043d\u0430 \u043d\u0438\u0445 \u043c\u043d\u043e\u0433\u043e\u043a\u0440\u0430\u0442\u043d\u043e \u0432\u043e\u0437\u0440\u0430\u0441\u0442\u0430\u043b\u0430.","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:28:01+00:00","article:modified_time":"2019-10-31T19:28:01+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"39133","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 00:53:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:55:36","updated":"2026-01-24 00:53:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/39133","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=39133"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/39133\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/29368"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=39133"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=39133"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=39133"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}