{"id":39133,"date":"2019-10-31T22:28:01","date_gmt":"2019-10-31T19:28:01","guid":{"rendered":"https:\/\/prohoster.info\/blog\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev\/"},"modified":"2019-10-31T22:28:01","modified_gmt":"2019-10-31T19:28:01","slug":"monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","title":{"rendered":"Pra\u0107enje + stres testiranje = predikcija i izostanak smetnji","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

IT odjel VTB-a suo\u010davao se s vanrednim situacijama u radu sustava, kada je optere\u0107enje na njima vi\u0161e puta pove\u0107ano. Stoga se pojavila potreba za razvojem i ispitivanjem modela koji bi predvi\u0111ao vr\u0161ne optere\u0107enja na kriti\u010dne sustave. Za to su IT stru\u010dnjaci banke postavili nadzor, analizirali podatke te nau\u010dili automatizirati prognoze. O kojim alatima smo koristili za predvi\u0111anje optere\u0107enja i jesmo li uspjeli optimizirati rad, govorit \u0107emo u ovom kratkom \u010dlanku.<\/p>\n

\"Pra\u0107enje
\n<\/a><\/noindex>
\nProblemi s visoko optere\u0107enim servisima javljaju se gotovo u svim industrijama, ali za financijski sektor su kriti\u010dni. U \u010das X sve borbene jedinice moraju biti spremne, pa je bilo potrebno unaprijed znati \u0161to se mo\u017ee dogoditi i \u010dak odrediti dan kada \u0107e optere\u0107enje porasti, kao i koji \u0107e sustavi s tim biti suo\u010deni. Treba se boriti protiv kvarova i sprije\u010diti ih, pa se potreba za implementacijom sustava prediktivne analitike nije ni raspravljala. Bilo je potrebno modernizirati sustave na temelju podataka nadzora.<\/p>\n

Analitika na brzinu<\/h2>\n

\nProjekt pla\u0107e je jedan od najosjetljivijih u slu\u010daju kvara. On je tako\u0111er najjasniji za predvi\u0111anje, pa se odlu\u010dilo zapo\u010deti s njim. Zbog visoke povezanosti, u trenucima vr\u0161nog optere\u0107enja mogli su nastati problemi i s drugim podsustavima, uklju\u010duju\u0107i daljinsko bankarstvo (DBO). Na primjer, klijenti koji su primili SMS o dolasku novca po\u010deli su ga aktivno koristiti. Optere\u0107enje je pritom moglo sko\u010diti za vi\u0161e od reda veli\u010dine.\u00a0<\/p>\n

Prvi model za prognozu izra\u0111en je ru\u010dno. Uzeli smo ispravku za posljednju godinu i izra\u010dunali na koje \u0107e dane do\u0107i do maksimalnih vrhova: na primjer, 1., 15. i 25. dana, kao i u posljednjim danima mjeseca. Ovaj model zahtijevao je ozbiljne radne napore i nije davao to\u010dnu prognozu. Ipak, otkrio je uska grla gdje je potrebno dodati \u201ehardver\u201c i omogu\u0107io optimizaciju procesa isplata, dogovoriv\u0161i se s klju\u010dnim klijentima: kako ne bismo ispla\u0107ivali pla\u010du \u201eodjednom\u201c, transakcije iz razli\u010ditih regija rasporedili smo po vremenu. Sada ih obra\u0111ujemo u dijelovima koje IT infrastruktura banke mo\u017ee \u201epro\u017evakati\u201c bez kvarova.<\/p>\n

Nakon \u0161to smo dobili prvi pozitivan rezultat, pre\u0161li smo na automatizaciju predikcija. \u010cekalo se jo\u0161 desetak kriti\u010dnih podru\u010dja.<\/p>\n

Sveobuhvatan pristup<\/h2>\n

\nU VTB-u su implementirali sustav pra\u0107enja tvrtke MicroFocus. Od tamo smo preuzeli prikupljanje podataka za predikciju, sustav pohrane i sustav za generiranje izvje\u0161taja. U su\u0161tini, pra\u0107enje je ve\u0107 bilo uvedeno, ostalo je samo dodati metriku, modul predikcije i stvoriti nove izvje\u0161taje. Ova rje\u0161enja podr\u017eava vanjski izvo\u0111a\u010d 'Tekhnoserv', stoga su glavni radovi na implementaciji projekta le\u017eali na njihovim stru\u010dnjacima, ali model smo gradili samostalno. Sustav predikcije izradili smo na temelju Propheta \u2014 ovog otvorenog proizvoda razvijenog u Facebooku. Jednostavan je za kori\u0161tenje i lako se integrira s na\u0161im postoje\u0107im sredstvima sveobuhvatnog pra\u0107enja i Vertica. U osnovi, sustav analizira grafikon optere\u0107enja i na temelju Fourierovih nizova radi njegovu ekstrapolaciju. Tako\u0111er postoji mogu\u0107nost dodavanja odre\u0111enih koeficijenata po danima, preuzetih iz na\u0161eg modela. Metrike se prikupljaju bez ljudskog sudjelovanja, jednom tjedno prognoza se automatski prera\u010dunava, novi izvje\u0161taji se \u0161alju adresatima.\u00a0<\/p>\n

Ovakav pristup otkriva osnovne cikli\u010dnosti, kao \u0161to su godi\u0161nje, mjese\u010dne, kvartalne i tjedne. Isplata pla\u0107a i avansa, razdoblja godi\u0161njih odmora, praznici i rasprodaje \u2014 sve to utje\u010de na broj poziva sustavima. Na primjer, otkriveno je da se neki ciklusi preklapaju, a glavno optere\u0107enje (75%) na sustavima dolazi iz Sredi\u0161nje federalne regije. Pravni i fizi\u010dki subjekti pona\u0161aju se razli\u010dito. Ako je optere\u0107enje od 'fizi\u010dkih' subjekata relativno ravnomjerno raspore\u0111eno tijekom dana u tjednu (to su mnoge male transakcije), onda 99,9% tvrtki pada na radno vrijeme, pri \u010demu transakcije mogu biti kratke ili se obra\u0111uju tijekom nekoliko minuta, pa \u010dak i sati.<\/p>\n

\"Pra\u0107enje
\n
\nNa temelju prikupljenih podataka odre\u0111uju se dugotrajni trendovi. Novi sustav je otkrio da se ljudi masovno prebacuju na DBO. To je svima poznato, ali nismo o\u010dekivali takve razmjere i isprva nismo vjerovali: broj zahtjeva u bankovnim uredima izuzetno brzo opada, a istovremeno se pove\u0107ava broj daljinskih transakcija. Tako\u0111er, optere\u0107enje na sustavima raste i nastavit \u0107e rasti. Trenutno prognoziramo optere\u0107enje do velja\u010de 2020. godine. Normalne dane uspijevamo predvidjeti s pogre\u0161kom od 3%, a vr\u0161ne dane s pogre\u0161kom od 10%. To je dobar rezultat.<\/p>\n

Skriveni problemi<\/h2>\n

\nBez pote\u0161ko\u0107a, kao i obi\u010dno, nije bilo. Mehanizam ekstrapolacije kori\u0161tenjem Fourierovih nizova slabo prelazi kroz nulu \u2014 znamo da tijekom vikenda pravne osobe generiraju malo transakcija, ali modul predikcije daje daleko od nule vrijednosti. Mogli smo ih prisilno ispraviti, ali takvi '\u0161takori' nisu na\u0161 pristup. Osim toga, morali smo rije\u0161iti problem bezbolnog prikupljanja podataka sa sustava-izvora. Redovito prikupljanje informacija zahtijeva ozbiljne ra\u010dunalne resurse, stoga smo izgradili brze ke\u0161iranje koriste\u0107i replikaciju, poslovne podatke dobivamo ve\u0107 s replikama. Odsutnost dodatnog optere\u0107enja na glavnim sustavima u takvim slu\u010dajevima je obavezni zahtjev.<\/p>\n

Novi izazovi<\/h2>\n

\nPostavljeni izazov za predikciju vrhunaca je rije\u0161en: od svibnja ove godine nije bilo povezani s preoptere\u0107enjem kvarova u banci, a novi sustav predikcije odigrao je zna\u010dajnu ulogu. Da, pokazalo se da to nije dovoljno, i sada banka \u017eeli razumjeti koliko su vrhunci opasni za nju. Potrebne su nam prognoze koriste\u0107i metrike iz testiranja optere\u0107enja, i otprilike za 30% kriti\u010dnih sustava to ve\u0107 funkcionira, dok su drugi u procesu dobivanja prognoza. U sljede\u0107em koraku planiramo predvidjeti optere\u0107enje sustava ne u poslovnim transakcijama, ve\u0107 u terminima IT-infrastrukture, tj. spustit \u0107emo se na sloj ni\u017ee. Osim toga, potrebno nam je potpuno automatizirati prikupljanje metrika i izgradnju prognoza temeljenih na njima, kako ne bismo morali obavljati ispite. U tome nema ni\u0161ta posebno \u2014 jednostavno mije\u0161amo pra\u0107enje i testiranje optere\u0107enja prema najboljim svjetskim praksama.
\n
Izvor: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

IT-\u0434\u0435\u043f\u0430\u0440\u0442\u0430\u043c\u0435\u043d\u0442\u0443 \u0412\u0422\u0411 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437 \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u043b\u043e\u0441\u044c \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0442\u044c\u0441\u044f \u0441 \u043d\u0435\u0448\u0442\u0430\u0442\u043d\u044b\u043c\u0438 \u0441\u0438\u0442\u0443\u0430\u0446\u0438\u044f\u043c\u0438 \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441\u0438\u0441\u0442\u0435\u043c, \u043a\u043e\u0433\u0434\u0430 \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043d\u0430 \u043d\u0438\u0445 \u043c\u043d\u043e\u0433\u043e\u043a\u0440\u0430\u0442\u043d\u043e \u0432\u043e\u0437\u0440\u0430\u0441\u0442\u0430\u043b\u0430. \u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u043e\u044f\u0432\u0438\u043b\u0430\u0441\u044c \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u044c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u0442\u044c \u0438 \u043e\u043f\u0440\u043e\u0431\u043e\u0432\u0430\u0442\u044c \u043c\u043e\u0434\u0435\u043b\u044c, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u0440\u0435\u0434\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u043b\u0430 \u0431\u044b \u043f\u0438\u043a\u043e\u0432\u0443\u044e \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0443 \u043d\u0430 \u043a\u0440\u0438\u0442\u0438\u0447\u043d\u044b\u0435 \u0441\u0438\u0441\u0442\u0435\u043c\u044b. \u0414\u043b\u044f \u044d\u0442\u043e\u0433\u043e IT-\u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0441\u0442\u044b \u0431\u0430\u043d\u043a\u0430 \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u043b\u0438 \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433, \u043f\u0440\u043e\u0430\u043d\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b\u0438 \u0434\u0430\u043d\u043d\u044b\u0435 \u0438 \u043d\u0430\u0443\u0447\u0438\u043b\u0438\u0441\u044c \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u044b. \u041a\u0430\u043a\u0438\u0435 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u043f\u043e\u043c\u043e\u0433\u043b\u0438 \u0441\u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0443 \u0438 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c \u043b\u0438 \u0441 \u0438\u0445 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29368,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-39133","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Pra\u0107enje + testiranje optere\u0107enja = predvi\u0111anje i izbjegavanje prekida | ProHoster","description":"IT odjel VTB-a nekoliko puta se suo\u010dio s vanrednim situacijama u radu sustava kada je optere\u0107enje na njima vi\u0161estruko poraslo.","canonical_url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hr_HR","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433 + \u043d\u0430\u0433\u0440\u0443\u0437\u043e\u0447\u043d\u043e\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 = \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0438 \u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432\u0438\u0435 \u0441\u0431\u043e\u0435\u0432 | ProHoster","og:description":"IT-\u0434\u0435\u043f\u0430\u0440\u0442\u0430\u043c\u0435\u043d\u0442\u0443 \u0412\u0422\u0411 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437 \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u043b\u043e\u0441\u044c \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0442\u044c\u0441\u044f \u0441 \u043d\u0435\u0448\u0442\u0430\u0442\u043d\u044b\u043c\u0438 \u0441\u0438\u0442\u0443\u0430\u0446\u0438\u044f\u043c\u0438 \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441\u0438\u0441\u0442\u0435\u043c, \u043a\u043e\u0433\u0434\u0430 \u043d\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043d\u0430 \u043d\u0438\u0445 \u043c\u043d\u043e\u0433\u043e\u043a\u0440\u0430\u0442\u043d\u043e \u0432\u043e\u0437\u0440\u0430\u0441\u0442\u0430\u043b\u0430.","og:url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/monitoring-nagruzochnoe-testirovanie-prognozirovanie-i-otsutstvie-sboev","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:28:01+00:00","article:modified_time":"2019-10-31T19:28:01+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"39133","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 00:53:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:55:36","updated":"2026-01-24 00:53:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/39133","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/comments?post=39133"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/39133\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media\/29368"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media?parent=39133"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/categories?post=39133"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/tags?post=39133"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}