{"id":53278,"date":"2019-11-28T00:00:00","date_gmt":"2019-11-27T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/9-podhodov-dlya-vyyavleniya-anomalij"},"modified":"2020-02-18T14:01:10","modified_gmt":"2020-02-18T11:01:10","slug":"9-podhodov-dlya-vyyavleniya-anomalij","status":"publish","type":"post","link":"https:\/\/prohoster.info\/pt\/blog\/news\/9-podhodov-dlya-vyyavleniya-anomalij","title":{"rendered":"9 abordagens para identificar anomalias","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Em artigo anterior<\/a><\/noindex> Discutimos sobre a previs\u00e3o de s\u00e9ries temporais. Uma continua\u00e7\u00e3o l\u00f3gica seria um artigo sobre a detec\u00e7\u00e3o de anomalias. <\/p>\n

Aplica\u00e7\u00e3o<\/h3>\n

\nA detec\u00e7\u00e3o de anomalias \u00e9 utilizada em \u00e1reas como: <\/p>\n

1) Previs\u00e3o de falhas de equipamentos<\/h3>\n

\nAssim, em 2010, as centr\u00edfugas iranianas foram atacadas pelo v\u00edrus Stuxnet, que configurou um modo de opera\u00e7\u00e3o n\u00e3o otimizado do equipamento e danificou parte dele devido ao desgaste acelera\u00e7\u00e3o. <\/p>\n

Se algoritmos de detec\u00e7\u00e3o de anomalias tivessem sido usados, as situa\u00e7\u00f5es de falha poderiam ter sido evitadas. <\/p>\n

\"9
\n<\/a><\/noindex>
\nA busca por anomalias no funcionamento dos equipamentos \u00e9 utilizada n\u00e3o apenas na ind\u00fastria nuclear, mas tamb\u00e9m na metalurgia e no funcionamento de turbinas aeron\u00e1uticas. E em outras \u00e1reas, onde o uso de diagn\u00f3stico preditivo \u00e9 mais barato do que as poss\u00edveis perdas decorrentes de falhas imprevis\u00edveis.<\/p>\n

2) Previs\u00e3o de atividades fraudulentas<\/h3>\n

\nSe o cart\u00e3o que voc\u00ea usa em Podolsk for usado para retirar dinheiro na Alb\u00e2nia, talvez seja necess\u00e1rio verificar as transa\u00e7\u00f5es com mais cuidado.<\/p>\n

3) Identifica\u00e7\u00e3o de padr\u00f5es de consumo anormais<\/h3>\n

\nSe parte dos clientes demonstrar um comportamento an\u00f4malo, pode haver um problema do qual voc\u00ea n\u00e3o est\u00e1 ciente.<\/p>\n

4) Identifica\u00e7\u00e3o de demanda e carga anormais<\/h3>\n

\nSe as vendas em uma loja de FMCG ca\u00edrem abaixo do limite do intervalo de confian\u00e7a da previs\u00e3o, \u00e9 necess\u00e1rio encontrar a causa do que est\u00e1 acontecendo.<\/p>\n

Abordagens para a detec\u00e7\u00e3o de anomalias<\/h2>\n

<\/p>\n

1) M\u00e9todo de vetores de suporte de uma classe \u2013 One-Class SVM<\/h3>\n

\nApropriado quando os dados no conjunto de treinamento seguem uma distribui\u00e7\u00e3o normal, enquanto os testes cont\u00eam anomalias.<\/p>\n

O m\u00e9todo de vetores de suporte de uma classe constr\u00f3i uma superf\u00edcie n\u00e3o linear ao redor da origem. \u00c9 poss\u00edvel definir um limite de corte para determinar quais dados s\u00e3o considerados an\u00f4malos.<\/p>\n

Com base na experi\u00eancia da nossa equipe DATA4, o One-Class SVM \u00e9 o algoritmo mais utilizado para resolver a tarefa de busca de anomalias. <\/p>\n

\"9
\n<\/p>\n

2) M\u00e9todo do isolamento de florestas \u2013 Isolate Forest<\/h3>\n

\nNo m\u00e9todo de constru\u00e7\u00e3o de \u00e1rvores 'aleat\u00f3rias', os outliers tendem a cair em folhas nas etapas iniciais (em uma pequena profundidade da \u00e1rvore), ou seja, os outliers s\u00e3o mais f\u00e1ceis de 'isolar'. A distin\u00e7\u00e3o de valores an\u00f4malos ocorre nas primeiras itera\u00e7\u00f5es do funcionamento do algoritmo.<\/p>\n

\"9
\n<\/p>\n

3) Envelope el\u00edptico e m\u00e9todos estat\u00edsticos<\/h3>\n

\nUtilizado quando os dados s\u00e3o normalmente distribu\u00eddos. Quanto mais pr\u00f3ximo a medi\u00e7\u00e3o estiver da cauda da mistura de distribui\u00e7\u00f5es, mais an\u00f4malo ser\u00e1 o valor.<\/p>\n

A esta classe tamb\u00e9m pertencem outros m\u00e9todos estat\u00edsticos.<\/p>\n

\"9
\n
\n\"9
\nImagem do site dyakonov.org<\/i><\/p>\n

4) M\u00e9todos m\u00e9tricos<\/h3>\n

\nOs m\u00e9todos incluem algoritmos como k-vizinhos mais pr\u00f3ximos, k-\u00e9simo vizinho, ABOD (dete\u00e7\u00e3o de outliers baseada em \u00e2ngulo) ou LOF (fator de outlier local).<\/p>\n

S\u00e3o adequados se a dist\u00e2ncia entre os valores nas caracter\u00edsticas \u00e9 equivalente ou normalizada (para n\u00e3o medir uma cobra em papagaios). <\/p>\n

O algoritmo k-vizinhos mais pr\u00f3ximos pressup\u00f5e que os valores normais est\u00e3o localizados em uma determinada \u00e1rea do espa\u00e7o multidimensional, e a dist\u00e2ncia at\u00e9 as anomalias ser\u00e1 maior do que at\u00e9 o hiperplano separador. <\/p>\n

\"9
\n<\/p>\n

5) M\u00e9todos de clusteriza\u00e7\u00e3o<\/h3>\n

\nA ess\u00eancia dos m\u00e9todos de clusteriza\u00e7\u00e3o \u00e9 que, se um valor se afasta dos centros dos clusters mais do que um valor determinado, o valor pode ser considerado an\u00f4malo. <\/p>\n

O principal \u00e9 usar um algoritmo que fa\u00e7a a clusteriza\u00e7\u00e3o correta dos dados, o que depende da tarefa espec\u00edfica. <\/p>\n

\"9
\n<\/p>\n

6) M\u00e9todo de componentes principais<\/h3>\n

\nAdequado onde dire\u00e7\u00f5es de maior varia\u00e7\u00e3o da vari\u00e2ncia s\u00e3o destacadas.<\/p>\n

7) Algoritmos baseados em previs\u00e3o de s\u00e9ries temporais<\/h3>\n

\nA ideia \u00e9 que, se um valor se desvia do intervalo de confian\u00e7a da previs\u00e3o, o valor \u00e9 considerado an\u00f4malo. Para prever s\u00e9ries temporais, s\u00e3o usados algoritmos como suaviza\u00e7\u00e3o tripla, S(ARIMA), boosting, etc.<\/p>\n

Os algoritmos de previs\u00e3o de s\u00e9ries temporais foram abordados no artigo anterior.<\/p>\n

\"9
\n<\/p>\n

8) Aprendizado supervisionado (regress\u00e3o, classifica\u00e7\u00e3o)<\/h3>\n

\nSe os dados permitirem, utilizaremos algoritmos desde a regress\u00e3o linear at\u00e9 redes neurais recorrentes. Mediremos a diferen\u00e7a entre a previs\u00e3o e o valor real e tiraremos conclus\u00f5es sobre como os dados se desviam da norma. \u00c9 importante que o algoritmo tenha capacidade de generaliza\u00e7\u00e3o suficiente e a amostra de treinamento n\u00e3o contenha valores an\u00f4malos.<\/p>\n

9) Testes de modelo<\/h3>\n

\nAbordaremos a tarefa de detec\u00e7\u00e3o de anomalias como uma tarefa de recomenda\u00e7\u00e3o. Decompor nossa matriz de caracter\u00edsticas usando SVD ou m\u00e1quinas de fatora\u00e7\u00e3o, e os valores na nova matriz, que diferem significativamente dos originais, ser\u00e3o considerados an\u00f4malos. <\/p>\n

\"9
\n
\nImagem do site dyakonov.org<\/i><\/p>\n

Conclus\u00e3o<\/h3>\n

\nNeste artigo, discutimos as principais abordagens para a detec\u00e7\u00e3o de anomalias. <\/p>\n

A busca por anomalias pode ser considerada uma arte. N\u00e3o existe um algoritmo ou abordagem perfeita que resolva todos os problemas. Frequentemente, utiliza-se um conjunto de m\u00e9todos para resolver um caso espec\u00edfico. A detec\u00e7\u00e3o de anomalias \u00e9 realizada por meio de m\u00e9todos de suporte vetorial de uma classe, florestas isoladoras, m\u00e9todos m\u00e9tricos e de cluster, al\u00e9m do uso de componentes principais e previs\u00e3o de s\u00e9ries temporais.<\/p>\n

Se voc\u00ea conhece outros m\u00e9todos, escreva sobre eles nos coment\u00e1rios do artigo.
\n
Fonte: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043c\u044b \u0433\u043e\u0432\u043e\u0440\u0438\u043b\u0438 \u043e \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445 \u0440\u044f\u0434\u043e\u0432. \u041b\u043e\u0433\u0438\u0447\u043d\u044b\u043c \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0435\u043d\u0438\u0435\u043c \u0431\u0443\u0434\u0435\u0442 \u0441\u0442\u0430\u0442\u044c\u044f \u043e \u0432\u044b\u044f\u0432\u043b\u0435\u043d\u0438\u0438 \u0430\u043d\u043e\u043c\u0430\u043b\u0438\u0439. \u041f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 \u0412\u044b\u044f\u0432\u043b\u0435\u043d\u0438\u0435 \u0430\u043d\u043e\u043c\u0430\u043b\u0438\u0439 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f \u0432 \u0442\u0430\u043a\u0438\u0445 \u043e\u0431\u043b\u0430\u0441\u0442\u044f\u0445 \u043a\u0430\u043a: 1) \u041f\u0440\u0435\u0434\u0441\u043a\u0430\u0437\u0430\u043d\u0438\u0435 \u043f\u043e\u043b\u043e\u043c\u043e\u043a \u043e\u0431\u043e\u0440\u0443\u0434\u043e\u0432\u0430\u043d\u0438\u044f \u0422\u0430\u043a, \u0432 2010 \u0433\u043e\u0434\u0443 \u0418\u0440\u0430\u043d\u0441\u043a\u0438\u0435 \u0446\u0435\u043d\u0442\u0440\u0438\u0444\u0443\u0433\u0438 \u0431\u044b\u043b\u0438 \u0430\u0442\u0430\u043a\u043e\u0432\u0430\u043d\u044b \u0432\u0438\u0440\u0443\u0441\u043e\u043c Stuxnet, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u0434\u0430\u043b \u043d\u0435\u043e\u043f\u0442\u0438\u043c\u0430\u043b\u044c\u043d\u044b\u0439 \u0440\u0435\u0436\u0438\u043c \u0440\u0430\u0431\u043e\u0442\u044b \u043e\u0431\u043e\u0440\u0443\u0434\u043e\u0432\u0430\u043d\u0438\u044f \u0438 \u0432\u044b\u0432\u0435\u043b \u0438\u0437 \u0441\u0442\u0440\u043e\u044f \u0447\u0430\u0441\u0442\u044c \u043e\u0431\u043e\u0440\u0443\u0434\u043e\u0432\u0430\u043d\u0438\u044f \u0437\u0430 \u0441\u0447\u0435\u0442 \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u043d\u043e\u0433\u043e \u0438\u0437\u043d\u043e\u0441\u0430. \u0415\u0441\u043b\u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-53278","post","type-post","status-publish","format-standard","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd479 abordagens para detec\u00e7\u00e3o de anomalias | ProHoster","description":"No artigo anterior, falamos sobre a previs\u00e3o de s\u00e9ries temporais. Uma continua\u00e7\u00e3o l\u00f3gica seria um artigo sobre a detec\u00e7\u00e3o de anomalias.","canonical_url":"https:\/\/prohoster.info\/pt\/blog\/news\/9-podhodov-dlya-vyyavleniya-anomalij","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"pt_PT","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd479 \u043f\u043e\u0434\u0445\u043e\u0434\u043e\u0432 \u0434\u043b\u044f \u0432\u044b\u044f\u0432\u043b\u0435\u043d\u0438\u044f \u0430\u043d\u043e\u043c\u0430\u043b\u0438\u0439 | ProHoster","og:description":"\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043c\u044b \u0433\u043e\u0432\u043e\u0440\u0438\u043b\u0438 \u043e \u043f\u0440\u043e\u0433\u043d\u043e\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445 \u0440\u044f\u0434\u043e\u0432. \u041b\u043e\u0433\u0438\u0447\u043d\u044b\u043c \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0435\u043d\u0438\u0435\u043c \u0431\u0443\u0434\u0435\u0442 \u0441\u0442\u0430\u0442\u044c\u044f \u043e \u0432\u044b\u044f\u0432\u043b\u0435\u043d\u0438\u0438 \u0430\u043d\u043e\u043c\u0430\u043b\u0438\u0439.","og:url":"https:\/\/prohoster.info\/pt\/blog\/news\/9-podhodov-dlya-vyyavleniya-anomalij","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-11-27T21:00:00+00:00","article:modified_time":"2020-02-18T11:01:10+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"53278","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 06:46:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:28:47","updated":"2026-01-24 06:46:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts\/53278","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/comments?post=53278"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts\/53278\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/media?parent=53278"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/categories?post=53278"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/tags?post=53278"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}