{"id":35274,"date":"2019-10-31T22:03:22","date_gmt":"2019-10-31T19:03:22","guid":{"rendered":"https:\/\/prohoster.info\/blog\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk\/"},"modified":"2019-10-31T22:03:22","modified_gmt":"2019-10-31T19:03:22","slug":"razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk","title":{"rendered":"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/8fe88b798a64e2ed8e77dbbc89558827.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKujtojm\u00eb se Elastic Stack bazohet n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash jo-relacionale Elasticsearch, nj\u00eb nd\u00ebrfaqe n\u00eb internet Kibana dhe nd\u00ebrlidh\u00ebs-dh\u00ebn\u00ebs (m\u00eb i njohur Logstash, Beats t\u00eb ndryshme, APM dhe t\u00eb tjera). Nj\u00eb prej shtesave t\u00eb k\u00ebndshme t\u00eb gjith\u00eb k\u00ebtij rrjeti produktesh \u00ebsht\u00eb analiza e t\u00eb dh\u00ebnave me ndihm\u00ebn e algoritmeve t\u00eb m\u00ebsimit makinerik. N\u00eb k\u00ebt\u00eb artikull, ne shqyrtojm\u00eb se \u00e7far\u00eb p\u00ebrfaq\u00ebsojn\u00eb k\u00ebto algoritme. Ju lutemi, lexoni m\u00eb tej.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\nM\u00ebsimi makinerik \u00ebsht\u00eb nj\u00eb funksion me pages\u00eb i Elastic Stack dhe p\u00ebrfshihet n\u00eb paket\u00ebn X-Pack. P\u00ebr t\u00eb filluar ta p\u00ebrdorni, mjafton pas instalimit t\u00eb aktivizoni nj\u00eb prov\u00eb 30-ditore. Pas skadimit t\u00eb periudh\u00ebs s\u00eb prov\u00ebs, mund t\u00eb k\u00ebrkoni mb\u00ebshtetje p\u00ebr zgatjen e saj ose t\u00eb blini nj\u00eb abonim. \u00c7mimi i abonimit nuk llogaritet sipas sasis\u00eb s\u00eb t\u00eb dh\u00ebnave, por sipas numrit t\u00eb nyjeve t\u00eb p\u00ebrdorura. Po, sasia e t\u00eb dh\u00ebnave natyrisht ndikon n\u00eb numrin e nyjeve t\u00eb nevojshme, por ende ky qasje p\u00ebr licencimin \u00ebsht\u00eb m\u00eb humane ndaj buxhetit t\u00eb kompanis\u00eb. N\u00ebse nuk keni nevoj\u00eb p\u00ebr performanc\u00eb t\u00eb lart\u00eb, mund t\u00eb kurseni.<\/p>\n<p>ML n\u00eb Elastic Stack \u00ebsht\u00eb shkruar n\u00eb C++ dhe funksionon jasht\u00eb JVM, n\u00eb t\u00eb cil\u00ebn funksionon vet\u00eb Elasticsearch. K\u00ebshtu q\u00eb procesi (ai \u00ebsht\u00eb quajtur autodetect) konsumon gjith\u00e7ka q\u00eb JVM nuk e p\u00ebrpunon. N\u00eb nj\u00eb demonstrim kjo nuk \u00ebsht\u00eb kaq kritike, por n\u00eb nj\u00eb ambient prodhimi \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb ndahen nyje t\u00eb ve\u00e7anta p\u00ebr detyrat ML.<\/p>\n<p>Algoritmet e m\u00ebsimit makinerik ndahen n\u00eb dy kategori \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/ru.wikipedia.org\/wiki\/%D0%9E%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D0%B5_%D1%81_%D1%83%D1%87%D0%B8%D1%82%D0%B5%D0%BB%D0%B5%D0%BC\">me mbik\u00ebqyrje<\/a><\/noindex> dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/ru.wikipedia.org\/wiki\/%D0%9E%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D0%B5_%D0%B1%D0%B5%D0%B7_%D1%83%D1%87%D0%B8%D1%82%D0%B5%D0%BB%D1%8F\">pa mbik\u00ebqyrje<\/a><\/noindex>. N\u00eb Elastic Stack algoritmi \u00ebsht\u00eb nga kategoria \"pa mbik\u00ebqyrje\". Me <noindex><a rel=\"nofollow\" href=\"http:\/\/www.ijmlc.org\/papers\/398-LC018.pdf\">k\u00ebt\u00eb lidhje<\/a><\/noindex> mund t\u00eb shihni aparatin matematikor t\u00eb algoritmeve t\u00eb m\u00ebsimit makinerik.<\/p>\n<p>P\u00ebr t\u00eb kryer analiz\u00ebn, algoritmi i m\u00ebsimit makinerik p\u00ebrdor t\u00eb dh\u00ebnat e ruajtura n\u00eb indekset Elasticsearch. T\u00eb dh\u00ebna p\u00ebr analiz\u00eb mund t\u00eb krijohen si nga nd\u00ebrfaqja Kibana ashtu edhe p\u00ebrmes API-s\u00eb. N\u00ebse e b\u00ebni k\u00ebt\u00eb p\u00ebrmes Kibana-s, disa gj\u00ebra nuk \u00ebsht\u00eb e nevojshme t\u00eb dihen. P\u00ebr shembull, indekset shtes\u00eb q\u00eb p\u00ebrdor algoritmi gjat\u00eb pun\u00ebs s\u00eb tij. <\/p>\n<p><b class=\"spoiler_title\">Indeksat shtes\u00eb t\u00eb p\u00ebrdorur gjat\u00eb procesit t\u00eb analiz\u00ebs<\/b>.ml-state \u2014 informacion mbi modelet statistikore (caktimet e analiz\u00ebs);<br \/>\n.ml-anomalies-* \u2014 rezultatet e pun\u00ebs s\u00eb algoritmeve ML;<br \/>\n.ml-notifications \u2014 caktimet e njoftimeve mbi rezultatet e analiz\u00ebs.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/a6a67b0c05c6650a158d99010c2872f1.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nStruktura e dh\u00ebnash n\u00eb baz\u00ebn Elasticsearch p\u00ebrb\u00ebhet nga indekse dhe dokumentet q\u00eb ruhen n\u00eb to. N\u00ebse e krahasojm\u00eb me nj\u00eb baz\u00eb t\u00eb dh\u00ebnash relationale, indeksi mund t\u00eb krahasohet me skem\u00ebn e baz\u00ebs s\u00eb t\u00eb dh\u00ebnave, nd\u00ebrsa dokumenti me nj\u00eb rekord n\u00eb tabel\u00eb. Ky krahasim \u00ebsht\u00eb i kusht\u00ebzuar dhe \u00ebsht\u00eb dh\u00ebn\u00eb p\u00ebr t\u00eb thjeshtuar kuptimin e materialeve t\u00eb m\u00ebtejshme p\u00ebr ata q\u00eb kan\u00eb d\u00ebgjuar vet\u00ebm p\u00ebr Elasticsearch.<\/p>\n<p>P\u00ebrmes API-s\u00eb \u00ebsht\u00eb i aksesuesh\u00ebm t\u00eb nj\u00ebjtin funksionalitet si p\u00ebrmes nd\u00ebrfaqes web, prandaj p\u00ebr qart\u00ebsi dhe kuptimin e koncepteve ne do t\u00eb tregojm\u00eb se si t\u00eb konfigurojm\u00eb p\u00ebrmes Kibana. N\u00eb menun\u00eb e majt\u00eb ekziston seksioni Machine Learning, n\u00eb t\u00eb cilin mund t\u00eb krijoni nj\u00eb detyr\u00eb t\u00eb re (Job). N\u00eb nd\u00ebrfaqen Kibana kjo duket si n\u00eb imazhin m\u00eb posht\u00eb. Tani do t\u00eb shqyrtojm\u00eb \u00e7do lloj detyre dhe do t\u00eb tregojm\u00eb llojet e analiz\u00ebs q\u00eb mund t\u00eb krijoni k\u00ebtu.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/c0c0bede29fbe38a72270d2128926122.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSingle Metric \u2014 analiza e nj\u00eb metri, Multi Metric \u2014 analiza e dy apo m\u00eb shum\u00eb metrikeve. N\u00eb t\u00eb dy rastet, \u00e7do metrik\u00eb analizohet n\u00eb nj\u00eb ambient t\u00eb izoluar, dmth. algoritmi nuk merr parasysh sjelljen e metrikeve q\u00eb shqyrtohen paralelisht ashtu si\u00e7 mund t\u00eb dukej n\u00eb rastin e Multi Metric. P\u00ebr t\u00eb kryer llogaritjen duke marr\u00eb parasysh korrelacionin e metrikeve t\u00eb ndryshme, mund t\u00eb aplikoni analiz\u00ebn e Popullat\u00ebs. Dhe Advanced \u2014 \u00ebsht\u00eb cil\u00ebsimi i im\u00ebt i algoritmeve me opsione shtes\u00eb p\u00ebr detyra t\u00eb caktuara. <\/p>\n<h2>Single Metric<\/h2>\n<p>\nAnaliza e ndryshimeve t\u00eb nj\u00eb metri t\u00eb vetme \u2014 \u00ebsht\u00eb gj\u00ebja m\u00eb e thjesht\u00eb q\u00eb mund t\u00eb b\u00ebni k\u00ebtu. Pasi t\u00eb klikoni Create Job, algoritmi do t\u00eb k\u00ebrkoj\u00eb anomali.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/d84d29539c3b97be7a15aac8708bc3be.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb fush\u00ebn <i>Agregimi<\/i> mund t\u00eb zgjidhni qasjen p\u00ebr t\u00eb gjetur anomali. P\u00ebr shembull, kur <i>Min<\/i> do t\u00eb konsiderohen si anomali vlerat n\u00ebn ato tipike. Ka <i>Max, Hign Mean, Low, Mean, Distinct<\/i> dhe t\u00eb tjera. P\u00ebrshkrimi i t\u00eb gjitha funksioneve mund t\u00eb shihet <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/guide\/en\/elastic-stack-overview\/current\/ml-functions.html\">n\u00eb lidhje<\/a><\/noindex>.<\/p>\n<p>N\u00eb fush\u00ebn <i>Fusha<\/i> tregon fush\u00ebn numerike n\u00eb dokument, mbi t\u00eb cil\u00ebn do ta realizojm\u00eb analiz\u00ebn.<\/p>\n<p>N\u00eb fush\u00ebn <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/blog\/explaining-the-bucket-span-in-machine-learning-for-elasticsearch\"><i>Bucket span<\/i><\/a><\/noindex> \u2014 granulariteti i intervaleve n\u00eb koh\u00eblin\u00eb, mbi t\u00eb cilat do t\u00eb b\u00ebhet analiza. Mund t\u2019i besoni automatizimit ose ta zgjidhni manualisht. N\u00eb imazhin m\u00eb posht\u00eb \u00ebsht\u00eb dh\u00ebn\u00eb nj\u00eb shembuj i granularitetit t\u00eb ul\u00ebt \u2014 mund t\u00eb humbni nj\u00eb anomali. Me an\u00eb t\u00eb k\u00ebtij cil\u00ebsimi mund t\u00eb ndryshoni ndjeshm\u00ebrin\u00eb e algoritmit ndaj anomali.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/a2808c0b451703a50d74f7737332ea8b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKoh\u00ebzgjatja e t\u00eb dh\u00ebnave t\u00eb mbledhura \u2014 \u00ebsht\u00eb \u00e7\u00ebshtje ky\u00e7e, e cila ndikon n\u00eb efikasitetin e analiz\u00ebs. Gjat\u00eb analiz\u00ebs, algoritmi p\u00ebrcakton intervalet e p\u00ebrs\u00ebritura, llogarit intervalin e besueshm\u00ebris\u00eb (bazat e linj\u00ebs) dhe identifikon anomali \u2014 devijime t\u00eb pazakonshme nga sjellja e zakonshme e metrikeve. Thjesht p\u00ebr shembull:<\/p>\n<p>Bazat p\u00ebrkat\u00ebse p\u00ebr nj\u00eb sasi t\u00eb vog\u00ebl t\u00eb dh\u00ebnash:<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/2a3cbeaa8cf08a2a5dec87071be27db8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKur algoritmi ka p\u00ebr t\u00eb m\u00ebsuar di\u00e7ka - bazat duken k\u00ebshtu:<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/13d636ab405a85d89e2ae4ebfe8bb6bd.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPas fillimit t\u00eb detyr\u00ebs, algoritmi p\u00ebrcakton devijimet anomale nga norma dhe i rendit ato sipas probabilitetit t\u00eb anomalis\u00eb (n\u00eb kllapa \u00ebsht\u00eb treguar ngjyra e etiket\u00ebs p\u00ebrkat\u00ebse):<\/p>\n<p>Warning (blu): m\u00eb pak se 25<br \/>\nMinor (e verdh\u00eb): 25-50<br \/>\nMajor (portokalli): 50-75<br \/>\nCritical (e kuqe): 75-100<\/p>\n<p>N\u00eb grafik\u00ebn m\u00eb posht\u00eb \u00ebsht\u00eb nj\u00eb shembull me anomali t\u00eb gjetura.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/ed87198f99f0cc93ffc16ce458054a9f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebtu shihet numri 94, i cili tregon probabilitetin e anomalis\u00eb. Kuptohet se, pasi vlera \u00ebsht\u00eb af\u00ebr 100, kemi nj\u00eb anomalie. N\u00eb kolon\u00ebn n\u00ebn grafik \u00ebsht\u00eb shfaqur nj\u00eb probabilitet ekstremisht i vog\u00ebl prej 0.000063634% p\u00ebr shfaqjen e atij vler\u00eb metri\u00e7.<\/p>\n<p>P\u00ebrve\u00e7 gjetjes s\u00eb anomalive, n\u00eb Kibana mund t\u00eb filloni parashikimin. Kjo b\u00ebhet leht\u00ebsisht dhe nga e nj\u00ebjta pamje me anomali - butoni <i>Forecast<\/i> n\u00eb k\u00ebndin e sip\u00ebrm t\u00eb djatht\u00eb.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/73429a947befb8834972ebbfe48c2d48.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nParashikimi nd\u00ebrtomaximumi n\u00eb 8 jav\u00ebt e ardhshme. Edhe n\u00ebse d\u00ebshiron shum\u00eb - nuk lejohet m\u00eb shum\u00eb p\u00ebr arsye dizajni.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/7622608e4b952be741be486f89d1bc43.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb disa situata parashikimi do t\u00eb jet\u00eb shum\u00eb i dobish\u00ebm, p\u00ebr shembull, kur ndiqet ngarkesa e p\u00ebrdoruesve n\u00eb infrastruktur\u00eb.<\/p>\n<h2>Multi Metric<\/h2>\n<p>\nKalojm\u00eb te mund\u00ebsia tjet\u00ebr e ML n\u00eb Elastic Stack - analiza e disa metrikave n\u00eb nj\u00eb grup. Por kjo nuk do t\u00eb thot\u00eb se do t\u00eb analizohet var\u00ebsia e nj\u00eb metrike nga tjetra. Kjo \u00ebsht\u00eb e nj\u00ebjt\u00eb si Single Metric, vet\u00ebm me shum\u00eb metrika n\u00eb nj\u00eb ekran p\u00ebr leht\u00ebsin\u00eb e krahasimit t\u00eb ndikimit t\u00eb nj\u00ebra mbi tjetr\u00ebn. P\u00ebr analiz\u00ebn e var\u00ebsis\u00eb s\u00eb nj\u00eb metrike nga nj\u00ebra tjetra, do t\u00eb flasim n\u00eb pjes\u00ebn Population.<\/p>\n<p>Pas klikimit n\u00eb katrorin e Multi Metric do t\u00eb shfaqet nj\u00eb dritare me cil\u00ebsimet. Ne do t\u00eb ndalemi m\u00eb n\u00eb detaje mbi to.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/eaa06c8b24b393209ef1fb5ba9cfbd62.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nFillimisht duhet zgjedhur fushat p\u00ebr analiz\u00eb dhe agregimin e t\u00eb dh\u00ebnave mbi to. Opcione t\u00eb agregimit k\u00ebtu jan\u00eb t\u00eb nj\u00ebjtat si p\u00ebr Single Metric (<i>Max, Hign Mean, Low, Mean, Distinct<\/i> dhe t\u00eb tjera). M\u00eb pas, t\u00eb dh\u00ebnat, n\u00ebse d\u00ebshirohet, ndahen sipas nj\u00ebrit prej fushave (fusha <i>Split Data<\/i>). N\u00eb shembullin ton\u00eb, e kemi b\u00ebr\u00eb p\u00ebr fush\u00ebn <i>OriginAirportID<\/i>. Vini re se tani grafiku i metrikave n\u00eb t\u00eb djatht\u00eb \u00ebsht\u00eb paraqitur n\u00eb form\u00ebn e shum\u00eb grafik\u00ebve.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/746a91b28a010a037c66a569fcf47a1b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nFusha <i>Key Fields (Influencers)<\/i> ndikon drejtp\u00ebrdrejt n\u00eb anomali t\u00eb gjetura. Nga ana tjet\u00ebr, gjithmon\u00eb do t\u00eb ket\u00eb t\u00eb pakt\u00ebn nj\u00eb vler\u00eb k\u00ebtu, dhe ju mund t\u00eb shtoni t\u00eb tjera. Algoritmi do t\u00eb ket\u00eb parasysh ndikimin e k\u00ebtyre fushave gjat\u00eb analiz\u00ebs dhe do t\u00eb tregoj\u00eb vlerat m\u00eb \"ndikues\".<\/p>\n<p>Pas fillimit, n\u00eb nd\u00ebrfaqen e Kibana do t\u00eb duket af\u00ebrsisht k\u00ebshtu.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/c6efd8f4815ad150b046c82780c36574.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKjo \u00ebsht\u00eb nj\u00eb hart\u00eb nxeht\u00ebsie e anomali\u00ebve p\u00ebr secil\u00ebn vler\u00eb t\u00eb fush\u00ebs <i>OriginAirportID<\/i>, e cila \u00ebsht\u00eb treguar nga ne. <i>Split Data<\/i>. Si n\u00eb rastin e Single Metric, ngjyra tregon nivelin e devijimit anormal. Nj\u00eb analiz\u00eb e ngjashme \u00ebsht\u00eb e dobishme, p\u00ebr shembull, p\u00ebr stacionet e pun\u00ebs p\u00ebr t\u00eb ndjekur ato, ku ka shum\u00eb autorizime t\u00eb dyshimta, etj. Ne kemi shkruar tashm\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/galssoftware\/blog\/447522\/\">p\u00ebr ngjarjet e dyshimta n\u00eb EventLog Windows<\/a><\/noindex>, t\u00eb cilat gjithashtu mund t\u00eb mblidhen dhe analizohen k\u00ebtu.<\/p>\n<p>N\u00ebn hart\u00ebn termike, lista e anomali, nga \u00e7do nj\u00eb mund t\u00eb kaloni n\u00eb pamjen e Single Metric p\u00ebr analiz\u00eb t\u00eb detajuar.<\/p>\n<h2>Popullsia<\/h2>\n<p>\nP\u00ebr t\u00eb k\u00ebrkuar anomali n\u00eb mes t\u00eb korelacioneve midis metrikave t\u00eb ndryshme n\u00eb Elastic Stack, ekziston analiza e specializuar t\u00eb Popullsis\u00eb. Me an\u00eb t\u00eb saj, mund t\u00eb k\u00ebrkoni vlera anormale n\u00eb performanc\u00ebn e ndonj\u00eb serveri n\u00eb krahasim me t\u00eb tjer\u00ebt n\u00eb rast se, p\u00ebr shembull, rritet numri i k\u00ebrkesave n\u00eb sistemin e synuar.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/00542d57f2a937d424b2db5e692a6289.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb k\u00ebt\u00eb ilustrim, n\u00eb fush\u00ebn e Popullsis\u00eb \u00ebsht\u00eb treguar vlera, ndaj s\u00eb cil\u00ebs do t'i referohemi metrikave q\u00eb po analizohet. N\u00eb k\u00ebt\u00eb rast, \u00ebsht\u00eb emri i procesit. Si rezultat, do t\u00eb shohim si ngarkesa e procesorit nga secili nga proceset ka ndikuar n\u00eb nj\u00ebri-tjetrin.<\/p>\n<p>Vini re se grafiku i t\u00eb dh\u00ebnave t\u00eb analizuar \u00ebsht\u00eb ndryshe nga rastet me Single Metric dhe Multi Metric. Kjo \u00ebsht\u00eb b\u00ebr\u00eb n\u00eb Kibana me q\u00ebllim p\u00ebr nj\u00eb perceptim m\u00eb t\u00eb mir\u00eb t\u00eb shp\u00ebrndarjes s\u00eb vlerave t\u00eb t\u00eb dh\u00ebnave t\u00eb analizuar.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/4af704044bf544ca6cd221404288e6c4.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNga grafiku, duket se procesi ka sjell\u00eb anomali <i>stress<\/i> (p\u00ebr ta th\u00ebn\u00eb, i krijuar nga nj\u00eb utilitar t\u00eb ve\u00e7ant\u00eb) n\u00eb serverin <i>poipu<\/i>, i cili ndihmoi (ose ishte influenceri) n\u00eb shfaqjen e k\u00ebsaj anomie.<\/p>\n<h2>T\u00eb avancuara<\/h2>\n<p>\nAnalitika me cil\u00ebsi t\u00eb lart\u00eb. Gjat\u00eb analiz\u00ebs s\u00eb avancuar n\u00eb Kibana, shfaqen cil\u00ebsime t\u00eb tjera. Pas klikimit n\u00eb menun\u00eb e krijimit n\u00eb pllak\u00ebn Advanced shfaqet ky dritare me skeda. Sked\u00ebn <i>Detajet e Pun\u00ebs<\/i> e kemi l\u00ebn\u00eb t\u00eb kaluar q\u00ebllimisht, pasi aty ka cil\u00ebsime bazike q\u00eb nuk lidhen drejtp\u00ebrdrejt me cil\u00ebsimin e analiz\u00ebs.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/abe7941ad56acf3ee7ab992fa5fe7b94.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb <i>emri_fush\u00ebs_s\u00eb_p\u00ebrmbledhjes<\/i> mund t\u00eb specifikohet opsionalisht emri i fush\u00ebs nga dokumentet q\u00eb p\u00ebrmban vlerat e agreguara. N\u00eb k\u00ebt\u00eb rast \u2014 numri i ngjarjeve p\u00ebr minut\u00eb. N\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/guide\/en\/elastic-stack-overview\/current\/ml-configuring-categories.html\"><i>emri_fush\u00ebs_s\u00eb_kategoris\u00eb<\/i><\/a><\/noindex> specifikohet emri i vler\u00ebs s\u00eb fush\u00ebs nga dokumenti, i cili p\u00ebrmban nj\u00eb vler\u00eb t\u00eb caktuar. Sipas mask\u00ebs s\u00eb k\u00ebsaj fushe, mund t\u00eb ndajm\u00eb t\u00eb dh\u00ebnat e analizuar n\u00eb n\u00ebngrupe. Vini re butonin <i>Shto detektor<\/i> n\u00eb ilustrimin e m\u00ebparsh\u00ebm. M\u00eb posht\u00eb \u00ebsht\u00eb rezultati i klikimit n\u00eb k\u00ebt\u00eb buton.<\/p>\n<p><img decoding=\"async\" alt=\"Po merremi me Machine Learning n\u00eb Elastic Stack (ai q\u00eb \u00ebsht\u00eb Elasticsearch, ai q\u00eb \u00ebsht\u00eb ELK)\" src=\"\/wp-content\/uploads\/2019\/06\/cce9a6401d6baa72c68b8e0c3c6ab209.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebtu \u00ebsht\u00eb nj\u00eb bllok shtes\u00eb cil\u00ebsimesh p\u00ebr t\u00eb konfigurimin e detektor\u00ebve t\u00eb anomalis\u00eb p\u00ebr nj\u00eb detyr\u00eb t\u00eb caktuar. Raste specifike p\u00ebrdorimi (ve\u00e7an\u00ebrisht p\u00ebr sigurin\u00eb) ne planifikojm\u00eb t'i shqyrtojm\u00eb n\u00eb artikujt e ardhsh\u00ebm. Si shembull, <noindex><a rel=\"nofollow\" href=\"https:\/\/discuss.elastic.co\/t\/dec-4th-2018-en-ml-rarity-analysis-with-machine-learning\/158979\">shikoni<\/a><\/noindex> nj\u00eb nga rastet e shqyrtuara. Ai lidhet me gjetjen e vlerave q\u00eb shfaqen rrall\u00eb dhe realizohet <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/guide\/en\/elastic-stack-overview\/7.1\/ml-rare-functions.html\">me funksionin rare<\/a><\/noindex>.<\/p>\n<p>N\u00eb fush\u00ebn <i>function<\/i> mund t\u00eb zgjidhet nj\u00eb funksion i caktuar p\u00ebr gjetjen e anomalis\u00eb. P\u00ebrve\u00e7 <i>rare<\/i>, ka edhe disa funksione interesante t\u00eb tjera \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/guide\/en\/elastic-stack-overview\/7.1\/ml-time-functions.html\"><i>time_of_day<\/i> dhe <i>time_of_week<\/i><\/a><\/noindex>. Ato zbulojn\u00eb anomali n\u00eb sjelljen e metrike gjat\u00eb dit\u00ebs ose jav\u00ebs p\u00ebrkat\u00ebsisht. Funksionet e tjera t\u00eb analiz\u00ebs <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/guide\/en\/elastic-stack-overview\/7.1\/ml-functions.html\">jan\u00eb n\u00eb dokumentacion<\/a><\/noindex>.<\/p>\n<p>N\u00eb <i>field_name<\/i> specifikon fush\u00ebn e dokumentit, sipas s\u00eb cil\u00ebs do t\u00eb b\u00ebhet analiza. <i>By_field_name<\/i> mund t\u00eb p\u00ebrdoret p\u00ebr ndarjen e rezultateve t\u00eb analiz\u00ebs sipas \u00e7do vlere t\u00eb ve\u00e7ant\u00eb t\u00eb fush\u00ebs s\u00eb dokumentit t\u00eb specifikuar k\u00ebtu. N\u00ebse mbushni <i>over_field_name<\/i> do t\u00eb rezultoj\u00eb n\u00eb nj\u00eb analiz\u00eb populacioni, t\u00eb cilin ne e shqyrtuam m\u00eb lart. N\u00ebse specifikoni nj\u00eb vler\u00eb n\u00eb <i>partition_field_name<\/i>, at\u00ebher\u00eb p\u00ebr k\u00ebt\u00eb fush\u00eb dokumenti do t\u00eb llogariten bazat e ve\u00e7anta p\u00ebr \u00e7do vler\u00eb (si vler\u00eb mund t\u00eb sh\u00ebrbej\u00eb, p\u00ebr shembull, emri i serverit ose procesit n\u00eb server). N\u00eb <i>exclude_frequent<\/i> mund t\u00eb zgjidhet <i>t\u00eb gjitha<\/i> ose <i>none<\/i>, q\u00eb do t\u00eb thot\u00eb p\u00ebrjashtim (ose p\u00ebrfshirje) e vlerave t\u00eb shpeshta t\u00eb fushave t\u00eb dokumenteve.<\/p>\n<p>N\u00eb k\u00ebt\u00eb artikull p\u00ebrpiqem t'i ofrojm\u00eb nj\u00eb pasqyr\u00eb t\u00eb shkurtuar t\u00eb mund\u00ebsive t\u00eb m\u00ebsimit automatik n\u00eb Elastic Stack, ka ende shum\u00eb detaje pas sken\u00ebs. Na tregoni n\u00eb komentet se cilat raste arrit\u00ebt t'i zgjidhni me ndihm\u00ebn e Elastic Stack dhe p\u00ebr cilat detyra e p\u00ebrdorni. P\u00ebr t\u00eb na kontaktuar, mund t\u00eb p\u00ebrdorni mesazhe personale n\u00eb Habr\u00eb ose <noindex><a rel=\"nofollow\" href=\"https:\/\/gals.software\/solutions\/elasticstack\">form\u00ebn e feedback-ut n\u00eb faqen e internetit<\/a><\/noindex>.<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/galssoftware\/blog\/455387\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041d\u0430\u043f\u043e\u043c\u043d\u0438\u043c, \u0447\u0442\u043e \u0432 \u043e\u0441\u043d\u043e\u0432\u0435 Elastic Stack \u043b\u0435\u0436\u0430\u0442 \u043d\u0435\u0440\u0435\u043b\u044f\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0431\u0430\u0437\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 Elasticsearch, \u0432\u0435\u0431-\u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 Kibana \u0438 \u0441\u0431\u043e\u0440\u0449\u0438\u043a\u0438-\u043e\u0431\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 (\u0441\u0430\u043c\u044b\u0439 \u0438\u0437\u0432\u0435\u0441\u0442\u043d\u044b\u0439 Logstash, \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0435 Beats, APM \u0438 \u0434\u0440\u0443\u0433\u0438\u0435). \u041e\u0434\u043d\u043e \u0438\u0437 \u043f\u0440\u0438\u044f\u0442\u043d\u044b\u0445 \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0439 \u0432\u0441\u0435\u0433\u043e \u043f\u0435\u0440\u0435\u0447\u0438\u0441\u043b\u0435\u043d\u043d\u043e\u0433\u043e \u0441\u0442\u0435\u043a\u0430 \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u043e\u0432 \u2014 \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u043e\u0432 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043c\u044b \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u043c\u0441\u044f \u0447\u0442\u043e \u0438\u0437 \u0441\u0435\u0431\u044f \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u044f\u044e\u0442 \u044d\u0442\u0438 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u044b. \u041f\u0440\u043e\u0441\u0438\u043c \u043f\u043e\u0434 \u043a\u0430\u0442. \u041c\u0430\u0448\u0438\u043d\u043d\u043e\u0435 \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":26487,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35274","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041d\u0430\u043f\u043e\u043c\u043d\u0438\u043c.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0420\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u043c\u0441\u044f \u0441 Machine Learning \u0432 Elastic Stack (\u043e\u043d \u0436\u0435 Elasticsearch, \u043e\u043d \u0436\u0435 ELK) | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041d\u0430\u043f\u043e\u043c\u043d\u0438\u043c.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:03:22+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:03:22+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Po shqyrtojm\u00eb Machine Learning n\u00eb Elastic Stack (i njohur edhe si Elasticsearch, i njohur gjithashtu si ELK) | ProHoster","description":"Kujtojm\u00eb.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0420\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u043c\u0441\u044f \u0441 Machine Learning \u0432 Elastic Stack (\u043e\u043d \u0436\u0435 Elasticsearch, \u043e\u043d \u0436\u0435 ELK) | ProHoster","og:description":"\u041d\u0430\u043f\u043e\u043c\u043d\u0438\u043c.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/razbiraemsya-s-machine-learning-v-elastic-stack-on-zhe-elasticsearch-on-zhe-elk","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:03:22+00:00","article:modified_time":"2019-10-31T19:03:22+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"35274","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 22:37:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:07:28","updated":"2026-01-21 22:37:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/35274","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=35274"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/35274\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/26487"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=35274"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=35274"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=35274"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}