Tiyeni tikumbukire kuti Elastic Stack imachokera ku database ya Elasticsearch yosagwirizana, mawonekedwe a intaneti a Kibana ndi osonkhanitsa deta ndi ma processor (Logstash wotchuka kwambiri, Beats zosiyanasiyana, APM ndi ena). Chimodzi mwazowonjezera zabwino pagulu lonse lazinthu zomwe zalembedwa ndikusanthula deta pogwiritsa ntchito makina ophunzirira makina. M'nkhaniyi timvetsetsa zomwe ma algorithms awa ndi. Chonde pansi pa mphaka.
Kuphunzira pamakina ndi gawo lolipidwa la shareware Elastic Stack ndipo likuphatikizidwa mu X-Pack. Kuti muyambe kugwiritsa ntchito, ingoyambitsani kuyesa kwamasiku 30 mutakhazikitsa. Nthawi yoyeserera ikatha, mutha kupempha thandizo kuti muwonjezere kapena kugula zolembetsa. Mtengo wolembetsa umawerengedwa osati potengera kuchuluka kwa deta, koma kuchuluka kwa node zomwe zimagwiritsidwa ntchito. Ayi, kuchuluka kwa deta, ndithudi, kumakhudza chiwerengero cha node zofunika, komabe njira iyi yoperekera chilolezo ndi yaumunthu kwambiri pokhudzana ndi bajeti ya kampani. Ngati palibe chifukwa cha zokolola zambiri, mukhoza kusunga ndalama.
ML mu Elastic Stack imalembedwa mu C ++ ndipo imayenda kunja kwa JVM, momwe Elasticsearch imayendera. Ndiko kuti, ndondomekoyi (mwa njira, imatchedwa autodetect) imadya chirichonse chimene JVM sichimeza. Pa chiwonetsero chaziwonetsero izi sizofunikira kwambiri, koma m'malo opanga ndikofunikira kugawa ma node osiyana a ntchito za ML.
Ma algorithms ophunzirira makina amagwera m'magulu awiri - ΠΈ . Mu Elastic Stack, algorithm ili m'gulu la "osayang'anira". Wolemba Mutha kuwona zida zamasamu zama makina ophunzirira makina.
Kuti muwunike, makina ophunzirira makina amagwiritsa ntchito deta yosungidwa muzolozera za Elasticsearch. Mutha kupanga ntchito zowunikira kuchokera pa mawonekedwe a Kibana komanso kudzera pa API. Ngati muchita izi kudzera ku Kibana, ndiye kuti simuyenera kudziwa zinthu zina. Mwachitsanzo, ma index owonjezera omwe algorithm amagwiritsa ntchito panthawi yake.
Zizindikiro zowonjezera zomwe zimagwiritsidwa ntchito pofufuza.ml-state - zambiri za zitsanzo zowerengera (zokonda kusanthula);
.ml-anomalies-* - zotsatira za ma aligorivimu a ML;
.ml-notifications - zokonda pazidziwitso zochokera ku zotsatira zowunikira.
Kapangidwe ka data mu nkhokwe ya Elasticsearch imakhala ndi ma index ndi zolemba zosungidwa momwemo. Poyerekeza ndi nkhokwe yaubale, index ikhoza kufananizidwa ndi schema ya database, ndi chikalata cholemba patebulo. Kufananitsaku kuli ndi zovomerezeka ndipo kumaperekedwa kuti muchepetse kumvetsetsa kwazinthu zina kwa iwo omwe adangomva za Elasticsearch.
Zochita zomwezo zimapezeka kudzera mu API monga kudzera pa intaneti, kotero kuti timveke bwino komanso kumvetsetsa mfundozo, tidzasonyeza momwe tingakonzere kudzera ku Kibana. Mu menyu kumanzere pali Machine Learning gawo kumene mukhoza kupanga Ntchito yatsopano. Mu mawonekedwe a Kibana zikuwoneka ngati chithunzi pansipa. Tsopano tisanthula mtundu uliwonse wa ntchito ndikuwonetsa mitundu ya kusanthula yomwe ingamangidwe apa.
Single Metric - kusanthula kwa metric imodzi, Multi Metric - kusanthula ma metric awiri kapena kupitilira apo. Muzochitika zonsezi, metric iliyonse imawunikidwa pamalo akutali, i.e. ma aligorivimu saganiziranso khalidwe la ma metric omwe amawunikidwa mofanana, monga momwe zingawonekere pa Multi Metric. Kuti muwerenge potengera kulumikizika kwa ma metric osiyanasiyana, mutha kugwiritsa ntchito kusanthula kwa chiwerengero cha anthu. Ndipo Advanced ndikukonza ma aligorivimu ndi zina zowonjezera pazosankha zina.
Metric Yokha
Kusanthula zosintha mu metric imodzi ndi chinthu chosavuta chomwe chingachitike apa. Pambuyo podina Pangani Ntchito, algorithm idzayang'ana zolakwika.
M'munda Mgwirizano mukhoza kusankha njira kufufuza anomalies. Mwachitsanzo, pamene Mphindi Makhalidwe omwe ali pansipa amaonedwa kuti ndi odabwitsa. Idyani Max, High Mean, Low, Mean, Distinct ndi ena. Mafotokozedwe a ntchito zonse angapezeke .
M'munda Field ikuwonetsa gawo la manambala mu chikalata chomwe tidzasanthula.
M'munda - granularity of intervality pa nthawi imene kusanthula kudzachitidwa. Mutha kukhulupirira zodzichitira nokha kapena kusankha pamanja. Chithunzi chomwe chili pansipa ndi chitsanzo cha granularity kukhala yotsika kwambiri - mutha kuphonya zovutazo. Pogwiritsa ntchito izi, mutha kusintha kukhudzika kwa algorithm kukhala zolakwika.
Kutalika kwa deta yomwe yasonkhanitsidwa ndi chinthu chofunika kwambiri chomwe chimakhudza kugwira ntchito kwa kusanthula. Pakuwunika, algorithm imazindikira kubwerezabwereza, kuwerengera nthawi yodalirika (zoyambira) ndikuzindikira zolakwika - zopatuka kuchokera kumayendedwe wamba a metric. Mwachitsanzo:
Zoyambira zokhala ndi data yaying'ono:
Pamene ma algorithm ali ndi china choti aphunzirepo, zoyambira zimawoneka motere:
Pambuyo poyambitsa ntchitoyi, ma aligorivimu amasankha zopatuka modabwitsa kuchokera pamwambo ndikuziyika molingana ndi kuthekera kwa kusokonezeka (mtundu wa chizindikiro chofananira ukuwonetsedwa m'makolo):
Chenjezo (buluu): zosakwana 25
Zochepa (zachikasu): 25-50
Chachikulu (lalanje): 50-75
Zovuta (zofiira): 75-100
Grafu ili m'munsiyi ikuwonetsa chitsanzo cha zolakwika zomwe zapezeka.
Apa mutha kuwona nambala 94, yomwe ikuwonetsa kuthekera kwa zovuta. Zikuwonekeratu kuti popeza mtengo uli pafupi ndi 100, zikutanthauza kuti tili ndi vuto. Mzere womwe uli pansipa graph ukuwonetsa kuthekera kwakung'ono kocheperako kwa 0.000063634% ya mtengo wa metric womwe ukuwonekera pamenepo.
Kuphatikiza pakusaka zolakwika, mutha kuyendetsa kulosera ku Kibana. Izi zimachitika mosavuta komanso kuchokera kumalingaliro omwewo ndi anomalies - batani Mapa mu ngodya yapamwamba kumanja.
Zoloserazo zimapangidwira kwa masabata 8 pasadakhale. Ngakhale ngati mukufunadi, sizingatheke ndi mapangidwe.
Nthawi zina, kuneneratu kudzakhala kothandiza kwambiri, mwachitsanzo, poyang'anira kuchuluka kwa ogwiritsa ntchito pazomangamanga.
Multi Metric
Tiyeni tipitirire ku gawo lotsatira la ML mu Elastic Stack - kusanthula ma metric angapo mugulu limodzi. Koma izi sizikutanthauza kuti kudalira kwa metric imodzi pa ina kudzawunikidwa. Izi ndizofanana ndi Single Metric, koma zokhala ndi ma metric angapo pa sikirini imodzi kuti mufananize mosavuta kukhudzika kwa wina ndi mnzake. Tikambirana za kusanthula kudalira kwa metric imodzi pa ina mu gawo la Population.
Mukadina pa lalikulu ndi Multi Metric, zenera lokhala ndi zoikamo lidzawoneka. Tiyeni tione mwatsatanetsatane.
Choyamba muyenera kusankha minda yowunikira ndi kusonkhanitsa deta pa iwo. Zosankha zophatikiza apa ndizofanana ndi Single Metric (Max, High Mean, Low, Mean, Distinct ndi ena). Komanso, ngati mukufuna, deta imagawidwa m'modzi mwa magawo (field Gawani Data). Mu chitsanzo, tinachita izi ndi munda OriginAirportID. Zindikirani kuti ma metrics graph kumanja tsopano akuperekedwa ngati ma graph angapo.
m'munda Key Fields (Othandizira) mwachindunji zimakhudza wapezeka anomalies. Mwachikhazikitso nthawi zonse padzakhala mtengo umodzi pano, ndipo mukhoza kuwonjezera zina. Algorithm idzaganiziranso kukhudzidwa kwa magawowa posanthula ndikuwonetsa zikhalidwe "zamphamvu" kwambiri.
Pambuyo poyambitsa, chinthu chonga ichi chidzawonekera mu mawonekedwe a Kibana.
Izi ndi zomwe zimatchedwa kutentha mapu a anomalies pa mtengo uliwonse munda OriginAirportID, zomwe tidaziwonetsa Gawani Data. Monga Single Metric, mtundu umasonyeza mulingo wa kupatuka kwachilendo. Ndikosavuta kusanthula kofananira, mwachitsanzo, pamalo ogwirira ntchito kuti mufufuze omwe ali ndi zilolezo zokayikitsa, ndi zina. Tinalemba kale , zomwe zingathenso kusonkhanitsidwa ndikuwunikidwa pano.
Pansi pa mapu a kutentha pali mndandanda wa zolakwika, kuchokera pa chilichonse chomwe mungasinthe kupita ku Single Metric view kuti muwunike mwatsatanetsatane.
Anthu
Kuti muwone kusiyana pakati pa ma metrics osiyanasiyana, Elastic Stack ili ndi kusanthula kwapadera kwa chiwerengero cha anthu. Ndi chithandizo chake mutha kuyang'ana zinthu zosasangalatsa pakugwira ntchito kwa seva poyerekeza ndi ena pomwe, mwachitsanzo, kuchuluka kwa zopempha kuzomwe mukufuna kumawonjezeka.
Muchifanizo ichi, gawo la Population likuwonetsa mtengo womwe ma metric omwe akuwunikidwa angagwirizane nawo. Pankhaniyi ndi dzina la ndondomeko. Zotsatira zake, tiwona momwe kuchuluka kwa purosesa kwa njira iliyonse kumakhudzira wina ndi mnzake.
Chonde dziwani kuti graph ya data yomwe yasanthulidwa imasiyana ndi yomwe ili ndi Single Metric ndi Multi Metric. Izi zidachitika ku Kibana popanga malingaliro owongolera a kugawa kwazinthu zomwe zawunikidwa.
Grafu ikuwonetsa kuti ntchitoyi idachita bwino kupanikizika (mwa njira, yopangidwa ndi ntchito yapadera) pa seva poipa, yemwe adalimbikitsa (kapena adakhala wolimbikitsa) kuchitika kwa vuto ili.
zotsogola
Analytics ndi kukonza bwino. Ndi kusanthula Kwapamwamba, zosintha zina zimawonekera ku Kibana. Pambuyo podina pa Advanced tile mu menyu yolenga, zenera ili ndi ma tabu likuwonekera. Tabu Zolemba za Job Tidalumphira dala, pali zosintha zoyambira zomwe sizikugwirizana mwachindunji ndi kukhazikitsa kusanthula.
Π summary_count_field_name Mwachidziwitso, mutha kutchula dzina la gawo kuchokera muzolemba zomwe zili ndi ma aggregated values. Mu chitsanzo ichi, chiwerengero cha zochitika pamphindi. MU imasonyeza dzina ndi mtengo wa gawo kuchokera mu chikalata chomwe chili ndi mtengo wosinthika. Pogwiritsa ntchito chigoba pamundawu, mutha kugawa zomwe zawunikidwa m'magulu ang'onoang'ono. Samalani ndi batani Onjezani chowunikira m'fanizo lapitalo. Pansipa pali zotsatira zakudina batani ili.
Pano pali chipika chowonjezera cha zoikamo kuti mukhazikitse chowunikira cha anomaly pa ntchito inayake. Tikukonzekera kukambirana milandu yeniyeni yogwiritsira ntchito (makamaka chitetezo) m'nkhani zotsatirazi. Mwachitsanzo, chimodzi mwazinthu zomwe zasinthidwa. Zimalumikizidwa ndikusaka kwazinthu zomwe sizikuwoneka kawirikawiri ndipo zimakhazikitsidwa .
M'munda ntchito Mukhoza kusankha ntchito yeniyeni kuti mufufuze zolakwika. Kupatulapo zosawerengeka, pali ntchito zina zingapo zosangalatsa - . Amazindikira zolakwika pamakhalidwe a metrics tsiku lonse kapena sabata, motsatana. Ntchito zina zowunikira .
Π field_name limasonyeza gawo la chikalata chimene kusanthula kudzachitidwa. By_field_name angagwiritsidwe ntchito kulekanitsa zotsatira za kusanthula kwa mtengo uliwonse wa zolemba zomwe zatchulidwa pano. Ngati mudzaze over_field_name mumapeza kuchuluka kwa anthu komwe takambirana pamwambapa. Ngati mufotokoza mtengo mu partition_field_name, ndiye pa gawo ili la chikalata choyambira chosiyana chidzawerengedwa pa mtengo uliwonse (mtengo ukhoza kukhala, mwachitsanzo, dzina la seva kapena ndondomeko pa seva). MU kupatula_nthawi zambiri angathe kusankha onse kapena palibe, zomwe zitanthauza kusaphatikiza (kapena kuphatikiza) zikalata zomwe zimachitika pafupipafupi.
Munkhaniyi, tayesera kupereka lingaliro lalifupi momwe tingathere za kuthekera kwa kuphunzira pamakina mu Elastic Stack; padakali zambiri zomwe zatsala m'mbuyo. Tiuzeni mu ndemanga zomwe mwakwanitsa kuthetsa pogwiritsa ntchito Elastic Stack ndi ntchito ziti zomwe mumazigwiritsa ntchito. Kuti mulankhule nafe, mutha kugwiritsa ntchito mauthenga anu pa HabrΓ© kapena .
Source: www.habr.com