Bari mu tuna cewa Elastic Stack yana dogara ne akan bayanan Elasticsearch maras alaƙa, ƙirar gidan yanar gizon Kibana da masu tattara bayanai da masu sarrafawa (mafi shaharar Logstash, Beats daban-daban, APM da sauransu). Ɗaya daga cikin kyawawan abubuwan da aka haɓaka ga duk tarin samfuran da aka jera shine nazarin bayanai ta amfani da algorithms koyon inji. A cikin labarin mun fahimci abin da waɗannan algorithms suke. Don Allah a ƙarƙashin cat.
Koyon inji siffa ce ta biya ta shareware Elastic Stack kuma an haɗa ta cikin Fakitin X. Don fara amfani da shi, kawai kunna gwajin kwanaki 30 bayan shigarwa. Bayan lokacin gwaji ya ƙare, zaku iya neman tallafi don tsawaita shi ko siyan biyan kuɗi. Ana ƙididdige kuɗin biyan kuɗi ba bisa adadin bayanai ba, amma akan adadin nodes ɗin da aka yi amfani da su. A'a, adadin bayanai, ba shakka, yana rinjayar adadin nodes ɗin da ake buƙata, amma har yanzu wannan hanyar yin lasisi ya fi ɗan adam dangane da kasafin kuɗin kamfanin. Idan babu bukatar high yawan aiki, za ka iya ajiye kudi.
ML a cikin Elastic Stack an rubuta shi a cikin C++ kuma yana gudana a wajen JVM, wanda Elasticsearch kanta ke gudana. Wato, tsarin (ta hanyar, ana kiran shi autodetect) yana cinye duk abin da JVM ba ya haɗiye. A kan nunin nuni wannan ba shi da mahimmanci, amma a cikin yanayin samarwa yana da mahimmanci a ware nodes daban don ayyukan ML.
Algorithms na koyon inji sun faɗi kashi biyu - и . A cikin Stack Elastic, algorithm yana cikin nau'in "marasa kulawa". By Kuna iya ganin kayan aikin lissafi na algorithms na koyon inji.
Don yin bincike, na'urar koyo algorithm tana amfani da bayanan da aka adana a cikin fihirisar Elasticsearch. Kuna iya ƙirƙirar ayyuka don bincike duka daga mahaɗin Kibana kuma ta hanyar API. Idan kun yi haka ta hanyar Kibana, to ba kwa buƙatar sanin wasu abubuwa. Misali, ƙarin fihirisar da algorithm ke amfani da shi yayin aikinsa.
Ƙarin fihirisa da aka yi amfani da su a cikin tsarin bincike.ml-state - bayani game da ƙirar ƙididdiga (saitunan nazari);
.ml-anomalies-* - sakamakon ML algorithms;
.ml-sanarwa - saituna don sanarwa dangane da sakamakon bincike.
Tsarin bayanai a cikin Elasticsearch database ya ƙunshi fihirisa da takaddun da aka adana a cikinsu. Idan aka kwatanta da bayanan da ke da alaƙa, ana iya kwatanta fihirisa da tsarin tsarin bayanai, da kuma daftarin aiki zuwa rikodi a cikin tebur. Wannan kwatancen sharadi ne kuma an bayar da shi don sauƙaƙe fahimtar ƙarin abu ga waɗanda kawai suka ji game da Elasticsearch.
Ana samun ayyuka iri ɗaya ta hanyar API kamar ta hanyar haɗin yanar gizo, don haka don tsabta da fahimtar ra'ayoyin, za mu nuna yadda za a daidaita shi ta hanyar Kibana. A cikin menu na hagu akwai sashin Koyon Injin inda zaku iya ƙirƙirar sabon Aiki. A cikin mu'amalar Kibana yana kama da hoton da ke ƙasa. Yanzu za mu bincika kowane nau'in aiki kuma mu nuna nau'ikan bincike da za a iya ginawa a nan.
Single Metric - nazarin awo ɗaya, Multi Metric - nazarin ma'auni biyu ko fiye. A kowane hali, kowane ma'auni ana nazarinsa a cikin keɓe muhalli, watau. Algorithm din ba ya la'akari da halayen ma'auni da aka bincika a layi daya, kamar yadda ake iya gani a yanayin Multi Metric. Don aiwatar da lissafin la'akari da daidaitawar ma'auni daban-daban, zaku iya amfani da ƙididdigar yawan jama'a. Kuma Advanced yana daidaita algorithms tare da ƙarin zaɓuɓɓuka don wasu ayyuka.
Ma'auni guda ɗaya
Yin nazarin canje-canje a cikin awo guda ɗaya shine mafi sauƙin abin da za'a iya yi anan. Bayan danna Ƙirƙiri Ayuba, algorithm zai nemi abubuwan da ba su da kyau.
A cikin filin Tarayya za ka iya zabar hanyar binciko abubuwan da ba su da kyau. Misali, lokacin min Za a yi la'akari da dabi'un da ke ƙasa da ƙima mara kyau. Ku ci Matsakaicin, Babban Ma'ana, Ƙananan, Ma'ana, Bambanci da sauransu. Ana iya samun bayanin duk ayyuka .
A cikin filin Field yana nuna filin lamba a cikin takardar da za mu gudanar da bincike a kai.
A cikin filin - granularity na tazara a kan tsarin lokaci wanda za a gudanar da bincike. Kuna iya amincewa da aikin sarrafa kansa ko zaɓi da hannu. Hoton da ke ƙasa misali ne na ƙwaƙƙwaran ƙanƙanta da yawa - ƙila za ku rasa rashin ƙarfi. Amfani da wannan saitin, zaku iya canza hankalin algorithm zuwa abubuwan da ba su da kyau.
Tsawon lokacin bayanan da aka tattara shine muhimmin abu wanda ke shafar tasirin bincike. A yayin bincike, algorithm ɗin yana gano tazara mai maimaitawa, yana ƙididdige tazarar amincewa (baselines) kuma yana gano abubuwan da ba su dace ba - sabawa dabi'u daga dabi'un da aka saba na awo. Misali kawai:
Baselines tare da ƙaramin yanki na bayanai:
Lokacin da algorithm yana da wani abu don koyo daga, tushen tushe yayi kama da haka:
Bayan fara aikin, algorithm ɗin yana ƙayyade ɓarna mara kyau daga al'ada kuma yana sanya su gwargwadon yuwuwar anomaly (launi na alamar da ke daidai yana nuna a cikin bakan gizo):
Gargaɗi (blue): ƙasa da 25
Ƙananan (rawaya): 25-50
Manyan (orange): 50-75
Mahimmanci (ja): 75-100
Jadawalin da ke ƙasa yana nuna misalin abubuwan da aka samu.
Anan zaka iya ganin lamba 94, wanda ke nuna yuwuwar kamuwa da cuta. A bayyane yake cewa tun da darajar ta kusan kusan 100, yana nufin cewa muna da anomaly. Shagon da ke ƙasa da jadawali yana nuna ƙaramin yuwuwar 0.000063634% na ƙimar awo da ke bayyana a wurin.
Baya ga neman abubuwan da ba su da kyau, kuna iya gudanar da hasashen kibana. Ana yin wannan a sauƙaƙe kuma daga ra'ayi ɗaya tare da anomalies - maballin forecast a kusurwar dama ta sama.
Ana yin hasashen na tsawon makonni 8 a gaba. Ko da da gaske kuna so, ba zai yiwu ta ƙira ba.
A wasu yanayi, hasashen zai zama da amfani sosai, misali, lokacin sa ido kan nauyin mai amfani akan abubuwan more rayuwa.
Multi Metric
Bari mu matsa zuwa fasalin ML na gaba a cikin Stack Elastic - nazarin ma'auni da yawa a cikin tsari ɗaya. Amma wannan ba yana nufin cewa za a yi nazarin dogaron awo ɗaya akan wani ba. Wannan daidai yake da Single Metric, amma tare da ma'auni masu yawa akan allo ɗaya don sauƙin kwatanta tasirin ɗayan akan wani. Za mu yi magana game da nazarin dogaron awo ɗaya akan wani a cikin ɓangaren yawan jama'a.
Bayan danna kan square tare da Multi Metric, taga mai saituna zai bayyana. Bari mu duba su dalla-dalla.
Da farko kuna buƙatar zaɓar filayen don bincike da tattara bayanai akan su. Zaɓuɓɓukan tarawa anan iri ɗaya ne da na Single Metric (Matsakaicin, Babban Ma'ana, Ƙananan, Ma'ana, Bambanci da sauransu). Bugu da ari, idan ana so, an raba bayanan zuwa ɗaya daga cikin filayen (filin Rarraba Bayanai). A cikin misalin, mun yi wannan ta filin OriginAirportID. Yi la'akari da cewa ma'aunin ma'aunin da ke hannun dama an gabatar da shi azaman jadawali da yawa.
filin Mabuɗin Filaye (Masu Tasiri) kai tsaye yana shafar abubuwan da aka gano. Ta tsohuwa koyaushe za a kasance aƙalla ƙima ɗaya a nan, kuma kuna iya ƙara ƙarin. Algorithm din zai yi la'akari da tasirin waɗannan filayen lokacin da ake nazari da nuna mafi yawan ƙimar "tasiri".
Bayan ƙaddamarwa, wani abu kamar wannan zai bayyana a cikin ƙirar Kibana.
Wannan shi ne abin da ake kira zafi taswirar anomalies ga kowane filin darajar OriginAirportID, wanda muka yi nuni a ciki Rarraba Bayanai. Kamar yadda yake tare da Single Metric, launi yana nuna matakin rashin daidaituwa. Yana da dacewa don yin irin wannan bincike, alal misali, akan wuraren aiki don bin diddigin waɗanda ke da babban adadin izini, da sauransu. Mun riga mun rubuta , wanda kuma za'a iya tattarawa da yin nazari a nan.
Ƙarƙashin taswirar zafi akwai jerin abubuwan da ba su dace ba, daga kowane za ku iya canzawa zuwa duban Metric Single don cikakken bincike.
Population
Don nemo rashin daidaituwa tsakanin ma'auni daban-daban, Elastic Stack yana da ƙididdigar yawan jama'a na musamman. Tare da taimakonsa zaku iya nemo ƙima mara kyau a cikin aikin uwar garken idan aka kwatanta da wasu lokacin da, alal misali, adadin buƙatun ga tsarin manufa ya ƙaru.
A cikin wannan kwatancin, filin yawan jama'a yana nuna ƙimar da ma'aunin da aka bincika zai danganta. A wannan yanayin shine sunan tsarin. A sakamakon haka, za mu ga yadda nauyin sarrafawa na kowane tsari ya rinjayi juna.
Lura cewa jadawali na bayanan da aka bincika ya bambanta da lamuran tare da Single Metric da Multi Metric. An yi wannan a Kibana ta hanyar ƙira don ingantacciyar fahimta game da rarraba ƙimar bayanan da aka bincika.
Jadawalin ya nuna cewa tsarin ya yi rashin daidaituwa danniya (ta hanyar, ƙirƙirar ta hanyar amfani na musamman) akan uwar garken poipu, wanda ya rinjayi (ko ya zama mai tasiri) faruwar wannan anomaly.
Na ci gaba
Bincike tare da daidaitawa mai kyau. Tare da Babba bincike, ƙarin saituna suna bayyana a Kibana. Bayan danna kan Advanced tile a cikin menu na halitta, wannan taga tare da shafuka yana bayyana. Tab Ayyukan Ayyukan Mun tsallake shi da gangan, akwai saitunan asali waɗanda ba su da alaƙa kai tsaye da kafa bincike.
В summary_count_field_name Da zaɓin, zaku iya ƙididdige sunan filin daga takaddun da ke ɗauke da ƙima. A cikin wannan misali, adadin abubuwan da suka faru a minti daya. IN yana nuna suna da ƙimar fili daga takaddar da ke ɗauke da wasu ƙima mai ma'ana. Yin amfani da abin rufe fuska a kan wannan filin, zaku iya raba bayanan da aka bincika cikin ɓangarori. Kula da maɓallin Ƙara mai ganowa a cikin misalin da ya gabata. A ƙasa akwai sakamakon danna wannan maɓallin.
Anan akwai ƙarin toshe saitunan don saita mai gano abubuwan da ba su da kyau don takamaiman aiki. Muna shirin tattauna takamaiman batutuwan amfani (musamman na tsaro) a cikin talifofin da ke gaba. Misali, daya daga cikin abubuwan da aka tarwatsa. Yana da alaƙa da bincike don bayyana dabi'u da wuya kuma ana aiwatar da shi .
A cikin filin aiki Kuna iya zaɓar takamaiman aiki don bincika abubuwan da ba su da kyau. Sai dai rare, akwai wasu ƙarin ayyuka masu ban sha'awa - . Suna gano rashin daidaituwa a cikin halayen awo a cikin yini ko mako, bi da bi. Sauran ayyukan bincike .
В filin_name yana nuna filin daftarin aiki da za a gudanar da bincike. Da_filin_suna ana iya amfani da shi don raba sakamakon bincike don kowane ƙimar filin daftarin aiki da aka kayyade anan. Idan kun cika over_field_name kuna samun ƙididdigar yawan jama'a da muka tattauna a sama. Idan ka ƙayyade ƙima a ciki partition_field_name, to, don wannan filin na daftarin aiki, za a ƙididdige mahimman bayanai na kowane ƙima (ƙimar na iya zama, alal misali, sunan uwar garken ko tsari akan sabar). IN ware_yawanci iya zabar dukan ko m, wanda ke nufin ban da (ko haɗawa) ƙimar filin daftarin aiki akai-akai.
A cikin wannan labarin, mun yi ƙoƙarin bayar da taƙaitaccen ra'ayi game da iyawar koyon injin a cikin Stack Elastic; har yanzu akwai cikakkun bayanai da aka bari a bayan fage. Faɗa mana a cikin maganganun abubuwan da kuka sami nasarar warware ta amfani da Elastic Stack da waɗanne ayyuka kuke amfani da su. Don tuntuɓar mu, zaku iya amfani da saƙonnin sirri akan Habré ko .
source: www.habr.com