Ukuthuthukiswa kwezimboni zezinhlelo zesofthiwe kudinga ukunakwa okukhulu ekubekezeleleni amaphutha komkhiqizo wokugcina, kanye nokusabela ngokushesha ekuhlulekeni nasekuhlulekeni uma kwenzeka. Ukuqapha, yiqiniso, kusiza ukuphendula ukwehluleka nokwehluleka ngokuphumelelayo nangokushesha, kodwa akwanele. Okokuqala, kunzima kakhulu ukulandelela inani elikhulu lamaseva - inani elikhulu labantu liyadingeka. Okwesibili, udinga ukuqonda kahle ukuthi uhlelo lokusebenza lusebenza kanjani ukuze ubikezele isimo salo. Ngakho-ke, sidinga abantu abaningi abanokuqonda okuhle kwezinhlelo esizithuthukisayo, ukusebenza kwazo kanye nezici. Ake sicabange ukuthi ngisho noma uthola abantu abanele abazimisele ukwenza lokhu, kusathatha isikhathi esiningi ukubaqeqesha.
Okufanele ngikwenze? Yilapho ubuhlakani bokwenziwa buzosisiza khona. Isihloko sizokhuluma ngakho
Isingeniso
Isistimu yesofthiwe ethuthukisiwe ngokushesha noma kamuva iqala ukusebenza. Kubalulekile kumsebenzisi ukuthi uhlelo lusebenze ngaphandle kokwehluleka. Uma kwenzeka isimo esiphuthumayo, kufanele sixazululwe ngokubambezeleka okuncane.
Ukuze kube lula ukwesekwa kwezobuchwepheshe kwesistimu yesofthiwe, ikakhulukazi uma kunamaseva amaningi, izinhlelo zokuqapha zivame ukusetshenziswa ezithatha amamethrikhi ohlelweni lwesofthiwe esebenzayo, zenze kube lula ukuxilonga isimo salo futhi zisize ukunquma ukuthi yini ngempela ebangele ukwehluleka. Le nqubo ibizwa ngokuthi ukuqapha uhlelo lwesofthiwe.
Umfanekiso 1. Isixhumi esibonakalayo sokuqapha i-Grafana
Amamethrikhi ayizinkomba ezihlukahlukene zesistimu yesofthiwe, indawo yokusebenza kwayo, noma ikhompuyutha ephathekayo lapho uhlelo lusebenza ngaphansi kwesitembu sesikhathi sesikhathi lapho amamethrikhi atholwa khona. Ekuhlaziyeni okumile, lawa mamethrikhi abizwa ngochungechunge lwesikhathi. Ukuqapha isimo sohlelo lwesofthiwe, amamethrikhi aboniswa ngendlela yamagrafu: isikhathi siku-X eksisi, futhi amanani ahambisana ne-Y axis (Umfanekiso 1). Izinkulungwane ezimbalwa zamamethrikhi zingathathwa kusistimu yesofthiwe esebenzayo (kunodi ngayinye). Zakha isikhala samamethrikhi (uchungechunge lwesikhathi olunezinhlangothi eziningi).
Njengoba inani elikhulu lamamethrikhi liqoqwa kumasistimu esofthiwe ayinkimbinkimbi, ukuqapha okwenziwa ngesandla kuba umsebenzi onzima. Ukuze unciphise inani ledatha ehlaziywa umlawuli, amathuluzi okuqapha aqukethe amathuluzi okuhlonza ngokuzenzakalelayo izinkinga ezingaba khona. Isibonelo, ungamisa i-trigger ukuthi ivuthe lapho isikhala sediski samahhala siwela ngaphansi komkhawulo othile. Ungakwazi futhi ukuxilonga ngokuzenzakalelayo ukuvala shaqa kweseva noma ukwehla okubalulekile kwesivinini sesevisi. Eqinisweni, amathuluzi okuqapha enza umsebenzi omuhle wokuthola ukwehluleka osekwenzekile kakade noma ukukhomba izimpawu ezilula zokuhluleka esikhathini esizayo, kodwa ngokuvamile, ukubikezela ukwehluleka okungenzeka kuhlala kuyinadi enzima ukuyiqeda. Ukubikezela ngokuhlaziywa mathupha kwamamethrikhi kudinga ukubandakanywa kochwepheshe abaqeqeshiwe. Ukukhiqiza okuphansi. Ukwehluleka okuningi okungenzeka kungabonakali.
Muva nje, lokho okubizwa ngokuthi ukugcinwa kokubikezela kwezinhlelo zesofthiwe sekuye kwanda kakhulu phakathi kwezinkampani ezinkulu zokuthuthukisa isofthiwe ye-IT. Ingqikithi yale ndlela yokuthola izinkinga eziholela ekulimaleni kwesistimu ekuqaleni, ngaphambi kokuba ihluleke, kusetshenziswa ubuhlakani bokwenziwa. Le ndlela ayikushiyi ngaphandle ngokuphelele ukuqapha okwenziwa yisistimu. Isiza enqubweni yokuqapha iyonke.
Ithuluzi eliyinhloko lokuqalisa ukulungiswa okubikezelwayo kuwumsebenzi wokusesha okudidayo ochungechungeni lwesikhathi, kusukela lapho kwenzeka i-anomaly kudatha kukhona amathuba aphezulu okuthi ngemva kwesikhathi esithile kuzoba nokwehluleka noma ukwehluleka. Okudidayo ukuchezuka okuthile ekusebenzeni kwesistimu yesofthiwe, njengokuhlonza ukucekelwa phansi kwesivinini sokwenziwa sohlobo olulodwa lwesicelo noma ukwehla kwenani elimaphakathi lezicelo ezihlinzekwa ngesevisi ezingeni eliqhubekayo lezikhathi zamaklayenti.
Umsebenzi wokusesha okudidayo kumasistimu wesoftware unemininingwane yawo. Ngokombono, ohlelweni lwesofthiwe ngayinye kuyadingeka ukuthuthukisa noma ukulungisa izindlela ezikhona, njengoba ukusesha okungaqondakali kuncike kakhulu kudatha eyenziwa ngayo, futhi idatha yezinhlelo zesofthiwe iyahlukahluka kakhulu kuye ngamathuluzi okusebenzisa uhlelo. , kuze kufike kuyiphi ikhompyutha esebenza kuyo.
Izindlela zokusesha okudidayo lapho ubikezela ukwehluleka kwezinhlelo zesoftware
Okokuqala, kufanelekile ukusho ukuthi umqondo wokubikezela ukwehluleka uphefumulelwe yi-athikili
Wonke amamethrikhi athathwa ohlelweni kusetshenziswa igraphite. Ekuqaleni, i-whisper database yayisetshenziswa njengesixazululo esijwayelekile se-grafana, kodwa njengoba isisekelo samakhasimende sikhula, i-graphite ayikwazanga ukumelana nayo, isiqede amandla esistimu engaphansi yediski ye-DC. Ngemva kwalokhu, kwanqunywa ukuthola isixazululo esisebenza kangcono. Isinqumo senziwe ngokuvuna
Umfanekiso 2. Uhlelo lokuqoqa amamethrikhi
Umdwebo uthathwe emibhalweni yangaphakathi. Ibonisa ukuxhumana phakathi kwe-grafana (i-UI yokuqapha esiyisebenzisayo) ne-graphite. Ukukhipha amamethrikhi kuhlelo lokusebenza kwenziwa ngesofthiwe ehlukene -
Uhlelo Lokuhlanganiswa Kwewebhu lunezici ezimbalwa ezidala izinkinga zokubikezela ukwehluleka:
- Umkhuba uvame ukushintsha. Izinguqulo ezihlukahlukene ziyatholakala kulolu hlelo lwesofthiwe. Ngamunye wabo uletha izinguquko engxenyeni yesofthiwe yesistimu. Ngokufanelekile, ngale ndlela, onjiniyela bathonya ngokuqondile amamethrikhi esistimu enikeziwe futhi bangabangela ushintsho lwethrendi;
- isici sokuqalisa, kanye nezinhloso amaklayenti asebenzisa ngazo lesi simiso, ngokuvamile kubangela okudidayo ngaphandle kokuwohloka kwangaphambilini;
- iphesenti lokudidayo elihlobene nayo yonke isethi yedatha lincane (< 5%);
- Kungase kube khona izikhala ekutholeni izinkomba ohlelweni. Kwezinye izikhathi ezimfushane, isistimu yokuqapha iyehluleka ukuthola amamethrikhi. Isibonelo, uma iseva igcwele kakhulu. Lokhu kubalulekile ekuqeqesheni inethiwekhi ye-neural. Kunesidingo sokugcwalisa izikhala ngokwenziwa;
- Izimo ezinokudida ngokuvamile zifaneleka kuphela ngosuku/inyanga/isikhathi esithile (isizini). Lolu hlelo lunemithethonqubo ecacile yokusetshenziswa kwalo ngabasebenzisi. Ngokufanelekile, amamethrikhi asebenza ngesikhathi esithile kuphela. Uhlelo alukwazi ukusetshenziswa njalo, kodwa ezinyangeni ezithile kuphela: ngokukhetha kuye ngonyaka. Izimo ziphakama lapho ukuziphatha okufanayo kwamamethrikhi esimweni esisodwa kungaholela ekuhlulekeni kwesistimu yesofthiwe, kodwa hhayi kwesinye.
Okokuqala, kwahlaziywa izindlela zokuthola okudidayo ekuqaphelweni kwedatha yezinhlelo zesoftware. Kuma-athikili kulesi sihloko, lapho iphesenti lokudidayo lilincane uma liqhathaniswa nayo yonke isethi yedatha, kuvame ukuphakanyiswa ukuthi kusetshenziswe amanethiwekhi e-neural.
Umqondo oyisisekelo wokusesha okudidayo usebenzisa idatha yenethiwekhi ye-neural uboniswa kuMfanekiso 3:
Umfanekiso 3. Isesha okudidayo kusetshenziswa inethiwekhi ye-neural
Ngokusekelwe kumphumela wesibikezelo noma ukubuyiselwa kwewindi lokugeleza kwamanje kwamamethrikhi, ukuchezuka kusukela kulokho okutholwe kusistimu yesofthiwe esebenzayo kuyabalwa. Uma kunomehluko omkhulu phakathi kwamamethrikhi atholwe kusistimu yesofthiwe nenethiwekhi ye-neural, singaphetha ngokuthi ingxenye yamanje yedatha ayijwayelekile. Uchungechunge lwezinkinga ezilandelayo luvela ekusetshenzisweni kwamanethiwekhi e-neural:
- ukuze usebenze kahle kumodi yokusakaza-bukhoma, idatha yokuqeqesha amamodeli enethiwekhi ye-neural kufanele ifake kuphela idatha "evamile";
- kuyadingeka ukuba nemodeli yakamuva ukuze itholwe kahle. Ukushintsha amathrendi kanye nesizini kumamethrikhi kungabangela inani elikhulu lokuhle okungamanga kumodeli. Ukuze uyibuyekeze, kuyadingeka ukucacisa ngokucacile isikhathi lapho imodeli isiphelelwe yisikhathi. Uma ubuyekeza imodeli ngokuhamba kwesikhathi noma ngaphambili, khona-ke, cishe, inani elikhulu lamaphothizithi angamanga lizolandela.
Akumelwe futhi sikhohlwe mayelana nokusesha nokuvimbela ukuvela kaningi kwemibono engamanga. Kucatshangwa ukuthi zizokwenzeka kakhulu ezimweni eziphuthumayo. Nokho, kungase futhi kube umphumela wephutha lenethiwekhi ye-neural ngenxa yokuqeqeshwa okunganele. Kuyadingeka ukunciphisa inani lamaphothizithi angamanga emodeli. Uma kungenjalo, ukubikezela okungamanga kuzomosha isikhathi esiningi somlawuli esihloselwe ukuhlola isistimu. Ngokushesha noma kamuva umlawuli uzovele ayeke ukuphendula ohlelweni lokuqapha "i-paranoid".
Inethiwekhi ye-neural evamile
Ukuze uthole okudidayo ochungechungeni lwesikhathi, ungasebenzisa
Umfanekiso 4. Isibonelo senethiwekhi ye-neural eqhubekayo enamaseli enkumbulo ye-LSM
Njengoba kungabonwa kuMfanekiso 4, i-RNN LSTM ikwazile ukubhekana nokuseshwa kokudidayo kulesi sikhathi. Lapho umphumela unephutha eliphezulu lokuqagela (iphutha lencazelo), kwenzeke okudidayo ezikhombi. Ukusebenzisa i-RNN LSTM eyodwa ngokusobala ngeke kwanele, njengoba isebenza enanini elincane lamamethrikhi. Ingasetshenziswa njengendlela eyisizayo yokusesha okudidayo.
I-Autoencoder yokubikezela ukwehluleka
Umfanekiso 5. Isibonelo sokusebenza kwe-autoencoder
Ama-autoencoder aqeqeshwa kudatha evamile futhi abese ethola okuthile okuxakile kudatha enikezwe imodeli. Okudingayo nje kulo msebenzi. Okusele nje ukukhetha ukuthi iyiphi i-autoencoder elungele lo msebenzi. Indlela elula yokwakhiwa kwe-autoencoder iyinethiwekhi ye-neural eya phambili, engabuyi, efana kakhulu
Kodwa-ke, umehluko phakathi kwama-autoencoder nama-MLP uwukuthi ku-autoencoder, isendlalelo esiphumayo sinenani elifanayo lamanodi njengesendlalelo sokufakwayo, nokuthi esikhundleni sokuqeqeshwa ukubikezela inani eliqondiwe elingu-Y elinikezwa okokufaka u-X, i-autoencoder iyaqeqeshwa. ukuze yakhe kabusha ama-X ayo. Ngakho-ke, ama-Autoencoder angamamodeli okufunda angagadiwe.
Umsebenzi we-autoencoder ukuthola izinkomba zesikhathi u-r0 ... rn ezihambisana nezinto ezididayo ku-vector yokufaka X. Lo mphumela ufinyelelwa ngokusesha iphutha eliyisikwele.
Umfanekiso 6. I-autoencoder evumelanisiwe
Okokufaka khodi okuzenzakalelayo kukhethiwe
Indlela yokunciphisa ukuhlehlisa okungamanga
Ngenxa yokuthi izimo ezihlukahlukene ezingavamile ziphakama, kanye nesimo esingaba khona sokuqeqeshwa okunganele kwenethiwekhi ye-neural, ukuze kuthuthukiswe imodeli yokuhlonza okungaqondakali, kwanqunywa ukuthi kwakudingeka ukuthi kusungulwe indlela yokunciphisa amaphuzu angamanga. Lo mshini usekelwe kusisekelo sesifanekiso esihlukaniswa umlawuli.
Umgomo oyinhloko wokunciphisa amaphuzu angamanga ukuqoqa isizindalwazi samazinga ngosizo lomsebenzisi ohlukanisa amacala asolisayo atholwe kusetshenziswa amanethiwekhi emizwa. Okulandelayo, izinga elihlukanisiwe liqhathaniswa necala elitholwe yisistimu, bese kwenziwa isiphetho mayelana nokuthi icala lingamanga noma liholela ekwehlulekeni. I-algorithm ye-DTW isetshenziswa ngokunembile ukuqhathanisa uchungechunge lwezikhathi ezimbili. Ithuluzi elikhulu lokunciphisa lisahlukanisa. Kulindeleke ukuthi ngemva kokuqoqa inombolo enkulu yamacala okubhekisela, uhlelo luzoqala ukubuza opharetha kancane ngenxa yokufana kwamacala amaningi kanye nokuvela okufanayo.
Ngenxa yalokho, ngokusekelwe ezindleleni zenethiwekhi ye-neural ezichazwe ngenhla, kwakhiwe uhlelo lokuhlola ukuze kubikezelwe ukwehluleka kohlelo lwe-“Web-Consolidation”. Umgomo walolu hlelo bekuwukusebenzisa ingobo yomlando ekhona yokuqapha idatha nolwazi mayelana nokwehluleka kwangaphambilini, ukuhlola amandla ale ndlela yezinhlelo zethu zesofthiwe. Uhlelo lohlelo lwethulwe ngezansi kuMfanekiso 7.
Umfanekiso 7. Isikimu sokubikezela sokwehluleka ngokusekelwe ekuhlaziyweni kwesikhala semethrikhi
Kumdwebo, amabhulokhi amabili amakhulu angahlukaniswa: ukusesha izikhathi ezingaqondakali ekusakazweni kwedatha yokuqapha (amamethrikhi) kanye nendlela yokunciphisa amaphuzu angamanga. Qaphela: Ngezinjongo zokuhlola, idatha itholwa ngoxhumo lwe-JDBC kusuka kusizindalwazi lapho igraphite izoyigcina khona.
Okulandelayo yi-interface yesistimu yokuqapha etholwe ngenxa yentuthuko (Umfanekiso 8).
Umfanekiso 8. Isixhumi esibonakalayo sohlelo lokuhlola lokuhlola
Isixhumi esibonakalayo sibonisa iphesenti lokudidayo ngokusekelwe kumamethrikhi atholiwe. Esimweni sethu, irisidi ifanisiwe. Sesivele sinayo yonke idatha emavikini ambalwa futhi siyilayisha kancane kancane ukuze sihlole isimo sokudida okuholela ekuhlulekeni. Ibha yesimo ephansi ibonisa lonke iphesenti ledatha engaqondakali ngesikhathi esithile, esinqunywa kusetshenziswa i-autoencoder. Futhi, iphesenti elihlukile liyaboniswa kumamethrikhi abikezelwe, abalwa i-RNN LSTM.
Isibonelo sokutholwa okudidayo okusekelwe ekusebenzeni kwe-CPU kusetshenziswa inethiwekhi ye-neural ye-RNN LSTM (Umfanekiso 9).
Umfanekiso 9. Ukutholwa kwe-RNN LSTM
Icala elilula, empeleni elingaphandle elijwayelekile, kodwa eliholela ekuhlulekeni kwesistimu, libalwe ngempumelelo kusetshenziswa i-RNN LSTM. Isikhombi esididayo kulesi sikhathi singama-85–95%; yonke into engaphezu kuka-80% (umkhawulo unqunywe ngokuhlolwa) kuthathwa njengokudidayo.
Isibonelo sokutholwa okudidayo lapho isistimu ingakwazi ukuqalisa ngemva kokubuyekezwa. Lesi simo sitholwa yi-autoencoder (Umfanekiso 10).
Umfanekiso 10. Isibonelo sokutholwa kwe-autoencoder
Njengoba ubona emfanekisweni, i-PermGen ibambelele ezingeni elilodwa. I-autoencoder ithole lokhu kuxakile ngoba yayingakaze ikubone okufana nalokhu ngaphambilini. Lapha i-anomaly ihlala i-100% kuze kube yilapho uhlelo lubuyela esimweni sokusebenza. Okudidayo kuboniswa kuwo wonke ama-metrics. Njengoba kushiwo ngaphambili, i-autoencoder ayikwazi ukwenza okwasendaweni okudidayo. Umsebenzisi uyacelwa ukuthi enze lo msebenzi kulezi zimo.
isiphetho
I-PC "Web-Consolidation" ibilokhu ithuthukiswa iminyaka eminingana. Uhlelo lusesimweni esizinzile, futhi inani lezehlakalo ezirekhodiwe lincane. Kodwa-ke, bekungenzeka ukuthola okudidayo okuholela ekuhlulekeni emizuzwini emi-5 - 10 ngaphambi kokuthi kwenzeke ukwehluleka. Kwezinye izimo, ukwaziswa ngokwehluleka kusenesikhathi kungasiza ukonga isikhathi esihleliwe esabelwe ukwenza umsebenzi “wokulungisa”.
Ngokusekelwe ekuhloleni okwenziwe, kusesekuseni kakhulu ukuthi ufinyelele iziphetho zokugcina. Kuze kube manje, imiphumela iyangqubuzana. Ngakolunye uhlangothi, kusobala ukuthi ama-algorithms asekelwe kumanethiwekhi e-neural ayakwazi ukuthola okudidayo “okuwusizo”. Ngakolunye uhlangothi, kusele iphesenti elikhulu lezinto ezingamanga, futhi akuzona zonke izinto ezididayo ezitholwe uchwepheshe oqeqeshiwe kunethiwekhi ye-neural ezingatholwa. Ukungalungi kufaka phakathi iqiniso lokuthi manje inethiwekhi ye-neural idinga ukuqeqeshwa nothisha ngokusebenza okuvamile.
Ukuqhubeka nokuthuthukisa uhlelo lokubikezela ukwehluleka futhi ululethe esimweni esanelisayo, kungacatshangwa izindlela ezimbalwa. Lokhu ukuhlaziya okunemininingwane eminingi kwamacala ane-anomalies eholela ekuhlulekeni, ngenxa yalokhu kungezwa ohlwini lwamamethrikhi abalulekile athonya kakhulu isimo sohlelo, nokulahlwa okungadingekile okungawuthinti. Futhi, uma siya ngale ndlela, singenza imizamo yokwenza ama-algorithms akhethekile ezimweni zethu ezinokudida okuholela ekuhlulekeni. Kukhona enye indlela. Lokhu wukuthuthukiswa kwezakhiwo zenethiwekhi ye-neural futhi ngaleyo ndlela kukhulisa ukunemba kokutholwa ngokunciphisa isikhathi sokuqeqeshwa.
Ngizwakalisa ukubonga kwami kozakwethu abangisizile ukuthi ngibhale futhi ngigcine ukufaneleka kwalesi sihloko:
Source: www.habr.com