Zita rangu ndiPavel Parkhomenko, ndiri mugadziri weML. Muchikamu chino, ndinoda kutaura pamusoro pehutano hwebasa reYandex.Zen uye kugovera kuvandudzwa kwehutano, kushandiswa kwekuita kwaita kuti zvikwanise kuwedzera kunaka kwekurumbidza. Kubva pane ino positi iwe uchadzidza nzira yekuwana iyo inonyanya kukosha kune mushandisi pakati pemamirioni ezvinyorwa mune mashoma milliseconds; maitiro ekuita kuenderera kwekuparara kwematrix hombe (ine mamiriyoni emakoroni uye makumi emamiriyoni emitsara) kuitira kuti zvinyorwa zvitsva zvigamuchire vector yavo mumakumi emaminitsi; mashandisiro ekushandisazve mushandisi-chinyorwa matrix decomposition kuti uwane yakanaka vector inomiririra yevhidhiyo.
Yedu yekurudziro dhatabhesi ine mamirioni ezvinyorwa zveakasiyana mafomati: zvinyorwa zvinyorwa zvakagadzirwa papuratifomu yedu uye zvakatorwa kubva ekunze masaiti, mavhidhiyo, rondedzero uye mapfupi mapfupi. Kuvandudzwa kwebasa rakadaro kunobatanidza nehuwandu hwematambudziko ehutano. Hezvino zvimwe zvacho:
- Govanisa mabasa emakomputa: ita zvese zvinorema mashandiro kunze kwenyika, uye munguva chaiyo ita nekukurumidza mashandisiro emamodheru kuitira kuti uve nemutoro we100-200 ms.
- Kurumidza funga nezvezviitwa zvemushandisi. Kuti uite izvi, zvinodikanwa kuti zviitiko zvese zvinounzwa pakarepo kune anorumbidza uye zvinokanganisa mhedzisiro yemamodheru.
- Gadzira chikafu kuitira kuti kune vashandisi vatsva inokurumidza kuchinjika kune maitiro avo. Vanhu vachangojoinha sisitimu iyi vanofanirwa kunzwa kuti mhinduro yavo inokanganisa kurudziro.
- Kurumidza kunzwisisa kuti ndiani wekukurudzira chinyorwa chitsva.
- Pindura nekukurumidza kune nguva dzose kubuda kwezvinhu zvitsva. Makumi ezviuru ezvinyorwa anoburitswa zuva rega rega, uye mazhinji acho ane mashoma ehupenyu (taura, nhau). Izvi ndizvo zvinovasiyanisa kubva kumafirimu, mimhanzi uye zvimwe zvenguva refu uye zvinodhura zvemukati kugadzira.
- Shandura ruzivo kubva kune imwe nzvimbo yekuenda kune imwe. Kana sisitimu yekurudziro ine mamodheru akadzidzira ezvinyorwa uye isu tikawedzera vhidhiyo kwairi, tinogona kushandisa zvakare mamodheru aripo kuitira kuti mhando nyowani yemukati ive nani.
Ndichakuudza kuti takagadzirisa sei matambudziko aya.
Kusarudzwa kwevanoda kukwikwidza
Nzira yekudzikisa sei nhamba yemagwaro ari pasi pekutariswa nezviuru zvenguva mumamilliseconds mashomanana, pasina kana kuderera kwehutano hwezvirongwa?
Ngatitii takadzidzisa mhando dzeML dzakawanda, takagadzira maficha kubva pazviri, uye takadzidzisa imwe modhi inoronga magwaro emushandisi. Zvose zvingava zvakanaka, asi iwe haugoni kungotora uye kuverenga zviratidzo zvose zvemagwaro ose munguva chaiyo, kana pane mamiriyoni ezvinyorwa izvi, uye mazano anoda kuvakwa mu 100-200 ms. Basa nderekusarudza imwe subset kubva kumamirioni, iyo inozoverengerwa mushandisi. Danho iri rinowanzonzi sarudzo yevamiriri. Pane zvakati wandei zvinodiwa pazviri. Chekutanga, kusarudzwa kunofanirwa kuitika nekukurumidza, kuitira kuti nguva yakawanda sezvinobvira yasarira iyo renji pachayo. Chechipiri, tadzikisira zvakanyanya huwandu hwemagwaro ezvinzvimbo, isu tinofanirwa kuchengetedza magwaro akakodzera kumushandisi zvizere sezvinobvira.
Nheyo yedu yekusarudzwa kwevamiriri yavepo, uye panguva ino tasvika pahurongwa hwematanho akawanda:
Kutanga, zvinyorwa zvose zvakakamurwa kuva mapoka, uye zvinyorwa zvakakurumbira zvinotorwa kubva kuboka rimwe nerimwe. Mapoka anogona kuva masaiti, misoro, masumbu. Kune umwe neumwe mushandisi, zvichienderana nenhoroondo yake, mapoka ari pedyo naye anosarudzwa uye zvinyorwa zvakanakisisa zvinotorwa kubva kwavari. Isu tinoshandisawo kNN index kusarudza zvinyorwa zviri padyo nemushandisi munguva chaiyo. Pane nzira dzinoverengeka dzekugadzira kNN index; yedu yakashanda zvakanyanya (Hierarchical Navigable Diki World graphs). Iyi imhando yehierarchical iyo inokutendera iwe kuti uwane iyo N yepedyo mavheta emushandisi kubva mudura remamiriyoni mumamillisecond mashoma. Isu tinotanga tandesesa dhatabhesi redu rese pasina Indaneti. Sezvo kutsvaga muindex kuchishanda nekukasira, kana paine akati wandei akamisikidzwa akasimba, unogona kugadzira akati wandei indekisi (indekisi imwe yekumisikidzwa kwega kwega) uye kuwana imwe neimwe yadzo munguva chaiyo.
Tichine makumi ezviuru zvemagwaro emushandisi wega wega. Izvi zvichiri zvakawanda kuverenga ese maficha, saka panguva ino isu tinoshandisa mwenje chinzvimbo - yakareruka inorema chinzvimbo modhi ine mashoma maficha. Basa nderekufanotaura kuti ndeapi magwaro anorema modhi achave nawo kumusoro. Zvinyorwa zvine akanyanya kufanotaura zvichashandiswa mune inorema modhi, ndiko kuti, padanho rekupedzisira renzvimbo. Iyi nzira inokubvumira kudzikisa dhatabhesi yezvinyorwa zvakatariswa kumushandisi kubva kumamirioni kusvika kuzviuru mumakumi emamilliseconds.
ALS inopindira munguva yekumhanya
Maitiro ekufunga nezve mushandisi mhinduro nekukurumidza mushure mekudzvanya?
Chinhu chakakosha mukurudziro inguva yekupindura kune mhinduro yemushandisi. Izvi zvinonyanya kukosha kune vashandisi vatsva: kana munhu achangotanga kushandisa hurongwa hwekurudziro, anogamuchira isiri-yemunhu feed yemagwaro ezvinyorwa zvakasiyana. Paanongoita kudzvanya kwekutanga, iwe unofanirwa kukurumidza kufunga izvi uye kugadzirisa kune zvaanofarira. Kana iwe ukaverenga zvese zvinhu zvisiri pamhepo, kukurumidza sisitimu mhinduro haigoneke nekuda kwekunonoka. Saka zvinodikanwa kugadzirisa zviito zvemushandisi munguva chaiyo. Nezvinangwa izvi, isu tinoshandisa iyo ALS nhanho panguva yekumhanya kuvaka vector inomiririra yemushandisi.
Ngatifungei isu tine vector inomiririra kune ese magwaro. Semuyenzaniso, tinokwanisa kugadzira zvinomisikidzwa pasina Indaneti zvichibva pane zvinyorwa zvechinyorwa tichishandisa ELMo, BERT kana mamwe mamodheru ekudzidza muchina. Tinogona sei kuwana vector inomiririra yevashandisi munzvimbo imwechete zvichienderana nekudyidzana kwavo muhurongwa?
General nheyo yekugadzira uye kuparara kwemushandisi-gwaro matrixNgativei nevashandisi m uye n magwaro. Kune vamwe vashandisi, hukama hwavo kune mamwe magwaro hunozivikanwa. Ipapo ruzivo urwu runogona kumiririrwa semxn matrix: mitsara inoenderana nevashandisi, uye makoramu anoenderana nemagwaro. Sezvo munhu wacho asina kuona magwaro akawanda, akawanda ematrix masero acharamba asina chinhu, asi mamwe achazadzwa. Pachiitiko chega chega (senge, kusada, tinya) kumwe kukosha kunopihwa mumatrix - asi ngatifungei muenzaniso wakareruka umo kufanana kunofanana ne1, uye kusada kunofanana ne -1.
Ngatiparadze matrix kuita maviri: P (mxd) uye Q (dxn), uko d iri dimension yevector inomiririra (kazhinji nhamba diki). Zvadaro chinhu chimwe nechimwe chichaenderana ne d-dimensional vector (kune mushandisi - mutsara mune matrix P, yegwaro - mutsara mune matrix Q). Aya mavheji anozove embeddings ezvinhu zvinoenderana. Kufanotaura kana mushandisi achada gwaro, unogona kungowedzera iwo embeddings.
Imwe yedzinogona nzira dzekuora matrix ndeye ALS (Alternating Madiki Makwere). Isu tichagadzirisa iyo inotevera kurasikirwa basa:
Pano rui ndiko kusangana kwemushandisi u negwaro i, qi ndiye vector yegwaro i, pu ndiye vector yemushandisi u.
Ipapo iyo yakakwana mushandisi vector kubva pakuona kweanoreva square mhosho (yeyakagadziriswa gwaro vectors) inowanikwa nekuongorora nekugadzirisa inoenderana mutsara regression.
Izvi zvinodaidzwa kuti "ALS nhanho". Uye iyo ALS algorithm pachayo ndeyekuti isu tinogadzirisa imwe yematrices (vashandisi uye zvinyorwa) uye nekuvandudza imwe, tichiwana iyo yakakwana mhinduro.
Sezvineiwo, kutsvaga mushandisi vector inomiririra ibasa rinokurumidza iro rinogona kuitwa panguva yekumhanya uchishandisa vector rairo. Iri zano rinokutendera iwe kuti utore pakarepo mhinduro yemushandisi muaccount mune chinzvimbo. Iko kudzvanya kumwe chete kunogona kushandiswa muindex yekNN kuvandudza kusarudzwa kwevamiriri.
Distributed Collaborative Sefa
Maitiro ekuita ekuwedzera akagovera matrix factorization uye nekukurumidza kuwana vector inomiririra yezvinyorwa zvitsva?
Zvemukati handizvo zvega zvinopa zviratidzo zvekurudziro. Imwe tsime rakakosha ruzivo rwekubatana. Akanaka chinzvimbo maficha anogona kuwanikwa kubva mukuparara kwemushandisi-gwaro matrix. Asi pakuedza kuita kuora kwakadaro, takasangana nematambudziko:
1. Tine mamirioni ezvinyorwa uye makumi emamiriyoni evashandisi. Iyo matrix haina kukwana zvachose pamushini mumwe, uye kuora kunotora nguva yakareba kwazvo.
2. Zvizhinji zvemukati muhurongwa zvine hupenyu hupfupi: zvinyorwa zvinoramba zvakakosha kwemaawa mashoma. Naizvozvo, zvinodikanwa kuvaka vector yavo inomiririra nekukurumidza sezvinobvira.
3. Kana iwe ukavaka kuora pakarepo mushure mokunge gwaro racho rabudiswa, nhamba yakakwana yevashandisi haizove nenguva yekuiongorora. Naizvozvo, iyo vector inomiririra ingangove isiri yakanaka kwazvo.
4. Kana mushandisi achida kana asingadi, isu hatizokwanisi kukurumidza kutora izvi munhoroondo mukuora.
Kuti tigadzirise matambudziko aya, isu takaita yakagovaniswa kuparara kwemushandisi-gwaro matrix neanogara achiwedzera ekuwedzera. Zvinonyatsoshanda sei?
Ngatitii isu tine cluster yeN michina (N iri mumazana) uye isu tinoda kuita yakagovaniswa decomposition yematrix pavari isingakwane pamushini mumwe. Mubvunzo ndewokuti ungaita sei kuora uku kuitira kuti, kune rumwe rutivi, pane data yakakwana pamushini wega wega uye, kune rumwe rutivi, kuitira kuti maverengero azvimirire?
Isu tichashandisa iyo ALS decomposition algorithm inotsanangurwa pamusoro. Ngatitarisei maitiro ekuita nhanho imwe yeALS nenzira yakagoverwa - mamwe matanho achafanana. Ngatiti isu tine yakagadziriswa matrix yemagwaro uye isu tinoda kuvaka matrix yevashandisi. Kuti tiite izvi, tichaikamura kuita zvikamu zveN nemitsara, chikamu chimwe nechimwe chichava nenhamba yakafanana yemitsara. Isu tichatumira kumushini wega wega maseru asina chinhu emitsara inoenderana, pamwe nematrix ekumisikidzwa kwegwaro (zvakazara). Sezvo saizi yaro isiri yakakura kwazvo, uye mushandisi-gwaro matrix kazhinji iri shoma, iyi data inokwana pamushini wenguva dzose.
Uhu hunyengeri hunogona kudzokororwa pamusoro penguva dzinoverengeka kudzamara modhi yachinja, ichichinjanisa iyo yakagadziriswa matrix imwe neimwe. Asi kunyangwe zvakadaro, kuparara kwematrix kunogona kutora maawa akati wandei. Uye izvi hazvigadzirise dambudziko raunoda kukurumidza kugashira embeddings ezvinyorwa zvitsva uye kugadzirisa embeddings yeaya aive neruzivo rudiki pakuvaka modhi.
Kuunzwa kwekukurumidza kuwedzera modhi yekuvandudza kwakatibatsira. Ngatitii tine modhi yakadzidziswa parizvino. Kubva pakudzidziswa kwake, pave nezvinyorwa zvitsva izvo vashandisi vedu vakadyidzana nazvo, pamwe nezvinyorwa zvaive nekudyidzana kushoma panguva yekudzidziswa. Kukurumidza kuwana zvakamisikidzwa zvezvinyorwa zvakadaro, isu tinoshandisa mushandisi akamisikidza akawanikwa panguva yekutanga kudzidziswa kukuru kwemuenzaniso uye kuita nhanho imwe yeALS kuverenga gwaro rematrix rakapihwa yakagadziriswa mushandisi matrix. Izvi zvinokutendera kuti ugamuchire embeddings nekukasira - mukati memaminitsi mashoma mushure mekuburitswa kwegwaro - uye kazhinji kugadzirisa kumisikidzwa kwemagwaro achangoburwa.
Kuita kurudziro nekukasira funga nezve zviito zvevanhu, munguva yekumhanya isu hatishandise mushandisi embeddings anowanikwa kunze kwenyika. Pane kudaro, tinoita nhanho yeALS uye tinotora iyo chaiyo mushandisi vector.
Chinja kune imwe nzvimbo domain
Maitiro ekushandisa mhinduro yemushandisi pane zvinyorwa zvemavara kugadzira vector inomiririra vhidhiyo?
Pakutanga, isu takakurudzira zvinyorwa zvemavara chete, saka mazhinji ealgorithms edu akagadzirirwa kurudzi urwu rwemukati. Asi pakuwedzera mamwe marudzi ezvinyorwa, isu takatarisana nekudiwa kwekugadzirisa mamodheru. Takagadzirisa sei dambudziko iri tichishandisa muenzaniso wevhidhiyo? Imwe sarudzo ndeye kudzoreredza mamodheru kubva pakutanga. Asi izvi zvinotora nguva yakareba, uye mamwe maalgorithms ari kuda pahukuru hwesample yekudzidziswa, iyo isati yave kuwanikwa muhuwandu hunodiwa hwemhando nyowani yezvinyorwa munguva dzekutanga dzehupenyu hwayo pabasa.
Takaenda neimwe nzira ndokushandisazve matext models evhidhiyo. Ihwo humwe hunyengeri hweALS hwakatibatsira kugadzira vector inomiririra yemavhidhiyo. Takatora inomiririra vector yevashandisi zvichibva pane zvinyorwa uye takaita nhanho yeALS tichishandisa ruzivo rwekuona vhidhiyo. Saka isu takawana nyore inomiririra vector yevhidhiyo. Uye panguva yekumhanya isu tinongoverengera padhuze pakati pemushandisi vector inowanikwa kubva muzvinyorwa zvinyorwa uye vhidhiyo vector.
mhedziso
Kugadzira musimboti weiyo chaiyo-nguva yekurudziro sisitimu inosanganisira akawanda matambudziko. Iwe unofanirwa kukurumidza kugadzirisa data uye kushandisa nzira dzeML kuti ushandise iyi data nemazvo; kuvaka masisitimu akaomeswa akagovaniswa anokwanisa kugadzirisa masaini evashandisi uye zvikamu zvitsva zvemukati munguva shoma; nemamwe mabasa mazhinji.
Muhurongwa hwazvino, dhizaini yandakatsanangura, mhando yekurudziro yemushandisi inokura pamwe nebasa rake uye kureba kwekugara pabasa. Asi chokwadi, pano pane dambudziko guru: zvakaoma kuti sisitimu inzwisise zvinofarirwa nemunhu ane zvishoma zvishoma nezviri mukati. Kuvandudza kurudziro yevashandisi vatsva ndicho chinangwa chedu chikuru. Isu tichaenderera mberi nekugadzirisa maalgorithms kuitira kuti zvirimo zvine chekuita nemunhu zvipinde mukudya kwake nekukurumidza, uye zvisina basa hazviratidzike.
Source: www.habr.com