Amanethiwekhi e-Neural embonweni wekhompiyutha athuthuka ngenkuthalo, izinkinga eziningi zisakude nokuxazululwa. Ukuze ube yinjwayelo emkhakheni wakho, vele ulandele abagqugquzeli ku-Twitter futhi ufunde izindatshana ezifanele ku-arXiv.org. Kodwa sibe nethuba lokuya kwi-International Conference on Computer Vision (ICCV) 2019. Kulo nyaka ibanjelwa eSouth Korea. Manje sifuna ukwabelana nabafundi bakaHabr ngalokho esikubonile nesikufundile.
Sasibaningi lapho sivela ku-Yandex: abathuthukisi bezimoto ezizishayelayo, abacwaningi, nalabo ababhekana nemisebenzi ye-CV ezinsizeni beza. Kodwa manje sifuna ukwethula umbono weqembu lethu ogxile kancane - iMachine Intelligence Laboratory (Yandex MILAB). Kungenzeka ukuthi abanye abafana babheke ingqungquthela ngombono wabo.
Yenzani ilabhorethri?Senza amaphrojekthi okuhlola ahlobene nokukhiqizwa kwezithombe nomculo ngezinjongo zokuzijabulisa. Sithanda kakhulu amanethiwekhi e-neural akuvumela ukuthi uguqule okuqukethwe kumsebenzisi (ezithombeni, lo msebenzi ubizwa ngokuthi ukukhohlisa kwesithombe). umphumela womsebenzi wethu ovela enkomfeni ye-YaC 2019.
Kunezingqungquthela eziningi zesayensi, kodwa eziphezulu zigqamile, okuthiwa izingqungquthela ze-A *, lapho izihloko ezikhuluma ngobuchwepheshe obuthakazelisa kakhulu nobubalulekile zivame ukushicilelwa. Alukho uhlu oluqondile lwezingqungquthela ze-A*, nalu uhlu olulinganiselwe nolungaphelele: I-NeurIPS (ngaphambilini eyayiyi-NIPS), ICML, SIGIR, WWW, WSDM, KDD, ACL, CVPR, ICCV, ECCV. Abathathu bokugcina bagxile esihlokweni se-CV.
I-ICCV shazi: amaphosta, okokufundisa, ama-workshops, izitendi
Ingqungquthela ithole amaphepha angu-1075, kwakukhona abahlanganyeli abangu-7500. Abantu abangu-103 bavela eRussia, kwakukhona izihloko ezivela kubasebenzi baseYandex, Skoltech, Samsung AI Center Moscow naseSamara University. Kulo nyaka, ababaningi abacwaningi abaphezulu abavakashele i-ICCV, kodwa, isibonelo, u-Alexey (Alyosha) Efros, ohlala eheha abantu abaningi:
Izibalo
Kuzo zonke izingqungquthela ezinjalo, izihloko zethulwa ngendlela yamaphosta ( mayelana nefomethi), kanti ezinhle kakhulu nazo zethulwa ngendlela yemibiko emifushane.
Nansi eminye yemisebenzi evela eRussia
Ngezifundo ungakwazi ukungena endaweni yesifundo esithile; kukhumbuza isifundo enyuvesi. Ifundwa ngumuntu oyedwa, ngokuvamile ngaphandle kokukhuluma ngemisebenzi ethile. Isibonelo sesifundo esihle ():
Ezingqungqutheleni, ngokuphambene nalokho, bakhuluma ngezihloko. Ngokuvamile lena yimisebenzi esihlokweni esithile esincane, izindaba ezivela kumakhanda aselabhorethri mayelana nawo wonke umsebenzi wakamuva wabafundi, noma izindatshana ezingamukelwanga engqungqutheleni enkulu.
Izinkampani ezixhasayo ziza e-ICCV nezitendi. Kulo nyaka, i-Google, i-Facebook, i-Amazon nezinye izinkampani eziningi zamazwe ngamazwe zeza, kanye nenani elikhulu leziqalo - isiKorea nesiShayina. Kube neziqalo eziningi ikakhulukazi ezigxile ekumaka kwedatha. Kukhona amakhonsathi ezitendi, ungathatha okuthengiswayo ubuze imibuzo. Ngezinhloso zokuzingela, izinkampani ezixhasayo ziba namaphathi. Ungangena kuzo uma uqinisekisa abaqashi ukuthi unentshisekelo nokuthi ungaphumelela izingxoxo. Uma ushicilele isihloko (noma, ngaphezu kwalokho, uyethulile), uqale noma uqeda i-PhD, lokhu kuhlanganisa, kodwa ngezinye izikhathi ungaxoxisana endaweni yokuma ngokubuza imibuzo ethakazelisayo konjiniyela benkampani.
Amathrendi
Ingqungquthela ikuvumela ukuthi ubheke yonke inkambu ye-CV. Ngenani lamaphosta esihlokweni esithile, ungahlola ukuthi isihloko sishisa kangakanani. Ezinye iziphetho ziziphakamisa ngokwazo ngokusekelwe kumagama angukhiye:
I-Zero-shot, i-one-shot, i-shot-shot, iyaziqondisa futhi igadwe kancane: izindlela ezintsha zemisebenzi efundiwe isikhathi eside
Abantu bafunda ukusebenzisa idatha ngempumelelo kakhulu. Ngokwesibonelo, ku kungenzeka ukukhiqiza isimo sobuso sezilwane ebezingekho kusethi yokuqeqeshwa (ekusetshenzisweni, ngokuhlinzeka ngezithombe ezimbalwa zereferensi). Imibono ye-Deep Image Ngaphambili ithuthukisiwe, futhi manje amanethiwekhi we-GAN angaqeqeshwa esithombeni esisodwa - sizokhuluma ngalokhu ngezansi. . Ungasebenzisa ukuzigada ekuziqeqesheni kwangaphambilini (ukuxazulula inkinga ongakwazi ukuhlanganisa ngayo idatha eqondaniswe, njengokubikezela i-engeli yokuzungezisa yesithombe) noma ufunde ngesikhathi esisodwa kudatha enelebulayo nengenawo amalebula. Ngalo mqondo, lesi sihloko singabhekwa njengomqhele wendalo . Futhi nakhu ukuqeqeshwa kwangaphambilini ku-ImageNet kuyasiza.
I-3D ne-360Β°
Izinkinga ezixazululwe kakhulu ezithombeni (ukuhlukaniswa, ukutholwa) zidinga ucwaningo olwengeziwe lwamamodeli e-3D namavidiyo e-panoramic. Sibone izindatshana eziningi zokuguqula i-RGB ne-RGB-D ibe yi-3D. Ezinye izinkinga, njengokulinganisa kokuma komuntu, zingaxazululwa ngokwemvelo ngokuthuthela kumamodeli e-3D. Kodwa akukho ukuvumelana okwamanje mayelana nendlela yokumelela amamodeli e-XNUMXD - ngendlela ye-mesh, ifu lephoyinti, ama-voxels noma i-SDF. Nansi enye inketho:
Kuma-panorama, ama-convolutions on the sphere ayathuthuka (bona. ) bese ucinga izinto ezibalulekile kuhlaka.
Ukutholwa kwe-Pose nokubikezela ukunyakaza kwabantu
Sekuvele kube nenqubekelaphambili ekutholweni kwe-pose ku-2D - manje sekugxilwe ekusebenzeni ngamakhamera amaningi naku-3D. Isibonelo, ungakwazi futhi ukubona uhlaka lwamathambo odongeni ngokulandelela izinguquko kusiginali ye-Wi-Fi njengoba idlula emzimbeni womuntu.
Mningi umsebenzi owenziwe emkhakheni wokuthola amaphuzu angukhiye wesandla. Kuvele amasethi edatha amasha, okuhlanganisa lawo asekelwe kumavidiyo ezingxoxo phakathi kwabantu ababili - manje ungakwazi ukubikezela ukuthinta kwezandla emsindweni noma kumbhalo wengxoxo! Kuye kwenziwa inqubekelaphambili efanayo emisebenzini yokulandelela iso (ukulinganisa kwamehlo).
Umuntu angakwazi futhi ukukhomba iqoqo elikhulu lemisebenzi ehlobene nokubikezela ukunyakaza kwabantu (isibonelo, noma ). Umsebenzi ubalulekile futhi, ngokusekelwe ezingxoxweni nababhali, ngokuvamile usetshenziselwa ukuhlaziya ukuziphatha kwabahamba ngezinyawo ekushayeleni okuzimele.
Ukukhohlisa nabantu ezithombeni nakumavidiyo, amagumbi okulingana abonakalayo
Umkhuba oyinhloko ukushintsha izithombe zobuso ngokuvumelana nemingcele ehumukayo. Imibono: i-deepfake esekelwe esithombeni esisodwa, ukushintsha indlela yokukhuluma ngokusekelwe ekuhumusheni kobuso (), i-feedforward-shintsha imingcele (isibonelo, ). Ukudluliswa kwesitayela kusukile esihlokweni kuya ekusetshenzisweni komsebenzi. Amakamelo okufaka izinto ezibonakalayo ayindaba ehlukile; cishe ahlala esebenza kabi, amademo.
Isizukulwane kusuka kumidwebo/emagrafu
Ukuthuthukiswa kombono othi "Vumela igridi ikhiqize okuthile ngokusekelwe kokuhlangenwe nakho kwangaphambilini" kwaba okunye: "Masibonise igridi ukuthi iyiphi inketho esiyithandayo."
ikuvumela ukuthi wenze upende oqondisiwe: umsebenzisi angaqeda ukudweba ingxenye yobuso endaweni esuliwe yesithombe futhi athole isithombe esibuyiselwe kuye ngokuqedwa.
Enye yama-athikili e-Adobe angama-25 ye-ICCV ihlanganisa ama-GAN amabili: eyodwa iqedela umdwebo womsebenzisi, enye ikhiqiza isithombe se-photorealistic kusuka kumdwebo ().
Ngaphambilini, amagrafu abengadingeki ekukhiqizeni izithombe, kodwa manje enziwe isitsha solwazi mayelana nendawo. Umklomelo we-Best Paper Honourable Mentions osuselwe emiphumeleni ye-ICCV nawo uwinwe yi-athikili . Ngokuvamile, ungazisebenzisa ngezindlela ezahlukene: khiqiza amagrafu ezithombeni, noma izithombe nemibhalo emagrafu.
Ukuphinda kukhonjwe abantu nezimoto, kubalwa usayizi wesixuku (!)
Ama-athikili amaningi anikezelwe ekulandeleleni abantu nokuphinda kukhonjwe abantu nemishini. Kodwa okwasimangaza kwakuyinqwaba yezihloko eziphathelene nokubalwa kwesixuku, zonke zivela eChina.
Amaphosta
Kodwa i-Facebook, ngokuphambene nalokho, ichaza isithombe. Futhi lokhu ikwenza ngendlela ethakazelisayo: iqeqesha inethiwekhi ye-neural ukuze ikhiqize ubuso obungenayo imininingwane eyingqayizivele - efanayo, kodwa akufani kakhulu kangangokuthi ingabonakala ngendlela efanele ngezinhlelo zokuqaphela ubuso.
Ukuvikelwa ekuhlaselweni yizitha
Ngokuthuthukiswa kwezinhlelo zokusebenza zombono wekhompiyutha emhlabeni wangempela (ezimotweni ezizishayelayo, ekuboneni ubuso), umbuzo wokuthembeka kwezinhlelo ezinjalo uya uphakama. Ukuze usebenzise ngokugcwele i-CV, udinga ukuqiniseka ukuthi isistimu imelana nokuhlaselwa izitha - yingakho bezingekho izindatshana ezimayelana nokuvikelwa kuzo kunangokuhlasela ngokwako. Kube nomsebenzi omningi ekuchazeni izibikezelo zenethiwekhi (imephu ye-saliency) nokulinganisa ukuzethemba kumphumela.
Imisebenzi ehlanganisiwe
Emisebenzini eminingi enomgomo owodwa, amathuba okuthuthukisa ikhwalithi ayaphela; enye yezinkomba ezintsha zekhwalithi eqhubekayo ukufundisa amanethiwekhi e-neural ukuxazulula izinkinga ezimbalwa ezifanayo ngasikhathi sinye. Izibonelo:
- isibikezelo sesenzo + isibikezelo sokugeleza kwe-optical,
β isethulo sevidiyo + isethulo solimi (),
- .
Kukhona nezindatshana ezikhuluma ngokuhlukaniswa, ukuzimisela kokuma kanye nokuhlonzwa kabusha kwezilwane!
Amaphuzu avelele
Cishe zonke izindatshana zaziwa kusenesikhathi, umbhalo wawutholakala ku-arXiv.org. Ngakho-ke, ukwethulwa kwemisebenzi efana ne-Allbody Dance Now, FUNIT, Image2StyleGAN kubonakala kuyinqaba - lena yimisebenzi ewusizo kakhulu, kodwa ayimisha. Kubonakala sengathi inqubo yakudala yokushicilelwa kwesayensi iyaphuka lapha - isayensi ihamba ngokushesha kakhulu.
Kunzima kakhulu ukunquma imisebenzi engcono kakhulu - miningi yayo, izifundo zihlukile. Kutholwe izindatshana ezimbalwa .
Sifuna ukugqamisa imisebenzi ethokozisayo ngokombono wokukhohliswa kwesithombe, njengoba lesi kuyisihloko sethu. Kuvele ukuthi kusha futhi kuthakazelisa kithi (asizenzi sinenjongo).
I-SinGAN (umklomelo wephepha ohamba phambili) kanye ne-InGAN
I-SinGAN: , , .
I-INGAN: , , .
Ukuthuthukiswa Kwesithombe Esijulile Umbono wangaphambili ovela ku-Dmitry Ulyanov, u-Andrea Vedaldi no-Victor Lempitsky. Esikhundleni sokuqeqesha i-GAN kudathasethi, amanethiwekhi afunda ezingxenyeni zesithombe esifanayo ukuze akhumbule izibalo ezingaphakathi kwawo. Inethiwekhi eqeqeshiwe ikuvumela ukuthi uhlele futhi uphilise izithombe (i-SinGAN) noma ukhiqize izithombe ezintsha zanoma imuphi usayizi kusuka emibhalweni yesithombe sangempela, ulondoloze isakhiwo sendawo (I-InGAN).
I-SinGAN:
I-INGAN:
Ukubona Lokho I-GAN Engakwazi Ukukhiqiza
.
Amanethiwekhi e-Neural akhiqiza izithombe ngokuvamile athatha i-vector yomsindo ongahleliwe njengokufakwayo. Kunethiwekhi eqeqeshiwe, ama-vector amaningi okufaka enza isikhala, ukunyakaza okuncane okuholela ekushintsheni okuncane esithombeni. Ngokusebenzisa ukwenza kahle, ungakwazi ukuxazulula inkinga ephambene: thola i-vector efanelekayo yesithombe esivela emhlabeni wangempela. Umbhali ubonisa ukuthi cishe akunakwenzeka ukuthola isithombe esifanelana ngokuphelele kunethiwekhi ye-neural. Ezinye izinto ezisesithombeni azenziwa (ngokusobala ngenxa yokuhlukahluka okukhulu kwalezi zinto).
Umbhali ulinganisa ukuthi i-GAN ayimbozi sonke isikhala sezithombe, kodwa isethi encane kuphela, egcwele izimbobo, njengoshizi. Uma sizama ukuthola izithombe ezivela emhlabeni wangempela kuwo, siyohlala sehluleka, ngoba i-GAN isakhiqiza hhayi izithombe zangempela ngokuphelele. Umehluko phakathi kwezithombe zangempela nezakhiwe unganqotshwa kuphela ngokushintsha izisindo zenethiwekhi, okungukuthi, ngokuyiqeqeshela kabusha isithombe esithile.
Uma inethiwekhi iqeqeshelwa isithombe esithile, ungazama ukukhohlisa okuhlukahlukene ngalesi sithombe. Esibonelweni esingezansi, iwindi lengeziwe esithombeni, futhi inethiwekhi iphinde yakha ukubonakaliswa kweyunithi yekhishi. Lokhu kusho ukuthi inethiwekhi, ngisho nangemva kokuqeqeshwa okwengeziwe kwezithombe, ayizange ilahlekelwe ikhono lokubona ukuxhumana phakathi kwezinto ezisesigcawini.
I-GANalyze: Ibhekise Izincazelo Ezibonakalayo Zezakhiwo Zesithombe Sokuqonda
, .
Ngokusebenzisa indlela ephuma kulo msebenzi, ungakwazi ukubona ngeso lengqondo futhi uhlaziye lokho okufundwe yinethiwekhi yezinzwa. Ababhali bahlongoza ukuqeqesha i-GAN ukuze idale izithombe lapho inethiwekhi izokhiqiza khona ukuqagela okucacisiwe. I-athikili isebenzise amanethiwekhi amaningana njengezibonelo, okuhlanganisa i-MemNet, ebikezela ukukhumbuleka kwesithombe. Kwavela ukuthi ukuze kukhunjulwe kangcono, into esesithombeni kufanele:
- abe seduze nendawo
- abe nomumo oyindilinga noma oyisikwele kanye nesakhiwo esilula,
- ube nesizinda esifanayo,
- aqukethe amehlo acacile (okungenani ezithombe zezinja),
- kukhanye, kugcwale kakhulu, kwezinye izimo, kube bomvu.
I-Liquid Warping GAN: Uhlaka Oluhlanganisiwe Lokulingisa Ukunyakaza Komuntu, Ukudluliswa kokubukeka kanye ne-Novel View Synthesis
, , .
Ipayipi lokukhiqiza izithombe zabantu isithombe esisodwa ngesikhathi. Ababhali babonisa izibonelo eziphumelelayo zokudlulisa ukunyakaza komuntu oyedwa komunye, ukudlulisa izingubo phakathi kwabantu nokukhiqiza ama-angles amasha omuntu - konke kusuka esithombeni esisodwa. Ngokungafani nemisebenzi yangaphambilini, lapha asisebenzisi amaphuzu abalulekile ku-2D (i-pose), kodwa i-3D mesh yomzimba (i-pose + shape) ukuze senze izimo. Ababhali baphinde bathola ukuthi bangaludlulisela kanjani ulwazi kusuka esithombeni sokuqala kuya kwesenziwe (I-Liquid Warping Block). Imiphumela ibonakala ihloniphekile, kodwa ukulungiswa kwesithombe esiwumphumela kungu-256x256 kuphela. Uma kuqhathaniswa, i-vid2vid, evele ngonyaka odlule, iyakwazi ukukhiqiza ngokulungiswa okungu-2048x1024, kodwa idinga imizuzu eyi-10 yokuqoshwa kwevidiyo njengedathasethi.
I-FSGAN: Isihloko Sokushintshanisa Ubuso be-Agnostic kanye Nokulingiswa
, .
Ekuqaleni kubonakala sengathi akukho okungavamile: i-deepfake enekhwalithi evamile noma engaphansi. Kodwa impumelelo eyinhloko yomsebenzi iwukuba esikhundleni sobuso besithombe esisodwa. Ngokungafani nemisebenzi yangaphambilini, ukuqeqeshwa kwakudingeka ezithombeni eziningi zomuntu othize. Ipayipi libonakale linzima (ukulingisa futhi ukuhlukaniswa, ukubuka ukuhumusha, ukudweba, ukuhlanganisa) kanye nama-hacks amaningi ezobuchwepheshe, kodwa umphumela uwufanele.
Ukuthola Okungalindelekile nge-Image Resynthesis
.
I-drone ingaqonda kanjani ukuthi into ivele ngokuzumayo phambi kwayo engangeni kunoma yisiphi isigaba sokuhlukaniswa kwe-semantic? Kunezindlela eziningana, kodwa ababhali bahlongoza i-algorithm entsha, enembile esebenza kangcono kunangaphambili. Ukuhlukaniswa kwesemantic kubikezelwa kusukela kusithombe somgwaqo esifakiwe. Iphakelwa njengokufakwayo ku-GAN (pix2pixHD), ezama ukubuyisela isithombe sokuqala kuphela kumephu ye-semantic. Okudidayo okungaweli kunoma yimaphi amasegimenti kuzohluka kakhulu kokuphumayo nasesithombeni esikhiqiziwe. Izithombe ezintathu (ezangempela, ukuhlukaniswa, nokwakhiwa kabusha) zibe sezifakwa kwenye inethiwekhi ebikezela okudidayo. Idathasethi yalokhu yenziwe kusukela kudathasethi eyaziwa kakhulu ye-Cityscapes, ishintsha ngokungahleliwe amakilasi ekuhlukaniseni kwe-semantic. Kuyathakazelisa ukuthi kulesi silungiselelo, inja emi phakathi nomgwaqo, kodwa ihlukaniswe kahle (okusho ukuthi kunesigaba sayo), akuyona into edidayo, njengoba isistimu ikwazile ukuyibona.
isiphetho
Ngaphambi kwengqungquthela, kubalulekile ukwazi ukuthi yiziphi izintshisekelo zakho zesayensi, yiziphi izethulo ongathanda ukuya kuzo, nokuthi ukhulume nobani. Khona-ke yonke into izokhiqiza kakhulu.
I-ICCV, okokuqala nokubaluleke kakhulu, inethiwekhi. Uyaqonda ukuthi kunezikhungo eziphezulu neminyango yesayensi ephezulu, uqala ukuqonda lokhu, wazi abantu. Futhi ungafunda izindatshana ku-arXiv - kanti-ke, kuhle kakhulu ukuthi asikho isidingo sokuthi uye noma yikuphi ukuze uthole ulwazi.
Ngaphezu kwalokho, engqungqutheleni ungangena ujule ezihlokweni ezingasondeli kuwe futhi ubone izitayela. Hhayi-ke, bhala uhlu lwezihloko okufanele uzifunde. Uma ungumfundi, leli yithuba lokuthi uhlangane nothisha ongase abe uthisha, uma uvela embonini, bese kuba nomqashi omusha, futhi uma inkampani, bese uzibonisa.
Bhalisela ku ! Lena iphrojekthi yomuntu siqu: siyihola ndawonye . Sithumele yonke imisebenzi esiyithandile ngesikhathi sengqungquthela lapha: .
Source: www.habr.com