Iyo Riffusion purojekiti iri kugadzira vhezheni yemuchina wekudzidza system Yakagadzikana Diffusion, yakagadziridzwa kugadzira mimhanzi pachinzvimbo chemifananidzo. Mimhanzi inogona kugadzirwa kubva pakutsanangurwa kwemavara mumutauro wechisikigo kana zvichibva pane yakarongwa template. Mimhanzi synthesis zvikamu zvakanyorwa muPython uchishandisa iyo PyTorch chimiro uye inowanikwa pasi peMIT rezinesi. Iyo interface inosunga inoitwa muTypeScript uye zvakare yakagoverwa pasi peMIT rezinesi. Mamodheru akadzidziswa anopihwa rezenisi pasi pemvumo yeCreative ML OpenRAIL-M rezinesi rekushandiswa kwekutengesa.
Iyo purojekiti inonakidza nekuti inoramba ichishandisa "mavara-ku-mufananidzo" uye "mufananidzo-ku-mufananidzo" modhi kugadzira mimhanzi, asi inoshandura ma spectrogram semifananidzo. Mune mamwe mazwi, classic Stable Diffusion inodzidziswa kwete pamifananidzo nemifananidzo, asi pamifananidzo ye spectrograms inoratidza shanduko mu frequency uye amplitude yeizwi wave nekufamba kwenguva. Saizvozvo, spectrogram inoumbwa zvakare pane inobuda, iyo inozoshandurwa kuita inomiririra inomiririra.
Iyo nzira inogona zvakare kushandiswa kugadzirisa iripo ruzha nziyo uye kugadzira mimhanzi kubva kumuenzaniso, yakafanana nekugadzirisa mufananidzo muStable Diffusion. Semuyenzaniso, chizvarwa chinogona kuenzanisa spectrograms nereference style, kubatanidza masitaera akasiyana, kuita shanduko yakatsetseka kubva kune imwe chitaera kuenda kune imwe, kana kuita shanduko kune iripo ruzha kugadzirisa matambudziko akadai sekuwedzera huwandu hwezviridzwa zvega, kushandura rhythm, uye kuchinja. zviridzwa. Samples dzinoshandiswawo kugadzira nziyo dzekutamba kwenguva refu, dzinoumbwa nenhevedzano yendima dzakanyatso paradzana dzinosiyana zvishoma nekufamba kwenguva. Ndima dzakapatsanurwa dzinosanganiswa kuita rukova runoenderera uchishandisa kududzira kwemukati maparamita emuenzaniso.
Fourier shanduko ine hwindo inoshandiswa kugadzira spectrogram kubva muruzha. Paunenge uchidzokorora ruzha kubva kune spectrogram, dambudziko rinomuka nekugadzirisa chikamu (chete frequency uye amplitude iripo pane spectrogram), pakuvakazve iyo iyo Griffin-Lim approximation algorithm inoshandiswa.
Source: opennet.ru