Txoj haujlwm Riffusion tsim ib qho txawv ntawm Stable Diffusion tshuab kev kawm tau yoog los tsim cov suab paj nruag es tsis txhob ntawm cov duab. Suab paj nruag tuaj yeem tsim los ntawm cov lus piav qhia hauv hom lus lossis raws li tus qauv qhia. Cov suab paj nruag synthesis yog sau rau hauv Python siv PyTorch lub moj khaum thiab muaj nyob rau hauv daim ntawv tso cai MIT. Kev khi nrog lub interface yog siv hauv hom lus TypeScript thiab tseem raug faib raws li MIT daim ntawv tso cai. Cov qauv kev cob qhia raug tso tawm raws li Creative ML OpenRAIL-M daim ntawv tso cai rau kev siv lag luam.
Qhov project yog nthuav nyob rau hauv hais tias nws tseem siv cov "ntawv nyeem-rau-duab" thiab "duab-rau-duab" qauv rau suab paj nruag tiam, tab sis manipulates spectrograms raws li cov duab. Hauv lwm lo lus, classic Stable Diffusion yog kev cob qhia tsis yog ntawm cov duab thiab cov duab, tab sis ntawm cov duab ntawm spectrograms uas cuam tshuam qhov kev hloov pauv ntawm qhov zaus thiab qhov loj ntawm lub suab yoj lub sijhawm. Raws li, ib qho spectrogram kuj yog tsim los ntawm cov zis, uas yog tom qab ntawd hloov mus rau hauv lub suab sawv cev.
Cov txheej txheem kuj tseem siv tau los hloov cov suab paj nruag uas twb muaj lawm thiab cov qauv suab paj nruag synthesis, zoo ib yam li kev hloov kho duab hauv Stable Diffusion. Piv txwv li, tiam neeg tuaj yeem teeb tsa cov qauv spectrograms nrog cov qauv siv, sib xyaw ua ke sib txawv, ua qhov kev hloov pauv ntawm ib hom mus rau lwm qhov, lossis hloov mus rau lub suab uas twb muaj lawm los daws cov teeb meem xws li nce qhov ntim ntawm cov twj paj nruag, hloov lub suab thiab hloov cov twj paj nruag. Cov qauv kuj tseem siv los tsim cov kev sib tw ua si ntev, tsim los ntawm cov kab lus uas nyob ze rau ib leeg, sib txawv me ntsis raws sijhawm. Cais generated fragments yog ua ke mus rau hauv ib tug tas mus li kwj los ntawm interpolating lub internal parameters ntawm tus qauv.
Txhawm rau tsim ib lub spectrogram los ntawm lub suab, lub qhov rais Fourier transform yog siv. Thaum rov tsim lub suab los ntawm lub spectrogram, muaj teeb meem nrog kev txiav txim siab theem (tsuas yog zaus thiab amplitude muaj nyob rau ntawm spectrogram), rau kev tsim kho dua tshiab uas siv Griffin-Lim approximation algorithm.
Tau qhov twg los: opennet.ru