Aikin Riffusion yana haɓaka bambance-bambancen tsarin koyo na inji Stable Diffusion wanda aka daidaita don samar da kiɗa maimakon hotuna. Ana iya haɗa kiɗa ta hanyar bayanin rubutu a cikin yare na halitta ko kuma bisa samfurin da aka ba da shawara. Abubuwan haɗin kiɗan an rubuta su cikin Python ta amfani da tsarin PyTorch kuma ana samun su ƙarƙashin lasisin MIT. Ana aiwatar da ɗaurin da ke dubawa a cikin yaren TypeScript kuma ana rarraba shi ƙarƙashin lasisin MIT. Ana fitar da samfuran horarwa a ƙarƙashin lasisin izini na Creative ML OpenRAIL-M don amfanin kasuwanci.
Aikin yana da ban sha'awa a cikin cewa yana ci gaba da yin amfani da samfurin "rubutu-zuwa-hoto" da "hoto-to-hoto" don tsara kiɗa, amma yana sarrafa spectrograms azaman hotuna. A takaice dai, ba a horar da Stable Diffusion na al'ada ba akan hotuna da hotuna ba, amma akan hotunan spectrograms waɗanda ke nuna sauyin mitar sauti da girman igiyar sauti akan lokaci. Saboda haka, ana kuma samar da spectrogram a wurin fitarwa, wanda sai a canza shi zuwa wakilcin sauti.
Hakanan za'a iya amfani da hanyar don gyaggyara abubuwan ƙirƙira na sauti da samfurin kiɗan kiɗa, kama da gyaran hoto a cikin Stable Diffusion. Misali, tsara za ta iya saita samfurin Spectogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogrogras, hada hanya mai kyau, ko yin canje-canje zuwa ga matsanancin kayan kida, canza irin kari, canza sautin mutum, canza kari. maye gurbin kayan aiki. Hakanan ana amfani da alamu don samar da abubuwan wasan kwaikwayo na dogon lokaci, waɗanda suka haɗa da jerin sassan da ke kusa da juna, suna ɗan bambanta kaɗan akan lokaci. An haɗa ɓangarorin da aka keɓance daban-daban a cikin rafi mai ci gaba ta hanyar haɗa sigogin ciki na ƙirar.
Don ƙirƙirar spectrogram daga sauti, ana amfani da canjin Fourier mai taga. Lokacin sake ƙirƙira sauti daga na'ura mai ba da hanya tsakanin hanyoyin sadarwa, akwai matsala tare da ƙayyadaddun lokaci (mita da girma kawai suna nan akan sikirin), don sake ginawa wanda aka yi amfani da algorithm na Griffin-Lim approximation.
source: budenet.ru