{"id":106128,"date":"2022-12-16T15:36:45","date_gmt":"2022-12-16T13:36:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/sistema-mashinnogo-obucheniya-stable-diffusion-adaptirovana-dlya-sinteza-muzyki"},"modified":"2022-12-16T15:36:45","modified_gmt":"2022-12-16T13:36:47","slug":"sistema-mashinnogo-obucheniya-stable-diffusion-adaptirovana-dlya-sinteza-muzyki","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-mashinnogo-obucheniya-stable-diffusion-adaptirovana-dlya-sinteza-muzyki","title":{"rendered":"The Stable Diffusion machine learning system has been adapted for music synthesis.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

The Riffusion project develops a variant of the machine learning system Stable Diffusion, adapted for generating music instead of images. Music can be synthesized from a natural language textual description or based on a provided template. The components for music synthesis are written in Python using the PyTorch framework and are available under the MIT license. The interface wrapper is implemented in TypeScript and is also distributed under the MIT license. The trained models are open under the permissive Creative ML OpenRAIL-M license, allowing for commercial use. <\/p>\n

The project is interesting because it continues to use models for generating music based on \"text to image\" and \"image to image\" but manipulates spectrograms as images. In other words, the classic Stable Diffusion is trained not on photographs and pictures, but on images of spectrograms that reflect changes in frequency and amplitude of sound waves over time. Accordingly, the output is also a spectrogram, which is then transformed into sound representation.

\"The<\/center> <\/p>\n

The method can also be used to modify existing sound compositions and synthesize music by template, similar to how images are modified in Stable Diffusion. For example, during generation, spectrogram samples with a reference style can be set, different styles can be combined, smooth transitions from one style to another can be executed, or changes can be made to existing sounds to address tasks such as increasing the volume of individual instruments, changing the rhythm, and replacing instruments. Samples are also used to generate long-playing compositions, composed of a series of segments that are close to each other and slightly change over time. Separately generated segments are combined into a continuous stream using interpolation of the model's internal parameters.

\"The<\/center> <\/p>\n

The Fourier window transform is used to create a spectrogram from sound. When reconstructing sound from the spectrogram, a phase determination problem arises (the spectrogram contains only frequency and amplitude); to reconstruct it, the Griffin-Lim approximation algorithm is used.

<\/center>
<\/center>
\n
Source: opennet.ru<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041f\u0440\u043e\u0435\u043a\u0442\u043e\u043c Riffusion \u0440\u0430\u0437\u0432\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u0430\u0440\u0438\u0430\u043d\u0442 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f Stable Diffusion, \u0430\u0434\u0430\u043f\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0434\u043b\u044f \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u0438 \u043c\u0443\u0437\u044b\u043a\u0438 \u0432\u043c\u0435\u0441\u0442\u043e \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439. \u041c\u0443\u0437\u044b\u043a\u0430 \u043c\u043e\u0436\u0435\u0442 \u0441\u0438\u043d\u0442\u0435\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u0442\u0435\u043a\u0441\u0442\u043e\u0432\u043e\u0433\u043e \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u044f \u043d\u0430 \u0435\u0441\u0442\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u044f\u0437\u044b\u043a\u0435 \u0438\u043b\u0438 \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0435 \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u043d\u043e\u0433\u043e \u0448\u0430\u0431\u043b\u043e\u043d\u0430. \u041a\u043e\u043c\u043f\u043e\u043d\u0435\u043d\u0442\u044b \u0434\u043b\u044f \u0441\u0438\u043d\u0442\u0435\u0437\u0430 \u043c\u0443\u0437\u044b\u043a\u0438 \u043d\u0430\u043f\u0438\u0441\u0430\u043d\u044b \u043d\u0430 \u044f\u0437\u044b\u043a\u0435 Python \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a\u0430 PyTorch \u0438 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b \u043f\u043e\u0434 \u043b\u0438\u0446\u0435\u043d\u0437\u0438\u0435\u0439 MIT. \u041e\u0431\u0432\u044f\u0437\u043a\u0430 \u0441 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441\u043e\u043c \u0440\u0435\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u0430 \u043d\u0430 \u044f\u0437\u044b\u043a\u0435 TypeScript \u0438 \u0442\u0430\u043a\u0436\u0435 \u0440\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u044f\u0435\u0442\u0441\u044f […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":106129,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-106128","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47The Stable Diffusion machine learning system is adapted for music synthesis | ProHoster","description":"The Riffusion project develops a variant of the Stable Diffusion machine learning system adapted for generating music instead of images.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-mashinnogo-obucheniya-stable-diffusion-adaptirovana-dlya-sinteza-muzyki","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0421\u0438\u0441\u0442\u0435\u043c\u0430 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f Stable Diffusion \u0430\u0434\u0430\u043f\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0430 \u0434\u043b\u044f \u0441\u0438\u043d\u0442\u0435\u0437\u0430 \u043c\u0443\u0437\u044b\u043a\u0438 | ProHoster","og:description":"\u041f\u0440\u043e\u0435\u043a\u0442\u043e\u043c Riffusion \u0440\u0430\u0437\u0432\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u0430\u0440\u0438\u0430\u043d\u0442 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f Stable Diffusion, \u0430\u0434\u0430\u043f\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0434\u043b\u044f \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u0438 \u043c\u0443\u0437\u044b\u043a\u0438 \u0432\u043c\u0435\u0441\u0442\u043e \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439.","og:url":"https:\/\/prohoster.info\/en\/blog\/news\/sistema-mashinnogo-obucheniya-stable-diffusion-adaptirovana-dlya-sinteza-muzyki","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2022-12-16T13:36:47+00:00","article:modified_time":"2022-12-16T13:36:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":[],"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/106128","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=106128"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/106128\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/106129"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=106128"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=106128"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=106128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}