A group of researchers from Tel Aviv University has released the source codes related to the Motion Diffusion Model (MDM), a machine learning system that enables the generation of realistic human movements. The code is written in Python using the PyTorch framework and is distributed under the MIT license. For experimentation, both pre-trained models and the option to train models from scratch using provided scripts are available, for example, utilizing the HumanML3D collection of 3D human images. A GPU with CUDA support is required for system training.
The application of traditional methods for animating human movements is complicated by challenges stemming from the vast variety of possible movements and the difficulty in formally describing them, as well as the high sensitivity of human perception to unnatural movements. Previous attempts to use generative machine learning models faced issues with quality and limited expressiveness.
The proposed system attempts to utilize diffusion models for generating movements, which are fundamentally better suited for simulating human actions but are not without drawbacks, such as high computational resource demands and management complexity. To minimize the downsides of diffusion models, MDM employs a transformer architecture neural network and sample prediction instead of noise prediction at each stage, simplifying the prevention of anomalies, like loss of surface contact with the foot.
To control generation, the system allows for the use of natural language text descriptions of actions (e.g., 'a person walks forward and bends down to pick something up from the ground') or typical actions like 'running' and 'jumping.' The system can also be used to edit movements and fill in lost details. Testing was conducted with participants who were asked to choose the higher-quality result from several options—in 42% of cases, people preferred synthesized movements over real ones.
Source: opennet.ru
