A group of researchers from Shanghai University of Technology toolkit , using machine learning methods to simulate human movements on static images, as well as to change clothing, transfer to different environments, and alter the perspective from which the object is viewed. The code is written in Python
using the framework . It also requires and the CUDA Toolkit.
The tool receives a two-dimensional image as input and synthesizes a modified result based on the selected model. Three types of transformations are supported:
Creating a moving object that mimics the movements on which the model was trained. Transferring appearance elements from the model to the object (for example, changing clothing). Generating a new perspective (for instance, synthesizing a profile image based on a frontal photo). All three methods can be combined; for example, a photo can be used to generate a video that simulates performing a complex acrobatic trick in different clothing.
During the synthesis process, operations for isolating the object in the photograph and forming missing background elements when moving are also executed. The model for the neural network can be trained once and used for various transformations. For usage, pre-trained models are available, allowing immediate use of the toolkit without prior training. A GPU with at least 8GB of memory is required.
Unlike transformation methods based on keypoint transformations that describe body placement in two-dimensional space, the Impersonator attempts to synthesize a three-dimensional mesh that describes the body, using machine learning methods.
The proposed method allows for manipulations considering the personalized body shape and current pose, simulating natural limb movements.
To preserve original information such as textures, styles, colors, and facial recognition during transformation, a (). Information about the original object and parameters for its precise identification are extracted by applying .
Source: opennet.ru
