A group of researchers from several American universities published a draft (Pixel-Aligned Implicit Function), which allows you to apply machine learning methods to build a 3D model of a person from one or more two-dimensional images. The system allows you to recreate complex clothing options, such as pleated skirts and high heels, and various hairstyles, independently restoring the texture and shape in areas not visible in the projection from which the 3D model is built. To increase the quality and detail of the final 3D model, several images from different angles can be used. The project code is written in Python using the PyTorch framework and under the MIT license.
A neural network is used as a source for the reconstruction of a three-dimensional layout, which allows choosing the most probable shape and inventing hidden elements, starting from a model trained on various options for existing objects. In parallel, the project provides an algorithm for matching the obtained volumetric layout with the textures in the provided 2D images, which aligns the pixels of the 3D image according to their position on the XNUMXD object and generates the most likely missing textures. Any image can be encoded For
surface reconstruction applied architecture "", a
architecture-based neural network used for texture mapping .
Ready-made trained model used by researchers for free download, but the original data on which the training was carried out remains closed, as it is based on the results of commercial 3D scanning. As a source for self-training of the model can be used people from the Renderpeople project.
Source: opennet.ru