A group of researchers from Tel Aviv University has introduced HyperStyle, an inverted version of the machine learning system StyleGAN2 developed by NVIDIA, which is reworked to reproduce missing parts when editing real images. The code is written in Python using the PyTorch framework and is released under the MIT license.
While StyleGAN allows synthesizing realistically-looking new human faces by specifying parameters such as age, gender, hair length, smile characteristics, nose shape, skin color, glasses, and photo angle, HyperStyle enables changing similar parameters in existing photographs without altering their distinctive features and maintaining the recognizability of the original face. For example, with HyperStyle, one can simulate a person's age change in a photograph, change hairstyles, add glasses, a beard, or mustache, give the image the appearance of a cartoon character or a painting, or modify the facial expression to be sad or happy. Moreover, the system can be trained not only for altering human faces but also for any objects, such as editing images of cars.
The proposed method aims to address the problem of reconstructing missing parts of an image during editing. In previously suggested methods, the compromise between reconstruction and editability was addressed through fine-tuning the image generator to insert parts of the target image while recreating originally missing editable areas. A drawback of such approaches is the requirement for extensive targeted training of the neural network for each image.
The method based on the StyleGAN algorithm allows using a standard model pre-trained on general image collections to generate elements characteristic of the original image with a level of accuracy comparable to algorithms that require individual training of the model for each image. Additional advantages of the new method also include the ability to modify images with performance close to real-time.
Pre-trained models are ready for detecting faces of people, machines, and animals based on collections like Flickr-Faces-HQ (FFHQ, 70,000 high-quality PNG images of human faces), Stanford Cars (16,000 images of cars), and AFHQ (animal photos). Additionally, tools for training your own models are provided, along with pre-trained models compatible with them, such as standard encoders and generators. For instance, generators are available to create images in the style of Toonify, Pixar characters, sketch formation, and even for stylizing as Disney princesses.
Source: opennet.ru
