Nvidia's neural network turns simple sketches into beautiful landscapes

The waterfall of the smoker and the waterfall of a healthy person

We all know how to draw an owl. First you need to draw an oval, then another circle, and then - you get a gorgeous owl. Of course, this is a joke, and a very old one, but Nvidia engineers tried to make the fantasy come true.

New development, which is called GauGAN, creates gorgeous landscapes from very simple sketches (really simple - circles, lines and all). Of course, this development is based on modern technologies - namely, generative adversarial neural networks.

GauGAN allows you to create colorful virtual worlds - and not only for fun, but also for work. So, architects, landscape designers, game developers - they can all learn something useful. Artificial intelligence immediately “understands” what a person wants and complements the original idea with a huge amount of detail.

“Brainstorming in terms of design development is much easier with the help of GauGAN, since a smart brush can complement the initial sketch by adding quality images,” said one GauGAN developer.

Users of this tool can change the original idea, modify the landscape or other image, add sky, sand, sea, etc. Everything your heart desires, and the addition takes just a couple of seconds.

The neural network was trained using a database of millions of images. Thanks to this, the system can understand what a person wants and how to achieve what they want. Moreover, the neural network does not forget about the smallest details. So, if you schematically draw a pond and some trees next to it, then after the landscape is revived, all nearby objects will be reflected in the mirror of the pond water.

You can tell the system what the visible surface should be - it can be covered with grass, snow, water or sand. All this can be transformed in a second, so that the snow becomes sand and instead of a snowy wasteland, the artist gets a desert landscape.

“It's like a coloring book that says where to place the tree, where is the sun, and where is the sky. Then, after the initial task, the neural network animates the picture, adds the necessary details and textures, draws reflections. All this is based on real images,” says one of the developers.

Although the system lacks "understanding" of the real world, the system creates spectacular landscapes. This is because two neural networks are used here, a generator and a discriminator. The generator creates an image and shows it to the discriminator. He, based on millions of previously seen images, chooses the most realistic options.

That is why the generator "knows" where the reflections should be. It is worth noting that the tool is very flexible and equipped with a large number of settings. So, with it, you can paint, adjusting to the style of a particular artist, or just play around with the quick addition of sunrise or sunset.

The developers claim that the system does not just take images from somewhere, add them together and get the result. No, all received "pictures" are generated. That is, the neural network “creates” like a real artist (or even better).

So far, the program is not freely available, but soon it will be possible to test it in work. This can be done at the GPU Technology Conference 2019, which is currently underway in California. The lucky ones who were able to visit the exhibition can already test GauGAN.

Neural networks have long been taught to take part in the creative process. For example, last year, some of them could create 3D models. In addition, the developers from DeepMind trained the neural network to restore three-dimensional spaces and objects from drawings, photographs, and sketches. In order to recreate a simple figure, one picture is enough for the neural network, to create more complex objects, five pictures are required for “training”.

As for GauGAN, this tool will clearly find a worthy commercial application - many areas of business and science have a need for such services.

Source: habr.com