Today, for many, it is no longer a problem to remove an interfering element from a photograph. Basic skills in Photoshop or trendy neural networks today can solve the problem. However, in the case of video, the situation is complicated, because you need to process at least 24 frames per second of video.
And here on Github a utility that automates these actions, allowing you to remove any moving objects from the video. It is enough just to select an extra object with a frame using the cursor, and the system will do the rest. The utility is called plainly - video-object-removal. However, it is based on advanced technologies.
The system uses a neural network that processes the video frame by frame, replacing an unnecessary object or person with the background. The program can change up to 55 frames per second, completing the background based on the surrounding image. Although on closer inspection it becomes clear that the method of removing objects is far from perfect, the results are impressive.
Some frames show that a transparent or translucent phantom trace remains in the place of the βremoteβ person. The fact is that the system analyzes only the available background and so far can not always adequately finish it. It depends on the complexity of the background - the simpler and more uniform it is, the better the final result.
For testing, we used Ubuntu 16.04, Python 3.5, Pytorch 0.4.0, CUDA 8.0, and the processing was carried out on an NVIDIA GeForce GTX 1080 Ti video card. The sources themselves are open and can be used by everyone. At the same time, we note that this technology can also be used for malicious purposes. For example, to βhideβ traffic violations or other crimes caught on cameras.
Source: 3dnews.ru