Nowadays, removing unwanted elements from photos is no longer a problem for many. Basic skills in Photoshop or trendy neural networks today can solve the issue. However, when it comes to video, the situation is more complicated, as at least 24 frames need to be processed per second of video.
And here on Github is a utility that automates these actions, allowing users to remove any moving objects from video. It’s as simple as highlighting the unwanted object with the cursor, and the system does the rest. The utility is simply named — video-object-removal. However, it is based on advanced technology.
The system uses a neural network that processes video frame by frame, replacing the unwanted object or person with the background. The program can change up to 55 frames per second, reconstructing the background based on the surrounding image. Though upon closer inspection, it becomes clear that the object removal method is far from perfect, the results are impressive.
In some frames, it can be seen that a transparent or semi-transparent phantom trace remains where the 'removed' person was. The system only analyzes the available background and as of now, it does not always accurately reconstruct it. This depends on the complexity of the background — the simpler and more uniform it is, the better the final result.
For testing, Ubuntu 16.04, Python 3.5, Pytorch 0.4.0, and CUDA 8.0 were used, with processing done on an NVIDIA GeForce GTX 1080 Ti graphics card. The source code is open and can be used by anyone. It is worth noting that such technology can also be misused. For example, to 'hide' traffic violations or other crimes caught on camera.
Source: 3dnews.ru
