NVIDIA Neural Network Reconstructs Images With Missing Parts

Advancements in machine learning and computer imaging are making impressive but impossible scenes in crime drama shows finally possible. Like the infamous "zoom in, enhance" CSI method. The latest achievement comes from NVIDIA, who has been aggressively pushing its silicon, like the Tesla V100 GPU as the processor of machine learning research. Unlike the aforementioned enhancing technique, this deep learning technique is instead able to fill in holes and missing parts in images with something else that looks deceptively part of the original.

The process is called "image inpainting" and it's not exactly new. Naturally, NVIDIA researchers are boasting their new technique to be better than any other so far and they definitely have the examples to show for it. While previous image inpainting methods are limited to small rectangular areas near the center of the original image, NVIDA's technique can fill up any missing space in whatever size, shape, position, or even quality.

To train the neural network, NVIDIA researchers fed it more than 55,000 random masks, basically filters, of varying streaks and holes. It also used thousands of images for testing where the masks from the earlier set were applied to create holes in the images. And just to make sure the neural network wasn't cheating, researchers also used completely new holes and missing parts that weren't part of the earlier set. The results were definitely impressive and convincing, if not a bit freaky at times.

NVIDIA envisions that such an application would greatly benefit photo editing software in restoring photos or removing unwanted content. Of course, it could also be used for less innocent purposes, which could make identifying fake content, like fake news, a bit harder.