Facebook Taps AI To Turn Almost Any 2D Photo Into A 3D Image

A couple of years ago, Facebook announced support for 3D Photos, but the feature was limited to images captured using a smartphone with a dual-lens 'portrait mode' feature. Things are changing thanks to the company's machine learning technology, it explained in an announcement on Friday. Using AI, Facebook said it can turn almost any 2D image into a 3D Photo, expanding the feature to more users.

Though dual-lens smartphone cameras are becoming more common, Facebook notes that most mobile devices still only feature a single rear camera. The new machine learning-powered system is able to create a 3D image from a 2D image regardless of whether it was captured with a smartphone or it's an old photo captured with a film camera and later digitized.

This means that old family photos can be experienced in entirely new ways, Facebook points out. As well, high-end phones with dual-rear cameras but only single front-facing cameras can still be used to create 3D selfies. Facebook says its 'enhanced' 3D Photos capabilities are available to anyone using a mid-range or higher Android phone or at least an iPhone 7 smartphone.

The technology behind the new capabilities involves a convolutional neural network that can rapidly determine the 3D positions of subjects featured within a 2D image. The system is optimized to run in a fraction of a second on an ordinary tablet or smartphone, something made possible in part by Facebook's ChamNet and FBNet techniques.

Facebook says that it is also looking forward to the use of these technologies to generate high-quality depth estimates of videos captured using smartphones and tablets. This is quite a bit trickier than the similar work with images, however, because a video is made from many individual frames and each frame must have consistent depth data relative to the ones that come before and after it.