Facebook Thinks AI Can Replace Expensive Dual-Camera Phones

Dual cameras are becoming a must-have feature on today's top-tier smartphones – to the point that their absence is one of a few big complaints about the Galaxy S8 – but Facebook is looking to replace one of their most common uses with AI. At F8 2017 today, the social network's annual developer conference, Facebook showed off one of its potential implementations of artificial intelligence. While smart computers may most commonly be associated with Facebook's work on bots, in fact the applications are far broader.

In an on-stage demo, Facebook showed how AI could replicate one of the currently fashionable effects on smartphones: virtual bokeh. Best known, perhaps, as the iPhone 7 Plus' Portrait mode, currently the best implementations of those effects use two cameras. By offsetting the two lenses, and thus adding the ability for the phone to recognize depth, the subject of the frame can be distinguished from what's behind them.

Facebook's alternative uses AI algorithms to analyze a scene, and from a single lens figure out the depth of what's in it. From that, it can do the blurring calculations. On-stage, the mockup app had a slider which could be used to adjust the degree of bokeh to taste.

Now, doing a portrait mode with a single camera isn't new. The Galaxy S8 has a mode for that, in fact, despite only using a single lens. Where Facebook's system goes further is in how it applies the effect in real-time: if you move around to reframe the scene, the blurring is automatically generated within the camera preview.

It's all particularly important now, given Facebook's focus on extending the camera. Already it's been added to all of the company's apps. Now, at F8, the company is talking about how computational photography, augmented reality, and more will use the expanding camera API to blend real and virtual components in the same scene. That could be everything from face filters through to digital sticky notes that could be pinned to real locations for others to find.

To encourage developers to get onboard, Facebook has open-sourced Caffe2 along with Qualcomm, NVIDIA, Intel, and others. A deep learning framework, it allows developers to tap into advanced AI processing, with server-side GPU crunching, a suite of software, and a device-side element. It opens the door to a huge range of AI implementations.

Indeed, artificial bokeh is only the start of it. Facebook is also working on using AI to power algorithms for casual 3D capture on any phone. Rather than require a multi-camera system – like, indeed, the x24 Facebook Surround 360 camera also announced today, a huge ball studded with 24 different lenses and sensors – the system could allow any phone to generate 3D environments which users could move around, even with a single camera.