Snapchat's iPhone X AR Lenses Are Finally Here

Snapchat has launched its special iPhone X Lenses, with the more accurately-placed augmented reality masks taking advantage of Apple's TrueDepth front camera. The new iPhone X Lenses were part of Apple's big reveal for the smartphone in September last year, and indeed one of the few things the company demonstrated that tapped into the complex camera array.

That camera system doesn't just have a regular, color camera, but various depth sensors too. Apple uses it for Face ID, mapping each user's face and then creating a biometric key from that data. However, it's also opened up for third-party apps through iOS' camera APIs, allowing for more accurate tracking of whoever is using the phone.

In Snapchat's case, that opens the door to three new masks. Although they're not the first augmented reality add-ons that Snapchat has included, they do a much better job of molding to the shape of the user's face, and then tracking it as they move. That's down to the TrueDepth camera array.

Apple's cameras work by projecting a field of dots onto the face, using an emitter built into the "notch" in the iPhone X's display. Although invisible to the naked eye, they can be picked up by a separate camera alongside that emitter. By tracking the dots, iOS can build a 3D map of the face.

That mapping data is then passed on to apps, like Snapchat, which can use it for more precise modeling. Traditionally, AR overlays such as those Snapchat has made popular – like the doggy face or floating sunglasses – use basic facial feature tracking to figure out where they should appear. That can be confused, however, especially with faster-moving subjects, leading to misalignment of the AR Lenses.

It's worth remembering that, though the TrueDepth camera is used in both this case and for Face ID, there are different layers of security at work. Apple's unlock system for the iPhone X – which is also shared as the biometric security for things like Apple Pay – taps into the Secure Enclave, with the image data of the registered face encrypted. What developers like Snapchat get, meanwhile, is the facial mapping data alone, with more than fifty micro-movements of things like lips, eyelids, and the nose.

Right now, only the iPhone X can use the three new Snapchat Lenses. However, according to the rumor mill, all of Apple's 2018 iPhone range – tipped to include a new, larger "iPhone X Plus" and a more affordable model – are expected to have the TrueDepth camera. That will open up AR features like these to far more people.