The Future Of Smartphone Cameras Is AI

Even before Google demonstrated its frighteningly believable and controversial Duplex AI, artificial intelligence and machine learning was already a big topic on smartphones. But with very few exceptions AI on smartphones have so far been limited to one common use case: photography. You might be tired of hearing about it every new phone launch event but, sooner than later, it's going to cease being just a buzzword. Pretty soon, "camera AI" will be something that all but the cheapest entry-level smartphone will have as a basic feature. Just like dual cameras.

At its most basic, camera AI on smartphones is used for scene recognition. You fire up the camera app, point your phone at your subject, and it will automatically tune the settings for the best possible shot, depending on what category the subject falls under and other factors like lighting conditions. It's pretty much the camera's mode on steroids but that belies the heavy computational processing that happens, especially even before the camera app is installed on a smartphone.

Like any other AI, there are two parts to the process. The first part happens at the manufacturer's or software developer's side. This is where the AI learns and builds up its mental models. In this specific case, the AI is fed thousands upon thousands of images in order to teach it how to properly categorize them into, for example, food, landscape, people, and more. This is the most computationally intensive part, as the neural network has to learn in just a few minutes, at most hours, something that humans learn over years.

Image courtesy of XDA

The other half of the AI is the actual application of what it has learned and happens on the phone when you launch the camera app. This is the part that applies the mental models developed in the learning stage to whatever image is being fed through the camera sensor. While there is still some algorithmic wizardry happening behind the scenes, it's not as extensive as the learning process.

That last bit is important for smartphone makers. It practically means that almost every modern smartphone from the mid-range and higher is already equipped to handle camera AI applications. Sure, you will benefit from having a dedicated "neural processing unit" ala Huawei's Kirin 970, but you don't have to wait for cheap silicon to become available.

Much of the heavy lifting has already been done on the developer's side, with models simply being downloaded on the device and taking up not much space. Even without offloading the AI processing to the cloud like virtual assistants, which would introduce unnecessary latency, the actual scene and object recognition can easily happen on the smartphone itself. Even tiny smart hearing aids, which only have a fraction of the processing power and storage size of smartphones, are able to have some AI-powered capabilities because of this.

It will only be a matter of time before mobile processor makers make such an AI a staple feature on their silicon. Qualcomm has already begun touting first its Spectra image processor followed by the Snapdragon 845's AI chops. Of course, there's also the aforementioned Kirin 970. Expect MediaTek to follow suit soon and for Qualcomm to make the feature available on the Snapdragon 600 series as well.

So it's possible and even easy to implement camera AI on almost all smartphones moving forward. It's no longer a question of "can" but of "should". The smartphone market is moving towards this kind of mobile photography not just because of the AI hype but also because it's what users never knew they needed.

While there's a great deal of professional mobile photography going around, where settings are manually and carefully tweaked to frame the perfect shot, majority of smartphone users only have a few seconds to press the shutter button before the moment disappears forever. They're not going to spend those precious seconds dealing with ISO levels, white balance, aperture, and the like. They will simply point, maybe pause, and shoot and hope that it turns out great.

Camera AI is meant to work during that pause to think the things humans would be too slow to think of in a split second. While we are naturals at identifying objects and scenes, it takes months even years of training to immediately know what camera settings to use in a given situation. And that's just for one shot. Imagine going on a tour where each shot happens in a different place under different conditions and on different subjects. Human brains would overload. That tiny chip in your smartphone would jump in glee.

That's not to say there is no room in the mobile world for "pro" manual controls. With manufacturers putting their focus on camera AI, however, those will become more luxuries than staples. They will become selling points and differentiating factors in a sea of smartphones that all have some camera AI feature.

This may be well and good, but one conversation around camera AI doesn't seem to be happening. Which is pretty ironic considering the latest issues regarding user data privacy. While the machine learning process doesn't have to happen on the device itself, the mental model could still do with some adjustment and refinement after the phone or the camera app has shipped. This means feeding the AI additional images. Your images, that is.

If you ask Google, it will always answer that is AI, and pretty much all its services, can be improved for all if you agree to upload pieces of your data to its servers. In fact, in some cases Google makes it mandatory because processing happens on its servers. Sharing text and voice data has already raised a few red flags. Imagine the concern when more personal, and sometimes intimate, photos are involved.

Some manufacturers may have subtle notices and disclaimers about sending (presumably) anonymized data for the sake of improving services. Like almost any disclaimer or warning, users are prone to just dismiss them without a second thought. Given the potential for abuse, whether intentional or accidental, it should be something manufacturers should be clearer about and user should be more aware of.

It might all seem like a fad for now. After all, everyone is jumping on the AI train. Pretty soon, however, Camera AI isn't going to be on a banner but simply a bullet point. And smartphones that don't have the future will be shunned as an artifact of the past. It will undoubtedly be useful, helping users take even better photos without really thinking much about it. But as with anything these days, every new, sophisticated feature comes with a price. And often that price is your data.