Adobe Enhance Details Sees AI Go All CSI On Your Photos
Adobe is setting an AI loose on your photos, with its new Enhance Details feature added to Lightroom CC among other apps to auto-magically increase their resolution by up to 30-percent. Powered by Sensei, the new system takes raw photos and applies a whole new way of processing them.
To see the benefit, therefore, you'll need a raw image file. Adobe says that Enhance Details works well on either Bayer files – as captured by cameras from Canon, Nikon, Sony, and others – and Fujifilm X-Trans files.
Where it differs from the previous demosaicing process is in how Enhance Details uses machine learning. In short, Adobe trained a convolutional neural network, or CNN, with over a billion different photos. That allowed it to build up best-practices for dealing with difficult demosaicing of raw images, and re-apply those lessons to new photos.
Demosaicing is the process by which raw images from a camera – which consist of a monochrome image and then an overlapping set of data about the color of each pixel, from the color filter array. Since only one of the three color values is stored for each pixel, the software doing the processing has to figure out the remaining two values.
While there are several traditional methods by which to do that, Adobe points out that they all share the same common issues. That includes the challenge of dealing with small-scale detail close to the resolution limit of the camera's sensor; false colors where the software gets adjoining pixel colors wrong; and zippering along the edges. Adobe Sensei, the company's AI platform, was therefore trained on images with one or more of those problems.
To be fair, some of the existing methods for advanced demosaicing have been impressively adept. The problem is that they've also been hugely computationally intensive. Where Enhance Details tries to set itself apart is in tapping into machine learning systems built into the latest versions of macOS and Windows 10, to boost performance.
There are other ways to help it, too. A faster GPU, Adobe points out, can introduce a big improvement; if you're using a laptop, an eGPU "can make a big difference" the company says.
Meanwhile images that have artifacts, and lots of fine detail, are the best source shots to show the most improvements. Get it right, and Enhance Details should bring out more accurate edges and details, less moiré patterning and fewer false colors.
Enhance Details is being added to Adobe Camera Raw, Lightroom Classic, and Lightroom CC for Windows and Mac. There are actually two different AI models, one for Bayer and one for X-Trans, though the software will choose the correct one automatically. You'll also need either a Mac running macOS 10.13 or later, or a PC running Windows 10 (1809) or later, since the new feature is based on Core ML and Windows ML respectively.