The HoloLens 2 Secret Sauce Just Got Revealed

Microsoft's reboot of Kinect may be giving drones smarter vision and bringing AI to edge computing, but it'll also be at the heart of the next-generation HoloLens 2. Unveiled during Build 2018 this morning, the new Project Kinect for Azure combines an updated version of Microsoft's time-of-flight (ToF) depth sensor with new AI talents.

According to Alex Kipman, Microsoft Technical Fellow of the Windows and Devices Group and key advocate for HoloLens, it's been earmarked for the new version of the AR headset that the company has in the pipeline. Indeed, the HoloLens team had already previewed some of the news about the new sensor earlier in the year, beating CEO Satya Nadella to the punch.

The sensor has 1024 x 1024 resolution, Kipman says, and a global shutter for improved performance in sunlight conditions. It has the highest figure of merit, too – that is to say, the highest modulation frequency and modulation contrast – which means lower power consumption. Indeed, overall system power is just 225-950 mw.

That's despite adding things like automatic per-pixel gain selection, for a broader dynamic range. That will give HoloLens 2 better abilities at capturing both near and far objects. There's multiphase depth calculation support, which will preserve accuracy even if the chip, laser, or power supplies vary.

Finally, and arguably most importantly if HoloLens is ever to make it into the mainstream, there'll be low peak current operation, even at high-frequency. That, Kipman explains, will help lower the cost of modules.

"With HoloLens, we saw incredible results when we took some of the magic of Kinect and applied it in a mixed reality context," Kipman writes. "The current version of HoloLens uses the third generation of Kinect depth-sensing technology to enable it to place holograms in the real world. With HoloLens we have a device that understands people and environments, takes input in the form of gaze, gestures and voice, and provides output in the form of 3D holograms and immersive spatial sound."

Now this fourth-generation Kinect will better integrate HoloLens 2 into the cloud, yet also deliver improvements on the intelligent edge. That means more onboard smarts, more effective AI even when you have limited or no connectivity whatsoever, and more efficient use of bandwidth and cloud processing when it does become available.

There's still no word on when, exactly, we can expect the next-generation HoloLens 2 to launch. Microsoft has already confirmed that the new augmented reality headset will use an ARM-based chipset for improved power consumption, running a custom version of Windows 10 to drive its Windows Holograph and Mixed Reality UIs. The expectation is that HoloLens 2 will launch early in 2019.