Google Project Tango Hands-On: 3D Mapping With Phones

Google loves smartphones, but with Project Tango it's hoping to make them even smarter and more aware. The handiwork of Google's skunkworks Advanced Technology and Projects Group, it explores what happens when you add high-resolution 3D mapping sensors to an Android device, and what the potential is for gaming and more. We caught up with Johnny Lee, technical program lead at Google, to find out more about the project and get some hands-on time with the prototype device.

The Project Tango team build its first prototype in late Q2 2013, a 7-inch Android tablet effectively sandwiching the mainboard between two slices of acrylic, and then fitted out with automotive-spec sensors. On the front there was a fisheye camera, while on the back was a high-resolution main camera – similar to what you might find on a regular tablet – together with a motion-tracking camera with a fish-eye lens, and a depth sensor.

That was replaced more recently by the second-gen prototype, the same 5-inch Project Tango smartphone we saw in Google's video back in February. Despite looking a little like an oversized Sony from a few years back, it's actually custom-made for Google, Lee told us, rather than a modified off-the-shelf device.

Lee wouldn't confirm exactly how many are out in the wild – Google in February said roughly 200 handsets had been seeded to developers – though did say that he wished there were more as demand has been strong. Despite being considerably smaller than the tablet, the Android smartphone actually contains all the same sensors along with a couple of extra vision-processing chips.

Those senses mimic how the human eye works, both giving broad peripheral vision and a more detailed, central section of foveal vision. The depth sensor actually cycles on and off during use, to help reduce battery consumption and processing load, though Lee said Google is still assessing different depth-sensing technologies.

Those sensing abilities actually get better over time. Although Project Tango is designed to do real-time mapping, if given a little room for some post-processing the results get even more accurate.

Google isn't talking about how much each handset has cost it, but Lee does refer to it as a consumer-level device. Compared to a GPS signal, after mapping two floors separated by a five-story staircase, Project Tango's computer vision-only mapping was just 1-percent out. The device's depth sensor was only 0.6-percent out compared to a professional laser scanner.

That's all well and good, but what Project Tango needs is a "killer app" to show people why they want these sort of high-fidelity depth sensors on their device. The ATAP team aren't game designers, Lee pointed out, but has cooked up a few game-style demos to show some of the possibilities. In one, you help an on-screen character explore a fantasy land that's constrained within the limits of the physical space you're in, first kneeling down to introduce yourself to your short companion, and then walking around with them as they navigate.

Another demo shows how Project Tango can make virtual environment assumptions based on your actual surroundings. Panning the handset around plants "digital grass" on the floor, but not on walls or any other objects. That way, Lee points out, developers could rely on Tango's inherent smarts to automatically define the spaces that gamers are in.

The research is only around a year old – the ATAP division as a whole was started up as part of Motorola Mobility after the Google acquisition, and then shifted within Google after the phone business was sold off to Lenovo – and there's still some way to go before it's ready for an Android device in your pocket. Still, as new phones like the HTC One M8 demonstrate, there's already real-world experimentation going on with different kinds of data such as depth, even if the fidelity isn't to the same accuracy as Google's custom handsets can achieve.

One thing is clear, though: our smartphones are already smart enough to do the raw crunching, they just need the sensors and software. "The compute is here," Lee pointed out. "Project Tango is an attempt to pull forward the stage of sensing hardware and software on mobile."