Inside The Nokia HERE HD Maps Putting Google On Notice

Where exactly am I? That's the question Nokia is facing, not in its position following the Microsoft deal that will see it shed its phone business, but in how it's constructing the next-generation of mapping data that's clever enough to give Google Maps sleepless nights. The HERE team calls it an "HD Map" – taking traditional cartography and putting it on steroids – and it's one of the main pillars the Nokia platform of tomorrow is expected to stand on.

Nokia has had a finger in the navigation pie for almost seven years now, starting with the acquisition of mapping data company NAVTEQ late in the last decade. It took until 2011 before Nokia brought NAVTEQ fully in-house, and wasn't until the following year that the HERE brand was created for the division formerly known as Nokia Location & Commerce.

While it's relatively unknown in comparison to, say, Nokia's smartphone team creating Lumia Windows Phones, HERE has already made a significant contribution to the company's mobile portfolio. HERE Maps on Windows and Windows Phone is obviously the company's handiwork, but Bing Maps also uses the HERE engine, in addition to the in-car navigation systems from most OEMs.

The current state of mapping is relatively straightforward. In effect, once you strip away the graphics of things like lanes and road edges, they're a collection of connected nodes. Each has multiple attributes piled on top, such as the maximum speed permitted, the lane size, number of lanes, whether there is a divider in-between, and other details.

That's sufficient for the basics, but HERE has been looking ahead and quickly realized that it wasn't going to be enough for the next-generation of uses, such as hyper-specific location-based services and self-driving cars.

"For the needs of navigation, you have the map you're familiar with today," HERE VP of Connected Driving Ogi Redzic explained to us. "For automated driving you need a completely different map, and that's HD Map."

HD Map is Nokia's next-generation of data-rich cartography, and it's created using the same sort of LIDAR laser mapping technology that we've already seen from the company's 68-megapixel street view mapping cars, not to mention on various self-driving vehicles.

LIDAR uses a rapidly-spinning laser array to bounce light off the surrounding area and then measure its distance. Each LIDAR unit is expensive – Nokia says the combined camera and laser rig on each of its second-generation mapping cars costs about the same as the vehicle it's attached to – but its benefit is huge accuracy and precision.

As the mapping car drives, it builds up a point cloud of its surroundings: a flurry of densely packed individual points which each indicates a reflected laser spot. It's incredibly precise, but it's also huge in terms of storage size. Each of Nokia's cars collects around 140GB of data a day, comprising LIDAR and image data, which is far too unwieldy to be manipulated readily.

Instead, the HERE team builds a vector model based on the laser data, trimming what would be petabytes for any reasonably sized map down to something more reasonable. However, where regular maps would be a couple of intersecting lines, an HD Map goes many steps beyond.

"The next generation map, used for automated driving, actually is built on top of this model," Redzic says. "It's fully three-dimensional, so each row doesn't have X/Y, it has X/Y/Z, so it's elevated in space. The map is modeled in a way that we actually take each surface of the road and model each individual point. There's no longer a center-line; each lane is modeled individually."

That means the exact width of each lane is known, along with its connectivity with other nearby lanes. So, if there's a left turn ahead, but only the leftmost lane is allowed to take it, that precise metadata is understood.

Equally importantly, Nokia adds in a third-dimension. Instead of treating the map as a flat surface on which roads are squiggled, HERE's HD Map takes into account elevation, too. That can help when planning routes which avoid unnecessary up- and down-shifting through the gears, saving on gas in the process.

As Redzic pointed out, for a trucking company that could make a big difference to what's spent on fuel each year, with drivers able to plot the most economical routes rather than simply the fastest or shortest.

"If you want to go from my house to my work, you might want to take the fastest route, but you may want to take the route that has the minimum number of stops. So, you don't stop necessarily and waste fuel at the traffic lights."

Nokia HERE Traffic: Inside the team tracking America's roads

Meanwhile, the high-resolution cameras on Nokia's cars are also snapping images in all directions, which the HERE team can combine with LIDAR data in interesting ways.

"We align images that we're taking at the same time as we're collecting this LIDAR," Jane MacFarlane, head of HERE Research at Nokia – who we talked to back in August 2013 about her work on turning smartphones into "probes" for tracking traffic and more – told us. "And we line up those images with the point cloud, and that allows us to color those dots. So now you get a colorized version of the place that you just drove. What we want to do is pull out doorways and things like that, which will help us with pedestrian navigation."

However there's also the possibility for the sort of 3D stereo panoramic images that simply aren't possible with rival street view cars, such as Google's. Existing 3D cameras are set up in one position and then rotate while grabbing a stereo image; however, both the camera and the scene around it need to be still in order for the panorama to come out.

In contrast, Nokia's system grabs photography and LIDAR from all directions simultaneously, and is in effect able to generate stereo panoramas while on the move. The HERE team can also colorize the LIDAR graphics in real-time – useful for car manufacturers, who might want their own unique on-screen aesthetic to match the rest of the dashboard – and apply effects like pen & ink.

That all pays dividends for the connected dashboards we're beginning to see emerge from the auto industry today, but it's also essential information for an autonomous vehicle. Out in the real world, Nokia's Redzic points out, there's only so much that local sensors on your bumpers can tell you.

"There's no automated driving at scale, and with a comfort level, without being able to properly place yourself in the real world," Redzic argues. "You've got to get that 10cm accuracy, so that the vehicle can properly position itself not only within the lane – which is critical – but also where exactly on the road you are, so you can make the right driving maneuvers."

For autonomous cars, driving dumb is key

"You can trust your sensors to a certain extent, but if there's a truck in front of the car that means you can't really see anything in front. So how do you trust your sensors to make the next maneuver? You have to know what the road looks like, what's coming ahead, how many lanes are there? How likely is it that there's ice on the road, the weather conditions?"

That data can all be delivered with a mixture of vehicle-to-vehicle communication and keeping in contact with the cloud, in effect giving the car a bird's eye view of its surroundings. "You want to extend the visibility of the sensors," Redzic explained. "The best sensors will give you maybe 100-150 yards; if you're going at highway speeds, it's not going to be good at 200kph on the Autobahn, it means you can only see so much. You have to know what's coming up next as well."

"For us that means provide information from sensors in vehicles ahead of you to vehicles behind, and also take into account all the real-time feeds you already have – like traffic – so you can come up with better driving strategies. What is the car going to do next?"

Nokia is keen to point out that we shouldn't expect to see a car with its badge on the grille, and that there are no plans to jump into the auto business just as it exits the phone industry. "We have no ambition to build a car ourselves," Redzic concludes, suggesting that instead Nokia will play to its strengths and build on its car industry partnerships, just as we saw with Mercedes-Benz's real-world trial of a self-driving vehicle last year. "We are building elements to help OEMs build connected cars sooner rather than later."