FOVE's Eye-Tracking VR Headset Is Finally Shipping: Here's Why It's Special

Plenty of virtual reality headsets have hit the market since we first saw FOVE back in May 2015, but none can do what the Samsung-backed VR system can. Differentiating itself in the increasingly crowded entertainment space by its potentially game-changing eye tracking system, FOVE sets out to address what it calls the "owl problem". If you want to look around in a VR world then, traditionally, you have to move your head.

That may be a fairly easy thing to grasp as a control dynamic, but it's not especially lifelike. FOVE 0 differentiates itself by adding a complex system of LED illumination and cameras within the headset, which – together with plenty of processing – track your pupils. That means not only does a game know at which angle your head is pointing, it knows to what, exactly, in the scene you're paying attention.

Things have changed considerably since I tried an early prototype of FOVE 0 last year. Since then, on the business side the company has been through a successful crowdfunding campaign, raising almost half a million dollars. It also caught the attention of Samsung Ventures, the electronics giant's investment arm, which dumped an unspecified amount of cash into the project.

All the same, it hasn't been entirely smooth sailing. The Kickstarter originally promised devices would ship in March 2016; in February, though, it admitted it wouldn't make that deadline, and offered refunds for those disappointed. At the same time, it also announced it would be using a homegrown positional tracking system rather than, as originally expected, the same Lighthouse system as HTC VIVE relies upon.

Now, though, we have a release date, and preorders kick off today. FOVE 0 is priced at $599, though if you slap your credit card down between now and 8am PT on November 9, you'll get fifty bucks off. The first headsets are expected to ship by the end of the year, though backers of the original Kickstarter are, rightly, being prioritized; the expectation is that preorder customers will get their FOVE 0 around the end of January 2017.

I caught up with the team earlier this week to take a look at the latest FOVE 0 headsets, now more compact and less cumbersome than the prototype I donned last year. The company's own positional tracking system is up and running and working well; FOVE is actually quite proud of the fact that its approach needs neither wireless pairing via Bluetooth or similar, or a sync cable, unlike most of the competition. Instead, a small camera watches the array of LEDs – invisible to the human eye – shining out through the black or white housing.

FOVE 0 will ship with one such camera, and as a result be able to track head movement at sharp oblique angles to the tracker, though not if you turn all the way around. I'm told adding a second is theoretically quite straightforward, however. Even with one, drift correction is an area of particular pride for the FOVE team: you can take the headset off, shake it around violently, and the system will still keep track of it.

Of course, the big difference between FOVE and other VR systems is what goes on inside the eyepiece. Now, the 120 fps infrared eye-tracking is accurate to less than a degree of movement, but with a simplified calibration system that – requiring 5 points rather than the previous 9 – is both faster and more stable. Impressively, it's also increasingly able to handle prescription glasses.

FOVE won't be officially supporting glasses-wearing at launch, but the unofficial message is that there's a fair chance you might have a good experience all the same. The prescription of the lenses and the size of the frames makes the most significant impact, I'm told; the pair worn by one of FOVE's engineers presented no problem, but my own weren't so keen to play ball. New post-processing is in the pipeline to smooth out lens-related jitter, and there's a next-gen idea simmering about how to deal with it more comprehensively, including selectively shutting off the IR LEDs when there's a reflection in the way of the tracking system.

That's something for the future, though. Right now, FOVE's eye-tracking offers a few primary advantages over other headsets. One of the most significant is so-called foveated rendering, where because the game or app knows where you're looking, it can concentrate graphical processing on that specific point.

Because of the way the human eye works, our peripheral vision is at a far lower quality compared to what we're looking at directly. Foveated rendering takes advantage of that fact: the point being focused on is at the highest resolution the system can supply, but it saves GPU cycles and power by lowering the resolution of everything else. In the demo I tried, I couldn't notice any real difference between how the scene looked when foveated rendering was toggled on or off, which is just what you would want to be the case.

The potential savings are significant. FOVE says it could cut up GPU use by up to 75-percent, in fact, though in practice what generally happens is that some of that saving is then spent on making the scene more visually effective. A depth-of-field effect, for instance, would be impractical if applied to the whole scene universally, but can be implemented while still saving 50-percent over a non-foveated view.

According to FOVE, game developers shouldn't have too much trouble implementing the system. In fact, if they're already using video multi-resolution sampling and compliant with the relevant background APIs in Unreal and Unity, there's a fair chance FOVE will be able to "just work" with them with no real modification. All the same, there's a good reason why studios might want to give more thought to eye-tracking.

That's because, as well as foveated rendering, FOVE is also exploring the possibilities when games know exactly where you're looking. In a game I tried, a first-person shooter that saw me blasting enemies from my spider-like mech, I could control the turret by moving my head but aim with my eyes. It very quickly becomes second-nature, to the point that, when the eye-tracking was disabled in the demo, the sheer clumsiness of controlling the system solely by head motions proved abundantly clear.

It's not just about cutting down what thumb-twiddling you need to do on a gamepad. FOVE and early software partners are looking at how new gameplay dynamics can take advantage of eye-tracking. In one demo, for instance, players are kidnapped by terrorists who demand to know which of three photos on the table show an informant. The player has to develop sufficient "eye discipline" not to look too closely at the informant's picture, lest their gaze give the identity away.

Later on, while still being held captive, the player must avoid glancing up at a rescue attempt being staged in the corner of the warehouse. If they pay too much attention there, the kidnappers will realize something is happening and the extraction will fail.

FOVE expects the system will gain traction in in-game conversations, whether with computer-controlled avatars or other players. Those characters will know when they're being stared at, for instance, or if you're focused on a particular item they're carrying. The company plans to add full face-tracking in a later generation, which could theoretically map your entire expression into a game or social app.

If that sounds ambitious to you, FOVE is aware of it too. The company had originally talked about the potential advantages of things like foveated rendering when dealing with VR on relatively low-power platforms, like smartphones. Now, it tells me it plans to license its eye-tracking system to others, so that it can focus on high-end, PC-based experiences now, and things like emotional interaction.

That's not to say it doesn't have proofs-of-concept. I wasn't shown them, but apparently there are mobile prototypes already in existence, with support for a variety of mainstream SoCs including Qualcomm's Snapdragon and Samsung's Exynos. The minimum requirement is Vulkan support.

Whether that will pan out to, say, a future Samsung Gear VR which can track eye movement remains to be seen. The FOVE 0 headset shipping in a month or so will be tethered to a PC – it needs at least a Core i5-4590 processor, 8GB of memory, and an NVIDIA GeForce GTX 970 or AMD Radeon R9 290 or greater. It's Windows-only, and you'll need a spare HDMI 1.4, USB 3.0, and two USB 2.0.

Inside the headset, a single 2560 x 1440 OLED screen has a 70 Hz refresh rate. That's 40-percent higher resolution than what Oculus or Vive offers, the FOVE team points out. Comfort has increased over the last prototype I tried, with the FOVE 0 tipping the scales at 520g.

NOW READ: Google is making an eye-tracking VR headset

Although delivery may be in sight, we're still a fair way from seeing FOVE on Best Buy shelves. Right now, the company has to convince developers that its vision of VR is more than just a gimmick, and that it makes sense to develop eye-tracking games and interfaces rather than just prioritize the bigger, more established names. Meanwhile, it won't have the eye-tracking space to itself for long, with other companies working on similar technology.

From what I've seen, eye-tracking does make a significant difference to how you perceive virtual environments. Going back to a regular headset is jarring, in fact; human beings really aren't owls. Whether it'll be FOVE's technology that powers that next generation of eye-tracking headsets remains to be seen, but I'd be very surprised if it's not a feature of tomorrow's VR experience.