[UPDATE: Google’s Project Tango shut down in March 2018 as focus shifted to ARCore. An early forerunner of location-based AR, we’re in a vastly different place today with ARCore and Apple’s ARkit.]

It’s clear that Google VR is going to be a lot more than Cardboard and some apps – and Project Tango may be the clue to future developments. Samsung is the only major player in the mid-range space with the $99 Gear VR headset, and Google dominates the low end of the market with Cardboard. But with leadership changes in place, Google VR may stake a position between Samsung and Oculus. Mobile VR is predicted to reach $861 Million this year – not huge by Google’s standards, but it’s the ground floor of a rapidly expanding market.

Head-tracking in VR

So how would Project Tango fit into a Google VR project? One of the challenges in running virtual reality off your phone is head-tracking. A virtual space needs to know where your head is (literally), otherwise, you’re not really in the VR space. A admittedly speculative article on Google’s plans by Dieter Bohn in The Verge describes the challenge:

The Gear VR and Google Cardboard can detect how your head is moving with a gyroscope and compass, but those sensors aren’t very precise. More importantly, they don’t detect your head’s position in space. Doing that second part is what enables headsets like the Rift and the Vive to feel fully immersive, letting you duck and dodge and have your real movements precisely mapped to the virtual world.

Oculus Rift and HTC Vive get around this using “outside-in” tracking. That’s the reason for the small dots on the headsets and hand controllers – it’s how the platform knows where your head is. And that’s part of the reason for the deeply immersive experience you get with high-end VR (the other is much higher video and audio quality). SmartPhone based VR doesn’t give you this. As a result, you don’t feel the need to duck when something is coming at you.

Dieter Bohn continues,

But those cameras represent extra hardware you need to set up and configure and mean that you can only use the associated headset in the room where they are. It would be far better to have the headset itself handle positional tracking, to do it “inside-out.” You wouldn’t need the extra hardware and you could use it in any room, not just in the VR cave where you’ve set it up. So far, we haven’t seen any consumer-grade hardware that can do it.

But if Google develops a new VR headset that leveraged Project Tango, you’d have a version of “inside-out” tracking and a much more powerful mobile VR experience.

Google VR and Project Tango

Google’s been working on Project Tango for some time, and Lenovo announced a Project Tango enabled phone at CES 2016. project with lenovo. Here’s a video of the strange Project Tango world:

What could make Project Tango incredibly powerful is that Google wants to crowdsource the data. Instead of relying on public buildings, art museums and businesses to provide access, our phones would automatically provide the data. In effect, our indoor spaces would be mapped out the way that Waze maps out traffic data for you. Our indoor spaces change much more than the outdoor world does in Streetview, and crowdsourcing would keep it up to date. For your home, you would map out the space with your own phone.

Project Tango at the Museum

Conference goers at Mobile World Congress got a chance to see Google’s work at Barcelona’s Museu Nacional d’Art de Cataluny. It didn’t work perfectly but was still an eye-opener. Chris Velazco describes the experience in Engadget,

One tap was all it took to launch an augmented-reality view, while pointing the tablet at the floor revealed a trail of blue, digital breadcrumbs leading us through the museum’s twisty corridors. Upon reaching one of the pre-marked spots and parking in front of portraits of an artist’s barstool comrades, I learned more about their histories by tapping a luminescent spot in the app’s AR view.

It was around then that Tango’s true potential sort of slapped me in the face — these sorts of experiences could easily be scaled beyond just stunning museum exhibits. In a few years, we could be finding our seats at football games without having to ask someone. Or feeding augmented-reality puppies while we wait for the L train to arrive.

But it also means that our homes would be mapped out. Combine that with a good mid-range VR headset and a whole new world opens up for low-cost virtual reality. You could be at the bottom of the ocean exploring coral reefs and dodging fish or examining the solar system from different vantagepoints. And all of it through your Smartphone and a VR headset.

We have few clues to what Google VR might lead to, but Project Tango could leverage the power of our mobile devices. Stay turned for the Google I/O conference in May.