We’re fascinated with creative hacks, and this one is too good to pass up. Try combining HTC Vive VR and HoloLens to synchronize both virtual and merged reality.

The HTC Vive is one of the best VR headsets available with deeply immersive experiences. Microsoft’s HoloLens is redefining augmented reality – so much that they refer to it as mixed reality. Put the two together and you have a shared reality where a HoloLens user can interact with the virtual space of the VR user.

It’s more than just hacking a couple of platforms together. It has tremendous potential for collaborative work in art, design and other areas. And it shows where we’re headed with virtual reality in the future. Not just immersive experiences, but shared immersive environments.

We can do better than VR

The fascinating experiment comes from Drew Gottlieb, a software developer. As Gottlieb notes on his blog, one of the main frustrations in virtual reality is watching someone else play around in it. At best, you’re the outside observer watching what they see on a flat screen:

Virtual reality is a pretty magical experience when it comes to making art. However, if you have friends in the room watching you, the magic is lost on them. They can only see the experience by looking at a distorted preview of the player’s perspective on a computer monitor.

A friend was recently painting a 3D submarine in Tilt Brush. “Look at this periscope!” she said. I told her to look closer at it, and I leaned into my computer monitor to get a glimpse of what she made.

We can do better than this. Why do I have to get up off the couch to see what my friend is creating? Why can’t I just lean back and see the art floating in the middle of the room?

So he got to work, building a small Tilt Brush type application. He then got HoloLens to track the Vive hand controller. The two different worlds are synchronized through Unity’s built-in networking and matchmaking service. It’s not the most elegant solution, but it works. And works surprisingly well.

Here’s the short demo:

As Engadget notes:

. . . there’s no limit to how many HoloLens observers can join in. The VR headset also becomes unnecessary, so anyone with a HoloLens can pick up the motion controllers and start drawing shapes floating in the real world.

VR and HoloLens together

If you’re thinking what we’re thinking, there’s incredible possibilities here. As Gottlieb suggests, two people using VR and HoloLens could work collaboratively in art and other disciplines. You could have multiple HoloLens users working on a same project. And the virtual and physical hand controllers can be shared across the two platforms.

. . . people can collaborate in within the same space, across virtual and holographic environments. As a spectator with a HoloLens, you can say to the VR player “I have an idea, hand me a controller” and then interact with the art yourself. Or if you’re ignoring the VR headset entirely and just interacting as two people wearing HoloLenses, each of you can take one controller and you can build very large shapes – with some communication.

Drew Gottlieb’s experiment points out the power of Unity as a creative platform. It also points to where we’re headed. Not a world of single-user VR headsets and isolated experiences. But an immersive world of shared experiences. It’s a remarkable hack of virtual and mixed reality that brings us closer to the future.

You can read the full account on Drew Gottlieb’s site and download the source code from Github: https://github.com/dag10/HoloViveObserver