The HTC Vive is one of the best VR headsets available with deeply immersive experiences. Microsoft’s HoloLens is redefining augmented reality – so much that they refer to it as mixed reality. Put the two together and you have a shared reality where a HoloLens user can interact with the virtual space of the VR user.
It’s more than just hacking a couple of platforms together. It has tremendous potential for collaborative work in art, design and other areas. And it shows where we’re headed with virtual reality in the future. Not just immersive experiences, but shared immersive environments.
We can do better than VR
The fascinating experiment comes from Drew Gottlieb, a software developer. As Gottlieb notes on his blog, one of the main frustrations in virtual reality is watching someone else play around in it. At best, you’re the outside observer watching what they see on a flat screen:
Virtual reality is a pretty magical experience when it comes to making art. However, if you have friends in the room watching you, the magic is lost on them. They can only see the experience by looking at a distorted preview of the player’s perspective on a computer monitor.
A friend was recently painting a 3D submarine in Tilt Brush. “Look at this periscope!” she said. I told her to look closer at it, and I leaned into my computer monitor to get a glimpse of what she made.
We can do better than this. Why do I have to get up off the couch to see what my friend is creating? Why can’t I just lean back and see the art floating in the middle of the room?
So he got to work, building a small Tilt Brush type application. He then got HoloLens to track the Vive hand controller. The two different worlds are synchronized through Unity’s built-in networking and matchmaking service. It’s not the most elegant solution, but it works. And works surprisingly well.
Here’s the short demo:
As Engadget notes:
. . . there’s no limit to how many HoloLens observers can join in. The VR headset also becomes unnecessary, so anyone with a HoloLens can pick up the motion controllers and start drawing shapes floating in the real world.
VR and HoloLens together
If you’re thinking what we’re thinking, there’s incredible possibilities here. As Gottlieb suggests, two people using VR and HoloLens could work collaboratively in art and other disciplines. You could have multiple HoloLens users working on a same project. And the virtual and physical hand controllers can be shared across the two platforms.
. . . people can collaborate in within the same space, across virtual and holographic environments. As a spectator with a HoloLens, you can say to the VR player “I have an idea, hand me a controller” and then interact with the art yourself. Or if you’re ignoring the VR headset entirely and just interacting as two people wearing HoloLenses, each of you can take one controller and you can build very large shapes – with some communication.
Drew Gottlieb’s experiment points out the power of Unity as a creative platform. It also points to where we’re headed. Not a world of single-user VR headsets and isolated experiences. But an immersive world of shared experiences. It’s a remarkable hack of virtual and mixed reality that brings us closer to the future.
Emory Craig is a writer, speaker, and VR consultant with extensive experience in art, new media, and higher education. He speaks at global conferences on innovation, education, and ethical technology in the future. He has published widely and worked with the US Agency for International Development, the United Nations, and the Organization for Economic Co-operation and Development (OECD). Living at the intersection of learning, games, and immersive storytelling, he is fascinated by AI-based avatars, digital twins, and the ethical implications of blurring the boundaries between the real and the virtual.