Oculus Quest hand tracking has finally arrived as promised at OC6, and we’re delighted. We had the opportunity to demo this breakthrough feature at the conference this fall. While it’s not perfect, it’s the future of virtual reality. Hand controllers, once so innovative, will end up looking like so many floppy disks.

Quest Hand Tracking

Here’s the short demo video from Oculus.

And the announcement from the Oculus blog:

From gesturing and communicating with others to picking up and manipulating objects, our hands play an important role in how we interact with the world—and they’re key to unlocking the feeling of true presence in a virtual space. We first brought your hands into VR with Oculus Touch controllers, so you could engage in VR in a more natural way. Now, we’re taking the next step with hand tracking on Oculus Quest—letting you interact in VR without controllers, using your real hands.

Getting access couldn’t be easier. Just update your headset to version 12 and access the Quest hand tracking under the Experimental Features menu. It’s easy to switch back to the standard Touch Controllers, so you’re not locking yourself into anything.

A Limited Rollout

If there’s a catch (okay, there usually is), it’s that access is limited during the initial rollout. You’ll only be able to use the Quest hand tracking feature with a few Oculus apps and while navigating the Oculus Library and Store. The all-important SDK will only be released to developers next week. It will be up to them to integrate it into other VR experiences. And Oculus updates don’t roll out universally – you may have to wait a few days before you see version 12 on your headset.

Unfortunately, you won’t be able to use the hand tracking feature with virtual experiences on the Rift. At OC6, John Carmack said it would be limited to the Quest as the digital signal processor (DSP) in the Qualcomm chipset had unused capabilities. Moreover, Oculus is maintaining compatibility between the Rift and the Rift S. Without onboard cameras on the former, hand tracking won’t come to the Rift S for now.

Here’s Ars Technica’s description of how you’ll use your hands in VR.

Right now, Quest turns your empty hands into laser pointers that can manipulate menus. Leave your hands somewhat open, like you’re about to pinch a pesky fly buzzing around, to make a pointer appear on a distant menu, as aimed by your hands’ orientation. Quickly pinch your index finger and thumb to “click” any menu buttons, or hold your pinch to drag menu elements like a scrolling list or volume slider.

The Future of VR is Hand Tracking

If the future of virtual reality is standalone headsets, the future of virtual gestures is controller-free tracking. We have a way to go before that becomes ubiquitous, but the Quest hand tracking feature is still a breakthrough.

In VR Scout, Bobby Carlton listed some of the challenges Oculus has to overcome. 

. . .I did find that really quick movements caused the headset to lose hand tracking fairly easily. There would either be some latency issues with the fingers and hands, or the headset would just lose track altogether, causing the VR hands to flip or spin backward as the headset attempted to recalibrate.

Though it only took a few seconds to recalibrate, it was enough to break the immersion for me, and I found it a little frustrating.

That paralleled our experience in using it at OC6. The recalibration was quick, but you ended up consciously moving your hands a little slower to keep up the sense of immersion. So while it was an amazing experience, latency issues were always a background factor.

But these are temporary obstacles that will soon be resolved.

The Hands-Free Haptic Feedback Challenge

However, there is a more significant challenge here, and it’s a fascinating question. What happens to all those developments trying to bring haptic feedback to our hands? Hands-free also happens to be feedback-free. High-end virtual reality experiences that require precision will still need some form of haptic feedback device. Medical, engineering, and other fields come to mind. One possible solution is the recent Wearable Skin breakthrough from the École Polytechnique Fédérale de Lausanne’s (EPFL) Reconfigurable Robotics Lab (RRL). Another is the Skin Patch under development at Northwestern University. But bulkier devices such as the Plexus Gloves will have to slim down to match the Oculus development. If not, you’re just replacing one device with another.

The Barriers to VR are Falling

Hand controllers were great when they first arrived. But they primarily appealed to the gaming community, not the general public. Time and again, we’ve watched people at VR events, immersive art exhibits, and film festivals fumble with the controllers. If you knew what you were doing, fine. But since you often couldn’t see the buttons while wearing an HMD, it was challenging for many. Hand tracking will end one of the significant barriers to access by the public, boosting the broader acceptance of virtual reality.

We were not expecting Oculus hand tracking until spring 2020. Count it as an early holiday gift from the rapid pace of immersive tech developments. Oculus continues to push virtual reality forward, and we can’t wait to see what developers do with this new feature.