The very popular Clash of Clans 360 video (over 32 million views) gives you a quick sense of what simple VR games will be like. While you can watch it through your browser, the best experience will be through the YouTube app on your Smartphone. This is not full VR, just 360 video, so flip your phone to landscape mode, stand up and turn around. It’s definitely an immersive experience.

Clash of Clans 360 video

Clash of Clans falls into that breed of mobile gaming which depends less on strategy than how much you’re willing to spend on upgrades. In an old review from Modojo, John Bedford notes how it’s become a money-grabbing fest,

This is a game that follows in the footsteps of no small number of titles that have made feverish demands on our wallets in exchange for just a slightly thicker slice of the gameplay.

As a vision of what a VR game might be, the video is highly limited. It’s nothing like the experience of gaming using a VR headset such as Oculus Rift or HTC Vive.

The challenge in VR games

The Clash of Clans 360 video points to a challenge in VR Games and 360 video in general – where should I look? When you’re watching a virtual reality video such as The New York Times, The Displaced, the action is limited. You’re there as a viewer to experience and understand the scene.

Take your time, look around. Watch it again. It’s a reflective experience.

It’s a whole different story in the action driven environment of VR gaming. Look one way and you completely miss what is happening behind you. That can quickly ruin gameplay. One solution is shown in the Clash of Clans 360 Video, where a character serves as a guide. You tend to look where she looks, keeping you focused on the action. Another approach is seen in the VR Hunger Games trailer, where you are moved through the scenes. You can look behind you, but since the scene is receding you’ll instinctively turn around to see what is coming in front of you. It’s immersion without being completely free to move where you want.

Spatial audio in VR

A more innovative solution is the work that Oculus, Google and others are doing on audio in virtual reality. A truly immersive experience requires directional audio. There is where good VR headsets will excel. Oculus showed off 3D sound at CES 2016 though they haven’t announced a release date.

It’s possible to do immersive sound using an array of microphones – see the work of Chris Milk in this area. But new approaches to spatial audio will do it through the software, making it much easier for developers and VR filmmakers.

Google has been working on ways to give Smartphone-based virtual reality experiences spatial audio. Adi Robinson from The Verge,

Spatial audio is exactly what it sounds like: audio that sounds like it’s coming not just from different sides of your headphones, but from exact points. A telephone can convincingly ring behind you, for example, or birds can sing above. Google’s new software development kit will let people making Google Cardboard experiences specify the positions of sounds, as well as the characteristics of their environments.

This has nothing to do with Google Cardboard, of course, but in the development platform for Google’s VR device.

Audio has always been an essential element in cinema. So important that before it was technically possible, theaters provided live soundtracks for early movies. We’re now seeing just how important it is for VR games and experiences.

While we are becoming a highly visual culture, we will only see if we can equally hear.