We were excited to hear that the groundbreaking VR tech and entertainment company Within hired noted game developer, Dave Cowling, as their new VP of Engineering. The move sparked interest among VR content developers and got us thinking about games, technical innovation, and creativity in virtual reality.

We recently had the opportunity to talk with Dave about his new position at Within. Not surprisingly, we covered everything from his early work with Neversoft and Activision to current and future developments in VR.

Dave Cowling’s been playing with computers for decades and has an enviable track record in the video game industry. He’s been involved in five generations of console platforms from the Sega 32X through PlayStation 4 + Pro / Xbox One. He has over 20 AAA titles to his credit, including top rated video games such as Tony Hawk’s Pro Skater, Guitar Hero and Call of Duty: Infinite Warfare.

It’s a record that speaks not only to his technical ability but his creative vision. And that seems to be a perfect fit for the innovative spirit that drives the work at Within.

As Within CTO Aaron Koblin said in Medium,

Dave brings the quintessential combination of creative and technological expertise that drives Within — creative ideas pushing technical innovations. It’s that diverse array of experience that will no doubt keep pushing us further. We’re thrilled to bring his talent and leadership to our team.

Read on for a fascinating conversation!

Interview with Dave Cowling

From Indie and AAA Games to VR

Digital Bodies: It’s a pleasure to talk with you today. We appreciate you taking time out of your busy schedule. We know you’re not going to have a lot of details about Within but we’re very curious as to why they brought you in and what attracted you.

Dave Cowling: So just very brief background on me. I have been working in console gaming space for a little over two decades. I started in Sega in the U.K. and moved to the States in the mid-90s and luckily very rapidly landed a role at Neversoft Entertainment, at that time a very small independent game developer. Over time we had the fortune to pair with Activision and work on a number of very successful titles like Tony Hawk’s Pro Skater. So I became an Activision employee by virtue of the Neversoft acquisition and moved around Activision in various technology roles most recently working as the CTO at Infinity Ward.

My experience in the games industry has been one of starting off with a very small scrappy developer, doing rapid prototyping on a technology stack that albeit now is over 15 years old. And in an environment where everyone had to pitch in and wear a number of hats. All the way up through my most recent triple A console experience which was a multi-year monolithic engineering development process with a very large team of engineers on a very high-quality complex piece of entertainment. So all of those things were super fun but having worked in the console gaming space for such a long time I had decided I was ready to take a break and had semi-retired, taking much-needed recuperation time with my family.

What happened next?

So I wasn’t actually actively looking for employment; I was having plenty of fun with my own side projects. Then I was approached by a recruiter who said Chris and Aaron were looking for someone, ideally with a background in interactive entertainment experiences, to take on the role of an engineering lead. The first step there was really coming into the studio and meeting with Chris and Aaron. Even before coming in, I had watched their TED Talks and was extremely impressed with the level of creativity and innovation in the studio. But in coming in and getting a sense of the vibe of the studio, it was interesting – in many ways, it reminded me a lot of the early scrappy startup days in the games industry. And there was a real positive energy around innovation and really good team vibe.

It sounds like you were very intrigued.

The interesting thing was, there were really two barriers to entry. One was: did I want to go back to work? But having met Chris and Aaron I was super excited to work with them. The other interesting thing was, I hadn’t spent a lot of time considering VR or dipping my toe in the VR pond just because I had been so deeply embedded in traditional console gaming. So I did a great deal of due diligence before coming here. I think I increased Aaron’s angst level a little as I wanted to take my time to make sure it was the right call.

So I went to a couple of conferences and did some reading and talking to people. I was certainly picking up the background noise and wondering if this thing is a fad, or are we in the trough of disillusionment? The uptake sounds great but is this going to just wither on the vine.

I think it’s a question many of us consider!

I spoke to a lot of knowledgeable people on the subject. And having bought my own headsets and experienced some of the content all the way from Google Cardboard to the premium HMD’s, I am convinced it has a good shot. Let’s say that. I have found it to be an incredibly compelling medium all the way from very simple 360 videos to high-quality interactive projects. There are many pieces that have actually moved me or impressed me in ways that more traditional electronic extended content has not.

We can see where the current environment could attract you. There’s a lot of creative people in the VR space and few boundaries. There’s tremendous opportunity to experiment and do some innovative work.

The super interesting thing and I’ve thought a lot about this – you know, back in the day. Certainly, at the dawn of the Sony console era, there was always a desire to prototype rapidly and for want of a better expression, throw a bunch of sh*t at the wall and see what sticks. And one of the sources of friction to that was in order to do any prototyping, you really had to develop your own tech stack, from soup to nuts. There really wasn’t much back then in the way of middleware.

And today, it’s very different.

Yes, I actually gave a talk at my daughter’s school the other day and told the kids how lucky they are, that these days, the process of development has been democratized enormously through things like Unity. Even as a single lone developer I can go home, I can download Unity and instantly inherit many, many person years of development that allows me to be much more creative. And that’s super exciting, not just for the VR industry but for the electronic entertainment industry as a whole.

Code and VR

That leads us to our next question. You have a strong reputation for your programming skills and you’ve written about performance critical code and load-hit-store penalties. Do you see ways to improve VR by tightening up the code?

That’s a good question. It is generally the case that if you’re using a middleware tech stack, if you’re using Unity or Unreal engine, you trade a little in terms of performance to gain a lot in flexibility. It is always invariably the case that the more general purpose the technology the less it is good at doing one specific thing. There’s always a tradeoff. I think the more you want to present the end user with a very specific experience and certainly, if you want that experience to turn the dial up to 11 in terms of the quality of the visuals or the gameplay, the more it behooves you to use a more custom tech stack. With the things that we’re doing here at Within right now I think there’s much more focus on creativity and uniqueness of experience such that we don’t need to invest in those things.

But without going too far into the weeds, you can look at some of the things done recently in terms of lens-centric or lens-match rendering. You’re essentially rendering to a non-planar projection surface that more closely matches the shape of a lens. Then when do your lens specific wrap at the end you’re wasting many fewer pixels. That’s a great example of technology that is aimed at allowing any engine to throw more quality right in front of the user.

 

 

That’s a great example. Shifting perspective, a lot of VR also relies on streaming which has tremendous implications for bandwidth.

Compression and bandwidth are super important for compelling VR experiences. I mean you’ve seen the holographic video playback that requires multiple gigabytes per frame. If these are ever going to become viable at a consumer retail level versus a dedicated installation level, we’re going to have to get much better at compressing our data and sending it down a pipe to somebody’s house.

All those players might be on different devices. Some are on premium HMD’s, some are on mobile devices. There’s even potentially a mixture of VR and AR, with everyone engaging in the same experience. Obviously, that’s going to require careful network infrastructure planning. And ideally, ways to minimize bandwidth.

Avatars and Social VR

In the past, you’ve done presentations on Avatar rendering which got us thinking about the possibilities after experiencing Chris Milk’s incredible Life of Us installation at Tribeca Film Festival. Since it’s a multi-user experience, I was wondering if that was an area you might end up working on at Within.

Well, I certainly think that there is room to improve what’s happening there, not in Life of Us, but in general. There’s a lot of interesting tech here. If I’m playing with another player and don’t actually have visuals of that player, I have to infer from a very small amount of information, typically head orientation and controller position and orientation, what pose that person might be in. Technology could help drive predictive models here. What pose is someone likely in given their recent movements on those three data points? So that’s something that’s interesting.

Certainly, it seems that as time goes by, input controllers will become more complex and provide us with more and more data points. And certainly, with wearable tech one can imagine that the data will increase. Of course, we’ll have to have the bandwidth to send this information. But in time, we will certainly get to more accurate representations of what a remote player is actually doing.

Related to that, we were wondering about your thoughts on social VR which is just getting under way. We’ve been spending time in AltSpaceVR and are fascinated by it.

Again, I don’t feel like I have a lot of insightful commentary from a non-technological standpoint. But one of the things I would point to is the recent funding round for Improbable who have raised an enormous amount of money with the premise that they are much like Unity or the other middleware tech stacks, that they can democratize basic development. Their goal is to become a platform that reduces the barrier of entry substantially to people that are looking to build much higher density multiplayer experiences.

We were impressed with the news out of Apple’s WWDC conference. Within was highlighted for the Goldilocks Story. It was an interesting turn – we didn’t expect to see Within move so quickly into the AR space. Is that something you might be working on in the future?

I say that without going into details, we are always interested in looking at new pieces of tech. Obviously, ARKit and other developments that Apple unveiled at WWDC. If we can find a way to use that tech and continue our mission statement of telling incredible stories, I think you can look forward to us investigating that space.

VR, AR, and Games

Since you have such an innovative track record in video games, we’d love to hear your perspective on gaming and immersive environments. Do you think games will move toward fully immersive VR or will it be more AR and mixed reality?

That’s a great question. I think you only have to look at the PSVR ecosystem. I feel like a lot of innovation in the VR space is actually coming out of the game development community. They are the ones most actively looking into how can we approach issues of locomotion across states and worlds without inducing nausea or worse. How can we deal with the fact that VR is such a visceral experience? Many times in games there are pretty horrific things going on that are orders of magnitude more horrific when you experience them in a VR space.

I think the traditional gaming industry will continue to investigate ways to make compelling experiences in VR. To me, partly by virtue of the fact there is so much activity around the AR space right now I’m sure we’ll see new stuff in augmented reality. The game industry has always been on the forefront of the adoption of new and interesting tech to make compelling pieces of entertainment.

VR and the Road Ahead

To wrap up, here’s the speculative question. How do you see the future of virtual reality? Any ideas on where you think we’re going? Things change so rapidly, just in the short time that AR has become popular. Pokemon Go appears and next thing you know, everybody’s out in Central Park at midnight or down at an abandoned pier chasing digital entities.

If I could predict this with any certainty, I’d have my own private island right now! But just a quick note on Pokemon Go. For Mother’s Day as my wife wanted to go to the Getty so we went up with the kids. We were surprised on entry to this historically accurate courtyard to see this big sign asking people to turn Pokemon Go off on their phones so that they could avoid walking into any priceless vases. It was shocking to me that the game had reached such a level of saturation that the Getty Villa felt they had to have a sign asking people to disable it. But there you go.

The environment is so fluid . . .

I’ll say this. It’s often the case when I talk to people who are not actively developing for VR that they bemoan the current state of the hardware. They say that the resolution is too low or there are all those wires. To me, these are growing pains. I think they are easily surmountable in the coming years.

Those things seem self-evident to me providing the hardware development velocity continues. To me, the much more interesting question is really about content and design. The ambition of design isn’t outpacing the hardware yet. I feel like even with existing hardware, admitting that it’s by no means perfect, I feel like we haven’t tapped into what can be done from an experience level. So I’m super curious to see what we can do.

And so are we! 

As an industry, we will start to coalesce around a design language, around standard ways that people expect to locomote through spaces and the way the whole UX / UI works. Getting to a point where I don’t have to remove my headset to sign into an account. Or have to unplug and plug back in my USB cables to embark on a VR experience.

There’s enormous room to leverage even existing hardware in much more exciting and compelling ways. That’s my hope and I wouldn’t be here if I didn’t think it was possible. In the short term as an industry, we have to start a common language in the use of VR. And from that platform, we can reap the benefits of much more advanced hardware as time goes on.

We totally agree. There’s so much we can do with the hardware we have right now.

Yes. Right.

Dave, this has been a wonderful conversation. It’s great to hear your take on games, virtual reality, and the creativity and innovation that drives the work at Within. Thank you so much! 

Thank you! I will go back to trying to get my team to make the next amazing VR experience!