The realism of virtual reality is behind Elon Musk’s widely shared comment this week that we are computer generated players inside another entity’s video game. Speaking at Recode’s Code Conference 2016 Musk proposed that while we think we’re real, and think we live in a physical world, it’s just the power of the simulation we find ourselves in.

The idea has a long history. You see its shadow in Plato’s Allegory of the Cave, Descartes’ late night musings in the Meditations, among many other philosophers and Sci-Fi writers. More recently, you’ll find the discussion and resources at the Simulation Argument website highlighting the work of Nick Bostrom and others on the topic (and on AI developments). Bostrom holds an appointment at Oxford in the appropriately named Future of Humanity Institute.

And of course, there’s our favorite version in The Matrix. Except that Musk seems to suggest that there is not an option for the Red pill.

You take the blue pill, the story ends. You wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in wonderland, and I show you how deep the rabbit hole goes.

Whether you agree or not with the Tesla CEO and SpaceX visionary’s point, it says a lot about where we’re headed with virtual reality and more. There’s a wide range of rapidly evolving display technologies – from augmented reality to mixed reality – that will impact and transform our visual experience of the world.

If you want to see the full interview with Musk covering everything from space travel (a one way rocket to Mars in 2018), self-driving cars, and artificial intelligence, it’s posted at the end of this article. But let’s dwell for a moment on virtual reality.

The Realism of Virtual Reality

Here’s Musk’s full statement from Vox,

The strongest argument for us being in a simulation probably is the following. Forty years ago we had pong. Like, two rectangles and a dot. That was what games were.

Now, 40 years later, we have photorealistic, 3D simulations with millions of people playing simultaneously, and it’s getting better every year. Soon we’ll have virtual reality, augmented reality.

If you assume any rate of improvement at all, then the games will become indistinguishable from reality, even if that rate of advancement drops by a thousand from what it is now. Then you just say, okay, let’s imagine it’s 10,000 years in the future, which is nothing on the evolutionary scale.

So given that we’re clearly on a trajectory to have games that are indistinguishable from reality, and those games could be played on any set-top box or on a PC or whatever, and there would probably be billions of such computers or set-top boxes, it would seem to follow that the odds that we’re in base reality is one in billions.

Musk goes on to say that we should hope we are not holders of the base reality. Otherwise, we will create multiple realities indistinguishable from the one we know.

So maybe we should be hopeful this is a simulation, because otherwise we are going to create simulations indistinguishable from reality or civilization ceases to exist.

The Dream of Virtual Reality

Elon Musk’s comment about the realism of our simulations goes back to the very beginning of the developments in virtual reality. The goal has always been to create simulations indistinguishable from the real, creating scenes and activities that affect us in the same way the “real world” does.

The idea surfaced in the early work of Ivan Sutherland, one of the founders of VR. Along with Bob Sproull, he created the first virtual reality display system. It was joking referred to as the “Sword of Damocles” since it had so much hardware you had to stand underneath it.

Sutherland was fascinated by the direction display technology was headed. In 1965, Sutherland published a very short (one and half pages), groundbreaking paper titled The Ultimate Display. And here’s what he said about where our visual displays were headed:

The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.

A remarkable statement by a remarkable individual – he gave us Sketchpad (a groundbreaking interactive design program), the first object-oriented program and a host of other innovations. His grad students went on found companies like Silicon Graphics, Netscape, and Pixar.

How close we get to that goal depends as he says, on the “appropriate programming”. But what ultimately matters is only that we think the simulation is true. And in that regard, virtual reality will get us there. When it does, we will live in a profoundly different world. A world where the boundary between simulation and “reality” disappears.

Elon Musk and Simulated Worlds

In one respect, all Elon Musk was saying is, there’s an excellent chance that we’re already part of someone else’s “ultimate display” or simulation. There’s no way to prove the statement right or wrong. His point was just that we are a lot better off if it is true.

I’m not sure which side I fall on in the reality vs. illusion debate but it’s a main reason why we find virtual reality so interesting. The implications for entertainment, the workplace, education – indeed, every area of human life – are fascinating.

Here’s the full interview with Elon Musk at Code Conference 2016: