Microsoft’s Flight Simulator in VR arrives this December. At 38 years old, the remarkable simulation is the company’s longest-running software product line. At that age, it even predates Windows by three years. Its history is an amazing time capsule of the evolution of computer graphics – from the blocky renderings of the early versions to today’s deeply realistic simulation – and it’s soon to become a VR experience.

Here is a screenshot of version 2 in 1984, a year after PC Magazine praised version 1 as “extraordinarily realistic … a classic program, unique in the market.” The scenes were composed of four (yes, four!) glorious colors and four hatched tones; if you hooked it up to a “color composite” monitor (i.e., a TV), you could get up to 15 colors. Ah, the abundant gifts of early technology.

By 1996, Flight Simulator for Windows 95 finally achieved a degree of realism that vaguely looked like the real thing. NYC had buildings that approximated a few well-known landmarks with non-volumetric clouds in the distance. We were flying over Manhattan. More or less.

Fast forward through decades of development (with Microsoft shutting down the project and bringing it back to life), we have the 2020 version with its photorealistic reproduction of New York City’s buildings and monuments, with a beautifully rendered sky reflected in the surface of the water. As a flatscreen computer simulation, Flight Simulator 2020 achieves a breathtaking level of fidelity and accuracy. And it’s only going to get better.

As you look at these images, you can see them as a roadmap for augmented and virtual reality. Every area of immersive tech, or XR, will undergo the same transformation in the coming decades. There’s only one difference: the pace of change is significantly faster now than it was in the 80s and 90s. In 2019, we got a standalone VR headset that shattered the limitations VR had labored under with cables and external base stations. Only 19 months later, we have the Quest Quest 2, which achieved a new standard in VR experiences. And even that device just received its first major update, bringing 90Hz support, a fitness tracker, and new features.

For the moment, Microsoft Flight Simulator in VR will be just another VR game – though an incredibly good one. But with its ambitious scope to digitize the entire world, the potential is for far more. We’re moving one step closer to recreating the world in a machine-readable 1-to-1 scale model – a digital twin of our world. And at some point in that development, we will no longer be able to distinguish between the computer-generated version and the real one. At first, that will only happen on the visual level since haptic developments are moving at a much slower pace. But as the latter evolves, we’ll reach a point where the immersive simulation is no different from piloting a real aircraft. Or doing any of the other thousand and one other activities that humans pursue on the small orbiting rock that serves as our home. 

Microsoft Flight Simulator In VR

What makes the new MS flight sim remarkable is how it combines real world satellite data and Azure AI to create a deeply realistic environment. Aircraft and buildings incorporate shadows and reflections, and urban landscapes have accurately reproduced structures or auto-generated content based on satellite data. Play the game online, and it adds in localized weather conditions, along with air and ground traffic to further the realism. That 747 you see on the runway in front of you? Those ominous low-hanging clouds you’re about to fly into? All there in real life and now part of the simulation.

Incorporate all this into virtual reality, and suddenly, you have an environment you step into instead of staring at a screen (or three as many avid users arrange their flight sim setups). Here’s a quick video on the upcoming release.

A further video with details on the VR implementation will be available on December 17, with the actual update coming online around the end of December. The original announcement limited the compatible VR headsets; but the latest news is that it will now be available where it needs to be – on all platforms, including Oculus and SteamVR.

The Sense of Immersion

The sense of immersion in VR comes from the feeling of being physically present in a virtual world. How far Microsoft can take this in a flight simulation will depend on a multitude of technical details, few of which we have answers to now. Can we reach out and flip the switches on the instrument panel? Can we open the door of a Cessna in flight and get the dreaded sense of vertigo that would come from the real experience? Can Microsoft Flight Simulator in VR render the scenery as well as the desktop version?

It’s hard to imagine that this will work with anything less than a high-end PC and the latest Nvidia graphics cards. And that will be a challenge when the VR version arrives – VR inevitably increases the digital divide when it first arrives. But as is always the case with the technology, today’s high-end specs will be tomorrow’s laughable limitations. There was a time when getting 15 colors on a color composite monitor was an achievement.

The Digital Twin Arrives

With Flight Simulator in VR, everything on the earth’s surface will be available, with buildings, monuments, and landscape features rendered in photorealistic accuracy supplemented with AI-generated content. It’s a little like Facebook’s work on foveated rendering in VR – recreate the central details in hi-res, and the rest can be automated through AI. As the updates continue, the photorealistic scenery will expand, and the AI contributions will become more accurate. 

Ultimately, Microsoft’s Flight Simulator in VR is creating a digital twin of the world we live in. Not an object or a portion of the world, but the entire environment of our planet with climate and water. In parallel with this, Facebook is also building a digital twin of our world – the on-the-ground side through its Live Maps initiative. While Microsoft will rely on satellite data for its portion, Facebook will vacuum up data from our smart devices, including our AR glasses. In effect, we become the onsite satellites for Facebook (a prospect at once fascinating and disconcerting).

There’s no chance in the near future that these two projects – both generating 1-to-1 scale models of our world with live updates  – will link together. But on the other hand, it’s inconceivable that these two realms will remain forever separated. You can envision a day in the future when we’ll be able to land the plane and step out into a completely rendered environment. It won’t just be the world seen outside a virtual plane’s cockpit, but an expansive world fully available to us in XR. Ready Player One, indeed.

And So Do The Questions

The Ready Player One scenario may seem far in the future, but it’s closer than you think. And it raises fascinating questions – especially when you remove the neat Hollywood trappings the movie is wrapped in.

• How will workforce training and education change when VR is no longer composed of one-off simulations but part of a larger virtual ecosystem? Think of XR not as the individual apps they are today, but as linked experiences and environments. Physical books and media were overwhelmed with digital versions giving us a wealth of resources. How will we handle the abundance of virtual experiences in the future when you can call up any object, environment, or activity at will?
• From the early days of Myst (which is also coming in VR) to Half-Life: Alyx, Games are increasingly realistic. What happens when the game/simulation becomes so real that it is indistinguishable from reality? As it is, we struggle to cope with the current onslaught of false information and conspiracy theories on our social media platforms. If fake text and videos can create what some have called an epistemological crisis, how will we live in a world where there is a 1-to-1 virtual model with fake experiences readily available on-demand?
• Will the status of real world objects and activities be impacted by our ability to access equally real versions in virtual environments? For example, VR is already bringing us closer to our national parks and nature, but will the significance of the physical version change when we can readily access the virtual in fully immersive experiences? Will we continue to preserve nature and historical monuments, or will the virtual versions be seen as sufficient?
• How will a digital twin of our world affect our understanding of culture and identity? In recent years, many of our national borders have sadly become synonymous with physical walls and barriers. At the same time, technology lets us breach them with increasing ease. In Flight Simulator, you can fly across North Korea, even land your plane on a street in Pyongyang without fear of reprisal or becoming the source of an international incident. Will physical borders become more impenetrable as technology makes them more porous? Or will it open the doors to empathy and understanding between cultures?

These are just a few of the questions that will begin to arise as the technology behind Microsoft’s Flight Simulator in VR expands into a richer and more all-encompassing virtual experience. Right now, it’s simply an amazing flight simulation. But in building a digital twin of our environment, it’s cracking open the door to the more far-reaching possibilities and challenges of our future.

See you in the virtual skies.