Alex Kipman has a unique talent for placing himself in remarkable digital stage sets, and the opening of Ignite conference was no different. Kipman used his virtual presentation to introduce Microsoft Mesh, the new platform to integrate Mixed Reality into our lives. If the vision is realized, Microsoft Mesh will give us the ability to:

• Feel presence with photorealistic avatars
• Experience together to collaborate, train, and draw upon remote expertise
• Collaborate from anywhere using a broad range of Mesh-enabled devices

In a Zoom-driven world still under the chokehold of a deadly pandemic, that would be a breakthrough – for both work and entertainment. And Microsoft made sure to highlight both with a demo of Pokémon GO in mixed reality and an emphasis on the workplace. It’s what was originally promised in the release of HoloLens One but with the recognition that people won’t just be using HoloLens. Mesh promises virtual collaboration and shared experiences through VR headsets, tablets, smartphones, and more.

Joanna Stern from The Wall Street Journal had the best one-sentence summary of the platform:

Think Zoom meets Facebook meets Roblox but with holograms.

Microsoft Mesh Arrives

Here’s the nicely done two-minute demo video of the possibilities, a video where every scene’s subtext is just what you’d expect it to be – our current isolated existence.

Microsoft’s blog summarizes the outlines of the vision for Mesh:

Microsoft Mesh will also enable geographically distributed teams to have more collaborative meetings, conduct virtual design sessions, assist others, learn together and host virtual social meetups. People will initially be able to express themselves as avatars in these shared virtual experiences and over time use holoportation to project themselves as their most lifelike, photorealistic selves, the company said.

The new platform is the result of years of Microsoft research and development in areas ranging from hand and eye tracking and HoloLens development to creating persistent holograms and artificial intelligence models that can create expressive avatars.

The first day of the Ignite conference included fascinating demos with John Hanke, CEO, and founder of Niantic, Cirque du Soleil co-founder Guy Laliberté, and the well-known filmmaker and ocean explorer, James Cameron. None of the demos offer publicly available content – they’re simply a glimpse of what’s to come. However, if you can’t handle the wait (and you’re a current customer), you can request a Mesh-enabled version of AltspaceVR.  

Alex Kipman’s Ignite talk was the first Microsoft keynote to use the Mesh platform, and he extolled the potential of mixed reality, which he’s been doing since 2016 (yes, it’s been that long):

This has been the dream for mixed reality, the idea from the very beginning . . . You can actually feel like you’re in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.

Microsoft’s Ambitious – But You Already Knew That

Microsoft is not the first to step into the virtual collaboration space. The company saved AltspaceVR from sudden death in 2017 and has a suite of collaboration apps for their HoloLens 2 mixed reality headset. But Mesh tries something very different. It seeks to bring together the currently fragmented collaboration space with co-presence based on photorealistic avatars. Most of our existing virtual world environments are only one step up from the original Second Life – helpful, temporary solutions which have bailed us out during a global health crisis. But they hardly live up to the full promise of XR. The avatars look cartoony, and while the environments deliver a sense of co-presence, you don’t have the realism that mixed reality provides.

If Microsoft Mesh can deliver, the potential impact on gaming, entertainment, education, and the workplace would be ground-breaking. As the Microsoft Blog noted,

Engineering or medical students learning about electric car engines or human anatomy could gather as avatars around a holographic model and remove parts of the engine or peel back muscles to see what’s underneath. Colleagues could simply get together and chat in a shared virtual space, or companies could use Microsoft Mesh-enabled apps to offer virtual all-hands meetings or trainings to employees around the world.

Staying at home during a pandemic wouldn’t mean being alone at all. You’d be present with others in your virtual classroom or office.

Let’s Acknowledge the Immense Challenges Here

Despite Microsoft’s somewhat spotty track record on software (see the latest glitches on its vaccine scheduling software), the company is well-positioned to leverage its rapidly expanding Azure cloud-computing platform. The virtual experiences and content live in the cloud, and Mesh is about getting devices to access them. It’s far ahead of Intel’s aborted effort with Project Alloy – which promised everything and gave us nothing more than some eye-popping videos.

But the challenges – never highlighted at tech conferences – abound. Foremost is bandwidth: cloud-based computing can often handle small amounts of lag time (think editing a shared Google Doc with a few-second delay) but shared virtual experiences rapidly disintegrate when they’re not instantaneous. The pandemic has been a sharp reminder that bandwidth remains a fundamental challenge even for many residents of high-income countries.

Our devices also present a fundamental challenge. There will be no inexpensive HoloLens in the foreseeable future, and our bulky headsets need to be much more comfortable to wear for extended virtual sessions. Moreover, trying to bring every device into the Microsoft Mesh ecosystem will require some heavy lifting. It will take cooperation from other companies (Facebook?) that may not be so eager to go this route. In the end, Mesh is Microsoft Teams for the XR era, and everyone has their own plans here. Yes, Zoom, you should be paranoid . . . very paranoid, as everyone wants to eat your lunch.

This was a day to dream big, and it’s hard not to resonate with that. Microsoft Mesh is a fascinating development that promises to deliver on the dream of mixed reality. The challenge will be in the implementation – expanding this out beyond the confines of tech companies with money to burn and well-endowed research labs. We’ll be watching it closely and keeping you up to date on the latest developments.