The latest Apple AR patent filing reveals fascinating details for their future AR Glasses. Of course, patents don’t always see the light of day, and this is only one of many that they’ve filed. But if it comes to pass, it would be a groundbreaking development for augmented reality headsets.
Apple is taking its time in developing an AR device, and despite concerns earlier this year (which we never bought into), their work continues. And in typical Apple fashion, this won’t be just another set of AR Glasses, but a device that outstrips everything on the market. Similar to what the iPhone did for mobile, and what their watch did for wearables, it could have a massive impact on the use of AR in business, education, and entertainment.
The Apple AR Patent News
Here are the technical details from UpLoadVR.
Based on our reading of the patent the idea appears to be a direct evolution of Akonia’s technology. While the startup didn’t give details of how their technology worked, it called the approach ‘HoloMirror’ and claimed it had “dramatically higher” field of view than competitors with lower production cost. As far back as 2017 the company was claiming a 60 degree field of view.
The system described in the patent is rather complex. A laser array projector emits light into entrance points of an optical “distribution” waveguide. This waveguide then passes the light to exit points which lead to MEMS scanning mirrors. These entrance and exit points can either be implemented using holographic film or as surface relief gratings via lithographic techniques. The MEMS mirrors then direct the light into layered waveguides, where holographic projectors project it at a “focus curve” which reflects into a holographic combiner.
Except for a device from Kura, no one has achieved anything close to 120° degrees FoV in AR Glasses. Kura would still top that with a 150° FoV (you can see a short video here). They promise an AR device by mid-2020 and plans on having a working demo at the upcoming CES conference in January. However, according to UpLoadVR, they are targeting enterprise instead of the consumer market.
Whether Apple or Kura gets there first, the FoV far surpasses what we have now in Microsoft’s HoloLens 2 and the Magic Leap One. And as for our VR headsets, most are stuck in 90° – 120° range. From our perspective, it’s critical that AR devices exceed the FoV we’re used to in VR. In the closed environment of virtual reality, the field of view is less noticeable. In AR experiences, you immediately see what you are missing out when large areas of your natural field of view are not augmented.
From the Apple AR patent info, it’s clear they are also working on foveated rendering. The idea behind this is that if you could track the movements of our eyes accurately enough, you could have displays that render hi-res details only where you are directly looking. Foveated devices would save power and make more efficient use of processors, both of which are ongoing challenges in developing AR devices.
Apple is working on this by incorporating peripheral projectors into the device. There’s still the challenge of using eye-tracking technology, and the patent suggests that the first devices may not include it. But if Apple achieved a breakthrough here – along with a wide FoV – it would completely upend the augmented reality market.
We’ve held to our prediction of Apple releasing AR Glasses in 2020, but it’s possible that date may slip by a year or two. Other companies are getting AR devices to the market now, but they’re supplements to your smartphone, not a replacement. We suspect Apple has something much more significant in mind.
Whatever the results of Apple’s AR patents, the next ten years will be the decade of spatial computing. We know from experience, they never need to be first to the market. But they have the vision and the resources to do it right.
Emory Craig is a writer, speaker, and VR consultant with extensive experience in art, new media, and higher education. He speaks at global conferences on innovation, education, and ethical technology in the future. He has published widely and worked with the US Agency for International Development, the United Nations, and the Organization for Economic Co-operation and Development (OECD). Living at the intersection of learning, games, and immersive storytelling, he is fascinated by AI-based avatars, digital twins, and the ethical implications of blurring the boundaries between the real and the virtual.