We’re still digesting the news out of LEAPcon, Magic Leap’s first public event for developers. Putting aside the idealism and offbeat approach, the most remarkable moment was the introduction of Mica, their Mixed Reality AI Assistant.

We’ve all (well, many of us) have been playing around with the quiet introduction of AI into our lives. We order from Amazon Echo, talk to Google Home, and yell at Siri (you can put the verbs wherever you like).

Our current AI is passable in some moments. Utterly frustrating in others. Don’t let the latter sentiment deceive you.

We have yet to see much of it in our immersive environments – probably because we are just getting acclimated to the virtual. But rest assured, AI is coming there, too – faster than you think. 

And if Magic Leap gets its way with Mica – we’ll see it in Mixed Reality. Or in other words, not just inside a VR headset but in our world.

It opens fascinating opportunities for how we’ll interact with the virtual. And how we learn. Instead of just experiencing static environments, we will respond to the AI-driven avatars within them. Indeed, we may actually live with them someday – as ongoing stories intertwined with our lives. You can catch another perspective on this in the remarkable work by Fable Studios with Wolves in the Walls. It still haunts us as one of the most visionary statements we’ve seen on the future of storytelling.

But Magic Leap is going far beyond the current efforts to get AI into VR.

This Mixed Reality AI is Different

Mica was introduced at LEAPcon as Magic Leap’s ongoing Mixed Reality AI project. But instead of the usual approach of having artificial intelligence drive some aspect of a virtual environment, they created a remarkably human-like character. Mica doesn’t talk yet, but she will. What she does do is interact with you through body language. She smiles, averts her gaze, stares at you, and yawns (and yes, she’s made people yawn in return).

Most AI projects focus on the “I” – the intelligence. We have to make it sound smart. What Magic Leap did is make it look – and act – like a human being.

All of what is missing in Siri and company is present in Mica. As Magic Leap said, the talking part will come. Include that, and you have a profoundly compelling avatar that goes beyond anything seen in a commercially available product.

We pulled out the short segment from the three-hour video of LEAPcon where the Mixed Reality AI project is discussed.

 

But the talk at LEAPcon doesn’t do the experience justice. Instead, read Adario Strange’s compelling description in Next Reality of what it’s like to meet Mica face-to-face.

Upon donning the Magic Leap One, I’m greeted by a virtual woman (Mica) sitting at the very real wooden table. Then, Mica, with an inviting smile, gestures for me to join her and sit in the chair opposite her. I oblige, and then a very weird interaction begins — she starts smiling at me, seemingly looking for a reaction. . .

. . . I’ll admit, I deliberately avoided smiling (though it was really hard, Mica seems so nice) and kept a poker face in an attempt to see if I could somehow throw the experience off by not doing the expected, that is, returning the smile.

Undaunted, Mica continued to look into my eyes and go through a series of “emotions” that, surprisingly, made me feel a bit guilty about being so stoic.

The Future Arrives

That would be enough for one encounter, but then it takes a turn toward what our future holds. Adario Strange suddenly finds him acting at the behest of a virtual being, rearranging objects in the real world.

That would have been enough to mildly impress me, but what came next was the kicker. She then pointed to a real wooden picture frame on the table, gesturing for me to hang it on a pin on the wall next to us. I did as asked, and… it was the eureka moment. This was a virtual human sitting at a real world table and she just got me to change something in the real world based on her direction.

But then it got better. Once I’d hung the empty frame, Mica got up (she’s about five feet six inches tall) and began writing a message inside the frame, which in context looked about as real as if an actual person had begun writing on the space.

Alas, I don’t remember what the message was (honestly, I was too blown away by what was happening), but I’m assuming it was somewhat profound, as Mica then looked to me in a way that seemed to ask that I consider the meaning of the message. After a few beats, the life-sized, augmented reality human walked out of the room. But she didn’t just disappear into a wall in a flurry of sparkly AR dust. Instead, she walked behind a real wall in the room leading to a hallway. It was a subtle but powerful touch that increased the realism of the entire interaction.

The ethical challenges ahead

Mica is hardly ready to venture out into the world. She doesn’t speak and has only limited actions she can perform now. She’s more like a programmed bot with a very human appearance.

Even if you’re purchasing the Magic Leap One this week, you won’t find her sitting on your couch – just yet.

But as Magic Leap’s artificial intelligence project develops, it will move AI into our world. This is not a simple disembodied AI program or responsive AI in our environment. It’s AI contextualized in human form, that looks at us. That reacts to our movements, responds to our gaze.

The ethical challenges are obvious.

Right now, we’re having trouble with straightforward text that passes itself off as “fake news.” And fabricated images that pass themselves off as photographic “truth.” I feel like Walter Benjamin and Susan Sontag were lone voices in the wilderness – and even they didn’t realize what was coming.

With Mixed Reality AI, we’re entering a virtual house of mirrors with avatars in our world . . . interacting with us . . . looking like us.

Or like the people who program them. Which is another issue altogether (as usual, no one mentioned this at LEAPcon).

Call it Plato’s Cave on steroids. A space of illusions that can be more real than reality itself. In the end, it leads to virtual avatars cohabiting with us.

What will they ask us to do? Will we always listen? Will we ultimately have the capacity to say no?

There are no easy answers here but a quick insight. Technology will bring us full circle back to ethics and the liberal arts. For it is Mica and her protegé that will force us to ask the most fundamental question of all – what does it mean to be human?