AR + AI in the latest Microsoft HoloLens to brings Digital avatars to life as Holograms!

• Kristin Houser in Virtuality

Now You See Me

You no longer need to speak another language to look like you’re fluent in it — to anyone, anywhere.

On Wednesday, Microsoft executive Julia White took the stage at the company’s Inspire partner conference to demonstrate how it’s now possible to not only create an incredibly life-like hologram of a persoun, but to then make the hologram speak another language in the person’s own voice.

This demo was possible thanks to a combination of two existing technologies — mixed reality and neural text-to-speech — and it foreshadows a future in which tech greatly diminishes existing barriers in human communication.

Team Effort

As for how Microsoft pulled off this remarkable demo, the first step was creating White’s seemingly identical, life-sized hologram.

To do that, she visited one of the company’s Mixed Reality capture studios, where she was recorded delivering her presentation in English.

That would be enough to allow anyone wearing one of Microsoft’s HoloLens devices to feel like White was standing in front of them, delivering her speech.

To then make it seem like White was speaking Japanese, Microsoft used its neural text-to-speech technology to create White’s “personalized voice signature.” Once it had that, it could have the hologram say anything, in any language, and the audio would sound like White’s voice.

The Future Is Here

Of course, an international audience could still benefit from a presentation like White’s even if Microsoft didn’t turn her into a hologram or use AI to replicate her voice — they could watch her on their television or computer screen, and listen along as a human translator repeated White’s words in their native tongue.

But this technology removes so many of those barriers between the speaker and the audience.

Imagine a world leader delivering a speech, and every person across the globe feeling like the leader was in the room with them and speaking the local language. Or a world-class professor giving a lecture that anyone could attend and understand — without leaving their homes and without learning their teacher’s language.

And then there’s the exciting possibility of what might come next. Mixed reality contact lensesthat do away with the need for a headset altogether? AI that can translate speech in the native speaker’s voice in real-time?

As White put it at the end of her demo, “All of these technologies exists today. The future is here.”