Eye tracking, immersive media, holoportation – is AR the future of communication?
From the holograms of our childhood to holoportation – how augmented reality is changing communication
Augmented reality — a buzzword, but one that entered our lives long ago. Whether it’s the TikTok filter or Google Lens on your phone, AR is everywhere. What potential does AR hold for our lives, for society? Will we give up old, familiar ways of communicating in order to surrender ourselves entirely to AR?
Whenever I hear augmented reality (AR for short), two lines come to mind.
- “You have failed me for the last time, Admiral.”
- “Help me, Obi Wan Kenobi, you are my only hope.”
Two movie quotes that have imprinted themselves on so many people — myself included — so deeply that we associate them inseparably with our youth. Mouths open in amazement, we followed the adventures of our heroes and heroines at the cinema or on VHS tape, and could not get enough of spaceships, alien creatures and technological marvels that still seemed to lie in the distant future. With the quotes above we remember glowing, flickering holograms that enabled real-time conversations with people waiting for our news at the other end of the galaxy.
Now, what does that have to do with augmented reality?
Unfortunately, to date we still haven’t managed to leave our home planet, and exotic alien creatures are also in rather short supply at the moment — but at least, even back then, we were allowed a glimpse of what the future of communication might look like. A handy little robot projects the image of our counterpart onto the table. An entire wall of a room turns into a projection screen, a theatre stage or an oversized cinema hall. Sounds very futuristic!
Yet the idea of AR is not nearly as new as one might think. The roots of AR reach back to the 1960s, when Ivan Sutherland coined the term “augmented reality.” Together with his student Bob Sproull he built the first “head mounted display system.” True, because of its enormous weight it still had to be lowered onto the user from the ceiling on chains (fittingly named the “Sword of Damocles”), and the visual events were limited to spatial wireframe renderings — but that first important step had been taken. Another significant milestone was the introduction of the GPS system, which made possible the precise tracking and positioning of objects in the real world.
But only thanks to the rapid technological progress of the last twenty years could devices be developed that, on the one hand, make the possibilities of AR affordable for everyone and, on the other, thanks to their light and comfortable design, can be taken everywhere. Augmented reality is already being used today in various areas of communication; one example is the advertising and marketing industry. Companies use AR to present their products and services in an innovative and interactive way. Using AR apps, customers can, for example, visualise pieces of furniture in their own rooms or try on garments virtually to get a better idea of how they look.
In the realm of social media, too, AR has become indispensable. Platforms such as Snapchat, TikTok and Instagram offer filters and effects that use AR to embellish photos and videos or add funny elements. This lets users communicate in a playful way and personalise their content, but on the other hand it can also lead to bizarre situations (we remember the lawyer who joined a court hearing live via Zoom and accidentally used a filter that swapped his face for that of a cat).
The further development of VR glasses goes one step further still, in that they no longer just offer passive support but are actively deployed in teaching and education. Teachers use AR by means of VR glasses to convey complex concepts more vividly. With the help of AR, three-dimensional models of organs can be examined in biology class, or historical events experienced in the form of virtual reconstructions. The “eye tracking” technology used here helps to highlight details of the displayed environment or to call up additional information without having to fiddle with hand gestures or user interfaces.
A remarkable example of the use of augmented reality in communication is Microsoft Holoportation. It allows people to present themselves in real time as holographic images at remote locations and to interact with one another. Far removed from the pixelated, distorted images of its predecessors, holoportation uses high-quality 3D models which, through the simultaneous use of VR glasses such as the HoloLens, can genuinely create the illusion that your conversation partner is actually in the same physical room. For this, holoportation uses a series of cameras and sensors to capture a person’s appearance, proportions, movements and sounds. This information is transmitted in real time and reconstructed as a holographic image at the destination. Virtual meetings between teams and conversation partners all over the world are thereby given undreamed-of ways to interact. Non-verbal communication and body language in particular are heavily curtailed and restricted by the use of online meetings via webcam. This is one reason why many people prefer “in-person” meetings. But global pandemics such as COVID-19 in turn show that direct contact between people can sometimes be very risky — professionally as well as privately. And this is where AR steps in to help. By creating an immersive and realistic experience, people can interact with one another across great distances as if they were physically present. This has the potential to change the way we deal with one another, collaborate and share our experiences, by overcoming physical barriers and enabling a deeper level of human interaction.
For cyberpunk and science-fiction fans, AR seems to make a dream come true. You stroll through a shopping street carrying nothing but a minimalist VR/AR headset. It no longer has a display but projects the digital images, or the user interface, directly onto your retina. Your favourite Italian around the corner points out via a pop-up that today he’s offering a 1+1 free pizza deal. A glance at the bank building opposite shows you the current share prices, how much you have in your account, and which investments might currently yield the most interest. Meanwhile, as if in a trance, you follow the big signpost projected onto the ground in front of you showing the shortest route to the nearest cinema. The call from your girlfriend, who is going to be a little late, shows her to you as a holographic projection right in front of you — almost as if she were standing there in flesh and blood. Payment by fingerprint? Works on the go too. Weather report? Top right of your field of view. A taxi for after the cinema? Already pre-booked with a movement of your eyes. Your favourite song in your ear, we move through the digital world. Pie in the sky? Yes — but the first steps have long since been taken.
How will we deal with AR? Will we, as so often depicted in dystopian films, succumb to the temptations of the digital world? Will we give up our social lives, be able to manage only virtual friendships, live only for the next kick online, hooked up with a cable jutting from our temple and humming machines, only a shadow of what was once called “human”?
I don’t think it will come to that. Physical contact, human closeness and warmth, emotions, the feeling of a real person standing before you — all these components of life, indispensable to humans, technology will not be able to replace. And it isn’t meant to, either. It’s meant to extend. To improve. To augment. And that, after all, is ultimately what the term augmented reality stands for.
Originally published at SEQIS Blog