Augmented Reality

Augmented Reality (AR) is poised to transform our interaction with the world, from aiding emergency services and simplifying navigation to enriching personal experiences with real-time information overlays. However, when combined with facial recognition, AR could profoundly impact social interactions and privacy, offering real-time behavioral insights and emotional transparency, necessitating a reevaluation of privacy norms.

This article was first published in The Mint. You can read the original at this link.


For years we’ve heard of the promise that an augmented reality, or AR, future will offer us. We’ve played games like Ingress and Pokemon Go that have, in stages, overlaid an alternative reality on the world around us, providing us new ways in which to enjoy our surroundings. We were given tantalizing glimpses of what Google Glass might make possible with its ability to overlay data in real time over what we can see in front of us, but it is only now, as major mobile platforms announce operating-system-level augmented reality toolkits for app developers, that it seems as if that AR is finally on the verge of going mainstream.

There are many real-world uses of AR that are already among us. In some places we already use this technology to provide real-time assistance to emergency services like the fire department, to help them navigate in real time through burning buildings to try and reach survivors. Elsewhere, law enforcement personnel use these technologies to supplement other wearable technologies as part of their daily routine. The time will not be far when ordinary technicians will be able to repair complex machines with the help of diagrams superimposed in augmented reality on top of the object that needs fixing, providing them instructions on how to deconstruct the broken part and what to do to fix it.

At a more personal level, AR will make our interaction with the physical world more seamless. It will simplify the way we navigate to the nearest petrol station or choose a restaurant. Where we use our phones to search for places today, AR will automatically generate an overlay in real time on top of any restaurant we glance at on the street, giving us the full details of its menu, customer reviews and recommendations. AR will allow us to understand the world around us differently, providing annotations so that if ever we need information about a flower or species of bird we come across, all we will need to do is look at it and call up an AR overlay of rich information in full encyclopaedic detail.

Museums will spill out onto the streets as AR will allow us to visualize historical locations just as they might have been hundreds or thousands of years ago. We will change the way in which we consume entertainment, allowing us to enjoy immersive artificial worlds on our living room floor in such rich detail that we will be able to interact with the narrative, zooming in and panning out on details when we want, as we choose. As technology advances, this computer-generated reality will be able to identify and interact with objects in a real world till they are able to join us, virtually, at the dining table or interact with us in our conference rooms at work.

However, it is in the area of social interactions that augmented reality is likely to have the greatest impact. In an earlier article for this column, I wrote about how far facial recognition has advanced, highlighting the heightened risk this poses for surveillance. Image processing algorithms can already identify faces in hundreds of different contexts. When this technology is combined with augmented reality, it will offer us remarkable identification capabilities, allowing us to instantly recognize people and bring up, in real time, all sorts of information about them from their social media feeds and other ambient sources of information.

This will drastically alter the shape of our social interactions, allowing us to form opinions and preconceived notions about each other before ever exchanging a word. It will impact our public behaviour, allowing all these layers of information to influence our decisions as we engage with first-time business partners and prospective employees.

We are not far away from a time when the combination of AR and facial recognition will offer us real-time behavioural insights of the people we are interacting with. When these algorithms are able to identify facial tics and tiny tells in real time, they will reveal more about our reactions to situations and the emotions we are feeling than is currently perceptible. With this sort of augmented understanding of the people we are negotiating with, advising or counselling, we will have an unprecedented ability to orient our behaviour to be most appropriately responsive to the situations we find ourselves in—eventually being able to use this knowledge to mould the behaviour of those around us to our advantage.

The impact that all of this will have on privacy will be far greater than anything that we are apprehensive about today. Once technology lays bare our emotions in real time, we will be rendered transparent in our social context. This will force us to change the structure of our social interactions and, while technology will undoubtedly help us evolve countermeasures, force us to re-evaluate our current boundaries of personal privacy.