If you thought surveillance capitalism was troubling, then Mark Pesce’s new book on augmented reality will give you a lot more to worry about.

In his usual easy, accessible style he shows just how data-hungry effective augmented reality will be and explores the implications of handing  both data and technologies over to corporations and governments.  He also points out that the freedom to to add a digital overlay to a scene is largely unconstrained, so that we have no effective way to stop anyone ‘writing’ on public or private spaces

What emerges is a vision of a world in which spaces, locations and even our gaze are  fully monitored by businesses and governments in order to give us access to digital overlays that will largely be created by those same corporations and governments,  as  late stage capitalism moves  from mere surveillance to omniscience

It’s a long way from the excitement of the early experiments in interactive computing, virtual reality and augmented reality that, coupled with the pervasive internet, have enabled this radical shift in how we engage with the both the world of data and the physical world.

Beginning with a description of the evening in July 2016 when the newly launched Pokemon Go brought hundreds of players to an unremarkable suburban park in Rhodes, New South Wales,  Augmented Reality gives us a well-informed history of the technologies that allow us to overlay computer-generated imagery on the world.  We hear about Sketchpad, the “Sword of Damocles” head-mounted display, and the Mother of All Demos, all key points in the history of computing and precursors of our smartphone- and sensor-laden world.

Apple’s iPhone and Microsoft’s Hololens mark turning points, both technologies that are capable of sensing their environment sufficiently accurately to position themselves in space and hence overlay imagery on that space,  revealing what Mark calls ‘digital depth’ – the hidden data flows that describe the real world but are normally invisible to us.

The book’s key point, which is obvious when you encounter it, is that  ‘augmented reality is a technology of networked surveillance’ and that this is not a side-effect like the data exhaust of web searches, but a fundamental design requirement. You can’t put an elephant in my visual field and let me walk around it unless you have scanned the room, captured my position, noted my head and eye movements, and watched my hands. All that data is being uploaded and processed in the cloud, with no real controls over what happens to it next.  This is not speculation: Facebook has already told Oculus users that it will use their data to tailor ads.

This makes the Hololens a much more  invasive surveillance device than the much-reviled Google Glass, constantly taking in vast amounts of data from the environment with no flashing red light to indicate that it’s in recording mode, yet carefully designed to make its camera and other sensors unobtrusive. However we are encouraged to see it as a delightful took for work, art, or relaxing games.

Along with the concerns over enhanced surveillance and the potential for the abuse of that data in all the many ways we have seen our clickstreams abused – imagine what Google could do with your gazestream – Mark also points out that we have no control  over who gets to overlay the real world: who has the ‘right to write’.

Powerful projectors are already being used for guerilla marketing and campaigning in the real world, with the UK’s House of Commons is a common target for the latter, but anyone can do what Niantic did with Pokemon Go and simply add a data overlay to public or private spaces.  In fact in 2016 the Auschwitz-Birkenau State Museum had to formally ask people not to play the game on its site, because Niantic had not bothered to fence off sensitive locations from being infested with Pokemon, but there has been no real discussion of what sort of rules we might want to impose in future as these technologies become more widespread and easier to use.

Reading Augmented Reality will give you the background you need to understand some of the less obvious implications of what remains an exciting and potentially transformative technology, just in time to engage in the serious debate that we need to have about what happens to the data amassed to make AR work, who gets to write in the digital sphere, and how we balance our individual interests, the wider public interest, and the desires of business to use these technologies commercially.

 

You can listen to Mark on the deeply entertaining and well-informed ‘Next Billion Seconds’ podcast at https://nextbillionseconds.com/

Augmented Reality by Mark Pesce is published by Polity Press