In February 2024, Vision Pro, Apple’s long-awaited extended reality (XR) headset, hit stores. It is Apple’s stab at the consumer XR market, but XR is not how Apple describes it. Instead, when it was announced last summer, Apple CEO Tim Cook said the headset marks the dawning of the era of spatial computing. “You’ve never seen anything like this before,” he added.
Greg Milner
That is not quite true.
The term spatial computing dates to the 1980s. Its modern definition entered the lexicon in 2003. Simon Greenwold, a graduate student in the Program in Media Arts and Sciences at the Massachusetts Institute of Technology (MIT), described spatial computing as “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces.”
But spatial computing extends back even further. It has been the cornerstone of geographic information system (GIS) technology since the software programs debuted in the late 1960s. Indeed, the theoretical foundation of GIS is that it is not only possible but inherently useful to retain and manipulate real objects within some form of virtual space.
In the first GIS programs, virtual space was synonymous with cartographic space. Spatial computing meant using maps to organize large amounts of data in a visually intuitive manner.
The Roots of Spatial Computing
Early geospatial technology pioneers applied the concepts of such theorists as Ian McHarg, who described the world as a series of layers of information that exist and interact in the same physical spaces. If we analyze any spot on Earth, we encounter such informational layers as elevation, soil type, hydrology, biology and land use.
GIS brought this idea to life. The technology allows us to visualize and analyze layers of data on a map. In this way, GIS has become a key integrator of information about our world, from science to engineering to commercial operations.
Through innovation, GIS has grown beyond the bounds of mere 2D map layers to generate maps that are, in effect, 1:1-scale 3D models we call geospatial digital twins.
The major benefit of geospatial digital twins is the ability to provide maximum context. This is especially useful for smart planning of our urban environments. For example, architects can use a digital twin to test how their proposals will fare in such situations as flooding and extreme heat brought about by climate change. City planners can understand the effects of large-scale shifts in the urban environment with interventions focused on enhancing livability. The combination of visualization and hard data allows them to predict impacts and modify plans before making expensive changes to the physical world.
Digital Twins
Each advance in GIS technology has improved our ability to visualize, link and manipulate real objects and spaces in a digital realm.
GIS has evolved to offer truly immersive experiences. In particular, the combination of GIS and game engines such as Unreal and Unity has transformed the process of large-scale infrastructure projects.
In Brisbane, Australia, for example, a digital twin of the ongoing subway construction has been used to display progress. People can walk virtually through planned subway tunnels and stations. This contextual experience helps project leads show Brisbanites how the work is shaping up.
The experience also allows planners, architects, engineers and construction workers to make decisions with more information than could be provided by a paper map or even a traditional digital twin. They can stand on a platform and see how the design elements of a station will look to people moving through it.
The overall effect, as Unity VP Matt Collins recently told Geo Week News, is a “visual narrative of space.” And not just a narrative, Collins added, but one that can “faithfully abide the laws of physics within the systems.”
Visualizing What Could Be
Digital twins can be crystal balls. The virtual spaces can be reconfigured to model different versions of an environment. In practical terms, digital twins allow various stakeholders to have the same vision. This is especially useful in the age of climate change.
Planners and architects can test different versions of a project. If they are designing a subdivision in a coastal community, they can calculate the flooding and storm surge that will likely occur from storms of different magnitudes. Just as important, they can visualize this data, inhabit it, and study it with maximum context.
At root, what they are doing is investigating spatial relations against a realistic backdrop of the world. For the subdivision, these objects include homes, streets, streetlights, and parks, and what matters is their existence in relation to water under multiple scenarios. This is spatial computing: manipulating referents to real objects in a virtual world that, unlike the real one, can be changed at will.
Visualizing Hidden Real Spaces
Immersive environments also offer the promise of displaying a world that is real and already exists yet remains largely invisible.
Public utilities and other companies involved with underground infrastructure have been some of the most enthusiastic adopters of digital twins because the experience can reveal critical connections buried beneath the earth — made visible without the need to dig.
In 2017, the Toms River Municipal Utilities Authority (TRMUA) in Toms River, New Jersey, began using mixed reality (MR) headsets to help crews find underground utility assets for electric, gas, water, telecommunications and sewer services.
GIS stores the location of these assets, and MR displays the underground infrastructure. Traditionally, utilities display this detail on a 2D map. What MR provides is maximum context. Workers in the field can visualize exactly what is under their feet—and see how it’s related spatially to what is all around them.
TRMUA credits MR with saving time and lowering the chances of breaking connections in the networks residents rely on for modern living — savings in the tens of thousands of dollars every day.
Many utilities have since followed TRMUA’s lead. MR setups serve multiple purposes, including training new employees and sharing information between teams in the field and staff in the office.
One utility industry publication recently noted that what these systems ultimately provide is the elimination of guesswork. The ability to know exactly where an asset is located — and to understand how changes will affect the area around it — leads to increased efficiency and customer satisfaction.
The World in Sharper Focus
Apple’s Vision Pro headset is not the only recent example of XR rebranded as spatial computing. Meta and Microsoft have also marketed their XR headsets — Quest 3 and Hololens, respectively — as spatial computers.
Spatial computing will continue its mainstreaming. Eventually, it will likely be the norm. As XR hardware increases in number and power, more organizations will look to unlock the value of all the spatial data recorded in GIS. Being able to experience data will add further value to the systems and workflows that create it.
GIS pioneers began exploring the outer limits of spatial computing a half-century ago. More recently they have realized its potential for smarter urban planning, climate risk mitigation, management of operations across industries and virtual exploration of real-world systems or scenarios via geospatial digital twins. Someday soon, those limits will be reachable by anyone.
As GIS users have learned through the decades, when we get a better sense of where we are in relation to things we care about, we can create the world we want to see.
To explore immersive spatial computing experiences in browser, visit 360 VR Experiences. Read more about Esri XR experiences in ArcGIS.
This article originally appeared on Esri Blog.
The post Mapping the future of spatial computing first appeared on GPS World.
