The recent and dramatic acceleration of technical research, technological promises, and corporate and public imaginaries in VR has led Silicon Valley executives, computer scientists, and the media to claim that the emerging “metaverse” will change human interaction as we know it” (Nardella 2019). At the same time, the emergence of a new set of wearable and half-transparent AR (e.g., Microsoft Holo-Lens or Magic Leap) devices together with VR headsets that use a video technology called “passthrough,” are radically configuring not only the experience of the user’s presence “between the digitality of VR and the concrete reality of their surroundings” (Saker and Frith 2020) but also embodied understandings of co-presence (Goffman 1959), “the conditions in which human individuals interact with one another face to face from body to body” (Zhao 2003).
Yet, the phenomenological experience of co-present, co-located (in the same physical space) sensory interaction in relationship with others in new VR/AR environments remains understudied. Indeed, “presence” in XR contexts has been long accepted as the sense of “being there” (Riva et al., 2003), “telepresence” (Minsky 1980), or what has been called the “place illusion:” the experience of being in a place (even though the real place one is in is usually irrelevant to the virtual experience) “in spite of the sure knowledge that you are not there” (Slater 2009). While there is a vast literature on such virtual presence, its focus is on “the degree to which a virtual environment submerges the perceptual system of the user” away from the “real world” (Biocca and Delaney 1995). Furthermore, studies of co-present interaction in VR/AR have overwhelmingly focused on purely virtual environments with avatars and virtual humans (Wang 2011; Freiwald et al. 2021; Shin and Dongsik 2019), virtual agents (Strojny et al., 2020) and video conferencing (Kim et al., 2014). In other words, the vast majority of research into presence and immersion in VR “has only a limited involvement in concrete space” (Saker and Frith 2020).
Using a recent XR-based theatre project called Animate which is focused on climate transformation as a case study, this presentation aims to present concepts and methods for grappling with what Ronald Azuma claims is the fundamental challenge as we move into increasingly mixed reality-based experiences where animation is indeed extended between the physical and computational worlds: “how to enable virtual content that is integrated with the surrounding real world, while users remain engaged with and aware of that ‘real world’” (2016).