Lana Tankosa Nikolic & Rameshnath Krishnasamy: Motion Capture in the Metaverse – Embodied co-present Hybrid Theater – CANCELED

This talk has been canceled and will not be presented at Synaesthetic Syntax.

We present the ‘Metastage’—a hybrid stage composed of a physical performance space, mediated and augmented through Metaverse-enabling technologies such as motion capture, virtual reality, and virtual production. We adopt a broad definition of ‘The Metaverse’ as an emerging concept, encompassing not only 100% virtual worlds but also virtual, augmented, and mixed reality. As such, we examine the Metaverse as various interconnected digital platforms (Ritterbusch & Teichmann 2023) enabling new forms of human connections and real-time performative potentials across multiple devices and platforms (Proulx et al 2022; Park & Kim, 2022).

The Metastage is a conceptual platform designed for hybrid theatrical performances, which involve actors wearing motion-capture suits, performing in front of live audiences who experience the performance through both physical and virtual reality elements while also being live-streamed to virtual reality audiences at other locations. The audience is able to participate in the performance through game-like interactions at crucial points, affecting various parts of the performance. Thus, the Metastage constitutes a mix of live non-digital performance, augmented virtual reality motion-capture performances, and distributed virtual reality to remote audiences—affected by the audience’s interactions throughout. The case for the Meta-stage is the experimental theater platform, White Hole Theater, and the three Danish digital hybrid performances: ‘Valdemars Story’, ‘The Battle #1’, and ‘The Battle #2’. We present how the Metastage has been utilized in the three cultural heritage-inspired theater performances in 2021–2023, where historical dramatic events from Denmark in 1146 are mixed into modern-day life through the hybrid performance (White Hole Theater, 2023).

We position this among current contemporary state of art in exploring immersive mediums as part of live performance arts (Montagud et al., 2020; Iudova-Romanov, 2023) and the role of virtuality in theatrical performances as a whole (Popat, 2016). The performative arts have long been trailblazers in combining digital technologies with live performances (Dixon, 2014; Jernigan et al., 2014; PluginHUMAN, Delbridge & Timpkins, 2012), and recent examples show how audience participation and performative interactions can be part of live theater performance.

The project’s central challenge has been to create engaging shared experiences for both live and remote audiences while balancing the physical presence of the actors and their motion capture and animated counterparts. This addresses a persistent issue of virtuality in performance arts – the question of choreography becoming intertwined with animation (e.g., Calvert & Mah, 1996) and whether the embodied experience of theater is affected by the ‘noise’ of immersive interfaces (Krivospitskaya, 2011; Smith, 2019; Popat, 2016). This is, in turn, to be judged against the augmentation potential of animating the performance space, as well as the inclusive potential of audiences interacting and participating remotely, making culture available beyond the availability of the physical stage itself.

From audience research, production post mortems, and plans for further iterations of the Metastage, we discuss these issues and propose that the potential for embodied, co-present hybrid stages stretches beyond theater alone but sets the stage for new ways of utilizing animation and digital media for a broad range of performative live events.

References:

https://docs.google.com/document/d/1bf_RxBH1SfQaB46veV7621XwQzqoqXu4kZ26hFXZhB0/edit?usp=sharing