The definition of puppetry in a traditional theatre or performance context implies the human manipulation of inanimate anthropomorphic objects such as puppets, dolls, marionettes, and figurines. This can be broken down into three elements – the inanimate figure, the movement imposed on it, and the vocal performance (Veltruský, 1983: 68-122 and Wagner, 2003: 47-48). In contrast to analog animation techniques, where inanimate objects are “brought to life” by frame-by-frame animation (Furniss 1998), a puppeteer performs in front of a live audience. According to (Tillis 1992:65), the audience also plays a crucial role because they must “suspend their disbelief and bring life to the object so that it becomes a puppet.” (Calvillo-Gámez and Cairns, 2008: 314). Although the live character must be suspended, this kind of “double vision” is also valid for an audience of animation films.
The proposed talk illustrates how these concepts can be combined and subverted to devise contemporary speculative fiction experiences. It revolves around the outcomes of the second iteration of the art-based research project VRinMotion – ExperiMotion 2 that applied and extended both techniques in an experimental virtual setup while also working with the proposed constraints as a guideline for narrative design to experience planetary scale concepts and more-than-human-actors, such as tectonic plates and labor. To achieve this, the experimental setup combined interactive, sensory experiences from contemporary puppetry with imaging methods from stop motion and timelapse photography. Arduino circuits were built to continuously stimulate inorganic matter, like using sand and vibration motors or colored fluids and micro fog machines in saltwater to create generative puppeteered stop-motion sequences that betray their actual scale. This forms the basis for a VR environment of an underground facility with a worker telling the participants about his day-to-day activity of cutting “frames” out of tectonic plates. Methods to create segmented phonetics of movements were developed for the flexible animation of this person. Integrating Kinect depth image technology allows the mapping of pixilated human motion onto a virtual character.
For VRinMotion, puppetry is an abstract systemic process where not movements but impulses go into social contexts, media contexts, or technical contexts and then lead to different consequences (Kirschner, 2023). Furthermore, according to Calvillo-Gámez and Paul Cairns (2018: 314), in video games, the player assumes the roles of the audience and a puppeteer with controllers, while the videogame itself assumes the role of the puppet. This also applies to the virtual environments of VRinMotion, because the interactive aspect between real and virtual worlds massively increases the range of immediate expressivity. The project developed new possibilities for interactive impulses such as eye-blinking, breathing, or sand-grabbing to influence and navigate the VR experience. The performative situation also resembles an immersive theatre in which ad-hoc reactions between the puppeteer/user and its environment are possible and must accommodate the storytelling.