Expanded Animation 2023

ASIFA Austria Forum – Pedro Harres: Storytelling and 2D Animation in 6DOF

A brief overview of what we have learned, sometimes the hard way, as a student team over the four years of making the 2D animated, interactive, 6DOF, VR-piece “From The Main Square”. Despite the support of the Babelsberg Film University, in which I was enrolled as a master student, in many senses, we were navigating uncharted waters and needed to invent a path of our own. As a seasoned animation director venturing himself in immersive storytelling for the first time, it was a deep shift in many previous notions regarding storytelling through moving images. All of the production tools and methods we've gathered from previous projects needed to be meaningfully adapted, if not discarded, with almost no exception. Much of the syntax we used to progress the story throughout the piece had to be invented during production, and therefore thoroughly tested. Some of these mechanisms shall be demonstrated live. From The Main Square is a multi-awarded VR-piece out in the festival circuit, since 2022. It gathered numerous recognitions including the Grand Jury Prize in Venice 79th, the Grand Prix in New Images 2023 and the Festivals Connexion VR Award at the International Animation Festival of Annecy recently. The piece lasts approximately 22 minutes and depicts the rise and downfall of a society shattered by polarisation. It is a compact illustration of social disruption in the form of an animated “Wimmelbild”. The script is a collage of occasionally funny, eventually brutal urban absurdities experienced in societies undergoing political polarisation and environmental collapse.

40 min

https://www.youtube.com/watch?v=hT68nsTXlWk

Pedro Harres

Pedro Harres (BR/DE) is a Berlin-based Brazilian animation director, scriptwriter, and multimedia artist. He studied film, animation, and philosophy. His work varies greatly in terms of style and formats, including installations, films, and XR. However, an allegorical use of space can be perceived as a constant throughout his work; space considered not only as support […]

ASIFA Austria Forum – Fanni Fazakas: Breaking Taboos with XR

How do we elevate the stories of underestimated people in XR? Through the case study of Missing 10 Hours VR - a multi-ending interactive piece about the Bystander Effect and date-rape drugs - I will share our rapid-prototyping process to help you move your ideas from scribble to story. We'll explore unconventional ways to use the medium for measurable positive impact, including worldbuilding, animation, and diverse and authentic character design. If you're tired of diversity panels that make no real change or if you're curious about the secrets of XR for Impact and the tools that make it more accessible, this talk is for you. Join me in the revolution!

40 min

https://www.youtube.com/watch?v=hT68nsTXlWk

Fanni Fazakas

Fanni Fazakas (HU) is an Unreal Engine Technical Artist, Director and Researcher. Her current project, Missing 10 Hours VR, is a non-fiction Virtual Reality experience with multiple endings that explores the use of GHB as a date-rape drug and the Bystander Effect. The project has earned recognition as the Best in Show VR at Siggraph […]

ASIFA Austria Forum – Kris Hofmann: Perform to Communicate

The most exciting feature of augmented reality is the technology’s ability to merge the physical world with an imaginary, virtual world. How can we blend the two in the best possible way to create a true sense of convergence? Most of the time, visual realism is not what we aim for as animators. Whether it is plants, animals or other people – the world reacts to our presence. Well thought-out interaction can make us part of the narrative and reinforce the notion that the experience is part of what we perceive as the ‘real’ world. Interaction empowers people who visit the worlds and characters we create and helps nudge passive observers into becoming story explorers. Kris will talk about her work with animation and augmented reality, focussing on an ongoing collaboration with the animal welfare organisation FOUR PAWS to create a virtual character that inhabits the animal sanctuaries the non-profit runs across the globe.

40 min

https://www.youtube.com/watch?v=hT68nsTXlWk

Kris Hofmann

Currently based in Vienna, Kris (AT) has been working as an animator and animation director since graduating from the Royal College of Art in London in 2010. Since 2018 her focus has been the design of immersive experiences. Her first site-specific augmented reality story, ‘The Bookworms’, was directed and produced for the Austrian National Library […]

Deep Space 8k – Animation Festival: Akiko Nakayama

Akiko Nakayama will comment on the side of video excerpts from previous performances giving insight into her technique called *Alive Painting*. The audience will witness her vivacious transformations of various mediums and colors and engage in a conversation about the curious relationship between painting, performance, and animation art.

30 min

Ars Electronica Center, Deep Space 8K

Akiko Nakayama

Akiko Nakayama (JP, *1988) is a painter who depicts the beauty of conveying energy metamorphosis through several media such as installation, videos, and performance. She brings the painting to life by combining the energy of movement and the vibrance of colors. Called “Alive Painting.” She depicts the resonance between shapes and textures using different types […]

Deep Space 8k – Animation Festival: Rebecca Merlic

Glitchbodies(Honorary Mention Prix Ars Electronica 2023) is an interactive game and multiplatform project encompassing new forms of feminism, LGBTQ+, and Drag transformations. While playing the game, participants become part of an interactive digital space. They are invited to join a fluid journey in which they take up non-heteronormative gender positions and step into a mesmerizing loop of diverse bodies, minds, and thoughts.

30 min

Ars Electronica Center, Deep Space 8K

Rebecca Merlic

Rebecca Merlic (DE/HR/AT, *1989 in Germany) is a European digital artist and architect and experimental filmmaker. She graduated with distinction at the Academy of Fine Arts Vienna, where she studied under Wolfgang Tschapeller, Francois Roché and Michael Hannsmeyer. During her masters she was able to study at Astushi KITAGAWARA Lab at Tokyo University of the […]

Prix Forum: Golden Nica – Ayoung Kim

Jury members in the Prix Category New Animation Art, Nora O Murchú and Helen Starr will moderate a discussion with Golden Nica winner Ayoung Kim (KR) and the two Award of Distinction winners Bassam Issa (IE) and SANGHEE (KR). Ayoung Kim (KR) is an artist with a very diverse practice, blending geopolitics, mythology, technology, and futuristic iconography. Her artworks take the forms of video, virtual reality (VR), game simulation, sonic fiction, and text. She synthesizes the outcomes of far-reaching speculation, establishing connections between biopolitics and border controls, the memories of stones and virtual memories, and ancestral origins and imminent futures.

20 min

https://www.youtube.com/watch?v=XJcC5jWVySc

Ayoung Kim

Ayoung Kim (KR) is an artist with a very diverse practice, blending geopolitics, mythology, technology, and futuristic iconography. Her artworks take the forms of video, virtual reality (VR), game simulation, sonic fiction, and text. She synthesizes the outcomes of far-reaching speculation, establishing connections between biopolitics and border controls, the memories of stones and virtual memories, […]

Nora O’ Murchú

Nora O’ Murchú (IE) is a curator & researcher.  In her curatorial work she explores online culture and the implications of technological developments. Her multidisciplinary practice embraces narratives, and fictions and results in objects, exhibitions, and interventions. Her research aims to help people understand how complex socio-technical systems are imagined, built & used. She has […]

Helen Starr

Helen Starr (TT/UK) is a Afro-Carib curator, producer and cultural activist from Trinidad, WI. She began curating exhibitions with artists such as Susan Hillier, Cindy Sherman and Marcel Duchamp in 1995. Helen founded The Mechatronic Library in 2010, to give marginalised artists access to technologies such as Game Engines, Virtual Reality (VR), Augmented Reality (AR). […]

Prix Forum: Award of Distinction – Bassam Issa

Jury members in the Prix Category New Animation Art, Nora O Murchú and Helen Starr will moderate a discussion with Golden Nica winner Ayoung Kim (KR) and the two Award of Distinction winners Bassam Issa (IE) and SANGHEE (KR). Bassam Issa (IE) works across digital animation, painting, sculpture, and textiles creating visions of resistance, transformation, and queer possibility. He completed a BA in Visual Art Practice from Dún Laoghaire Institute of Art, Design and Technology in 2016. His work is part of collections in IMMA, the Arts Council, and Dun Laoghaire Rathdown County Council. In 2021 he received the Golden Fleece Award, and he is currently a studio member at Temple Bar Gallery + Studios. Al-Sabah lives and works in Belfast and Dublin.

20 min

https://www.youtube.com/watch?v=XJcC5jWVySc

Bassam Issa

Bassam Issa (IE) works across digital animation, painting, sculpture, and textiles creating visions of resistance, transformation, and queer possibility. He completed a BA in Visual Art Practice from Dún Laoghaire Institute of Art, Design and Technology in 2016. His work is part of collections in IMMA, the Arts Council, and Dun Laoghaire Rathdown County Council. […]

Nora O’ Murchú

Nora O’ Murchú (IE) is a curator & researcher.  In her curatorial work she explores online culture and the implications of technological developments. Her multidisciplinary practice embraces narratives, and fictions and results in objects, exhibitions, and interventions. Her research aims to help people understand how complex socio-technical systems are imagined, built & used. She has […]

Helen Starr

Helen Starr (TT/UK) is a Afro-Carib curator, producer and cultural activist from Trinidad, WI. She began curating exhibitions with artists such as Susan Hillier, Cindy Sherman and Marcel Duchamp in 1995. Helen founded The Mechatronic Library in 2010, to give marginalised artists access to technologies such as Game Engines, Virtual Reality (VR), Augmented Reality (AR). […]

Prix Forum: Award of Distinction – SANGHEE

Jury members in the Prix Category New Animation Art, Nora O Murchú and Helen Starr will moderate a discussion with Golden Nica winner Ayoung Kim (KR) and the two Award of Distinction winners Bassam Issa (IE) and SANGHEE (KR). SANGHEE (KR) did photography while majoring in sociology. Her photos emphasize the objects and places functioning as extensions of the Human body. In line with such emphasis, She currently focuses on the affective gaps experienced when the physical input in the real world has an output into the virtual world or via the virtual world. Another main focus in her works is the way class distinction cooperates or conflicts with technology-intensive media. She uses various genres, such as VR, Sound performance, and Video games, as her medium.

20 min

https://www.youtube.com/watch?v=XJcC5jWVySc

SANGHEE

SANGHEE (KR) did photography while majoring in sociology. Her photos emphasize the objects and places functioning as extensions of the Human body. In line with such emphasis, She currently focuses on the affective gaps experienced when the physical input in the real world has an output into the virtual world or via the virtual world. […]

Nora O’ Murchú

Nora O’ Murchú (IE) is a curator & researcher.  In her curatorial work she explores online culture and the implications of technological developments. Her multidisciplinary practice embraces narratives, and fictions and results in objects, exhibitions, and interventions. Her research aims to help people understand how complex socio-technical systems are imagined, built & used. She has […]

Helen Starr

Helen Starr (TT/UK) is a Afro-Carib curator, producer and cultural activist from Trinidad, WI. She began curating exhibitions with artists such as Susan Hillier, Cindy Sherman and Marcel Duchamp in 1995. Helen founded The Mechatronic Library in 2010, to give marginalised artists access to technologies such as Game Engines, Virtual Reality (VR), Augmented Reality (AR). […]

Virtual Stages – Rebecca Merlic: GLITCHBODIES

The lecture at the Expanded Animation Symposium 2023 will explore GLITCHBODIES, a multi-platform project consisting of a visual novel game, VR-game experience, animation movie, and performance piece which presents the stories of 59 protagonists from different international cities. The project seeks to investigate new forms of feminism, queer transformations, and sensitive representations of the protagonists, to bring diverse perspectives into the male-dominated digital gaming world. A discussion of the project's interactive digital space, which provides a safe platform for exploring alternative gender positions from a queer and political perspective, will be included in the presentation. The primary focus will be on the project's mission of sharing the experiences of diverse individuals and celebrating its protagonists in the format of breaking the boundaries of performance art using game engines, live gaming, live sound creation. Created through collaborations and rituals GLITCHBODIES explores new forms of feminism, LGBTQ+, Drag transformations and intimate sensitive representations of the protagonists and aims to bring them to a wider audience trough the stage of performance. How can performers be represented in digital safer space and then be brought back to the analogue world to generate a new audiovisual Act of performance? What does it mean to provide a safer platform for non-heteronormative gender positions sharing the notion of queer as political attitude, providing new perspective for the player while celebrating its protagonists with motherly care? How can new technologies and co-creative inclusive production processes blur the boundaries between digitality and virtuality and reality.

40 min

https://youtube.com/live/UjC6Uxnrh-I?feature=share

Rebecca Merlic

Rebecca Merlic (DE/HR/AT, *1989 in Germany) is a European digital artist and architect and experimental filmmaker. She graduated with distinction at the Academy of Fine Arts Vienna, where she studied under Wolfgang Tschapeller, Francois Roché and Michael Hannsmeyer. During her masters she was able to study at Astushi KITAGAWARA Lab at Tokyo University of the […]

Virtual Stages – PlayOn!: New Storytelling

For four years, the theatre network PlayOn! has been exploring how traditional theatres can use new storytelling to expand their scope and possibilities. How audience can become co-creators of stories? What immersive technologies on stage demand from theatre artists and enrich audiences? The PlayOn! consortium consists of 9 European theatres and as many tech universities. Together they have so far produced almost two dozen theatre performances that combine new storytelling and immersive technologies. The key objectives of PlayOn! are

1. to develop new forms of creative storytelling building on narrative structures used in the creative game sector;
2. to merge new storytelling with a variety of immersive technologies;
3. to use this new format to extend theatre towards the public urban space with relevant stories.

40 min

https://www.youtube.com/watch?v=UjC6Uxnrh-I

Lucy Hammond & Dirk Neldner

Lucy (UK) is Projects Producer at Pilot Theatre and Creative Producer/Director for their projects that use immersive technology. Her work is focused on immersive and interactive storytelling. Her most recent projects include ’The Archive’, an interactive game exploring happiness and the VR experience ‘Monoliths’, celebrating the voices of northern women and the northern landscape. Lucy’s […]

Virtual Stages – minus.eins

Nils Corte, Nils Gallist and Roman Senkl will present minus.eins' digital and hybrid work.

40 min

https://www.youtube.com/watch?v=UjC6Uxnrh-I

Nils Corte, Nils Gallist & Roman Senkl

minus.eins are a Berlin based digital theatre and arts project, with a strong focus in innovative technologies and interactive storytelling. They research, develop and produce mixed and virtual reality theatre, interactive software and (post)digital media arts. Their latest productions have been nominated for FAUST Prize and for the Digital Showcase of Theatertreffen Berlin.

Synaesthetic Syntax – Welcome

Birgitta Hosea (SE/UK) and Juergen Hagler (AT) welcome you to this year's Synaesthetic Syntax.

5 min

https://youtube.com/live/60H0SBRRQTg?feature=share

Birgitta Hosea

Birgitta Hosea (SE/UK) is an artist, filmmaker, and researcher in expanded animation. Exhibitions include Venice & Karachi Biennales; Oaxaca & Chengdu Museums of Contemporary Art; InspiralLondon; Hanmi Gallery, Seoul. She has a solo exhibition at ASIFAKeil, Vienna in April 2020. Included in the Tate Britain and Centre d’Arte Contemporain, Paris, archives, she has been awarded […]

Juergen Hagler

Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a […]

Panel I – The Haptic Body In Action

Panel I consists of three presentations and a follow-up discussion.

1h 10m

https://www.youtube.com/watch?v=60H0SBRRQTg

Birgitta Hosea

Birgitta Hosea (SE/UK) is an artist, filmmaker, and researcher in expanded animation. Exhibitions include Venice & Karachi Biennales; Oaxaca & Chengdu Museums of Contemporary Art; InspiralLondon; Hanmi Gallery, Seoul. She has a solo exhibition at ASIFAKeil, Vienna in April 2020. Included in the Tate Britain and Centre d’Arte Contemporain, Paris, archives, she has been awarded […]

Camille Baker: Haptics and the role of the body as interface to the virtual experiences in immersive artistic experiences

While narrative and design grammars for making virtual reality and immersive experiences are still in development, designers and artists are starting to better understand what audiences respond to within the experiences they design. Some research is in development to make bespoke haptic garments as interfaces to send and receive sensation, enabling wearers to interact with each other’s bodies within immersive virtual and mixed reality spaces and contexts for performance and other social interaction. In most immersive VR experiences, embodiment or embodied presence ideally should involve the “…awareness of bodily sensations… [so that in] The combination of VR and interception …not only do you feel like you’re in a VR environment, but because you’ve consciously worked to integrate your bodily sensations into VR, it's a full, more vivid version of presence.” (Rubin, 2018:51) “Touch is the oldest sense, and most urgent sense” (Ackerman 1990). It can be argued that “we need continual intimate interaction with others through physical contact in order for the brain to be properly and continuously regulated; online or mobile connections do not suffice” (Baker CC 2018). Haptic interfaces are a major factor in feeling immersed, embodied being in and interacting with the world and others in virtual contexts. “…embodiment and its various states can be described as: (1) being in the body, the mind—body integration; and (2) being in the body, but also sensing and extending the mind, such as (3) …. within virtual worlds, space and other immersive environments – and to some extent through the internet and in social media…” and “ …the embodied, in-person aspects of liveness and how our bodies connect the technologies of modern living....that technology and the body meet, to interface to extend beyond the boundary of skin, and our engagement with technology occurs in the process. (Idhe (2002: 5–7, quoted in Baker 2018:60) In VR embodiment, also called embodied presence is …interception… awareness of bodily sensations… [so that in] The combination of VR and interception …not only do you feel like you’re in a VR environment, but because you’ve consciously worked to integrate your bodily sensations into VR, its a full, more vivid version of presence.”(Rubin, 2018:51) Touch is combined with the other sensory modalities to form a coherent and robust percept of the world. By creating emotional and physical connections within VR experiences through narrative and physical actuation, using haptic interfaces is the best way elicit a fully embodied experience for ‘immersants’(Char Davies’ 2004). In the author’s practice, creating an emotional and physical connection between VR experiences and narrative and physical actuation through haptic interfaces is the best way to create a fully immersive experience for audiences. This paper will discuss discoveries from the design and exhibition of the author’s virtual reality (VR) installation artwork in particular, INTER/her, as well as other personally experienced works, and the role of haptics in making audience experiences richer, deeper, visceral and personal.

20 min

https://www.youtube.com/watch?v=60H0SBRRQTg

Camille Baker

Camille Baker (UK) is an artist-performer/researcher/curator within various art forms: immersive experiences, participatory performance, interactive art, mobile media art, tech fashion/soft circuits/DIY electronics, responsive interfaces and environments, and emerging media curating. Maker of participatory performance and immersive artwork, Baker develops methods to explore expressive non-verbal modes of communication, extended embodiment, and presence in real and […]

Darren Woodland: Echoes of Harmonic Threads: Weaving Afrofuturist Narratives through Cloth, Dance, and Sonic Identity

Echoes of Harmonic Threads is an interdisciplinary arts-based research and performance project that investigates the confluence of animated cloth, sonic identity (LaBelle, 2018), and Afrofuturism in an immersive co-created environment. Set against a projected virtual backdrop that mirrors and responds to a live dancer's movements, this investigation explores engagement with human kinesthetics, the live body in motion, and themes of bondage, freedom, and speculative futures/pasts through a lens of poetry and musical motif rooted in the heritage and posterity of Black culture and the African diaspora. In this setting, the audience is presented with physical fragments of cloth with which they can interact through touch. Limited to a small number of participants at a time, their manipulations of these tangible threads influence the dancer's virtual canvas, creating a symbiotic, real-time dialogue. The audience's involvement with the physical cloth directly manipulates the properties of the virtual. The virtual cloth's visual and temporal properties, such as gravity, weight, texture, and its sonic attributes, pitch, amplitude, and timbre, evolve throughout the performance, reflecting the dancer's embodied experience and the audience's participatory interactions within the structure of Afrofuturist narrative themes, simultaneously addressing the human ritual of wrapping our bodies in cloth (Schneider, 1987). Drawing from the rich tapestry of Afrofuturism, the project delves into sonic performativity, animated materiality, and the deep connections between the human body, technology, and imagination. The dancer's costume, illuminated by the projected animations, serves as an extension and augmentation of their body, a reflection of their narrative identity (Brandstetter, 2015), and a bridge between the physical and the virtual worlds. With every ripple of the virtual cloth, a dialogue unfolds—echoing an improvisational symphony pulsating unpredictably. Through this exploration, the project aims to blur the once-concrete boundaries between flesh, costume, and digital matter, ushering us into an extraordinary tableau of human-digital coexistence. The practical implementation of the project emphasizes the dynamic interplay between the dancer, the virtual cloth, and audience participation in improvisational feedback with sound. This improvisation and generative structure are based on historical reflections of Black culture echoed in the forms of jazz music, blues music, poetry, and Afrofuturist literature (Hamilton, 2017). Echoes of Harmonic Threads contextualizes existing research in movement, body, sound, and space while exploring the importance of costume and cloth in dance and music performance, and the performativity of sound and identity. This project seeks to explore and celebrate Black cultural heritage while challenging traditional approaches to the medium and pushing the boundaries of Afrofuturism. References:

• Brandstetter, G. (2015). Dance costume and movement space: Spatial formulas and their metamorphoses into fabric. In Poetics of Dance: Body, Image, and Space in the Historical Avant-Gardes (pp. 131-148). Oxford Studies in Dance Theory. New York: Oxford Academic. https://doi.org/10.1093/acprof:oso/9780199916559.003.0008
• Hamilton, E. C. (2017). Afrofuturism and the Technologies of Survival. African Arts, 50(4), 18–23. https://www.jstor.org/stable/48547433
• LaBelle, B. (2018). Sonic agency: Sound and emergent forms of resistance. Goldsmiths, University London.

20 min

https://www.youtube.com/watch?v=60H0SBRRQTg

Darren Woodland

Darren Woodland Jr. (US) is an innovative multimedia designer, accomplished 3D artist, and emerging researcher. Their expertise lies in designing, animating, and sonifying assets for virtual reality, interactive installations, and education/training projects. Specializing in lighting, shading, VFX, audio, and procedural workflows, they possess a versatile skill set in art, design, 3D asset production, animation, illustration, […]

Tomo Sone: The marriage of the emerging technology and human body to be beyond boundaries – inspired by Japanese traditional Noh theatre

Through the pandemic, we remembered how vital is the connection between people. Performing arts expanded its value and possibilities beyond the theater. It is important that the performers on stage will not only become central figures but also feel the world, move forward, and bring about change with the audience. In need of an emerging audience with the rise of the digital age, an increasing number of choreographers and dance artists are exploring new ways to create a different dance experience from traditional production with digital technology. There are physical and mental boundaries between the performer and the audience; in some examples, choreographers have challenged these boundaries in contemporary dance history. Ohad Naharin, for example, repeatedly attempts to disrupt the 'picture frame' of the stage, the frame that divides the audience and the performer by breaking the fourth wall in pieces like “Anaphaza” (1993), “Mamootot” (2003), and “Kamuyot” (2003). The so-called “dancing with the audience” section from 'Deca-Dance' has been performed worldwide and is now one of the most popular parts of his choreographies. In this section, dancers generously invite the audience onto the stage to dance with them. The end of this section is ironic - dancers and audience members leave one by one, while the 'soloist,' the last audience member, stands alone in the middle of the stage. Nevertheless, it offers audiences a less formal and different perspective to dance. From the performer's side, this means that their performance should be different from the traditional way to do with the fourth wall. Ohad's words, “Dancing is about listening, not about telling,” describes everything. Bringing digital technology to concert dance can break the fourth wall and reach an emerging audience, but there is always tension between innovational technology and physical performance. Digital technology and physical performance can be placed on opposite sides, and the process of creation and aims of digital content and physical performance are different. Therefore, the author chose augmented reality (AR) to experience, as a performer, a new way to engage with the audience. It was here aiming to understand the meaning of bringing digital technology to the technology of the body and demonstrate the process of creating a live performance where the audience used AR to extend their imagination. Like any art, dance is the product of human social behavior and a romantic behavior of human thoughts and emotions in the virtual world. The interest in using AR is motivated by a need to engage the emerging audience who wants to engage in the art differently. Some potential audiences can be away from dance because of its traditional setting and procedure, and emerging audiences are interested in participating and engaging in dance and performing interactively. This paper examines the meaning of adding digital technology to the technology of the body and how AR can reach more audiences for dance through case studies. Moreover, and most importantly, it examines how AR can be helpful to make live performances beyond boundaries between performer and audience.

20 min

https://www.youtube.com/watch?v=60H0SBRRQTg

Tomo Sone

Tomo Sone (JP) is an independent choreographer, performer born in Kyoto. She started her career as a choreographer in Israel in 2008. In 2010, Tomo established Japan-Israel International Dance Project with Israeli choreographers and dancers, and the project has been producing performances and movement workshops. Nowadays, she is expanding her activity internationally while performing, creating […]

The Haptic Body In Action – Discussion

Birgitta Hosea will discuss with the participants of Panel I.

10 min

https://www.youtube.com/watch?v=60H0SBRRQTg

Birgitta Hosea

Birgitta Hosea (SE/UK) is an artist, filmmaker, and researcher in expanded animation. Exhibitions include Venice & Karachi Biennales; Oaxaca & Chengdu Museums of Contemporary Art; InspiralLondon; Hanmi Gallery, Seoul. She has a solo exhibition at ASIFAKeil, Vienna in April 2020. Included in the Tate Britain and Centre d’Arte Contemporain, Paris, archives, she has been awarded […]

Deep Space 8k – Delivery Dancer’s Sphere (Golden Nica Winner for New Animation Art 2023)

The algorithm-centric, hypervigilant economy of platform labour urges delivery drivers to defy space and time to meet the demands of fastest deliveries. Ernst Mo rushes with the speed of light through a labyrinth of endlessly shifting paths in techno-futuristic Seoul. An unexpected encounter with a double self cracks her reality, and she slides to another dimension beyond algorithmic computability. A dazzling, eerie journey begins, in which various spaces and time-zones collide.

30 min

Ars Electronica Center, Deep Space 8K

Deep Space 8k – Animation Festival with Data, Bodies Space

The selected shorts from Data, Bodies, Space fast-forward to a future which is fascinating and frightening at the same time. In The Needlecast Rhapsody, we witness a technology which can upload humans to distant locations in the galaxy. Moirai - Thread of Life also shows a journey to different cosmic dimensions and touches on quantum physics to explain notions of fate. Posthuman Hospital and My Mind As/Is Your Memory, My Body As/Is Your Substance take one to a nightmarish posthuman dystopia, whereas Rehousing Technosphere gives a more hopeful look into a new planetary ecology in a more-than-human future in which the digital is everpresent yet endued with haptic and sensual properties. La Limite est une facade, and IT’S DANGEROUS TO GO ALONE! TAKE THIS! depict mesmerizing journeys of fluidity and metamorphosis with a poetic approach, whereas CIRCVS MAXIMVS and Interchange feature a humorous take on virtual, computer-animated worlds and their relationship with the “real” world, highlighting the entanglements of power and control.

30 min

Ars Electronica Center, Deep Space 8K

Jonathan Armour

Jonathan Armour’s (IE) practice builds from a previous career in engineering and is an interplay between digital and oil-based works, including mappings of non-normative bodies, and digital investigations of the human body as a celebrated, objectified, abstracted form. Often the work is driven by a desire to get under the skin, to pare away the […]

Wang & Söderström

Wang & Söderström (SE) is a Swedish-born, Copenhagen-based artist and design duo consisting of Anny Wang and Tim Söderström. We are interested in investigating the relationship of digital technologies, materials and living things in the current and future digital and ecological shift. Our synthetic realities translates into sculptures, visual imagery and installations that bring craft, […]

Dorian Rigal Minuit

Minuit (Dorian Rigal, FR), light and digital artist, was born in 1987 in Montmorency, France. Graduate of the National School of Architecture of Paris Val de Seine (ENSAPVS) in 2011, he explores the nocturnal world. He proposes to inhabit the urban night, symbol of an uninhibited freedom, in opposition to the constraining social codes of […]

Maxime Chudeau

Maxime Chudeau (FR/US) is an independent French filmmaker based in Chicago. As a scenography MFA graduate from Haute École des Arts du Rhin (Strasbourg, FR), Maxime’s hybrid approach to film combines scenic design, sound design and computer animation. His work explores the boundaries of perception and often puts the viewer in the leading role of […]

Bassam Issa

Bassam Issa (IE) works across digital animation, painting, sculpture, and textiles creating visions of resistance, transformation, and queer possibility. He completed a BA in Visual Art Practice from Dún Laoghaire Institute of Art, Design and Technology in 2016. His work is part of collections in IMMA, the Arts Council, and Dun Laoghaire Rathdown County Council. […]

Panel II – Agency in Performance

Panel II consists of three presentations and a follow-up discussion.

1h 10m

https://www.youtube.com/watch?v=2GyHszClGPc

Juergen Hagler

Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a […]

Louis-Philippe Demers & Bill Vorn: Animating Flesh: Procedural Authorship in Participative Robotic Immersive Experiences

This presentation explores the incorporation of mechatronic devices such as exoskeletons and supernumerary limbs into live performances to animate human bodies on stage. We adapt motion capture (mocap) techniques and data gathered from bodies, crowds, and devices to control, augment and alter human movements as if we would be animating 3D virtual characters yet with the inevitable constraints of the physical world and human body integrity. The study acknowledges a shift in the arts towards a more experiential approach. To be actively involved in the work, beyond merely occupying the physical and temporal space, fundamentally changes the perception of action and point of view.  “Procedural authorship” refers to participation where instructions are given to audiences by artists [1], similar to the mechanics, dynamics, and aesthetics in game design. This research investigates how procedural authorship can be facilitated and expanded by mocap systems and wearables. Mocap, presented as a spectacle to audiences, highlights the ambiguities of the bodies involved, such as intersubjectivity and de-individuation [2]. The notion of free will is critically engaged, with scenarios either impeding or enhancing future actions, fostering collective responsibility within the performance. The flow of motion data navigates various routes: from bespoke input devices extending to the crowd, from chosen audience members to the entire gathering, through one-to-one "dialogues" facilitated via mobile applications, to a self-capturing and replay system of motion. This transforms audiences into interactive feedback systems and positions mocap as an instrument for crowdsourcing activities and collaborative creativity. Several productions have seamlessly integrated human and machine, transforming animated bodies into amalgamations of light, robot, sound, and group elements. "Grace State Machines" (Vorn, 2007) aligns human bodies with abstract machine forms. "Inferno" (Demers, Vorn, 2015) equips 24 novices with exoskeletons, transforming them into dancers, where live mocap facilitates better dynamics. "XLimbs" (Demers, 2017) employs wearables that self-record and replay movements through extra limbs. The upcoming "Godspeed" (2024) will empower spectators to become skilled performers within a larger ensemble, utilizing feedback mocap and subsequently quantizing each member's movements. There are affordance motives to be found in interoperable motor skills between participants, in kinesthetics resonances, and also in impossibilities [3]. Given this procedural authorship, participants’ brains expect certain prototypical features once the pattern of body data harvesting is suggested or given. Participative works aim to create a liminoid framework akin to a stage space. The heightened intersubjectivity is set forth by an invitation to join, followed by consent to participate. Audiences are literally “raptured” [4] by the wearables. This enhances immersion and triggers strong physiological responses marking transformative moments in the lives of participants.

1. White, G., Audience participation in theatre: Aesthetics of the invitation. 2013: Springer.
2. Mueller, F.F., et al., Towards Understanding the Design of Intertwined Human-computer Integrations. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023.
3. Garner Jr, S.B., J. Garner, and Rene, Kinesthetic Spectatorship in the Theatre. 2018: Springer.
4. Levy, E.K., Enraptured: Attention and Art, in Perception and Agency in Shared Spaces of Contemporary Art. 2017, Routledge. p. 171-183.

20 min

https://www.youtube.com/watch?v=2GyHszClGPc

Louis-Philippe Demers & Bill Vorn

Bill Vorn and Louis-Philippe Demers are prominent figures in the field of Robotic Art for the past 30 years. Their projects incorporate robotics, motion control, sound, lighting, video, and cybernetic processes. Their artistic exploration is strongly influenced by advancements in Human-Robot Interaction, with a focus on Relational Robotics and Audience Experiences. Vorn is a Full […]

Friedrich Kirschner, Julian Salhofer & Leoni Voegelin: Animating a Concept – employing stop-motion and puppetry lenses for more-than-human experiences

The definition of puppetry in a traditional theatre or performance context implies the human manipulation of inanimate anthropomorphic objects such as puppets, dolls, marionettes, and figurines. This can be broken down into three elements – the inanimate figure, the movement imposed on it, and the vocal performance (Veltruský, 1983: 68-122 and Wagner, 2003: 47-48). In contrast to analog animation techniques, where inanimate objects are “brought to life” by frame-by-frame animation (Furniss 1998), a puppeteer performs in front of a live audience. According to (Tillis 1992:65), the audience also plays a crucial role because they must “suspend their disbelief and bring life to the object so that it becomes a puppet.” (Calvillo-Gámez and Cairns, 2008: 314). Although the live character must be suspended, this kind of “double vision” is also valid for an audience of animation films. The proposed talk illustrates how these concepts can be combined and subverted to devise contemporary speculative fiction experiences. It revolves around the outcomes of the second iteration of the art-based research project VRinMotion – ExperiMotion 2 that applied and extended both techniques in an experimental virtual setup while also working with the proposed constraints as a guideline for narrative design to experience planetary scale concepts and more-than-human-actors, such as tectonic plates and labor. To achieve this, the experimental setup combined interactive, sensory experiences from contemporary puppetry with imaging methods from stop motion and timelapse photography. Arduino circuits were built to continuously stimulate inorganic matter, like using sand and vibration motors or colored fluids and micro fog machines in saltwater to create generative puppeteered stop-motion sequences that betray their actual scale. This forms the basis for a VR environment of an underground facility with a worker telling the participants about his day-to-day activity of cutting “frames” out of tectonic plates. Methods to create segmented phonetics of movements were developed for the flexible animation of this person. Integrating Kinect depth image technology allows the mapping of pixilated human motion onto a virtual character. For VRinMotion, puppetry is an abstract systemic process where not movements but impulses go into social contexts, media contexts, or technical contexts and then lead to different consequences (Kirschner, 2023). Furthermore, according to Calvillo-Gámez and Paul Cairns (2018: 314), in video games, the player assumes the roles of the audience and a puppeteer with controllers, while the videogame itself assumes the role of the puppet. This also applies to the virtual environments of VRinMotion, because the interactive aspect between real and virtual worlds massively increases the range of immediate expressivity. The project developed new possibilities for interactive impulses such as eye-blinking, breathing, or sand-grabbing to influence and navigate the VR experience. The performative situation also resembles an immersive theatre in which ad-hoc reactions between the puppeteer/user and its environment are possible and must accommodate the storytelling. Further details about the project VRinMotion: https://research.fhstp.ac.at/en/projects/vrinmotion/experimotion-2

20 min

https://www.youtube.com/watch?v=2GyHszClGPc

Friedrich Kirschner, Julian Salhofer & Leoni Voegelin

Friedrich Kirschner (DE) is a theatre director and software developer. He uses game structures and technology for participatory performances and interactive installations. Friedrich is head of the MA Spiel und Objekt as a Professor for digital media at the University of Performing Arts Ernst Busch in Berlin. Julian Salhofer (AT) is a junior researcher at […]

Lana Tankosa Nikolic & Rameshnath Krishnasamy: Motion Capture in the Metaverse – Embodied co-present Hybrid Theater – CANCELED

This talk has been canceled and will not be presented at Synaesthetic Syntax. We present the ‘Metastage’—a hybrid stage composed of a physical performance space, mediated and augmented through Metaverse-enabling technologies such as motion capture, virtual reality, and virtual production. We adopt a broad definition of ‘The Metaverse’ as an emerging concept, encompassing not only 100% virtual worlds but also virtual, augmented, and mixed reality. As such, we examine the Metaverse as various interconnected digital platforms (Ritterbusch & Teichmann 2023) enabling new forms of human connections and real-time performative potentials across multiple devices and platforms (Proulx et al 2022; Park & Kim, 2022). The Metastage is a conceptual platform designed for hybrid theatrical performances, which involve actors wearing motion-capture suits, performing in front of live audiences who experience the performance through both physical and virtual reality elements while also being live-streamed to virtual reality audiences at other locations. The audience is able to participate in the performance through game-like interactions at crucial points, affecting various parts of the performance. Thus, the Metastage constitutes a mix of live non-digital performance, augmented virtual reality motion-capture performances, and distributed virtual reality to remote audiences—affected by the audience's interactions throughout. The case for the Meta-stage is the experimental theater platform, White Hole Theater, and the three Danish digital hybrid performances: ‘Valdemars Story’, ‘The Battle #1’, and ‘The Battle #2’. We present how the Metastage has been utilized in the three cultural heritage-inspired theater performances in 2021–2023, where historical dramatic events from Denmark in 1146 are mixed into modern-day life through the hybrid performance (White Hole Theater, 2023). We position this among current contemporary state of art in exploring immersive mediums as part of live performance arts (Montagud et al., 2020; Iudova-Romanov, 2023) and the role of virtuality in theatrical performances as a whole (Popat, 2016). The performative arts have long been trailblazers in combining digital technologies with live performances (Dixon, 2014; Jernigan et al., 2014; PluginHUMAN, Delbridge & Timpkins, 2012), and recent examples show how audience participation and performative interactions can be part of live theater performance. The project's central challenge has been to create engaging shared experiences for both live and remote audiences while balancing the physical presence of the actors and their motion capture and animated counterparts. This addresses a persistent issue of virtuality in performance arts - the question of choreography becoming intertwined with animation (e.g., Calvert & Mah, 1996) and whether the embodied experience of theater is affected by the ‘noise’ of immersive interfaces (Krivospitskaya, 2011; Smith, 2019; Popat, 2016). This is, in turn, to be judged against the augmentation potential of animating the performance space, as well as the inclusive potential of audiences interacting and participating remotely, making culture available beyond the availability of the physical stage itself. From audience research, production post mortems, and plans for further iterations of the Metastage, we discuss these issues and propose that the potential for embodied, co-present hybrid stages stretches beyond theater alone but sets the stage for new ways of utilizing animation and digital media for a broad range of performative live events. References: https://docs.google.com/document/d/1bf_RxBH1SfQaB46veV7621XwQzqoqXu4kZ26hFXZhB0/edit?usp=sharing

20 min

https://www.youtube.com/watch?v=2GyHszClGPc

Lana Tankosa Nikolic & Rameshnath Krishnasamy

Lana (DK) is an established producer of award-winning animation and immersive multimedia projects embracing animation, digital arts, VR, theatre, and performance. She is the founder and owner of Late Love Production. She works part-time as a project leader in the Research and Development Department at The Animation Workshop, VIA University College. She is the producer […]

Agency in Performance – Discussion

Juergen Hagler will discuss with the participants of Panel II.

10 min

https://www.youtube.com/watch?v=2GyHszClGPc

Juergen Hagler

Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a […]

Break

A short break between the Synaesthetic Syntax sessions.

20 min

Panel III – Liveness and Procedural Animation

Panel III consists of three presentations and a follow-up discussion.

1h 10m

https://www.youtube.com/watch?v=w9YeZsQl6g0

Juergen Hagler

Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a […]

Max Hattler: Hattlerizer 4.D: Expanded Stereoscopic Abstract Animation Live Performance

When I started performing live visuals, I saw it as an enjoyable but slightly throwaway counterpoint to my experimental animation practice. Over the years, the live work has matured into a succinct and coherent audio-visual expression on equal par with the animation films and installation works. It deals with a very similar kind of inquiry, except that it happens in real-time, in a less controlled and much longer-form format. Improvisation within a set of rules, the search for synchresis, and the orchestration of time and movement in a musical expression of visual elements are at the heart of it. My latest audio-visual performance instrument, Hattlerizer 4.D, represents a consolidated real-time motion graphics setup with a precise and specific vocabulary of shapes, sounds, and motions. Inspired in equal parts by improvisational jazz, the scroll paintings of Hans Richter and Viking Eggeling, Norman McLaren’s optical sound experiments, and Laurenz Theinert’s visual piano performances, it provides a self-referential audio-visual system with a range of modifiers which enable real-time, improvised performance of multi-layered and fluid stereoscopic abstract animation and synchronised optical sound. Consisting of custom software components and hardware controllers, the physical input of the performer becomes an important part of the experience. In contrast to my fixed animation works which are meticulously micro-edited, in Hattlerizer performances, live improvisation – with all its flaws and imperfections – is foregrounded and becomes perceivable for the audience. This presentation discusses the influences, development, technical setup, and visual expressions of the Hattlerizer from the first version in 2010 to its latest 4.D iteration to reflect on questions of ‘liveness’ and improvisation in animation and questions of ‘animation’ in audio-visual performance.

20 min

https://www.youtube.com/watch?v=w9YeZsQl6g0

Max Hattler

Max Hattler (DE/HK) is an artist, researcher, curator, and educator who works with abstract and experimental animation, video installation, and audio-visual performance. After studying at Goldsmiths and the Royal College of Art, London, he completed a doctorate in fine art at the University of East London. He is an associate professor at the School of […]

Johannes DeYoung: The Endless Mile: Phantasmagoria, Shadow Play, and Performative Assembly

“The Endless Mile (Another Line of Becoming)” is an interactive computational artwork that blends live and prepared media through procedural graphics generation, real-time animation, and audio-visual synthesis. Each time its underlying program runs, the artwork generates new aesthetic assemblies from an index of prepared components, including animated elements, images and videos, virtual 3D models, and sounds, as well as live media inputs prompted through performative interaction or audience feedback. While each component initially comprises its own individual statement, the collective assembly yields variable and sometimes overlapping phrases.

The aesthetic arrest of audiovisual content sometimes misrepresents computational artworks as cinematic endeavors. Strictly approaching such work from within cinematic discourse misses an important conceptual framework. “The Endless Mile” is built upon a poetic combine structure that’s more akin to Robert Rauschenberg’s “The 1/4 Mile or 2 Furlong Piece” (1981–98) than to fixed cinematic narrative.  While presented as an animated video mural, the work’s real-time affordances confound strict linear flow and relinquish authorial control from within the momentary relationships of the scroll. Authorial influence instead resides in the design of the index and the system itself, which perpetually spawn new component combinations throughout its open-ended duration.

This presentation considers expansive frontiers for making and experiencing animation at the intersections of art, technology, and live performance.  Poetic frameworks in procedural real-time graphics, collective assemblies, and collaborative design situate the historical examples and contemporary works highlighted in this presentation.

References include: theatrical storytelling paradigms in Indonesian Wayang Beber and Wayang Kulit traditions; early cinematic actualities, novelty films, and the cinema of attractions; eighteenth and nineteenth-century theatrical immersions, including phantasmagorias, architectural panoramas, and mechanical scrolls; post-media tendencies of Expanded Cinema, the Neo-Avant-Garde, Fluxus, and contemporary art practice; behavioral studies in psychology and UX design; as well as computational ambiance acts, counter-gaming console hacks, and AI-generated cinematic arts practices.

Works Cited (abbreviated):

20 min

https://www.youtube.com/watch?v=w9YeZsQl6g0

Johannes DeYoung

Johannes DeYoung (US) is an internationally recognized artist and filmmaker who works at the intersection of computational and material processes. His moving-image works have been exhibited internationally at venues such as: Museo de Arte Contemporáneo de Alicante, Alicante, Spain; Festival ECRÃ, Rio de Janeiro, Brazil; National Taiwan Museum of Fine Arts, Taichung, Taiwan; B3 Biennale […]

Varvara Guljajeva & Mar Canet Sola: Exploring the Catastrophic Impact of Human Interaction on Nature: Navigating the Latent Space of Diffusion Models through the Gaze

Large AI models, such as Stable Diffusion, have compressed billions of nature images that bear traces of both the catastrophic impact of human interaction and the mesmerizing beauty of natural landscapes, seamlessly merging these contrasting aspects. The past history of humanity, available online, is stored in datasets like LAION-5B, which aggregated 5 billion images from the internet for training such large models like Stable Diffusion. The weights of the checkpoint file are the repository where all this knowledge remains compressed and stored. Artwork “Visions of Destruction” introduces an innovative real-time AI-driven interactive latent cinema experience to its audience. By harnessing the power of gaze and leveraging prompts that depict various ecological damages caused by humans, this groundbreaking project creates initial beautiful landscapes which mesmerize the flow of animated transformations to landscapes of ecological disasters. Guided by the viewers' gazes, which are seamlessly detected through an advanced eye-tracking system, the real-time animations unfold, offering a captivating exploration of the consequences of our actions on the environment. In other words, a simple act of observation results in transformation of imagery. The piece unveils the possibility space of a generative model when is fearlessly exploring the digitalized history of human transformations to nature captured in the training images. In the latent cinema of AI, dreams and nightmares intertwine as the generative model unravels hidden dimensions and unleashes new possibilities creating visual indeterminacy landscapes [1] of catastrophes to nature. It opens up a realm of limitless creativity and imagination, creating a unique experience for each viewer. It becomes a powerful tool, providing artists and filmmakers with a canvas to transmute their visions into tangible reality, blurring the line between the creator and the created. Moreover, gaze is a mode of interaction in this system, taking salient parts looked at by the viewers, but also the metaphor of humanity’s gaze over nature and its impact. In terms of its technological aspects, this art installation demonstrates recent advancements in text-to-image models. Moreover, the interactive art piece uses real-time deep learning technology for image generation and animations, skillfully combined with an innovative method of interaction through eye-tracking. To achieve this, we make use of Stable Diffusion Inpainting model by using masks created of the specific areas where the viewer looks in the image. These masks enable the transformation of the image on the viewer's focal points, guided by prompts that poignantly address the theme of human-induced destruction of nature and create interpolations to the new images generated. Finally, when a landscape has been destroyed by the gaze interaction, and it is not looked at for a few seconds, it regenerates into a new ‘beautiful landscape’. To sum put, the proposed presentation is about a case study that introduces real-time gaze-based navigation to AI-generated animation depicting human destructive events in the beautiful landscapes, which are actually synthetically generated, too. References [1] Hertzmann, Aaron. "Visual indeterminacy in GAN art." In ACM SIGGRAPH 2020 Art Gallery, pp. 424–428. 2020.

20 min

https://www.youtube.com/watch?v=w9YeZsQl6g0

Varvara Guljajeva & Mar Canet Sola

Liveness and Procedural Animation – Discussion

Juergen Hagler will discuss with the participants of Panel III.

10 min

https://www.youtube.com/watch?v=w9YeZsQl6g0

Juergen Hagler

Juergen Hagler (AT) studied art education, experimental visual design, and cultural studies at the University for Art and Design in Linz, Austria. He currently works as a professor of Computer Animation and Animation Studies in the Digital Media department at the Hagenberg Campus of the University of Applied Sciences Upper Austria. Since 2014 he a […]

Break

A short break between the Synaesthetic Syntax sessions.

20 min

Keynote – Ghislaine Boddington

60 min

https://www.youtube.com/watch?v=83eki6cQC_E

Ghislaine Boddington

Ghislaine Boddington (UK) is a curator, presenter and researcher, known for her pioneering work placing the body as the interface for digital technologies and exploring telepresence, digital intimacy and virtual physical blending since the early 1990s. Her research led practise, expert direction and curations include “Robots and Avatars” (EU/Nesta 2009–11), “me and my shadow” (National […]

Synaesthetic Syntax – Closing Note

Birgitta Hosea (SE/UK) concludes this year's Synaesthetic Syntax and Expanded Animation in a short closing note.

10 min

https://www.youtube.com/watch?v=83eki6cQC_E

Birgitta Hosea

Birgitta Hosea (SE/UK) is an artist, filmmaker, and researcher in expanded animation. Exhibitions include Venice & Karachi Biennales; Oaxaca & Chengdu Museums of Contemporary Art; InspiralLondon; Hanmi Gallery, Seoul. She has a solo exhibition at ASIFAKeil, Vienna in April 2020. Included in the Tate Britain and Centre d’Arte Contemporain, Paris, archives, she has been awarded […]