Merging symbolic, physical and virtual spaces: Augmented reality for Iannis Xenakis’ Evryali for piano
Event :
Courses & workshops

Merging symbolic, physical and virtual spaces: Augmented reality for Iannis Xenakis’ Evryali for piano

Tag : Talk

The paper presents interactive systems for the visualisation and optimisation of extreme score-based piano performance. The systems are founded on an ecological theory of embodied interaction with complex piano notation, under the title embodied navigation (Antoniadis, 2018a; Antoniadis and Chemero, 2020). The theory has materialised in a modular, sensor-based environment for the analysis, processing and real-time control of notation through multimodal recordings, called GesTCom (Antoniadis, 2018b; Antoniadis and Bevilacqua, 2016). The motion capture modeling is based on an one-shot learning Hidden Markov Model developed at Ircam and called Gesture Follower (Bevilacqua et al., 2010). At a later stage, mixed reality applications have been developed on the basis of existent visualisation methodologies for motion capture (Jégo, Meyrueis and Boutet, 2019), seeking to create a virtual concert environment. Drawing on music performance analysis, embodied cognition, movement modeling and augmented reality, we consider the concert experience as embodied navigation of performers and listeners in a hybrid environment. This environment capitalises on the isomorphisms and decouplings of physical, virtual and symbolic spaces, which merge in static and dynamic relationships: the performer’s gesture shapes music notation, music notation becomes an integral part of the concert space, a virtual avatar of the performer allows the audience to experience multimodal aspects of the performance which usually remain private, and so on. The main focus of this presentation will be on a recent performance of Iannis Xenakis’ solo piano work Evryali employing live motion capture and augmented reality1. This particular work problematises usual notions of virtuosity and performability, bears extra-musical references and is encoded in a unique graphic design. These features justify the task’s characterisation as extreme and demand a rethinking of technology-enhanced performance that combines sensorimotor learning, symbolic interpretation and multimodal feedback in novel ways.