Page  1 ï~~THE NETWORKED PERFORMANCE PROJECT FOR LO SPECCHIO CONFUSO DALL'OMBRA OF ROBERTO GIROLIN Antonio Camurri, Corrado Canepa, Paolo Coletta, Simone Ghisio, Roberto Girolin, Gualtiero Volpe Casa Paganini - InfoMus Lab University of Genova Genova, Italy 1. INTRODUCTION Communication of expressive and emotional content is a central aspect in music performance. An open research challenge in networked music performance is how to communicate such expressive, emotional content to remote audiences and performers. We carried out a preliminary study in this direction in the framework of the networked performance "Lo specchio confuso dall'ombra" by Roberto Girolin, presented for the first time at Casa Paganini, Genova, on the occasion of the opening concert of the 8th Intl Conference on New Interfaces for Musical Expression (NIMEO8), June 4th, 2008. The piece consists of two separate installations, one in the Foyer and another in the Auditorium of Casa Paganini. The problem of remote communication and social interaction between audience and/or performers between the two different locations is based on the extraction and transmission in real-time of qualitative movement descriptors (expressive cues): the EyesWeb XMI open software platform has been adopted for this aims. In "Lo specchio confuso dall'ombra" we investigate how the sound and visual rendering of a collection of qualitative expressive and emotional descriptors of fullbody movement and gesture of a user/performer are perceived and understood remotely by another user/performer and by the audience. Such descriptors have been chosen according to theories of psychologists [4], humanistic theories by choreographers and researchers on human movement and gesture (e.g., Laban's Theory of Effort), and results from previous projects at our Centre (see for example [1], [2]). Sound rendering is inspired from research on sound design and sonification; visual rendering is inspired from research into theories on animacy [3]. The expressive qualities from full-body movements and gesture are extracted in real-time, and concern cues on movement activity, contraction/expansion, symmetry, impulsiveness, postural attitude (Camurri et al 2005). Descriptors of the dynamic shape of such cues are also used, as well as techniques for higher-level analysis (Camurri et al.2008). The following section describes the concept of "Lo specchio confuso dell'ombra". The piece has been developed with the EyesWeb XMI open platform for eXtended Multimodal Interaction ( and has been partially supported by the EU Culture 2007 Project CoMeDiA. 2. CONCEPT Lo specchio confuso dall'ombra can be translated as "The mirror confused by its shadow": it lies between a distributed installation and a concert, in which opposing groups of performers in two remote places play solo or interact. The audience (two people at a time, one for each installation) activates video and sound transformations, depending on the space they occupy and their gesture. The two installations are in the Foyer and in the Auditorium, respectively, so the two persons from the audience cannot see and talk to each other. Multimodal data and expressive gesture cues are extracted in realtime by an EyesWeb XMI patch, interacting and playing with the electronic performer. The interaction occurs both between the electronic performer and the two places where the audience has access, and between the two remote installations. There are two different levels of intervention in the audio and video transformation: autonomous, depending on the single person and conditioned, depending on the behaviour and the actions occurring in the other, separate installation. Further, the entrance of the concert hall has microphones, which capture words, sentences, coughs, laughs or other noise, which are transformed in realtime and thus entering into the piece. Lo specchio confuso dall'ombra can't bind the audience to remain seated or follow a specific pattern in its behaviour. Its duration is indefinite: it changes every time it is performed.

Page  2 ï~~3. ACKNOWLEDGEMENTS The piece has been commissioned from Roberto Girolin by Casa Paganini - InfoMus Lab. The piece and the scientific and artistic research has been developed with the partial support of the EU Culture 2007 Project CoMe-Di-A. 4. REFERENCES [1] A.Camurri, G. De Poli, M. Leman, G. Volpe, "Toward Communicating Expressiveness and Affect in Multimodal Interactive Systems for Performing Art and Cultural Applications", IEEE MULTIMEDIA, Vol.12, No.1, pp.43-53, IEEE Computer Society Press, 2005. [2] G.Castellano, A.Camurri, K.Scherer, G.Volpe "Expressive Gesture and Music: Analysis of Emotional Behavior in Music Performance, Music Perception. In Press, 2008. [3] F. Heider, Simmel, M. "An Experimental Study of Apparent Behavior", American Journal of Psychology, 57:2, 1944. [4] H.G. Wallbott "Bodily expression of emotion", European Journal of Social Psychology. 28, 879-896, 1998.