Synthesis of expressive movement
Antonio Camurri, Paolo Coletta, Matteo Ricchetti, Gualtiero Volpe
DIST, Laboratorio di Informatica Musicale - University of Genova (http://musart.dist.unige.it)
Viale Causa 13, 16145 Genova, Italy
music@dist.unige.it colettap@musart.dist.unige.it ricchetti@eidomedia.com volpe@musart.dist.unige.it
ABSTRACT
We present experiments concerning the use of mobile robots in
music theatre artistic productions. The paper focuses on the
development of techniques for expressive movement synthesis
and on their hardware and software implementation in concrete
human-robot and human-virtual characters interaction. Different
kinds of synthesis of expressive content in movement are
considered:
(i) Expressive content conveyed through the movement of
robots and interacting with them (e.g. robots interacting with
dancers, artists, musicians on a stage): a particular focus will
be on the expressiveness raising not only from the movement
of a robot (i.e. its style of movement), but also from a global,
multimodal interaction between a robot and an artist (e.g. a
music performer or a dancer). The integration of music and
movement is also a crucial issues presented in the paper.
(ii) Expressive utilisation of mobile scenery in music theatre
applications. Research on synthesis of expressive movement
will take advantage from the results obtained in our Labs on
the analysis of expressiveness in human movement and
dance. Experiments based on small mobile robots (the
Pioneer 2 from Stanford Research Institute) are presented.
The paper describes issues developed in a multimedia artistic
production we partecipated with Virgilio Sieni Dance Company
(L'Ala dei Sensi, Ferrara, November 1999).
In particular, the interactive setup, the techniques, and the
lessons learned in experimenting the developed system will be
presented.
1.INTRODUCTION
One of the main goals of our research is to explore paradigms of
expressive interaction between humans and robots in the
framework of multimodal environments in music, theatre,
museal exhibitions, and art installations. In a previous work
(Suzuki et al, 1998), we experimented a small mobile robot on
wheels (the Pioneer 1 from Stanford Research Institute) as a
semi-autonomous agent capable of communicating by means of
several channels, including sound and music, visual media and
its style of movement (smooth/nervous, tailwagging, fast/slow,
etc.) with the visitors of a museum exhibit on contemporary art
("Arti Visive 2", Palazzo Ducale, Genova, October 1998).
In another experimental setup, we developed a "theatrical
machine" for the performance of "Spiral", by K.Stockhausen, for
trombone and radio output. The radio, audio amplifier and
loudspeakers were installed on top of the robot navigating on the
stage, thus creating effects of physical spatialization of sound
due to the movement of the robot during the performance
(trombone Michele Lo Muto, live electronics Giovanni Cospito,
Civica Scuola di Musica, Sezione Musica Contemporanea,
Milano, June 1996). The movements of the robot were
influenced by sound parameters and by the gesture of the
trombone performer: for example, a high "energy" content by
the trombonist's gesture and a high sound spectral energy were
stimuli for the robot to move away from the performer. Smooth
and calm phrasing and movements were stimuli attracting the
robot near and around the performer. Further, the robot sound
and music output can be part of the interaction process, i.e.
influenced in real time by the movement and sound from the
performer.
In this paper we investigate how the physical interaction of
performers, dancers and robotic scenery can participate as a
component of the music language. Interactive visual media and
robotic scenery are therefore part of the compositional project of
music theatre works, art installations, multimedia concerts. The
paper explores these directions by presenting the performance
environments developed for the multimedia performance "L'Ala
dei Sensi" (literally, "The Wing of the Senses"), held in Ferrara
(Italy) on November 1999.
Figure 1. The agent-robot in the art installation at the "Arti
Visive 2" museal exhibit, Palazzo Ducale, Genova, Oct 1998.
2. THE MULTIMEDIA PERFORMANCE "L'ALA DEI
SENSI"
We carried out experiments in the above described directions in
the framework of the work "L'Ala dei Sensi" (Ferrara, Italy,
November 1999). "L'Ala dei Sensi" is a multimedia
performance consisting of a percourse on the theme of human
perception from different perspectives. In a few words, it aims at
explaining scientific principles (on human perception
mechanisms) by means of the language of art (dance, music,
visual arts). Each episode focuses on a specific scientific issue
which is explained and demonstrated in a performance.
The episodes involving interactive dance/music performances
also made use of a small mobile on-wheel robotic platform
Pioneer 2 from Stanford Research Institute. Our EyesWeb
system for real-time gesture analysis and for the design of
gesture-music interaction has been used (Camurri et al, 1999;
2000), together our wireless technology for movement sensors,
visualisation and live electronics hardware (see our web site
http://musart.dist.unige.it).
These episodes consist of short dance performances in which the
dancer himself is involved in a dialogue with the robot, with
visual and musical clones on large videoprojection screens. In
particular, three episodes are here described and then discussed.
0
