DATA ANTICIPATION FOR GESTURE RECOGNITION IN
THE AIR PERCUSSION
Vincent Goudard, Christophe Havel, Sylvain Marchand,
and Myriam Desainte-Catherine
SCRIME - LaBRI, University of Bordeaux 1, 351 cours de la Lib6ration,
F-33405 Talence cedex, France
www. scrime.u-bordeaux. fr, www.labri.fr
ABSTRACT
The use of computers as Digital Musical Instruments
(DMI) rises many problems due to the separation between
gesture and sound production. However, computational
power allows now for complex real-time analysis and gesture anticipation and recognition. We present here an interactive performance system for a percussionist playing
in the air, with no physical percussion. Since the gesture
sensors are not perfect, we have to deal with unreliable
data. And since we expect a real-time sound synthesis
synchronous with the strikes of the percussionist, we have
to anticipate his gesture to some extent, in order to forecast his strikes, because we cannot wait for the strikes to
occur without degrading the auditory feedback. We show
that the use of linear prediction can help to both correct
the data from the gesture sensors and anticipate the gesture of the percussionist. The difficult problem of gesture
recognition is also discussed in the context of the "air percussion" project.
1. INTRODUCTION
If the expressiveness of an instrumentalist lies in his ability to play an instrument through an intuitive coordination
of his gesture, in response to an intimate musical feeling
and listening-understanding, one has to consider the long
experience of traditional acoustic instrument players.
At the present time, our choice is limited to a restricted
set of digital instrument interface imitating / modeling their
acoustic equivalent, the most common being the MIDI
keyboard for piano and organ. In particular, the percussion
finds no suitable equivalent in the electronic instrumentarium. Pads almost reduce the sharpness of the play to
a simple triggered event with velocity information, while
the contact can take plenty of other subtleties (e.g. depending on the point of strike on the percussion, the kind
of contact, etc.).
Furthermore, the contact is only a very small part of the
whole gesture. In the artistic approach presented here, we
choose to remove the contact and focus on the movements
of the sticks in the air.
A few projects have been led until now to alleviate
this problem, but technology was hardly able to render
both static (position) and dynamic (movement) information. From Boie and Matthews "Radio-Baton" [12] covering a reduced surface, to accelerometer solutions that
do not send position back [11], to video-based systems 1
1 http://www.gmem.org
which suffer from high computational load, the complex
gesture of the percussionist remains with no clear modeling.
If the digital interface takes into account the ergonomic
of the traditional instrumentarium, the composer is then in
a position to define a new sonic environment, with unexplored sounds and interactions, while keeping the articulation expertise of the traditional instrumentalists.
After a brief presentation of the "air percussion" project
in Section 2, we show in Section 3 some particularities of
this project due to the behavior of the percussionist and
give in 4 some interesting solutions for gesture data correction and anticipation based on a linear prediction technique. Finally, the difficult problem of gesture recognition
is discussed in Section 5.
2. THE "AIR PERCUSSION"
2.1. Artistic Project
Since 2000 the second author (Havel, composer) develops "metamorphoses", a long-term project of investigation and experimental creation with the ambition of defining, through a succession of works performed in public, a
process of musical writing for a group of chamber music
performers.
The setting is as following: to put the instrumentalists
in front of an electronic set-up with the same ergonomics
that their custom acoustical instruments but with sounds
and control completely different. In this case, the musicians must play according to the reactions of the electronic
instrument and the musical propositions of the other musicians inside a formal structure proposed by the composer.
The electronic instrument consists of a synthesis module associated to a set of interrelations (mapping) between
the gestures sensors and the control of the sound parameters. Each musician generally has many instruments (synthesis modules + mapping) at one's disposal, instruments
he could select during the performance.
2.2. System architecture
In the context of this musical research, the SCRIME (LaBRI, University of Bordeaux 1) developed a scientific research project for the recognition of the percussionist gesture.
An air-percussion device thus was implemented from
the obtained results, including a system of sensors connected to a computer which analyses the gestures of the
