ï~~Synthetic Rehearsal: Training the Synthetic
Performer
Barry Vercoe & Miller Puckette
Experimental Music Studio
Media Lab, M.I.T.
1. ABSTRACT
Computer tracking of live instruments aims to understand
the dynamics of live performance well enough to replace
any member of an ensemble by a synthetic performer
(computer model) so that the others cannot tell the
difference. Modelling the performer experience has 3
major parts: listen (extract temporal and other cues from
the other players), perform (organize a synchronized and
suitably matched performance), and learn (remember
enough of each experience to benefit future encounters).
While encouraging progress has been made on parts 1 and 2
(Vercoe, 1984), methods to date have relied exclusively on
highly-sensitive live tracking. There has been no learning
or training capacity to allow the benefits of previous experience. The synthetic performer is essentially "sight reading
every time" on the concert stage.
This paper outlines a method of integrating past and
present experience into new strategies of rehearsal and
improved performance. During a performance run, past or
learned information is used to sensitize the perceptual and
cognitive components, giving them a bias towards certain
expected live input behavior. Eventual pitch contour
matching and durational best fit are thus made easier and
more robust. As before, rhythmic elasticity, and both pitch
and rhythmic fault tolerance, are a necessary part of a
practical performance system.
1. BACKGROUND AND OVERVIEW
During 1983-84, research was conducted jointly at the Institut de Recherche et Coordination Acoustique/Musique in
Paris, France, by Barry Vercoe (MIT) and Lawrence Beauregard (flutist of the IRCAM Ensemble Intercontemporaine).
The objective was to understand the dynamics of live
ensemble performance well enough to replace any member
of a group by a synthetic performer (i.e. a computer
model) so that that the remaining live members could not
tell the difference. This aimed to break clear of the
"music-minus-one" syndrome that has characterized tape
and instrument pieces of the past, and to recognize the
machine's potential not as an amplifier of low-level switches
and keys, but as an intelligent and musically informed collaborator in live performance.
The research was carried out using standard repertoire
(flute sonatas by Handel and W.F.Bach) in which the flute
part was played live and the responding harpsichord part
was "performed" by a strictly synthetic accompanist. The
motivation for the work was an IRCAM commission for Vercoe to compose a work, Synapse, for Larry Beauregard and
the 4X real-time audio processor. Proof of how well the
computer can behave as chamber music player in both
traditional and contemporary contexts was then given in
public demonstration at the 1984 International Computer
Music Conference (Vercoe, 1984).
The live demonstration routinely showed a facile ability to
track and remain in sync with wildly changing tempos. It
also showed an ability to deal with heavy doses of live performer errors, sloppy rhythms and general distortion that
would likely throw off a live accompanist. However, in these
early implementations there was no data retention from
one performance run to another: there was no performance "memory", and no facility for the synthetic performer to learn from past experience. The synthetic performer was essentially sight reading on the concert stage
every time.
The aim of subsequent research has been to conceive and
model a working representation of music rehearsal and
music learning. This has meant rethinking the music
abstractions so as to more adequately represent strongly
structured scores, and to more easily model those elements of cognition and learning that depend heavily on
structure. However, since many contemporary scores are
only weakly structured (e.g. unmetered, or multi-branching
with free decisions), it has also meant development of score
following and learning methods that are not necessarily
dependent on structure. This has led to a flexible scorefollowing method in which dependence on temporal structure and on previous rehearsal information can be
parametrically controlled.
2. THE ANATOMY OF PERFOlANCE
2.1 Organizing one's own Contribution
Live music performance comprises three major parts:
Perception and cognition of external controls.
Organizing one's own performance.
Learning from the experience.
Although computer synthesis has traditionally specialized
in organizing a performance, it provides a very poor model.
Live music performance is decidedly more procedural, and
many of its aspects remain unevaluated until the last possible moment in time. The chronological value of a 2-beat
duration, for instance, is determinable only near the end of
its performance. This means that the processes which control time-sensitive parameters (e.g. crescendo over several
beats) must remain active throughout, and are susceptible
to outside influences (changing tempo, etc.) at any time. In
order to construct a synthetic performer, we must first
understand the anatomical principles that drive real performers.
275
ICMC '85 Proceedings
