ï~~Synthetic Rehearsal: Training the Synthetic Performer Barry Vercoe & Miller Puckette Experimental Music Studio Media Lab, M.I.T. 1. ABSTRACT Computer tracking of live instruments aims to understand the dynamics of live performance well enough to replace any member of an ensemble by a synthetic performer (computer model) so that the others cannot tell the difference. Modelling the performer experience has 3 major parts: listen (extract temporal and other cues from the other players), perform (organize a synchronized and suitably matched performance), and learn (remember enough of each experience to benefit future encounters). While encouraging progress has been made on parts 1 and 2 (Vercoe, 1984), methods to date have relied exclusively on highly-sensitive live tracking. There has been no learning or training capacity to allow the benefits of previous experience. The synthetic performer is essentially "sight reading every time" on the concert stage. This paper outlines a method of integrating past and present experience into new strategies of rehearsal and improved performance. During a performance run, past or learned information is used to sensitize the perceptual and cognitive components, giving them a bias towards certain expected live input behavior. Eventual pitch contour matching and durational best fit are thus made easier and more robust. As before, rhythmic elasticity, and both pitch and rhythmic fault tolerance, are a necessary part of a practical performance system. 1. BACKGROUND AND OVERVIEW During 1983-84, research was conducted jointly at the Institut de Recherche et Coordination Acoustique/Musique in Paris, France, by Barry Vercoe (MIT) and Lawrence Beauregard (flutist of the IRCAM Ensemble Intercontemporaine). The objective was to understand the dynamics of live ensemble performance well enough to replace any member of a group by a synthetic performer (i.e. a computer model) so that that the remaining live members could not tell the difference. This aimed to break clear of the "music-minus-one" syndrome that has characterized tape and instrument pieces of the past, and to recognize the machine's potential not as an amplifier of low-level switches and keys, but as an intelligent and musically informed collaborator in live performance. The research was carried out using standard repertoire (flute sonatas by Handel and W.F.Bach) in which the flute part was played live and the responding harpsichord part was "performed" by a strictly synthetic accompanist. The motivation for the work was an IRCAM commission for Vercoe to compose a work, Synapse, for Larry Beauregard and the 4X real-time audio processor. Proof of how well the computer can behave as chamber music player in both traditional and contemporary contexts was then given in public demonstration at the 1984 International Computer Music Conference (Vercoe, 1984). The live demonstration routinely showed a facile ability to track and remain in sync with wildly changing tempos. It also showed an ability to deal with heavy doses of live performer errors, sloppy rhythms and general distortion that would likely throw off a live accompanist. However, in these early implementations there was no data retention from one performance run to another: there was no performance "memory", and no facility for the synthetic performer to learn from past experience. The synthetic performer was essentially sight reading on the concert stage every time. The aim of subsequent research has been to conceive and model a working representation of music rehearsal and music learning. This has meant rethinking the music abstractions so as to more adequately represent strongly structured scores, and to more easily model those elements of cognition and learning that depend heavily on structure. However, since many contemporary scores are only weakly structured (e.g. unmetered, or multi-branching with free decisions), it has also meant development of score following and learning methods that are not necessarily dependent on structure. This has led to a flexible scorefollowing method in which dependence on temporal structure and on previous rehearsal information can be parametrically controlled. 2. THE ANATOMY OF PERFOlANCE 2.1 Organizing one's own Contribution Live music performance comprises three major parts: Perception and cognition of external controls. Organizing one's own performance. Learning from the experience. Although computer synthesis has traditionally specialized in organizing a performance, it provides a very poor model. Live music performance is decidedly more procedural, and many of its aspects remain unevaluated until the last possible moment in time. The chronological value of a 2-beat duration, for instance, is determinable only near the end of its performance. This means that the processes which control time-sensitive parameters (e.g. crescendo over several beats) must remain active throughout, and are susceptible to outside influences (changing tempo, etc.) at any time. In order to construct a synthetic performer, we must first understand the anatomical principles that drive real performers. 275 ICMC '85 Proceedings
Top of page Top of page