Page  340 ï~~The Artificially Intelligent Computer Performer and Parallel Processing Bridget Baird Box 5412 Connecticut College New London, CT 06320 bbbai@con nco I1. bitnet ABSTRACT The artificially intelligent computer performer is a program that enables a computer to listen to and accompany a live performer, reacting in real time to changes in the performer's tempo or mistakes in the performer's play. Parallel processing, via transputers on a Macintosh, is used to enable the computer performer to simultaneously listen to several live performers, match their inputs to the score, determine their positions, and then respond. Background A computer performer which is capable of playing intelligently with live musicians would not only be an extremely useful tool, but the process of designing such a performer would also teach us much about musical cognition. Several researchers (Dannenberg (1984), Vercoe (1984)) have worked on this problem. At the 1989 ICMC Baird, Blevins & Zahler (1989) demonstrated the artificially intelligent computer performer (AICP). The AICP listens to the input of a live performer, uses a pattern matching algorithm to decide on the performer's position in the score, and then decides on its response (tempo). The pattern matching algorithm looks at sequences of notes from the live performer and matches them to sequences of notes in the score. The algorithm also anticipates several kinds of mistakes that musicians are likely to make during a performance and uses these to make a better match. Since the ICMC demonstration the algorithm has been further refined and many of the performance parameters were tuned in order to achieve a better response by the computer. A major limitation of this earlier version of the AICP was that it could only listen to a single monophonic input. Clearly it would be desirable to have the performer listen to several inputs. The main problem in having the AICP do this was that of time. The AICP used roughly.04 seconds to make a pattern match. When more time was used to make a match (for example, by enlarging the scope of the ICMC 340

Page  341 ï~~pattern matching search) a definite musical degradation was noticed. Thus it seemed difficult to envision having the AICP listen to several performers, at least by using this model. Parallel Processing When musicians play in an ensemble they are obviously listening to several colleagues simultaneously; the human brain is massively parallel. Since the AICP attempts to emulate live musicians, it was a natural step to consider implementing parallel processing into the AICP. Most computers are sequential machines; parallel processing means that many processors work on a problem simultaneously. This technique has become increasingly important because of its speed and because it more nearly simulates human cognition. Issues such as the number of processors to be used, the power of each processor, configuration of the processors, and the speed and ease with which the processors communicate are extremely important. In musical applications parallel processing has been used in areas such as meter and rhythm perception, synthesis, and sound processing. For the AICP, parallelism could be used to process input from several live performers and could also be used to expand the number of possibilities for pattern matches in the score. Since the platform for the AICP is a Macintosh computer, and since parallel processing has recently become practical for implementation on this machine, introduction of parallel processing into the AICP became the next step in this project. Hardware and Software Setup The AICP is connected to several synthesizers and uses transputers for its parallel processing. The T800 transputer is a 32-bit single chip microprocessor from INMOS which contains its own memory and four links for communication. The transputer is an extremely fast microprocessor which contains a floating point unit, uses a variant of RISC architecture, and may have multiple processes running simultaneously. Up to four transputers may be put on a card and one or more cards inserted into nu-bus slots on the Mac. The transputers are configured by the user for communication with each other and with the Macintosh and thus create a parallel processing system for ICMC 341

Page  342 ï~~the Macintosh. Although transputers have been used extensively in Suns and PCs, use in Macs has been limited. The code that runs on the Macintosh is written in MPW C. The code for the transputers is written in a parallel version of Logical Systems C; it is separately compiled and sent to the transputers from the host Macintosh C program. As soon as the transputers receive their code, they start running. The Mac communicates with only a single transputer and it does this with driver calls; transputers communicate with each other via channels (four on each transputer). In order to have all transputers communicate with each other and with the Mac, fairly cumbersome routing code must be written. Parasoft produces a software package, Express, which greatly reduces this burden. However, the current version of Express added an intolerable cost to the communication time between the Mac and a transputer (on the order of.1 seconds) which was clearly unacceptable for a real time program. After numerous consultations with the people at Parasoft, the AICP now uses an alpha version of an update to Express which has greatly speeded up communication. So far (knock on wood) the system is operating without major bugs. Parallel Processing on the AICP The first task was to extensively reconfigure the AICP so that it would acknowledge input from multiple sources. It was decided that a MIDI channel would be the basic unit. The AICP accepts up to eight channels of information and will acknowledge up to eight transputers. A score is read into the program with standard MIDI format and channels are activated according to the score's specification. Each active channel is then capable, through menu choices, of accepting input from a live performer or file, of being played by the computer, or of being turned off. During a live performance there may be input from live performers via the synthesizers or from files. Inputs from files are actually played by the computer, but are treated as if they came from live performers. During a live performance the computer plays all remaining active channels. All live performances are recorded and saved by the program for replays of the performance, or to be used as input in a future performance. "Live" performances from files may be read in via standard MIDI format, or read in from files saved by the user. The program can play live performances, replay performances, or play selected parts of the score. Since the ICMC 342

Page  343 ï~~scores can be fairly large, the program does a preliminary pass through the score in order to dynamically allocate memory. If transputers are used, each transputer corresponds to one live performer channel. The score for this channel is sent to the transputer before the performance begins. As the performance progresses, musical input from performers, tempo changes, and other information is sent to the appropriate transputers. Each transputer does a pattern match to determine proper score position for the performer using that channel and then communicates the information back to the Macintosh. The Macintosh receives information from the transputers and then reconciles all of the information to decide on the appropriate computer response. Musicians, although they listen to all players, tend to follow one player; the choice of that player is determined by the importance of the player's instrument (eg. first violinist in a string quartet), the musical line of the player (melody or beat keeping), and confidence that the playing is correct. The AICP receives from each transputer not only an estimate of that performer's position in the score, but also a cost which quantifies the reliability of the estimate. This cost, the absence or presence of playing, and the importance of the instrument, are all used to assist the computer in deciding on its response. This aspect of the AICP project is in its infancy and will constitute the next major phase of research. A great deal of interesting musical exploration remains. Acknowledgments The author gratefully acknowledges the support of NSF for this project and also thanks three Connecticut College students, Christine Amorossi, Miriam Fendel, and David McClendon, for their invaluable work during the summer of 1991. Thanks also to Donald Blevins for his continuing assistance. References Baird, B., Blevins, D. & Zahler, N. (1989). The artificially intelligent computer performer on the Macintosh II and a pattern matching algorithm for real-time interactive performance. Proceedings of the 1989 ICMC, 13-16. Dannenberg, R. (1984). An on-line algorithm for real-time accompaniment. Proceedinas of the 1984 ICMC, 193-198. Vercoe, B. (1984). The synthetic performer in the context of live performance. Proceedings of the 1984 ICMC, 199-200. ICMC 343