Page  00000001 MUSICAL SCORE GENERATION IN VALSES AND ETUDES Dr. David Kim-Boyle University of Maryland, Baltimore County Department of Music 1000 Hilltop Circle, Baltimore, MD 21250, U.S.A. kimboyle@umbc.edu ABSTRACT The author describes a recent composition for piano and computer in which the score performed by the pianist, read from a computer display, is generated in real-time from a vocabulary of predetermined scanned score excerpts. The author outlines the algorithm used to choose and display a particular excerpt and describes some of the musical difficulties faced by the pianist in a performance of the work. 1. INTRODUCTION In Valses and Etudes, a recent work for piano and computer premiered at the 2005 Florida Electroacoustic Music Festival, the score performed by the pianist is generated in real-time from a variety of scanned score excerpts. These excerpts are taken from a number of existing works including Movement VI of Schoenberg's 6 Kleine Klavierstficke Op. 19, the Second and Third Movement of Webern's Variationen ffir Klavier Op. 27, several of Ravel's Valses Nobles et Sentimentales, John Cage's One 5, Chopin's Nocturne No. 6, Op. 15 Nr. 3, and Debussy's Prelude No. 10 ("La cathedrale engloutie"), Book One. A number of additional pieces are played by the computer, from pre-existing recordings, at several points in the work but are not called upon in the score generation process. The score selected for performance is conditioned by Markov chain probabilities and the actual score excerpt displayed for the pianist is determined through a Jitter patch. This excerpt is not fixed, but dynamically varies during the performance. Unlike previous works of the author in which the original source materials are extensively processed, in Valses and Etudes very little audio processing of the materials takes place. Rather, the musical complexity lies in the simultaneous performance of up to twelve or so pieces by the computer, and the interaction within this musical tapestry by the pianist. Aesthetically, the process recalls the mesostics of John Cage where pre-existing works are read through in a manner that creates new meanings from the amplification and omission of detail. [1] 2. SCORE GENERATION METHOD Unlike traditional methods of score generation in which algorithms generate a score from which the performers learn and then perform the piece, [2] in Valses and Etudes the score is generated in real-time from eight predetermined scanned score excerpts. How these excerpts are displayed and ordered is determined in realtime during the performance itself. This method of score generation recalls the musical dice games popular in the 18th century where precomposed measures were randomly assembled to form a score. The real-time generative process of Valses and Etudes extends the method considerably, however, as well as challenging traditional aesthetic norms of performance practice. The first page of each of the preselected scores was scanned, edited and saved as high-resolution jpeg files for further processing in the Cycling'74s Jitter environment. [3] Jitter is a set of external graphical objects for the Max/MSP programming environment which allows live video processing and other graphical effects to be integrated into the MSP audio platform. In Valses and Etudes, Jitter is used to frame particular score excerpts. This framing patch is illustrated in Figure la, where a small, random window is generated with the jit.lcd.object with values mapped to the alpha channel and then blended with the jpeg score file. Large Jitter matrices are used to maximize the screen resolution. A typical result, as view by the pianist, is illustrated in Figure lb.

Page  00000002 In Valses and Etudes, source scores are selected with a first order Markov Chain algorithm where probabilities are determined with eight multisliders. The interface for this is illustrated in Figure 3. a) b) Figure 1. a) Framing in Jitter, b) Typical framing result. An obvious feature of Figure lb is that the image needs to be rotated 90 degrees clockwise. This is not done in Jitter in order to maximize the on-screen display size of the image. Rather, for performance, the display from which the pianist reads is simply turned on its side. This is not as unwieldy a solution as it might appear however, especially if a laptop computer is used. Other possible solutions might involve the use of headmounted displays. [4] Another particularly important consideration is the need to obtain the maximum possible clarity of image given that the screen resolution will not be able to match the print resolution of a traditionally printed paper score hence the relatively large Jitter matrices. The windows mapped by Jitter are not fixed, as implied in Figure Ib, but dynamically change during performance. Their size, rate of size change, trajectory across the score page, and speed of movement are all definable. Each of these parameters can also be randomized. Clearly, different rates of change will produce qualitatively different interpretations. Trajectory paths for eight possible score selections are defined with one lecd object and can be determined during or prior to the performance. The interface used to define these values is shown in Figure 2. The author has also experimented with Jean-Baptiste Thiebaut's trajectory object. [5] This object requires a different implementation but facilitates geometrical trajectories which are not easily obtainable otherwise. Unfortunately, however, it does not allow multiple trajectories to be simultaneously defined with the one object which yields a more cumbersome interface. Figure 3. Markov chain interface. In Valses and Etudes each multislider in the above interface represents one score performed by the pianist. The eight sliders of each multislider represent the probability that another score will follow. For example, the top left multislider determines the probabilities that another score will follow an instance of the sixth movement of Schoenberg's 6 Kleine Klavierstficke. There is around a 50 percent probability that the Schoenberg score will be followed by the Schoenberg score or the Cage score, a slightly higher probability that it will be followed by the two Webern scores, and a decreasing probability that it will be followed by the Ravel, Debussy, or Chopin scores. The engine behind this interface system is based on the Max prob object. In Valses and Etudes one prob object is used to store all 64 possible transitions. The Markov chain process is able to lend the work a spontaneity that remains nevertheless musically coherent. 3. PERFORMANCE DIFFICULTIES One of the obvious difficulties the pianist faces in performing Valses and Etudes is the need to learn eight individual pieces. This prospect is made somewhat less daunting given that only the first page of each score need actually be learned and that the pieces chosen are not too technically demanding. The fact that the pianist has no knowledge of which piece will follow another affects their interpretation in a more significant way. Winkler notes a similar issue with his real-time score generation technique. [6] That the windowing process might also be different from score to score, for example with different trajectories, window sizes and speeds, adds another layer of complexity which further disrupts interpretative continuity from work to work. The most obvious challenge for the pianist, as mentioned, is the effect of the windowing process on the Figure 2. Window size/speed/trajectory interface.

Page  00000003 interpretation of each of the source pieces. This is particularly challenging as it goes against much in the way of traditional performance practice. To be faced with a score that dynamically changes during performance or where only a fragment of the score may be visible, or even to be faced with a score fragment that moves from the bottom of the page to the top, forces the pianist to abandon, to a certain extent, traditional interpretative concepts of form and development. The most musically effective solutions have involved simply performing coherent segments of a score in short phrases. This lends the pianist's performance a somewhat episodic quality which nevertheless blends seamlessly with the constantly varying computer part. 4. FUTURE DEVELOPMENT In Valses and Etudes the order in which the piano works are played back by the computer is fixed. In more recent work the author has begun experimenting with more open form Max patches that determine work selections based on an extension of the first order Markov chain process outlined in Section 2. Also of interest is the incorporation of genetic algorithms that will condition the score selection process. These techniques are being employed in a new work for cello and MaxMSP/Jitter. The author is also experimenting with the use of MIDI files rather than actual recordings. This could enable interesting transformations to occur between source pieces. A similar process has been undertaken by Muenz in his work "The Self Composer" [7] for sight-reading oboist and computer. Also being explored is a more responsive musical process whereby the performer is not simply responding to events determined by the computer but becomes more of a musical instigator. [6] This will necessarily involve gesture recognition [8][9] and score following [10][11] techniques to enable the computer to accurately determine which scores are being played and how they are interpreted. Other methods of displaying scores using various color-coding techniques are also being explored. 5. REFERENCES [1] Cage, J. I-IV. Cambridge, MA: Harvard University Press, 1990. [2] Maurer, J. "A Brief History of Algorithmic Composition," available at <http://ccrma www.stanford.edu/-blackrse/algorithm.html>. Winter 1999. [3] Zicarelli, D. "An Extensible Real-Time Signal Processing Environment for Max," in Proceedings of the 1998 International Computer Music Conference, Ann Arbor, MI: International Computer Music Association, pp. 463-466, 1998. [4] Clay, A., T. Frey, and J. Gutknecht. "GoingPublik: Using Realtime Global Score Synthesis," in Proceedings of the 2005 Conference on New Interfaces for Musical Expression (NIME-05). Vancouver. Available at <http://www.nime.org>. [5] Thiebaut, J.-B. <http://www.mshparisnord.org/themes/Environme ntsVirtuels/00070887-001E7526>. April 2004. [6] Winkler, G. E. "The Realtime Score. A Missing Link in Computer Music Performance," in Proceedings of the 2004 Sound and Music Computing Conference. Paris. Available at <http://smc04.ircam.fr/scm04actes/P3.pdf>. [7] Anderson, C. "Audible Interfaces Festival," (trans. P. Castine), in Computer Music Journal, Vol. 26 No. 4, Winter 2002. [8] Camurri, A. and G. Volpe (Eds.) Gesture-Based Communication in Human-Computer Interaction. Berlin: Springer-Verlag, 2004. [9] Traube, C., P. Depalle, and M. Wanderly. "Indirect Acquisition of Instrumental Gesture Based on Signal, Physical and Perceptual Information," in Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03). Montreal. Available at <http://www.nime.org>. [10] Dannenberg, R. "An On-Line Algorithm for RealTime Accompaniment," in Proceedings of the 1984 International Computer Music Conference. Paris: International Computer Music Association, pp. 193-198, 1984. [11] Orio, N., S. Lemouton, and D. Schwarz. "Score Following: State of the Art and New Developments," in Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03). Montreal. Available at <http://www.nime.org>.