Page  00000001 Automatic Discovery of Right Hand Fingering in Guitar Accompaniment Ernesto Trajano, Maircio Dahia, Hugo Santana and Geber Ramalho Centro de Informatica (CIn)-Universidade Federal de Pernambuco (UFPE) Caixa Postal 7851-CEP 50732-970-Recife-Pernambuco-Brazil {etl,mlmd,hps, glr}@cin.ufpe.br Abstract The analysis of musical interpretation, an important research topic in Computer Music, is usually focused on the study of musical dimensions like harmony and melody for piano music. Despite its obvious importance in expressive performance, rhythm is somewhat neglected as well as guitar music. This paper introduces a novel algorithm for the automatic discovery of right hand fingering in guitar accompaniment. This algorithm introduces the notion of hand position and uses the idea of minimizing the costs associated to each transition between hand positions as its basis. The tests we made showed that it correctly fingers all examples we used. 1 Introduction It is common sense that playing music in the exact way it is written in the score results in a mechanical and uninteresting succession of sounds. To make the written music interesting, the musician is required to make variations on low level musical parameters, such as: local tempo (accelerandi, ritardandi, rubato); dynamics; notes articulation (staccati, ligatures, etc.); micro-silences between the notes, etc. (Widmer 1998). Several researchers stress the importance, as well as the the difficulties, of studying this phenomenon, also known as expressive performance (Sundberg, Friberg, and Fryd6n 1991; Desain, Honing, and Timmers 2001; Widmer, Dixon, Goebl, Pampalk, and Tobudic 2003). These researches, in general, focus on the study of the relationship between musical dimensions (such as harmony and melody), low level parameters previously mentioned, and the expressive performance phenomenon. With some exceptions (MMM 2003; Dixon 2000), the role of rhythm in expressive performance has been somewhat neglected so far, despite its intuitive importance. Moreover, the research is almost exclusively devoted to the piano. Instruments, like acoustic guitar, and the music composed for them, have not been studied. In the case of studying how rhythm is used by the guitar player in accompaniment, the right hand of the performer is the immediate focus of attention, since this hand is responsible for rhythm generation (in right handed players). This paper presents an algorithm for automatic discovery of right hand fingering in guitar accompaniment. More precisely, given a MIDI guitar output, this algorithm must find out which (right hand) fingers may have been used. It is part of a research project that intends to study guitar rhythm in the accompaniment of Brazilian Popular Music (MPB), such as Bossa Nova and Samba. The fingering discovered by this algorithm, together with other metadata, is being used as the input of machine learning systems able to extract rhythmic patterns and induce general rhythmic accompaniment rules. Our algorithm follows the same principles of Sayegh's algorithm for finding the fingering of the guitar player's left hand (Sayegh 1991). The remainder of this paper is organized as follows: in Section 2 we discuss the motivation of this work. In Section 3, we describe the right hand fingering problem and the algorithm we designed. In Section 4, we describe and explain the set up we used for the algorithm's evaluation. In Section 5, we show the results of the algorithm's evaluation. And finally, in Section 6, we present some conclusions. 2 Motivation Expressive performance is guided by the musicians intuition, by some interpretation rules, and by some sort of analysis of the musical structure. There are many open questions about guitar and its music, despite its wide use and vast repertoire, specially non-classical accompaniment guitar music. Examples of typical questions we are interested in are: Are there rhythmic cells or patterns that are preferred by a certain performer or required for a certain music style? In which situations do they appear in the song? Do others dimensions (tempo, dynamics, melody, harmony, for instance) Proceedings ICMC 2004

Page  00000002 have any influence in rhythm generation and vice-versa? For answering these types of questions, we set up a project having three phases: data acquisition, data preprocessing and preparation, and knowledge extraction, as proposed in data mining projects. In the first phase, we used a MIDI acoustic guitar connected to a computer. In the second phase, we prepare the data, removing any existing noise and extending the initial MIDI representation with some relevant information (metadata). In the third phase, yet to be started, we will use techniques and algorithms from fields such as Machine Learning, Data Mining and Pattern Matching to extract knowledge from the prepared data. The algorithm described in this paper is part of the second phase of the project. 3 The Right Hand Fingering Algorithm Playing a song on guitar consists of two different tasks: choosing where the notes will be played and actually playing them. With the left hand, the guitar player selects the strings, the fret stop to be pressed (if any), and a certain combination of up to four of the fingers. The actual number of fingers depends on the number of notes that should be played and how these notes will be distributed in the fretboard. With the right hand, the player uses some combination of the five fingers in order to pluck the strings that were selected in the previous operation. The combination of fingers selected in each operation is called fingering. The problem we are interested in consists of automatically finding an appropriate right hand fingering for a given song. Although the left hand fingering discovery has already been studied (Sayegh 1991; Cabral, Zanforlin, Lima, Santana, and Ramalho 2001), up to our knowledge there is no algorithm that deals with the right hand fingering. It is important to notice that both left and right hand tasks have some degree of complexity: considering the right hand only, each note can be played by all fingers, but the little one. So, for a monophonic song with n notes, the simplest case, the number of different fingerings is 5n. In the worst case (five voices polyphony), this number yet increases to 120n possible fingerings. To automatically discover the left hand fingering, our algorithm follows a similar approach used by Sayeg. As described in (Sayegh 1991), each transition from a fingering to the next is assigned a cost. This cost represents the amount of "effort" required to change from fingering Fi to fingering Fi+1. This cost is built based on the combined difficulty/uniformity of a transition. Each possible fingering is represented as a node in a graph, whereas the costs are the edges weights. The fingering problem can then be reduced to the discovery a path that minimizes the overall cost. Before describing the algorithm itself, it is necessary to explain and define some concepts. First, we adopted the following representation for the fingers: T for thumb, F for fore finger, M for middle finger, R for ring finger, and L for the little finger. A sequence of notes N is the sequence of n notes (ni,..., In), and S is the set of strings used to play each note in N. Other concepts are defined as follows. Definition 1 Hand Position (hp) is the position of the right hand fingers regarding the string or strings they are about to pluck. It is represented as a 6-tuple, where each position in the tuple represents a string, being the first one string E2, the second string A2, and so on. For instance, to play the C major chord shown in Figure 1, hand position (x, T, F, M, R, x) is required, where x represents the absence of a finger. M R Figure 1: An illustration of a C Major chord (right hand fingers in squares). It is also possible that two or more different fingers play on the same string, like in solos. To play two consecutive notes on string G3, for instance, a possible hp is (x, x, x, 2, x, x). Definition 2 Set of Hand Positions (HP) is the set that represents all relevant hand positions hp. Definition 3 Sequence of Best Hand Positions (besthp) is a sequence of n hand positions, (besth,,...., besth). It is the result of the right hand fingering algorithm. Definition 4 Transformation Cost (T(hpi, hpi+1)) is a function that establishes a cost to change from hand position hpi to hand position hpi+1, where hpi, hpi+1 eG H Definition 5 Application Cost (a(hp, s)) is a function that computes the cost of applying hand position hp e G P to element se S. The algorithm that computes the sequence besthp is described in pseudocode as follows 1 foreach siG eS and besthp. e BestHP 2 foreach hpj e HP 3 a(hpj, si) min [a(hpj, si-) + T(hpj, hp)] hp,s'HTP 4 besthpi argmin[cijhp, si)] hp G'H P Proceedings ICMC 2004

Page  00000003 In our case, just as in Sayegh, each transition is assigned a cost. These costs were defined taking into account the following criteria: easy of play, flexibility, and agility. By ease of play, we mean that the transitions that preserve the fingers' positions as much as possible have lower costs. By flexibility, we mean that transitions that take into account future ones, by somehow preparing the proper fingers that will be used, have lower costs. By agility, we mean that transitions that are more suitable for solo passages have lower costs. It is important to say that these costs, as well as the set HP, also serve to model anatomical and cultural restrictions. The best sequence, then, is the one with the smallest cost possible. To compute the best sequence of transitions (line 4 in the algorithm), we used Viterbi's algorithm (Viterbi 1967). Using dynamic programming techniques, its complexity is O(n(7P)n), where n(H-P) is the number of elements of 7-P and n is the number of notes of the song. 4 MPB Guitar Fingering Before evaluating the algorithm, we have set up the algorithm's inputs: the set of hand positions HP, and functions 7 and a. This set up reflects our initial motivation, i.e., using the fingering discovered by the algorithm, together with other metadata, as the input of machine learning systems able to extract rhythmic patterns and induce general rhythmic accompaniment rules for MPB guitar music. Thus, set H-P was built by a simple observation of the fingerings commonly used by MPB guitar players. The bass line is always played with T and the rest of the chord with other fingers, usually in the following order: F for the note immediately above the bass, M for the note immediately above the one played by F, and R for the note immediately above the note played by M. The resulting set is formed 20 different hand positions, as can be seen in Figure 2. -P = {(T, x, F, M, R, x), (T, x,, F, M, R), (x, T, F, M, R, x), (x,T, x, F, M, R), (xx,, T,F, M, R), (T, x, x, 1, R, x), (2, T, 2,, R2), (2,, T,, 1, R, ), (T, 2, 2, 2, 1, R), (x, T, x, x, 1, R), (x, x, T, x, 1, R), (T, x, x, F, 2, x), (x, T, x, F, 2, x), (Xx,,T, F, 2, x), (T, x, x, x, 1), (x, T, x, x, x, 1), (x, x,, T,,, 1), (T, x, x, x, F, 2), (x, T, x, x, F, 2), (x, x, T, x, F, 2)} Figure 2: Set 7-(P used in the algorithm's evaluation. The function 7 (transformation cost) was designed in order to avoid hand position changes. It punishes with higher costs any transition and benefits hand position maintenances with lower costs. The application cost function a was very simple: it punishes with an infinite cost (the highest cost possible) hand positions that do not use the string in which the note in analysis was played, and with zero cost (the lowest cost possible) the opposite situation. 5 Algorithm's Evaluation In order to evaluate the algorithm, we tested it with in 13 MPB songs, all played by two different professional musicians on a MIDI guitar. The songs' titles are: Bim Bom, Eu Sei Que Vou Te Amar, 0 Barquinho, Samba de uma Nota So (One-note Samba), Desafinado, Garota de Ipanema (Girl from Ipanema), Sd Dango Samba, A Felicidade, Insensatez (How Insensitive), Wave, Corcovado, Tarde em Itapod and Chega de Saudade, most of them composed by Tom Jobim1. In Figure 3, it is possible to see part of the system's output. It is the result of the right hand fingering for the beginning of the song Insensatez, as played by player 1 (lower string is on top). E TF - A - D FG MB R -F -M -R -F--- [...] -M--- E ---------------------- Figure 3: Right hand fingering for the beginning of the song Insensatez. Figure 4 depicts the result of the right hand fingering for the beginning of the song Eu Sei Que Vou Te Amar, as played by player 2. A -- D FG MB R -T--T -T---T [... ] -M -R E ---------------------------- Figure 4: Right hand fingering for the beginning of the song Insensatez. The players have, then, analyzed the system output in order to find any error or unacceptable fingering (anatomically incorrect fingerings, for instance). With the set up described 1All scores were obtained from (Chediak 1990). Proceedings ICMC 2004

Page  00000004 in Section 4, the algorithm has correctly fingered all songs, that is, all fingerings the algorithm has discovered are acceptable, according to the guitar players. 6 Conclusion This paper presented a novel algorithm for automatically discovering right hand fingering in guitar accompaniment. It is part of a research where the influence of rhythm in the phenomenon of expressive performance is being studied, in particular the influence of rhythm in MPB guitar accompaniment. It is important to stress that, although the algorithm was tested having a well-defined set of restrictions, it is sufficiently general to find the fingering for other kinds of guitar music. It is only necessary to change the set of typical hand positions, which may vary according to cultural or stylistic characteristics. Although the algorithm was designed with a very well defined goal (provide metadata for a particular research), it could certainly be used for other purposes. Such algorithm can be used, for instance, for educational purposes in Instrument Performance Systems (IPS). These kinds of systems simulate a human instrumental performance, showing it on a virtual instrument on the computer screen (Cabral, Zanforlin, Lima, Santana, and Ramalho 2001). References Cabral, G., I. Zanforlin, R. Lima, H. Santana, and G. Ramalho (2001, Jul-Ago). Playing along with D'Accord Guitar. In Anais do VIII Simp6sio Brasileiro de Computacdo Musical (SBCM'O1), Fortaleza, pp. 858-866. Sociedade Brasileira de Computacio (SBC). Chediak, A. (Ed.) (1990). Songbook: Bossa Nova, Volume 1-5. Rio de Janeiro: Lumiar Editora. Desain, P., H. Honing, and R. Timmers (2001, Nov.). Music performance panel: NICI/MMM position statement. In MOSART Workshop on Current Research Directions in Computer Music, Barcelona, Spain. Dixon, S. (2000). A lightweight multi-agent musical beat tracking system. In Pacific Rim International Conference on Artificial Intelligence, pp. 778-788. Garcia, W. (1999). Bim Bom: A Contradigdo sem Conflitos de Jodo Gilberto. Editora Guerra e Paz. MMM (2003). Music, mind, machine group. http: //www.nici.kun.nl/mmm/. Last access date Mar 12 2004. Sandroni, C. (1988). O olhar do aprendiz: ObservaC6es sobre a pritica do violao popular no brasil, fartamente documentada por exemplos em partitura. Unpublished manuscript. Sayegh, S. L. (1991). Fingering for string instruments with the optimum path paradigm. In P. M. Todd and D. G. Loy (Eds.), Music and Connectionism, Cambridge (MA), pp. 243-255. The MIT Press. Sundberg, J., A. Friberg, and L. Fryd6n (1991). Common secrets of musicians and listeners: an analysis-by-synthesis study of musical performance. In P. Howell, R. West, and I. Cross (Eds.), Representing Musical Structure, pp. 161-197. Academic Press. Viterbi, A. J. (1967). Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory 13(2), 260-269. Widmer, G. (1998). Applications of machine learning to music research: Empirical investigations into the phenomenon of musical expression. In R. S. Michalski, I. Bratko, and M. Kubat (Eds.), Machine Learning, Data Mining and Knowledge Discovery: Methods and Applications. Chichester (UK): Wiley & Sons. Widmer, G., S. Dixon, W. Goebl, E. Pampalk, and A. Tobudic (2003). In search of the Horowitz factor. AI Magazine 24(3), 111-130. Proceedings ICMC 2004