Page  00000165 HARMONIC, MELODIC, AND FUNCTIONAL AUTOMATIC ANALYSIS Placido R. Illescas, David Rizo, Jose M. Inesta Dept. Lenguajes y Sistemas Informiticos Universidad de Alicante, E-03071 Alicante, Spain ABSTRACT This work 1 is an effort towards the development of a system for the automation of traditional harmonic analysis of polyphonic scores in symbolic format. A number of stages have been designed in this procedure: melodic analysis of harmonic and non-harmonic tones, vertical harmonic analysis, tonality, and tonal functions. All these informations are represented as a weighted directed acyclic graph. The best possible analysis is the path that maximizes the sum of weights in the graph, obtained through a dynamic programming algorithm. The feasibility of this approach has been tested on six J.S. Bach's harmonized chorales. 1. INTRODUCTION The musical analysis is a mean to better understand the thought of the composer when creating a piece. A musician must perform a good musical analysis to execute a correct interpretation of a work. The melodic, harmonic, and tonal function analyses are the basic elements in order to achieve an optimal musical analysis. Besides the interpretation, there are many applications of automatic analysis to diverse areas of music: education, score reduction, pitch spelling, harmonic comparison of works, etc. The automatic tonal analysis has been tackled under different approaches and objectives. Some works use grammars to solve the problem [6]. There are probabilistic models like that in [3] and others based on preference rules or scoring techniques [5]. Taube [4] solves the problem by means of model matching. A more comprehensive review of these works can be found in [1]. Our objective is not only to obtain a high percentage of correct analyses, but also to describe in a human-readable way the reasons why the system has chosen an analysis. The system described in this paper uses both a rule engine approach and a ranking method, along with a dynamic programming algorithm, to describe analytically a musical piece. The complexity and nature of the variables that must be utilized to perform a musical analysis are different among genres. However, the general scholastic method can be explained using the harmonized chorals of Bach. The presented work is devoted to test the ability of the proposed approach and has been founded on the baroque music rules and tested using some Bach chorals. 1 Supported by the projects GV06/166 and CICyT TIN2006-14932 -C02, partially funded by EU ERDF. {drizo, inesta} 2. METHODOLOGY In order to analyze a musical piece the system performs the following steps. Firstly, a melodic analysis is performed. The notes are tagged either as harmonic tones for those belonging to the current harmony at each time, or as non-harmonic tones for those ornamental notes. There can be notes with different possible analysis that will be disambiguated later. Secondly, a vertical analysis of the piece is performed: after segmenting each bar into a number of time windows, all possible chords are obtained from the notes in each individual window. The third step chooses which of the 24 different tonalities are candidates to be the central key in each window, given the accidentals of the notes involved. The fourth step has all the possible chords and tonalities for each window as input. From these data, a weighted acyclic directed graph (wDAG) organized by layers is built. Each layer represents a window. The nodes of the graph correspond to chords with tonal functions in a tonality. The edges of the graph contain the valid progressions between the nodes in successive layers, weighted according to the importance of those progressions in order to establish a tonality. The final step uses a dynamic-programming approach to compute the best path along the graph, discovering the best tonality and tonal function sequence. The output of this step is the Roman numeral analysis along with a tonality segmentation. Finally, those notes still having a multiple melodic analysis are disambiguated. Next, each of these steps are described more indepth. 2.1. Melodic analysis The melodic analysis tags each note as harmonic or nonharmonic-tone (NHT) based only on the melodic information, leaving aside any harmonic vertical information. Since a given note can have multiple different analysis, a confidence value is also computed. The NHT tags are given based on the type of ornament: neighbor, etc. The JBoss Drools 2 rule engine framework has been used to implement a series of rules written by the authors based on the music theory applicable to the baroque period. Those rules are based on meter, pulse, duration, and pitch interval information. Before detailing the rules some terms must be defined. 2 165

Page  00000166 Definition 2.1 rd(ni) =durationr(ni) /durationr(beat) The relative duration function determines the ratio between the duration of a note, ni, and the duration of a beat. Definition 2.2 ratio(ni) - rd(ni) rd(ni) rd(ni-1) rd(nii) The ratio function is used to compare the relative duration of a note with its next and previous notes. Definition 2.3 pitchName(ni) It defines the position of the name of the note ni in the ordered set {fC, D, E, F, G, A, B}. Definition 2.4 pitchClass(ni) Order of the note name including its accidental in within the octave. (e.g. pitch(Eý4) 3). Definition 2.5 pitch~ntv(i, ri+1) = d.s It computes the pitch interval between two notes ni and ni+ 1, where d pitchName(ni+1) -pitchName ni) + 1 and s pitchClass(ni+1) - pitchClass ni) specified with a resolution of two decimals. A positive value indicates an ascending interval. E.g, the unison interval pitchIntv(ni, Tii) 1.00 and the tritone is 4.06. Definition 2.6 prevI(ni) The previous interval is the interval between a note ni and its predecessor. prevI(ni) pitchIntv(nri-, Tii) Definition 2.7 nextl(ni) The next interval is the interval between a note ni and its successor. nextl(ni=) pitchIntv(ni, ri+1) Definition 2.8 subbeat(ni) It is a boolean function that is true when the note onset does not match the exact position of the beat. Definition 2.9 strong (ni) For quaternary meters a note is strong when its onset is located in the first or third beat of the measure. In ternary meters, it is strong if and only if it onsets in the first beat. For the compound meters, this function can be computed from these two situations. Definition 2.10 tied (ri) It is a boolean function that is true whenever the note m, is tied to the previous note using a prolongation tie. A NHT is tagged that way depending on its location in strong or weak beats, its duration, and the intervals it defines from its previous note and to the next one. There are 38 different rules that can be summarized in a simplified version in Table 1 3. Each rule in the whole set assigns a confidence to the harmonic tag selected from the following catalog: 's' for sure, 'h' for high, 'in' for medium, and '1' for low. This confidence value will be useful in next steps. 3 The whole set of complete rules has been stored in NHT kind rules appoggiatura strong A prevl = 1.00 A nextI C {-2.01, -2.02, +2.01} suspension strong A tied A prevI = 1.00 A (nextI E {-2.01, -2.02, +2.01}) passing tone (-istrong) A (ratio < 1) A ((prevI G {-2.01, -2.02}) A (nextI G {+2.01, +2.02})) V ((prevI C {+2.01, +2.02}) A (nextI C {-2.01, -2.02})) neighbor tone -istrong A (ratio < 1) A ((prevI C {+2.01, +2.02} A nextI C {+2.01, +2.02}) V (prevI C {-2.01, -2.02} A nextI C {-2.01, -2.02})) Table 1: Summary of melodic rules for a note ni. The parameter ni of the functions has been omitted to reduce space. Actually, a passing tone starting in a weak beat or sub-beat is allowed using a low confidence. 2.2. Chord extraction For each of the bars, the duration of the shortest figure or rest is selected as the time resolution for that bar. Then, the bar is divided into windows of that duration for all the voices in the score. This way, the musical piece is sliced into a sequence W of windows. For each window w C W, a set S,, of all notes sounding in it and a set Cs, of chords from all combinations of notes in S,, are built. Two adjacent notes of a chord ci C Cu, must hold (1): V note -1, notej C ci, (1) pitchIntv(notej_1, rnoted) e {3.s, 5.s} A backtracking scheme computes all combinations of note names without octave in S,, leaving out those orderings containing adjacent notes not holding that condition, and storing in C,, the set of valid chords. Note that the backtracking process reorders all notes in the chords: even if the original positions of the notes suggest an inversion, this process deletes it and returns the root note of the chord to the lowest note position. 2.3. Accidental analysis The accidental analysis filters out from the 24 possible tonalities those that are not feasible given the accidentals of a set of notes. This filtering is performed for each window. The following definitions will be used to describe how this step of the analysis works. Definition 2.11 mode(k) C {Maj, miri} The mode of a tonality can be major or minor. Definition 2.12 expectSem(g, m) The values of this function are detailed in table 2 as the set of valid semitone intervals from the first degree of a scale to the specified degree g given the mode m of a tonality. Definition 2.13 actualSem(ni, k) Its value corresponds to the actual number of semitones from the tonic of k to ni. 166

Page  00000167 Definition 2.14 deg(ni, k) It is defined as the degree of a note ni given a tonality k. Definition 2.15 isDiatonic(ni, k) This boolean function is computed as actualSem(ni, k) E expectSem(deg(ni, k), mode(k)) We introduce here the set K, of all valid tonalities for the set of notes in a given window, S,. A tonality k e K, if isDiatonic(ni, k) is true Vn e S,. expectSem I Maj 0 mmin 0 II I II 2,(1) 4 2,(1) 3,(4) IV v I viI vii 5 7 9 11 5 7 9,8 11,10 Table 2: Diatonic scales for the definition (2.12). The cells contain all the valid semitone differences from the tonic of the specified degree, the (1) value represents the Neapolitan, the (4) represents the Picardy ending. 2.4. Tonal functions Music is based on a continuous flow of tensions and relaxations. In tonal music, this is performed by adding dissonances to the tensions that are resolved to successive chords without dissonances (i.e., chords with tonic function). When passing from musical instability to stability, a transitory function named subdominant is included. Under our point of view, all notes, all chords, all music fulfill this principle. This way, a relation between tonal functions based on tensions and relaxations is established. Table 3 shows the relationship between chord degrees and tonal functions. The degree of the chord is extracted using the function g defined in definition 2.16. Definition 2.16 g,,,k = deg(root(ci), k) The degree of a chord ci is equal to the degree of the root of the chord. Chord deg. Tonal func. Chord deg. Tonal func. I T V D,T II,IV S VI S,T III S,T,D VII S,D Table 3: Possible tonal functions for each degree. 'T': tonic, 'D': dominant, 'S': subdominant. The V degree has a tonic function to allow half cadences that move the tonal relaxation to the dominant region. The III and VI degrees can assume different relations depending on the harmonic neighbours. Mediant chord The III degree chord is the one with more possibilities of analysis. Depending on the tonality mode, the chord will have a different tonal function. The mediant chord in a major tonality is minor perfect, where the fifth of the chord is the leading note of the tonality. A complete chord can be considered as dominant with a high level of confidence when the next chord is a tonic function. When the mediant chord is incomplete, if the previous chord is a dominant one, it can be tagged as tonic with a high level of confidence because the tension generated by the previous dominant resolves to that tonic. In addition to the dominant and tonic functions, the third degree can behave as subdominant. To the best of our knowledge, this treatment is not found in the music theory literature, however, we attribute this function to it because we consider the tonal functions as levels of tension and musical stability. From this point of view, and considering three levels of tension, when the third degree is located between a tonic and a dominant, it should be given a subdominant transitory function. VI chord As stated above, the VI chord can be assigned two tonal functions depending on its harmonic surroundings. The VI degree will be a subdominant function when the previous chord is a tonic or a subdominant. In this case, the next chord has no impact in this decision. Otherwise, if the previous chord is a dominant function, this VI chord is considered as tonic. 2.5. Weighted Acyclic Directed Graph As outlined in section 2.2, the piece is segmented in W time windows and, from each window, w E W, the sets of chords C, and valid tonalities K, are built. Now, following the principles exposed in 2.4, the set F,k (ci) of all feasible tonal functions (f) for all k e K, and all i E Cw is computed. The wDAG is constructed using these data. 2.5.1. Graph construction Let's define the wDAG as G = (V, E, D), where V is the set of nodes and each v e V is labeled with the values provided by F,,k(ci). Thus, we will use the labels (tonal functions) f to represent the nodes, instead of v. E C V x V is the set of edges, and D is the set of weights that are computed by a weight function d: E -- R. The vertices are partitioned into \W disjoint subsets Vi, 1 < i K< \W, in a way that, if(fa, fb) E E, then fa e Vi and fb e Vi+l. Each subset Vi includes the nodes from a given window w. This way, the graph is structured as a sequence of layers, Vi, representing the course of time along the score. The definition of the weight function d is explained below in the cadences section. Cadences Cadences are points of relaxation that have the function of reaffirming the tonality. A cadence takes place when it concludes with the tonic function, and a half cadence when the target is a dominant or subdominant. Cadences and half cadences are summarized in Table 4. The sequence of cadences and half cadences is a hint towards the recognition of a tonality in a given point. A sequence of chords analyzed from two different tonalities yields two different cadence / half-cadence successions. Weight information is introduced in the graph for the edge linking nodes fa and fb in adjacent layers by means of the 167

Page  00000168 weight function d(fa, fb). The more conclusive a cadence is, the higher the weight is assigned to it. Each sequence of tonal functions will produce a different weight value 4. The path with the highest sum of weights will be selected as the most suitable one. Those weights have been established empirically. A negative value is assigned to reflect the tonal regression D -- S. Tonality changes are allowed only when a cadence in a new tonality is found. Not feasible progressions (those specified in section 2.4) are weighted with -oc. Cadence Tonal funct. Degrees Perfect authentic D-T V-{I,i} Imperfect D-T vii-I, viid->i Interrupted D--T V- {vi, VI} Plagal S--T {IV, iv, VI, vi, iid, ii} -- {I,i} Dominant He T-D, {I,i,II,ii,III,ii,IV,iv,VI,vi} S--D --{V,v,vii,viid} Subdominant He T-S, {I,i,ii,III,ii,IV,iv,VI,vi} S->S ->{II,IV,iv,VI,vi} Table 4: Cadences. 'Hc' stands for half cadence. The degrees in lower case represent minor chords. Subindex d is applied to diminished chords. 2.5.2. Best path computation Once the graph is constructed, the selection of the best path is reduced to the problem of the computation of the best path in a graph using dynamic programming [2]. The nodes visited in this path are taken for the best analysis. 2.6. Melodic analysis disambiguation After the best path computation, the system has selected which chord, tonal function, and tonality are the best for each window. The first step of the analysis (the melodic analysis step) left some notes with conflicting rules unable to classify them in harmonic or non-harmonic tones. Now, having solved the harmony of the whole piece, those notes are tagged as NHT when they do not belong to the current chord at each moment. 3. EXPERIMENTS As mentioned above, to test the system the harmonized chorals from J.S.Bach have been uused. The input of the system was transcriptions of the chorals in MusicXML format found at the Humdrum Kern Scores Site 5. Due to the lack of a digitally annotated corpus of these chorals, we focused the analysis of results on the chorals: choral #25, BWV-0269, BWV-0367, BWV-0400, BWV 2-6. We did so in order to be able to compare our results to those of a relevant work named MTW (Music Theory Workbench) [4]. A full report of the obtained results can be downloaded from our website quoted below. 4 The actual weights can be found at 5 Since two different analyses sometimes can be both valid we cannot give success percentages in order to compare to those reported by MTW. For our point of view, the MTW fails in some tonal function progressions and seems to make mistakes when analyzing alternative tonalities by not solving the chord cadence (e.g. BWV 2-6 at bar 3, beats 2-4). Our system corrects those errors by means of the cadence scoring. However, we must correct some errors the MTW does not make. Anyway, our system has failed only in two chords in the analysis of choral #25. 4. DISCUSSION AND CONCLUSIONS This paper has described a system to analyze automatically a score from the melodic and harmonic points of view, providing the tagging of each note as harmonic or not harmonic, the tonality changes and the Roman numeral analysis of each chord along with its tonal function. The presented system is comparable in results to those reported by MTW [4], but it has the advantage to be ready to work with monodic melodies only adding more possibilities of analysis at each layer of the graph. We are working on it to compact the output so that the chords that have been split by the windowing process can be merged again. Currently the system does not show the inversion of chords, the implementation of this feature is straightforward and is being developed. We have defined the conditions for the melodic tags: double neighbor tone, cambiatta, escape tone, and fux, and currently we are implementing them. That may correct some of the erroneous analysis the system outputs nowadays. Other important line of work is to improve the merging of the melodic analysis with the harmonic steps of the process. Modulation points are currently displaced in a ~one chord distance. This is due to the dynamic programming algorithm. Hopefully, this problem will be solved by the use of a harmonic rhythm and a modulation subsystem. Other current work is the creation of a larger corpus of tagged pieces to be able to learn the scoring of the tonal function progressions from that corpus. The next process to be tackled is the redefinition of rules for other genres. References [1] Jerome Barthelemy. Figured bass and tonality recognition. In ISMIR, 2001. [2] G. Brassard and P. Bratley. Fundamentals ofAlgorithmics. Prentice-Hall, 1997. [3] Christopher Raphael and Joshua Stoddard. Functional harmonic analysis using probabilistic models. Comnput. Music J., 28(3):45-52, 2004. [4] Heinrich Taube. Automatic tonal analysis: Toward the implementation of a music theory workbench. Comnput. Music J., 23(4):18-32, 1999. [5] David Temperley and Daniel Sleator. Modeling meter and harmony: A preference-rule approach. Comput. Music J., 23(1): 10-27, 1999. [6] Terry Winograd. Linguistics and the computer analysis of tonal harmony. MIT Press, Cambridge, MA, USA, 1992. 168