Page  00000482 Musical Tension Curves and its Applications Min-Joon Yoo and In-Kwon Lee Department of Computer Science, Yonsei University, Abstract We suggest a graphical representation of the musical tension flow in tonal music using a piecewise parametric curve, which is a function of time illustrating the changing degree of tension in a corresponding chord progression. The tension curve can be edited by using conventional curve editing techniques to reharmonize the original music with reflecting the user's demand to control the tension of music. We introduce three different methods to measure the tension of a chord in terms of a specific key, which can be used to represent the tension of the chord numerically. Then, by interpolating the series of numerical tension values, a tension curve is constructed. In this paper, we show the tension curve editing method can be effectively used in several interesting applications: enhancing or weakening the overall feeling of tension in a whole song, the local control of tension in a specific region of music, the progressive transition of tension flow from source to target chord progressions, and natural connection of two songs with maintaining the smoothness of the tension flow. Madson and Fredrickson 1993; Krumhansl 1996). In these attempts, the tension at each part of music has been measured by manual inputs of the participants in the experiments. In our work, we developed several methods to automatically compute the tension of a chord by combining known experimental results. Then the series of tension values corresponding to the given chord progression in the input music are interpolated to construct a smooth piecewise parametric curve. We exploit the B-spline curve representation that is one of the most suitable method to model a complex shape such as tension flow. Once the tension curve is constructed, we can edit the tension flow of the original music by geometrically editing the tension curve, where the tension curve editing also generates the new reharmonization of the original chord progression. In this paper, we show our tension curve editing method can be effectively used in several interesting applications: enhancing or weakening the overall feeling of tension in a whole song, the local control of tension in a specific region of music, the progressive transition of tension flow from source to target chord progressions, and natural connection of two songs with maintaining the smoothness of the tension flow. All of these application results are achieved by appropriate geometrical curve editing methods such as curve translation, local curve shape control with space-time constraints, curve morphing, and shape blending. Consequently, our work shows the possibility of controlling the perceptual factor (tension) in music by using numerical methods. Note that most of the computations used in this paper are not expensive so they can be executed in realtime. Thus, the real-time modifications of the tenseness in background music could be possible according to the user's emotion or current scenario in the interactive environments such as games. Introduction Musical tension or dissonance is an important term in music analysis. In most music analysis books (Schoenberg 1954; Piston 1987), musical tension is analyzed by an examination of the intervals, the harmonic context, and the tonal motion. We divide musical tension in three categories: melodic tension (melody versus chord), harmonic tension (chord versus key), and chord tension (simultaneous tones) and we focus on harmonic tension and chord tension in this paper. we suggest a new method to represent the tension flow using a smooth piecewise parametric curve. The resulting tension curve, a numerical model, can be used to control the degree of tension in music. By applying various conventional curve editing techniques to the tension curve, we can construct a flexible model for locally or globally controlling the tension of music. The attempts to illustrate the tension in music has been executed mainly in the area of psychology (Nielson 1983; 482

Page  00000483 Computing Tension of Chords Using Lerdahl's Tonal Pitch Space 2.1 In Lerdahl's theory of tonal pitch space (Lerdahl 1988), he suggested a model to calculate the psychological distance of any pitch, chord, or region (key) from a given reference point. Let us consider the two chords x = {pl,p2,'. } in key kx and y = {qi, q2, *,} in key ky, where {Pi} and {gqi (i = 1, 2,... ) are the sets of pitch classes in the chord x and y, respectively. Lerdahl defined the distance between two chords x and y by: d(x y) = a + b + c, (1) where a is the number of steps from the key kx to kv on the circle of fifths, b is the number of steps from the chord x to y on the circle of fifths, and c is a specially weighted Hamming distance between the two sets of pitch classes x and y. If any of both chords has a seventh and/or a tension note, an additional value is added to the distance by Lerdahl's suggestion for calculating the surface dissonance (Lerdahl 1996). In our work, we consider a tension of an arbitrary chord x in terms of a specific key k by assuming that the higher the distance between a chord x and a tonic triad chord kt of the key k, the more tense the chord x. Thus, the tension of the chord x in the key k is defined by d(kt - x) computed by the Equation (1). We calculated the distance from C triad to twenty-four selected chords in the C key including triads, seventh, sixth, substitute chords, and etc., according to this rule. Shepard 1979). She suggested a numeral rating assigned by human subjects to certain pitches, that is, the rating of the tonal stability of certain pitches with respect to a given scale. We thought that this stability rating can also be applied to calculate the amount of the tension of chord because each chord consists of several pitches and the tension is made from them. Table 1 shows normalized instability values. C C# D D#I E F F# G G# A A# B Major 0 100 70 97 48 55 94 27 97 67 100 85 Minor 0 96 73 27 96 73 100 42 62 96 77 85 Table 1: The normalized instability values of each pitch in the C key. Although we got a good rating of each chord by the probe tone, this was not enough because the tension is made by not only the pitches but also the intervals among the pitches in the chord. So we also considered the rating of intervals using the traditional ranking of harmonic consonance. We assigned linearly increased tension values to the pitch intervals in the order: Unison, Perfect 5th,, Perfect 4th, Major 3rd, Major 6th, minor 3rd, minor 6th, Major 2nd, Minor 7th, Major 7th, minor 2nd, and tritone (see Table 2). Interval P1 m2 M2 m3 M3 P4 TRI P5 m6 M7 m7 M7 value 0 91 64 46 371 18 100 9 55 36 73 82 Table 2: The normalized instability values of each interval (P:perfect, m:minor, M:major, and TRI:tritone) 2.2 Using Chew's Spiral Array Model Chew (Chew 2000) suggested a 3D spiral array model of the circle of fifth that represents a hierarchical musical factors such as pitch, chord, and key in a geometric point of view. In this model, the distance between two musical factors can be measured by Euclidean distance between two points corresponding to the factors. Let x = {p,p2,". } be a chord in a key k. According to the Chew's work, the pitch classes Pl, P2, " " " and the key k can be represented as corresponding 3D points vl, v2, ~ ~ ~ and w in the spiral array model, respectively. In our work, we define the tension of the chord x in the key k as the sum of all Euclidean distances between the each pitch point and key point: In this model, the degree of tension of a chord is calculated as follows: d(x) = wl s s + w2 E r, (3) where s is the instability value of the pitches in the chord x (Table 1), r is the rank of intervals in the chord x (Table 2), and wi and w2 are weight constants for the two terms, respectively. In our implementation, we set w1 = w2 in order that two factors have influence on tension of the chord x equally. 2.4 Comparison of Three Methods Table 3 shows the ranking of the 24 selected chords in C key according to the tension values computed from the three methods. The amount of tension are normalized to the range [0, 1] for easy relative comparison. Lerdahl's method focus on harmonic tension and Chew's method is related to chord tension (remember the definitions of harmonic and chord tension in Section 1). Pitch-Interval method, however, can consider both tensions. d(x) = | \vi - w. (2) i 2.3 Using Pitch-Interval Method Psychologist Krumhansl introduced the probe tone technique to quantify the hierarchy of tonal stability (Krumhansl and 483

Page  00000484 Lerdahi's T Chew's Pitch-Interval [chords Jtension Tchords tension chords ~jtension] C J0.00 - C 0.00 C 0.00 CM7 0.07 Am 0.06 Am 0.08 C6 0.21 Em 0.50 Em 0.17 Am 0.50 C6 0.26 C6 0.35 Am7 0.57 Am7 0.26 Am7 0.37 Em 0.57 CM7 0.32 CM7 0.38 Em7 0.64 Em7 0.37 Em7 0.55 F~m7-5 0.86 F~m7-5 0.48 F~m7-5 0.67 F [0.36 ] F 0.13 F 0.09 FM7 0.43 Dm 0.18 Dm 0.24 F6 0.57 FM7 0.37 FM7 0.41 Dm 0.57 F6 0.39 F6 0.45 Fm6 0.64 Dm7 0.39 Dm7 0.47 Dm7 0.64 BbM7 0.56 BLbM7 0.66 BbM7 0.71 Fm6 J0.59 Fm6 [0.66 G 0.36 G 0.13 G 0.22 G7 0.43 Bin-S 0.24 G7sus4 0.42 G7sus4 0.57 G7 sus4 0.33 Bin-S 0.47 Bin-S 0.79 G7 0.44 G7 0.73 G7-5 0.79 Bm7-5 0.50 Bm7-5 0.81 G7-15 0.79 G7-15 0.58 G7-15 0.86 Bm7-5 0.86 Bdim7 0.70 DLb7 0.94 Bdim7 0.86 G7-5 0.74 G7-5 0.95 SDb7 1.00 Db7 1.00 Bdim7 1.00 Table 3: The ranking of the selected 24 chords in C key according to the tension values calculated by three methods. The chords are grouped into tonic, subdominant, and dominant. We compare these methods on a simple criterion. Generally the three chord families are known to have different degrees of tension: tonic < subdominant < dominant. We sorted the twenty-four chords according to the tension values calculated by each method and counted the number of chords that are out of sequence. There are subdominant chords and dominant chords with lower tension values than tonic chords, and dominant chords with lower tension values than subdomiinantt chords. Table 4 shows the number of chords that are out of sequence in each case. Counter sequence Lerdahi's_] Chew's IPitch-Interval T-S 19 17 13 T-D 9 15 13 S-D 12 j 14 21 Total 40 46 47 Table 4: The number of chords that are placed out of sequence by each method. T-S is the number of subdominant chords (S) with lower tension values than tonic chords(T); T-D and S-D have similar meanings, where D indicates the dominant. These results suggest that the pitch-interval method produces the most accurate ordering. But the difference between the methods is quite small, and all three methods represent valid ways to measure the tension of chords. Constructing Tension Curves The tension value of each chord in an original song is computed using one of the methods in the previous section. In our work, we enforce that any chord in the tonic family has less tension value than any chord in the subdominant family. Similarly, all subdominant family chords are treated to have less tension than any dominant family chord. In each chord family, the order of chords are determined by the rank shown in Table 3. So, for example, the order of chords in terms of the ascending tension value using Lerdahl's method is: C - CM7 -C6 -Am-Am7 -Em-Em7 -F~mi'5 -F-FM7 -F6 - - Bm7b'5 - Bdim7 - Db'7. This strict classification of chord family is reflecting the traditional tension-and-release concept. After computing the all tension values in a song, we can construct a smooth piecewise parametric curve by the curve interpolation algorithm. We exploit the famous B-spline interpolation technique (Cohen et al. 2001) to construct a iD cubic B-spline curve T(t). Figure 1 shows an example of the tension curves. Note that the horizontal axis represents the time parameter t, while the vertical axis represents the tension value T. Tension Value Time Figure 1: An example tension curve with a chord sequence: C - C - C -Am -Am -F -G -Am -Am -C -C -Am -Am -F -G -C. Editing Tension Curves 4.1 Enhancing the Tension The overall tension in original music can be enhanced by shifting the tension curve T(t) of the music up to T, (t) T(t) -H (0, AT) with positive AT or weakened using negative AT. The newly computed T, (t) represents the tension curve of the new chord progression for the original song. We can compute the new reharmonization of the song by sampling T, (t) at appropriate time instances. (see Figure 2). 484

Page  00000485 contrast with just moving the constraint point and then recomputing the spline. Constraint / Point (f) t(e),(d) - (c),(b)*/oi1 - - SII i i I I (a) C C C Am Am F G7 C (b) Am Am Am Em Em Dm Bm7-5 Am (c) C6 C6 C6 Am7 Am7 F6 Db7 C6 (d) Am7 Am7 Am7 CM7 CM7 Dm7 G7-5 Am7 (e) CM7 CM7 CM7 Em7 Em7 BbM7 Bdim7 CM7 (f) F#m7-5 F#m7-5 F#m7-5 F F G Bdim7 F#m7-5 Figure 2: An example of tension enhancing: (a): original chord sequence, and (b)-(f): chord sequences generated by enhanced tension curves. The tension rank of the chords is computed using the pitch-interval method. 4.2 Local Editing of Tension Curves Although the tension curves can be easily edited using wellknown control-point based methods (Cohen et al. 2001), we use another method called "space-time constraints method" (Witkin and Kass 1988) to control the local shapes of the tension curves. The idea of the space-time editing is using the optimization method to minimize the squared difference between the original curve and the new curve generated by editing. In this optimization problem, the editing constraints can be added to reflect the user's demands. Let T(t) be an existing tension curve. An editing constraint can be represented as T(t,) = v, which means we need that the tension curve has a specific tension value v at a specific time t,. Then, the optimization problem of the spacetime editing that can be used to find a newly computed curve S(t) is defined by: /I I I SI II (a) C C C C F F F Em7 F F F F (b) C C C C F#m7-5 BbM7 Dm Em7 F F F F Figure 3: Editing a tension curve using the space-time editing technique. The bold curve (a) is an original curve and the thin curve (b) is the edited curve that satisfies the constraint indicated by the star mark. 4.3 Morphing/Blending Two Chord Sequences We can compute the continuous steps to smoothly change a given source chord sequence into a target one using the curve morphing technique (Surazhsky and Elber 2002). A simple implementation is defined by: S(t) = s - Ti(t) + (1 - s) - T2(t), 0 < s < 1, (5) where S(t) represents an intermediate curve in the process of morphing from the source curve T (t) into the target curve T2(t). We can compute the series of intermediate curves using several steps of corresponding s values in [0..1]. This operation is useful for generating the smooth (tension) transition between two different chord sequences. Figure 4 shows an example of a tension curve morphing. minimize(T(t) - S(t))2 subject to T(t,) = v. (4) The above problem means that we want to keep the tension curve as original as possible while satisfying a local editing constraint. Using this technique, we can maintain the overall tension flow of the original music with editing the music to have some specific tension degree at a specific time. In this paper, the curves T(t) and S(t) are represented with the Bspline curves, thus, the unknown variables of the above problem are the control points of the newly computed S(t) curve. Figure 3 shows a resulting tension curve computed by space-time editing method. In figure 3 chord change occurs not only in target constraint chord but also in neighborhood of constraint chord. It makes much stronger local change of tension. This is the advantage of the space-time method in (e) (a) Bm-5 BbM7 CM7 Am Am F Bm-5 Am (b) BbM7 FM7 Am7 Am C6 F G C6 (C) F F#m-5 C6 C Am7 F Dm7 C6 (d) Am7 C6 Am C Em7 F FM7 Em7 (e) C C C C F F F F Figure 4: Tension is the target curve. curve morphing: (a) is the original (b)-(d) are intermediate curves. curve and (e) Two different chord segments also can be smoothly connected using the curve blending technique. In our implementation, we used the Hermite interpolation technique (Cohen et al. 2001) to compute the blending curve to connect the two tension curves (see Figure 5). 485

Page  00000486 (b) I~~ I I II I (a) G G G7 G7 G G G7 G7 G (b) CM7 CM7 C C CM7 CM7 C C (c) G G G7 G7 G Dm7 Dm Am7 CM7 CM7 C C Figure 5: Blending tension curve. (a) and (b) and (c) is a blending curve between (a) and (b). are original curves Conclusions and Future work References Chew, E. (2000). Towards a Mathematical Model of Tonality. Doctoral dissertation, Cambridge: MIT. Cohen, E., R.F.Riesenfeld, and G.Elber (2001). Geometric Modeling with Splines: An Introduction. Natick, Massachusetts: A K Peters. Felts, R. (2002). Reharmonization Techniques. Berklee Press. Krumhansl, C. (1996). A perceptual analysis of mozart's piano sonata k.282: Segmentation, tension, and musical ideas. Music Perception 13(3), 401-432. Krumhansl, C. L. and R. N. Shepard (1979). Quantification of the hierarchy of tonal functions within the diatonic context. Journal of Experimental Psychology: Human Perception and Performance 5(1), 579-594. Lerdahl, F. (1988). Tonal pitch space. Music Perception 5(1), 315-350. Lerdahl, F. (1996). Calculating tonal tension. Music Perception 13(1), 319-363. Madson, C. and W. Fredrickson (1993). The experience of musical tension: A replication of nielsen's research using the continuous response digital interface. Journal ofMusic Therapy 30(1), 46-63. Mazzola, G. (2002). The Topos ofMusic. Basel: Birkhauser. Nielson, F. V. (1983). Oplevelse of musikalsk spaeding(The experience of musical tension. Copenhagen: Akademisk Forlag. Piston, W. (1987). Harmony. New York: W.W.Norton & Company, Inc. Schoenberg, A. (1954). Structural Functions of Harmony. New York: Norton. Sethares, W. (2004). Tuning, Timbre, Spectrum, Scale. London: Springer. Surazhsky, T. and G. Elber (2002). Metamorphosis of planar parametric curves via curvature interpolation. The international Journal of Shape Modeling 8(2), 201-216. Witkin, A. and M. Kass (1988). Spacetime constraints. In ACM Proceedings of SIGGRAPH 1988, pp. 159-168. ACM. In this paper, we suggested a method for numerical modeling of tension in music with some interesting operations which can be used to edit the tension flow using various curve manipulation techniques. Because we only focus on the numerical representation of tension in a music, the resulting chord sequences sometimes may not match human intuition. Nevertheless, the resulting chord sequences can be a good start point to produce more interesting music. For better tension curve construction and editing, we may consider many musical rules such as chord progression or substitution rules in jazz harmony (Felts 2002). In this paper, we don't consider on melodic tension, that is, relation between melody and chord, so some resampled chords may not match with a given melody. We hope that our system is extended to the general system including all kind of tension more concretely, especially in the perspectives of mathematical and computational music theory such as the HarmoRubette of the software RUBATO(Mazzola 2002). In another point of view, there are some attempts of calculating the tension, or (more concretely) 'dissonance' of audio signal numerically(Sethares 2004). Although our paper deals with manipulating tension based on a score, it is interesting to manipulate tension of audio signal in the similar manner. Acknowledgements The Authors would like to thank the anonymous reviewers for their helpful comments. This research is accomplished as the result of the promotion project for culture contents technology research center supported by Korea Culture & Content Agency(KOCCA). 486