# Fourier Oracles for Computer-Aided Improvisation

Skip other details (including permanent urls, DOI, citation information)This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. Please contact mpub-help@umich.edu to use this work in a way not covered by the license. :

For more information, read Michigan Publishing's access and usage policy.

Page 00000099 Fourier Oracles for Computer-Aided Improvisation Emmanuel Amiot, Thomas Noll, Moreno Andreatta, and Carlos Agon CPGE Perpignan, Esmuc Barcelona, Ircam/CNRS manu.amiot@free.fr, noll@cs.tu-berlin.de, {andreatta, agonc}@ircam.fr Abstract A as a FT of its characteristic function: The mathematical study of the diatonic and chromatic universes in the tradition of David Lewin [9] and John Clough [6] is a point of departure for several recent investigations. Surprisingly, Lewin 's original idea to applyfinite Fourier transform to musical structures has not been further investigated for four decades (until [12]). It turns out that several musictheoretically interesting properties ofcertain types ofmusical structures, as the partial symmetry ofFourier balances, Maximal Evenness [5] and Well-Formedness [3], allow alternative characterisations in terms of their Fourier transforms. The paper explores two particularly interesting cases: vanishing Fourier coefficients as an expression of chord symmetry and maximal Fourier coefficients as a reinterpretation of maximally even scales. In order to experimentally explore the Fourier approach we design an interactive playground for rhythmic loops. We propose a Fourier-based approach to be integrated as an "Scratching"-interface in the OMAX environment (built on OpenMusic and MaxMSP) which allows to interactively change a rhythm through a gestural control of its Fourier image. A collection of theoretical tools in OpenMusic visual programming language helps the improviser to explore some new musical situations by inspecting mathematical and visual characteristics ofthe Fourier image. TA tF Y -2i~rkt/c keA For example, the FT of a diminished seventh D7 =(O 3 6 9) is 3 TD7 t Y, 6- (3k)2ikt/12 k=0 - 1- 2i3t 1 -eixt/2 k=O Notice TfD7(4) = 4 and.FD7 (1, 2,3) = 0. This prominence of this 4th Fourier coefficient means precisely that set A is 12/4 = 3-periodic. The usual properties of FT apply and are indeed elementary in this simple discrete case. It is worthwhile to note that the usual musical operations (transposition, retrogradation, even complementation) do not change the modulus (i.e. length) of the FT: this means already that |FA is a good musical invariant. The most interesting property is that the FT of the relative intervallic content of two chords (a function stating the number of occurences of any interval between A, B) is simply the product of their respective Fourier transforms, as FT turns convolution product into ordinary product: F(IC(A, B)) = FA X F-B David Lewin's call for Fourier In a few lines at the end of his very first paper [9], David Lewin cryptically alludes to Fourier transforms and convolution products to explain how he was led to the special symmetries he considers in his paper about the relative intervallic content of two chords. In the present paper we elaborate some of Lewin's ideas by applying Fourier Transform (FT) to several discrete musical structures, which may be (periodic) rhythms or scales. We begin in this section with the set of pitch classes, modeled by Z, (c = 12 in the equal division of the octave in twelve parts) and define, following Lewin, the Fourier transform of a subset Interestingly, this line of thought leads to a "one line" proof of Babbitt's hexacord theorem on which Babbitt and Lewin were both working hard at the same time. It was also the basis of Dan Tudor Vuza's original work [13] on 'Vuza canons' which has been also implemented in OpenMusic. 1.1 Vanishing Fourier Transform of Characteristic Functions Lewin's interest though was in a kind of inverse problem reconstructing (say) A from B and the intervallic content. This is possible when TF-B is non vanishing, and this excludes precisely the five cases that Lewin put forward as 'augmented-triad property' and the like in [9], and simply 99

Page 00000100 'FOURPROP(i)' in [10]. See also [12], chap. 3 for more details. Of interest to us is the fact that FB(k) will vanish essentially when set B exhibits a notion akin to periodicity, the Fourier balance. For instance the set B = (0, 4, 5), like the augmented triad (0, 4, 8), has FOURPROP(4), as it is evenly distributed among the three diminished seventh chords. Suffice to say here that the Fourier transform enables to give an interesting generalization of periodic subsets (or Messiaen transposition limited modes) of Zc. 1.2 Maximal Fourier Coefficients and Maximal Evenness On the other hand, when the FT is as far as possible from vanishing (for some value k of its argument), it must characterise other interesting subsets of Zc. Originally Maximally Even Sets (ME sets) are defined by [5] in terms of generic intervals having the Myhill property. To give a simple example, the major scale is ME as there are exactly two sizes of thirds (major and minor), fourths, seconds, and so on, inside the scale. As noticed in [12], and later formally proved by one of the authors, the subsets with largest Fourier transform are precisely the ME sets: Theorem 1 A subset A e Zc of cardinality d is Maximally Even if IA (d) I > FB (d) for all subsets B of cardinality d. Intuitively, this means that A approximates as well as possible for a discrete subset an even division of cardinality c by d terms. For instance, in the simplest case when d is a divisor of c, and A = {0, c/d, 2c/d,...} TA (d)= e2id/ = 1 + 1 +... d keA which is clearly maximal for a sum of d terms, each of which has length 1. In less clear-cut cases, like the major scale (c = 12, d = 7), the maximum (here 2 + /3 3.732) is strictly less than d. Aside from characteristic functions, there is another simple way to encode scales and periodic rhythms; namely as labeled points on the unit circle. In this approach we do not need a chromatic universe where the scales are embedded. The notion of maximal evenness can be defined relative to any collection of N-note scales. A scale X = {e2it,..., e27itN } will be called Maximally Even among all scales within a given family & if its Fourier coefficient ai has maximal radius among the corresponding Fourier coefficients a' for all members of the family &. Here we define the Fourier coefficients by ak = zj=1 e2it e-2ijk/N. This recovers the ordinary definition of maximally evenness, where & consists of all N-note subscales of a equally distributed chromatic universe. Scales in step order, whose zero'th Fourier coefficient ao is bigger than al can be characterized as clusters. The unit-circle representation of scales is the point of departure for our modeling of periodic rhythms in the following section. Fourier "Scratching" Musical Rhythms are often described as sequences of interonset intervals or simply as sets of musical onsets. The former manner of description focuses on the internal connectivity of a rhythm while the latter one conceives rhythms as sets of events. A sequential temporal order of the single events is then inherited from the total linear order of the onset dimension. The temporal sequentiality often interacts with other manners of sequential organization, such as in pitch, space, dynamics etc. The following model therefore considers a concept of sequentiality independently from the temporal order. Identity of temporal and sequential order serves only as a point of departure for the exploration of a variety of other kinds of solidarity between the two. For example, think of the linear order of musical pitch. An ascending scale exemplifies a strict solidarity between both orders. A systematic exchange of each odd note with its predecessor destroys this strict solidarity, but still exemplifies a generalized solidarity principle. The manipulation of the finite Fourier transforms of the rhythms thereby allows to intuively perform slight global changes on the rhythms without changing each single event "by hand". 2.1 Rhythmic Loops In this application we assume a cyclic organisation of musical time with a fixed abstract period d 1. Practically, the actual interpretation of this abstract period (in terms of physical time in a performance) might vary, but we do not elaborate this further. Let U = {t = e27rit 0 < t < 1} denote the unit circle and let UN = { k/N = e2ik/N 0 < k < N} denote the discrete subcircles of cardinality N (i.e. N'th roots of unity). Of all the parameters which might be postulated for the specification of a musical event we isolate just the following two: phase /t E U and intensity r > 0 (r E R). For all additional parameters we postulate a musical space S. The selected parameters phase and intensity may vary independently with one exception: vanishing intensity implies zero phase. This trick allows us to identify musical events with pairs (z, s) c C x S. Intensity and phase are thus jointly encoded in terms of complex numbers z = rot. 100

Page 00000101 The musical space S may encode several aspects such as pitch, timbre, direction (orthophony). In order to motivate the idea of rhythmic loops we assume a modality for creating loops (closed curves) within this space A: U - S or discrete loops AN: UN - S. Sometimes it is practical to assume assume that all the discrete loops AN are restrictions of a previously chosen continuous loop A. As in the case of the period d this loop may vary within the space S, but we do not elaborate on that here. By definition, we conceive a rhythmic N-loop as a map p x AN:UN - C x S. A simple example, to be used below again, consists of an arpeggiated ascending pentatonic scale with increasing intensity values. Its rhythm p: U5 -- C maps the 5-th roots of unity 1, i,..., to complex numbers with increasing phases and absolute values: p(qk): (ro + krl)qk (k = 0,..., 4). The pentatonic loop (modulo octave) has the form As5(~) = C, A5( ) "= D, A5( ) = E, A5( ) = G, A5(04):= A in an abstract pitch space. 5 2.3 Dragging, Eliminating and Inserting of Fourier Coefficients The section title "Fourier Scratching" refers to a manipulation of rhythms through gestural distortions of their Fourier images. The "Fourier DJ" grabs a Fourier coefficient and moves it in order to modify the entire rhythm in a subtle way. The theory guarantees the existence of the inverse transform. Of course, the musical interpretation depends on a suitably chosen quantification of the musical time and intensity. Figure 3 shows an example for a series of simple progressive rhythm deformations, where always one Fourier-coefficient is varied. 11.. JJ., i,J J' S4-- -- SJ '*,6 - U - I, _ *JI, - JW-J '. L Figure 2: The pentatonic arpeggio (c.f. Figure 1) serves as an illustrative example of dragging single Fourier coefficients. The three examples in the upper half of the Figure stand for progressive variations of the the Fourier-coefficients ao,al and a2, while the two examples in the lower half stand for analogous progressive variations of the the Fouriercoefficients a3 and a4. In each of these 5 figures the experiment is the same: The corresponding Fourier-coefficient is dragged four times in the same direction on the complex plane. Inverse Fourier Transform then leads to the depicted deformations of the rhythmic loops. The changes in the intensity values is not shown here. Figure 3 shows a slightly more complex example, of a progressively altered rhythm. Aside from a purely experimental exploration of the musical effects like in this example, we propose some theoretical feedback to the improviser (see next section). The number of events per rhythm does nod need to be fixed. If a family of discrete loops AN: UN -- S is inherited Figure 1: Temporal order, ascending pitch and crescendo are all in solidarity in this example of a rhythmic loop. While the ascending pitch stands for a loop in cyclic pitch space, the intensity is meant to vary in a linear, non-cyclic way. In the sequel we focus on the purely rhythmic part of this map, namely p: UN -- C. However, it is important to notice that the musical motivation for this is encoded in the other map AN: UN - S. which embodies a sequentiality with regard to the space S. We claim that the manipulation of Fourier Coefficients of p results in a manipulation of the musical solidarity between sequentiality in S on the one hand and cyclic time on the other. 2.2 Analysis and Resynthesis of Rhythms The application of finite Fourier transform is inspired by the surprising success of this method in sound processing and works more or less straightforward. However, finite Fourier transform serves in signal processing as a approximation of the continuous case. The present application to musical rhythm - such as the applications to Chords and Scales mentioned previously - is based on finite Fourier transform as an autonomous analogue to the continous case. The finite Fourier transform is a linear bijection of CN 101

Page 00000102 Figure 3: The OpenMusic-Voice object shows a snapshot from an experiment with progressively altered rhythms through dragging of Fouier coefficients. from a continuous loop A: Ul - S, the rhythmic loops may vary in cardinality. Filter techniques as well es zero-padding offer modalities for contiguous transitions between rhythms of different cardinality. 3 Towards a Guided Improvisation in the OMAX Environment Computer-Aided approaches to musical improvisation need not to be restricted to gestural interaction and sound processing. We would like to support the challenging idea of using theory-oriented tools, enhancing the improvisers' conscious knowledge about their own creative activity. In this section we describe a concrete instance of an architecture which combines the realtime capacities of MAX/MSP for live performance with the exploratory power of OpenMusic visual programming language. This architecture, which has been proposed by [1], allows the combination of realtime interaction with high level operations such as the inspection of musical properties of the ongoing process. Fourier transforms on rhythms and other musical objects provide a particularly interesting case study, as these give instantaneous snapshots of the global tendency of the musical process. This is well-known from a long tradition of successful applications of spectral analysis and manipulation in signal processing, where finite Fourier transform and especially FFT serve as computational approximations to the continous Fourier transform for complex functions on the circle. In the present investigations the finite Fourier transform is applied directly to abstract musical processes which are represented in CN. Aside from merely inspecting and manipulating their actual Fourier transform - as discussed in section 2 -; the environment offers possibilities for the interactive comparison of an actual diagnosis with families of previously investigated musical situations. Three simple examples may illustrate the general philosophy of such an interaction. 1. Suppose the improvisor attempts to turn his actual rhythmic pattern into a palindromic one in a smooth and economic way. Of course, there is a trivial solution to this problem, namely to append the retrograde to the original. But suppose the number of notes shouldn't be changed. While the visual inspection of the rhythm itself may be as complicated as listening to it, the Fourier transforms of palindromes correspond to alignments of the coefficients along lines. The best linear fit to the actual Fourier image can serve as an oracle for a path toward the desired palindromic state. Thus the improviser may drag single Fourier coefficients towards this line or he may leave it to an automated process to reach these positions by a morphing. 2. Some theoretical aspects of well-formed scales have rhythmic analogues (see [4]) and furthermore they have an expression in terms of Fourier transforms. To bring both links together provides challenging possibilities for theoretical transfer. We mention a special observation from where an investigation may depart. Wellformed N-scales are generated scales and are thus parametrized by a single parameter - for fixed N: the size of the generator. The diatonic scale can be generated by the fifth. If the size of this fifth is slightly varied, all 7 tones are slightly varied and consequently, their Fourier coefficients vary along parametrised curves as well. Figure 4 shows the diatonic scale as one instance of a family of generated 7-note scales. The diatonic scale is a palindrome and thus its 7 Fourier-coefficients are situated on a straight line. Variation of the generator corresponds to a rotation of this line. Thereby the Fourier-Coefficients - of a generation order representation of the scale - share a common curve on which they move. Well-formed scales are characterized by the fact that their Fourier-coefficients of the step order representation move along the same curve. This curve thus serves as a Fourier-oracle for well-formed N-scales. In this particular case the parametrization of the scales via generator is simpler than the parametrisation of their Fourier coefficients (this curve is the balanced sum of the first 7 (complex) partials). But if the variation of scales, rhythms or other musical objects within a family shall be more advanced, it is helpful to study or to define their variations in the Fourier-Image. 3. The improviser may find his actual music particularly fascinating without understanding it in the live situation. A Fourier snapshot can offer him a characteristic picture, which can be interactively manipulated (see the previous section) or saved as a potential component of an oracle. 102

Page 00000103 [7] Clough, J., Myerson, G., 2Musical scales and the Generalized Circle of Fifths, in: AM2M(1986), 93:9, 695701. -~ [8] Clampitt, D.,Carey, N., infinite generated scales, in: / / /[9] Lewin, D., Re. Intervalic Relations between two col/ / lections of notes, in: Journal of Music Theory, (1959) / / 3:298-301. [10] Lewin, D., Special Cases of the Interval Function be_ ~tween Pitch-Class Sets Xand Y~, in: Journal of Music I (~YTheory, (2001) 45: 129. // [11] Mazzola, G., The Topos of Music, Birkhiuser, Basel,,,, 2003. Temperaments, PhD Dissertation, Eastman School of -4- Music, 2004. [13] Vuza, D. T. "Supplementary Sets and Regular Complementary Unending Canons", Perspectives of New 2MuFigure 4: Localisation of the Fourier coefficients of the di-sins2()2-9301,84073(),0-25 atonic scale on the characteristic curve for generated 7-note 3() 7- 5 91 scales Although the full integration of this playground in OMAX is still under development we can already present some instructive OpenMusic implementations. References [1] Assayag, G., Using Factor Oracle for Machine Improvisation, G. Assayag, V. Cafagna, M. Chemillier (eds.), Formal Systems and 2Music special issue, Soft Computing 8, pp. September 2004. [2] Babbitt, M., Some Aspects of Twelve-Tone Composition, Score, 12 (1955), 53-61. [3] Carey, N., Clampitt, D., Aspects of WVell Formed Scales, in: 2Music Theory Spectrum, (1989), 1 l(2), 187-206. [4] Self-Similar Pitch Structures, their Duals, and their Rhythmic Analogues. Perspectives of New Music 34/2: 62-87. [5] Clough, J., Douthett, J., Maximlally even sets, in: Journal of Music Theory (1991), 35:93 173. [6] Clough, J., Myerson, G., Variety and 2Multiplicity in Diatonic Systems, in: Journal of Music Theory (1985) 29: 249-70. 103