Page  00000001 Understanding the Mathematics of the Frequency Transform: An Interactive Tutorial for Computer Musicians R. Gerard Pietrusko* and Richard C. Boulangert *Department of Electrical & Computer Engineering, Villanova University rpietru @ tDepartment of Music Synthesis, Berklee College of Music Abstract At the Berklee College of Music, students in the Music Synthesis Major acquire a great deal of experience working with sound design tools. They understand the terminology and develop a strong, musical intuition about the operation of various synthesis techniques through hands-on experience. However, rarely do they bring with them the mathematical knowledge needed for a true understanding of the DSP tools that are becoming increasingly important to the landscape of Computer Music. The Frequency Transform Tutorial was developed to address the needs of these musicians as they begin their study of the Discrete Fourier Transform (DFT). With a set of interactive software programs, important equations are presented as experiments and are explained using music synthesis terminology, thus providing the student a concrete connection between their intuitive understanding of sound design and the more mathematical approach that will be required for a thorough treatment of the DFT. 1 Introduction In teaching synthesis to musicians, it has been our approach to present material in a way that appeals to their musical knowledge. New techniques are developed through hands-on experiments and listening exercises. The student begins to develop a strong intuition for signal processing and sound design that is largely divorced from the underlying mathematics and algorithms. When presenting the foundations of Digital Signal Processing, we've found the transition from this intuitive approach to the necessary mathematical treatment to be difficult for many students, especially those with little prior mathematical experience. In addition, once familiar with the operation of higher level tools, such as the Phase Vocoder, students are often interested in further study that uncovers how these techniques work. To address this need, we have developed the Frequency Transform Tutorial as an introduction to the Discrete Fourier Transform (DFT). Through hands-on exercises using familiar synthesis tools, we build up the DFT's notation and conceptual operation. After working through the tutorial, our goal is that a student-despite their previous mathematical experience-will be prepared for a more thorough study of the topic, such as that presented by Steiglitz (1996). This paper acts as a small overview of the tutorial. Section 2 explains the tutorial's approach, while the remaining sections describe the tools and methods used to introduce the DFT. 2 The Tutorial's Approach The DFT is presented in terms of synthesis techniques already understood intuitively by the computer musicianspecifically, Ring Modulation (RM) and Averaging. Based on the multiplicative properties of Ring Modulation, we establish a conceptual foundation for the DFT. This is presented as a set of six Modulation Principles. The transform is developed incrementally and whenever relevant, new ideas are explained as instances of one or more of these principles. We combine Ring Modulation with Averaging to introduce the inner product and orthogonality. The DFT itself is first presented as a simplified, real-valued transform (in terms of the inner product). Then, a discussion of phase presents the DFT's complex basis as an augmentation to previously developed ideas. As stated in the introduction, the tutorial is designed for students with little prior mathematical knowledge and is therefore, implemented as a set of cross-platform, interactive applications in the Max/MSP environment. Equations are introduced graphically with explicit, parametric control of the variables as well as immediate and important visual feedback on how an equation is operating at the signal level. Through hands-on exploration of an equation, the student is able to visualize and ultimately predict its output. This approach builds a strong connection between mathematical notation and their musical intuition. Figures 1 and 2 show the tutorial's first interactive program. 3 Ring Modulation The student's knowledge of Ring Modulation is usually formulated in the frequency domain as sum and difference tones. We, therefore, use this terminology to introduce signal multiplication. In our interactive tutorial, equation 1 is presented along with a sidebar that briefly discusses sine and cosine functions as well as degrees and radians. Proceedings ICMC 2004

Page  00000002 term the student is asked to treat the amplitude of the second input as a unit constant. At various frequencies, the student confirms that the upper sideband, 2f, rides on a DC offset which is determined by the amplitude of input one, as shown in equation and figure 3. (3) A A A cos(2ft) x cos(2~kt)= - cos(4ft)+- - 2 2 f=k (1) A cos(2nft) x B cos(2ikt) Throughout the tutorial we use the variable k to denote the frequency of the second input; as the paper progresses, this input takes on the role of the basis function. (When the DFT is finally introduced, the notational consistency enforces connections to previous equations-all the way back to equation 1.) The program shown in figure 2 is introduced as a way to familiarize the student with equation 1 and its block diagram representation. This tool is used to conduct a number of experiments that will establish our Modulation Principles. It is worth noting that the program ensures both input waveforms start from the same angle at any frequency. This simplification allows us to later present the frequency transform on the real axis first. (We do not, however, address this in the tutorial until the phase and the complex basis section.) 0.5 - V- V I -\ r- -r - t7 figure 3: 2f sum tone with an A/2 DC offset Equation 3 is analyzed in depth explaining that the DC term arises from cos(o)= 1. (This property is revisited again when discussing phase.) To conclude the section, we present our first set of Modulation Principles: (1) Inputs of different frequency result in two sinusoidal sidebands. (2) Inputs of the same frequency result in one sinusoidal sideband oscillating about a DC offset. (3) Assuming the amplitude of B is unity; the magnitude of the DC offset is half the amplitude of the input, A. 4 Averaging In addition to being part of the tutorial's methodology, time-averaging provides the means for introducing the discrete-time index and the sigma notation; each of which is required for the student's understanding of the DFT. We first replace the continuous time indexer, t, in the above equations by the discrete version n/N and introduce the sigma notation in a sidebar for those students who may be unfamiliar with its operation. Through several software examples, we demonstrate that the average of a sinusoidal wave over any integer number of cycles is 0. This is expanded to include any weighted sum of sinusoidal waves. We relate this back to ring modulation showing the average of the resultant sum and difference tones to be 0 if the frequencies are different. We ultimately present equation 4 to the student. (4) figure 2: RM full program The hands-on exercises focus on two situations; when the inputs are at different frequencies and when the inputs are at the same frequency. We then relate these situations to the student's knowledge of sum and difference tones using the trigonometric identity in equation 2. (2) Acos(2ft)x B cos(2kt) = ABcos(2[f + k]t)+ ABcos(27[f - k]t) 2 2 Considering the case when the frequencies are the same, we explain the significance of obtaining a difference tone of zero frequency. During the discussion of this resultant DC S N-1 n2 2 0=- A cosf2nf - x Bcos 2Tk.k fak Following the same procedure used in the ring modulation section, we next ask the student to explore the case when the frequencies of the input are the same. The sidebands are first treated separately, where the upper sideband is a sinusoidal time-average (shown above to be 0) and the lower sideband becomes the time-average of a DC constant (the result being the constant itself). This is shown in equation 5. Proceedings ICMC 2004

Page  00000003 (5) A 2 1 [A cos 2cfJ x cos 2Tk j. N n=0 N N f=k The time-average software tool allows the student to adjust the amplitude and frequency of the inputs to confirm equations 4 and 5, visually. These ultimately become our next two Principles, after a short discussion of terminology in the next section. 4.1 Inner Products and Orthogonality We introduce the term inner product to describe the previous experiments demonstrating equations 4 and 5. This provides the student with a procedural understanding of how the inner product operates-a summed multiplication of two vectors. We look at equation 4 more closely and define it to be an instance of orthogonality-the case when the inner product of two signals is zero. We augment our original Principles using these new definitions: (4) Two signals modulated together do not have a DC difference tone if they are orthogonal. Therefore, (5) two sinusoidal signals of different frequencies are orthogonal. We acknowledge that the inner product we present to the student includes a nonstandard 1/N constant. This allowed us to approach both the inner product and the DFT in terms of averaging and thus increase the student's familiarity with the topic. 5 Frequency Transforms The first frequency transform presented is a simplified version of the DFT, using only the real component of the basis. Since all of our examples still force the input and the basis to start from a common angle, the resultant magnitudes will be the same as those computed by the DFT. We introduce the term basis and assign it to the second input of the modulator, explaining that the basis contains cosines at all of the frequencies that k can assume. The frequency transform is then broken into steps: (1) increment k; (2) measure the inner product between the basis and the input; (3) store this value in an array X(k)-the spectrum of the input. A program was designed to show this process using a single cosine input. The student sets the input to arbitrary frequencies and varies k to see how and when the spectrum changes. This interface is shown in figure 4. We then generalize the input allowing for more complicated spectra and demonstrate this using a sawtooth wave as the input. In addition to showing how the transform is a linear process (specifically, treating the input as a sum of cosine waves), it also provides us the opportunity to fully normalize the inner product ( 2/N) such that the magnitude of the sawtooth's harmonics are consistent with the student's prior knowledge. The final, real-valued transform is shown in equation 6. ngure 4: Interactive, real-valueda requency transtorm (6) X(k)= - x(n)cos 2nk - N no N) 5.1 Phase and the Complex Basis Before introducing the full DFT, the last topic addressed is that of phase which acts as a mediator between our realvalued basis and the complex basis used in the DFT. A simple program was designed that first demonstrates the orthogonality of sine and cosine waveforms, independent of frequency. We then present the imaginary component as a second, separate ring modulator with a sine wave as the basis. The student varies the phase of the input using a polar-form graphic interface and compares the inner products of the two basis functions. Figures 5 and 6 show the program interface and the block diagram, respectively. figure 5: interface for presenting the complex basis We use this software to introduce the idea of projection and to explain that the previous real-valued examples were constrained versions of this more general property. The trigonometric identity in equation 2 is then revisited and augmented, showing that the resultant DC sideband (when the inputs are of the same frequency) is also dependent on the inputs' relative phase. To place this within the context of the tutorial, we present equations 7 and 8-our two, separate inner products-and pose the question: Is it possible to combine these into a single representation resulting in a single magnitude value? Proceedings ICMC 2004

Page  00000004 input cosie wavefiWrm sine l( k A f X 0 k: (9a) 2 N-1[ n1 N-1[ X (k) =-Y\ x(n)cos 2nkn- - 1: x(n)sin 2/k&n N n=o N) N n=o N X(k) = 2 x(n) cos 2k "- j sinf 2nk" N n=o N N) (9b) (9c) reala imkgia ry inner product inner prodiuct 2 N-1 -j2'kn X (k)= Yx(n)e N Sn=0 The tutorial's concluding section then suggests a number of texts as good starting points for further study in Fourier Transforms and Digital Signal Processing. figure 6: signal diagram for the complex basis We introduce the student to j and the complex basis as the answer to this question. (7) 6 Conclusion & Future Work 2 N-1 Xo 2(k) = L x(n)cos2nk N (8) Xsin (k) 2 = x(n)sin 2k n The tutorial treats j only as a method of keeping the real and imaginary components of the basis separate; thus, we leave a more thorough mathematical explanation for the student's further studies using some of the texts mentioned in the reference. Several examples are, however, providedusing the graphic tool in figure 6-that demonstrate conversions from the complex Fourier coefficients to polar form. Lastly, our set of Modulation Principles are augmented to include phase differences: (6) The basis input 'signal' must be complex for the previous principles to hold, given an input of arbitrary phase. The tutorial explains this further using the trigonometric identity in equation 2. 5.2 The Discrete Fourier Transform By this point in the tutorial, we've explored the fundamental components of the DFT, and we have related these concepts back to familiar synthesis tools using handson exercises. Strong connections between mathematical notation and physical parameters have been established for the student that allows us to present the DFT as a natural extension of the previously covered material. It is introduced in an expanded form (equation 9a) in order to make this extension explicit. We then rewrite it in a more standard, sinusoidal form (equation 9b). Lastly, Euler's relation is presented as a way to convert the transform into the common, exponential basis form of equation 9c. This paper provided an overview of the Frequency Transform Tutorial which introduces the Discrete Fourier Transform through an interactive approach. Using a clearly guided sequence of instructions and experiments, mathematical concepts are gradually introduced through musically intuitive parameters. This allows students to modify equation variables directly and receive both aural and visual feedback on their operation. A set of six Modulation Principles are derived through cumulative experiments. These are then used to relate the DFT's mathematical notion to the intuition gained by the students during the tutorial. After completion, a student is better prepared for a more rigorous study of frequency transforms-as they are presented by Steiglitz (1996) or Roads (1996). This tutorial is available at and will be included with the forthcoming MIT Press book, Audio Programming in C and C++, edited by Richard Boulanger. Additionally, a companion tutorial is currently being developed that uses the conceptual framework described in this paper to introduce Windowing, Heterodyning and the Short-Time Fourier Transform. References Boulanger, R., editor. (2000). The Csound Book: Perspectives in Software Synthesis, Sound Design, Signal Processing, and Programming. Cambridge, MA: MIT Press. Ifeachor, E., and B. Jervis. (2002). Digital Signal Processing: A Practical Approach. Upper Saddle River, NJ: Prentice Hall Oppenheim, A. V., and A. S. Willsky (1996). Signals and Systems. Upper Saddle River, NJ: Prentice Hall. Roads, C. (1996). The Computer Music Tutorial. Cambridge, Massachusetts: MIT Press. Steiglitz, K. (1996). A Digital Signal Processing Primer. Boston, Massachusetts: Addison-Wesley. Zicarelli, D., G. Taylor, J.K. Clayton, et al. (2003). MSP 2: Reference. Proceedings ICMC 2004