Page  162 ï~~REAL-TIME TIMBRE MODIFICATIONS USING SINUSOIDAL PARAMETER STREAMS Lippold Haken CERL Sound Group, University of Illinois ABSTRACT: We present a new way of using sinusoidal models in real-time software synthesis. Many synthesis systems allow the user to operate on streams of stereo samples; instead, we use streams of sinusoidal partial parameters. We encode the parameters for the partials such that operations which are normally applied to sample streams can be applied to these parameter streams. Frequency and amplitude shifts are achieved by adding a constant to a parameter stream. Timbre morphing is achieved through a weighted average of several parameter streams. Applying modulators, resonators, and filtering operations to parameter streams results in a wide range of timbral modifications. INTRODUCTION: Sinusoidal synthesis allows the composer independent fine control over the amplitude and frequency characteristic of each partial in a sound. A large variety of modifications are possible with sinusoidal synthesis, including frequency shifting, time stretching, and timbre morphing. We have developed Kyma "Sound Objects" (Scaletti 1987, Hebel and Scaletti 1994) which efficiently implement sinusoidal synthesis in a user-friendly real-time environment. Sinusoidal synthesis is challenging for real-time systems because of high computation requirements. We use highly optimized DSP code, omit inaudible sinusoids during analysis (Fitz, Walker, Haken 1992), and provide a Sound Object which detects inaudible sinusoids in real-time and prunes them from the synthesis. The real-time detection of inaudible sinusoids takes into account what sounds are playing at what frequencies throughout a composition; during a solo passage, all analyzed sinusoids of a sound will be audible, but during a tutti, only a small number may be audible (Haken 1992). Traditionally, sinusoidal synthesis has been of limited use to composers because of the large amount of data (time-varying amplitudes and frequencies for a large number of sinusoids) which must be continually changed for timbral modifications. We have developed the idea of Sinusoidal Parameter Streams to simplify manipulation of this large amount of data. tuIdedx M i s los oscillator bank PARAMETER STREAMS: The Kyma graph shown above contains three Sound Objects and a speaker output. Real-time data flow is from the left to right. Each sample time, two 24-bit values (left and right channel) flow from one Sound Object to the next. The leftmost Sound Object, called "time index" in this graph, is a Kyma oscillator. Its parameters are set to produce a 0.5 Hz ramp wave form. The next Sound Object, called "analysis envelopes," reads sinusoidal parameters (amplitude and frequency envelopes). Its input (the ramp wave) determines how far into the analysis the sound has progressed. Each sample time it outputs a log-amplitude value in the left channel and a log-frequency value in the right channel. The output is called a "parameter stream" because it takes N sample times to transmit the parameters for a sound with N partials. The rightmost Sound Object, called "oscillator bank," is a bank of sinusoidal oscillators. Each oscillator computes a partial of the sound, and all oscillators are evaluated each sample time. The oscillator bank computes piecewise linear amplitude and frequency envelopes for each oscillator, if there are N partials, the piecewise linear segments are N samples long. The output of the oscillator bank (the sum of its oscillators) is sent to the speaker. The "time index" oscillator produces a low frequency ramp wave so that the sound plays from beginning to end, repeatedly. If the frequency of the ramp is decreased, time stretching will occur. If the ramp is changed to a triangle wave, the sound will play forwards and backwards. If more complex waveforms are used, some parts of the sound will be time stretched will others will be time compressed. The "time index" oscillator could be replaced by a Kyma graph which includes randomness or provides looping and sustain in the sound. Extreme time stretching is possible because the "analysis envelopes" Sound Object interpolates between frames in the analysis. The Kyma graph shown below implements timbre morphing between two timbres called "elect spect" and "neut spet." The "time index" at the left produces a 0.5 Hz ramp wave, as in the Kyma graph above. The time index connects to two Sound Objects, "elect spet" and "neut spet," which produce sinusoidal parameter streams. These streams go to a cross fader called "interpolate spectra." This cross-fader would normally be used to cross fade between two sample streams, but here we use it to cross fade between two sinusoidal parameter streams. The control for the cross fader comes from "morph function," a wavetable 162 I C M C P ROC EE DI N G S 1995

Page  163 ï~~which produces a single 10 second ramp. The output of the cross fade is a weighted average of two parameter streams, and is sent to the oscillator bank. When this is played in Kyma. five sounds are heard, the first sound having the "elect spect" timbre, and the later sound slowly morphing to the "neut spect" timbre. The "morph function" can be replaced by Kyma graphs which produce "timbral vibrato" or other more complex timbre morphs. The "time index" can be replaced with a Kyma graph which provides time stretching and compression needed for morphs between sounds with varying attack rates and vibrato rates (Tellman and Haken, 1995). not ot i,, t la!,, cillto- bank Other modifications are possible with parameter streams. To produce a frequency shift, add a constant to the log-frequency values in the right channel. To implement a spectral envelope filter, index into a waveshaper with the right channel (log frequency value) and add the result to the left channel (log-amplitude value). To implement other types of timbral morphing, use amplitude values from one sound and frequency values from another, or use alternate values from each sound. Sound synthesis for the Continuum keyboard (Haken, Abdullah, Smart 1992) requires morphing in a three-dimensional timbre space, which we implement by cross-fade between eight parameter streams. 14JSumOfSInes o Aa ss Aaalqslslliarph irPwrtkls BankSIze _____ E1Â~4 EI I EI Z Tindex Gafllr "SumOfSines" is a higher-level Sound Object which combines several other Sound Objects. Here we show the parameters for "SumOfSines" for a MIDI controlled morph between the "virtusos" and "tubesos" timbres. Thanks to Carla Scaletti for the Kyma displays used in this paper. REFERENCES: (Please see the following papers for a more complete list of references.) K. Fitz, W. Walker, and L. Haken, "Extending the McAulay-Quatieri Analysis for Synthesis with a Limited Number of Oscillators," Proc. 1992 International Computer Music Conference, 1992. L. Haken, "Computational Methods for Real-Time Fourier Synthesis," IEEE Transactions on Signal Processing, Vol. 40, Number 9, October 1992. L. Haken, R. Abdullah, and M. Smart, "The Continuum: A Continuous Music Keyboard," Proc. 1992 International Computer Music Conference, 1992. K. Hebel and C. Scaletti, "A Framework for the Design, Development, and Delivery of Real-time Softwarebased Sound Synthesis and Processing Algorithms," Audio Engineering Society, Preprint Number 3874(A-3), San Francisco, 1994. C. Scaletti, "Kyma: An Object-oriented Language for Music Composition," Proc. 1987 International Computer Music Conference, 1987. E. Teilman and L. Haken, "Morphing Between Timbres with Different Numbers of Features," to appear in Journal of the Audio Engineering Society, 1995. ICMC PROCEEDINGS 199516 163