Page  349 ï~~GABOR SYNTHESIS OF RECURRENT ITERATED FUNCTION SYSTEMS Michael Gogins 55 Pineapple Street Apt. 5-F " Brooklyn NY 11201 " 718-522-0081 " Abstract: The following introduces a universal, parametric algorithm for computing musical sounds globally, not distinguishing "notes" from "timbres." A recurrent iterated function system (RIFS) is randomly iterated to accumulate a fractal measure over a time-frequency-phase array. After numerous iterations have smoothed the measure, its phase dimension is interpreted as the real and imaginary amplitudes of time-frequency samples in a Gabor transform. This in turn is rendered as a soundfile by means of overlapping inverse fast Fourier transforms. Eleven years ago I saw Mandelbrot (1983) show slides of Julia sets, the chaotic boundaries of regions attracted to infinity by quadratic dynamical systems, and of the Mandelbrot set, the set of all parameters for connected Julia sets. Any region in the Mandelbrot set resembles Julia sets generated from parameters in that region (Peitgen, Jurgens and Saupe). I wondered: Is there an acoustical Mandelbrot set? It would be an atlas of sounds that could be explored from the broadest continents of form to the tiniest streams of timbre. Some points in that set would be not noise but music, and each such point would be an entire composition in all details. Pursuit of this notion (Gogins 1991, 1992) has now yielded the Gabor recurrent iterated function synthesis (GRIFS) algorithm. It realizes, in practical form, the dynamical system aspect of the dynamical system-Mandelbrot set duality. It also realizes, in theory, the Mandelbrot set aspect. The basic idea of GRIFS is to use one space for both the generation of fractals and the representation of sound. Every fractal is thus a sound, and every sound a fractal. The Gabor Transform Sound is commonly represented by waveforms or spectra_ These, however, are the one-dimensional limits of the two-dimensional Gabor (1946, 1947a, 194Th) transform, where any sound is the sum of elementary signals, with Gaussian envelopes and complex amplitudes c, at each cell of a time-frequency grid. The cells' time and frequency sizes are reciprocally related by aspect ratio, a: 1 t a00 0 (t-nAt)2 2nrkt At =,Af =. The Gabor transform is ye(t) = 7n1k cm exp - r cis--, where a_ 2r -0 _-W2(At)2 A y0(t) is the amplitude of the signal at time t, n is time, k is frequency, and c is the peak amplitude of the n,kth cell. So the space of GRIFS is complex amplitudes on a time-frequency grid. But this is the same as real amplitudes on a time-frequency-phase grid, on which fractal measures (i.e., densities of points) can be accumulated by dynamical systems, each cell's measure being represented by its amplitude. Iterated Function Systems A metric space {X, d} is a space, X, in which the distance between any two points is measured by a function, d. An iterated function system (IFS) is a metric space together with a special dynamical system, a finite set of affine transformations that contracts the space: {X; w, n=l,2,...,N}. In time-frequency-phase space, the transformations are of the form w =if < d j j f"fI I. The Hutchinson operator for any subset B of X is then N W(B) =Jw, (B); each transformation is applied to a set, and the result is the union of all the n=1 transformations. Barnsley (1993) and others proved every W, if contractive, to have a unique attractor under iteration, A = limn__ wo"(B), for any B in X. After infinitely many contractions, WoÂ~ l(B) ca get no smaller than WÂ~0 (B), so B moves no more. The attractor is thus the fixed point of the operator: A = W(A). The Collage Theorem. Barusley (1993) also proved that any compact set in a metric space is approximated within any desired error bound by the attractor of some IFS. Intuitively, any set can be I C M C PROCEE D I N G S 1995349 349

Page  350 ï~~covered as precisely as desired by shrunken, just-touching affine transformations of itself. Applying those transformations to the set just leaves it in place. An iterated function system with probabilities gives to each transformation in W a probability or weight: {X; w,, n=1,2,...,N; pm n=1,2,...,N}. If the weights of transformations differ, so do the measures on different parts of the attractor. Just as there is a collage theorem for IFS attractors, so there is a collage theorem for IFS measures, proving that any measure on a metric space can be approximated within any desired error bound by the measure on the attractor of some IFS with probabilities. Collage Theorem for Sounds. From the representation of every sound by some measure quantized as a Gabor transform, and from the collage theorem for IFS measures, it follows that any sound can be represented as accurately as desired by the measure on the attractor of some IFS with probabilities. Non-self-similar attractors, however, are more easily represented with recurrent iterated function systems (RIFS). Each transformation has, instead of a single weight for the next iteration, a vector of weights for each tranformation, {X; w,, n=1,2,...,N; pt E [0,1]; ij=1,2,...,N}, so that the matrix of weights is a recurrent Markov operator for the Hutchinson operator's transformations. Again, there is a collage theorem, and a collage theorem for measures, for RIFS (Barnsley). Being hierarchical, RIFS build up complex measures from disjunct parts and layers. The SGRIFSAlgorithm. Iterate a RIFS over a time-frequency-phase array with but 2 phase elements, until the measure in the array is smooth. Seek in the soundfile to the time of the current Gabor column. For each frequency in the column: Store the value of the first phase element into the real part of the corresponding Fourier coefficient, and the value of the second phase element into the imaginary part. Zero pad the Fourier coefficients for negative frequencies. Compute the inverse Fourier transform. Mix the waveform now stored in the real parts of the Fourier coefficients, modulated by a Gaussian window of aspect a, into the soundfile, by cycling once through the window and 5 times through the real coefficients. Repeat for each time column in the array. Applications. GRIFS codes afford new means of signal processing and compositional transformation. E.g., decreasing all frequency scaling factors in a code increases the separation and distinctness in frequency of every feature of a sound at all frequencies simultaneously. Or, GRIFS codes can be located in a virtual parameter space (Gogins 1991) by defining a sort order for codes and placing the lower of two codes at the low corner of the space, and the higher code at the high corner. The shorter code is zero padded to give both corners the same dimensionality. Interpolating between the corners yields new sounds intermediate not only in timbre, but also in structure. And of course music can be composed by working directly with GRIFS codes, creating sounds of heretofore unimaginable timbre and structure. The Acoustical Mandelbrot Set. The other aspect of the dynamical system-Mandelbrot set duality can now be defined. Set up a virtual parameter space, interpolate to derive actual parameters for points, and generate an attractor for each point. Sounds with no gaps of silence are black; sounds with gaps of silence are gray or white; the more silence, the whiter. The black region is the acoustical Mandelbrot set. References Barnsley, M. 1993 [1988]. Fractals Everywhere, Second Edition. Boston: Academic Press. Gabor, D., 1946. "Theory of Communication," The Journal of the Institution of Electrical Engineers, Part HI, 93:429-457. Gabor, D. 1947a. "New Possibilities in Speech Transmission," The Journal of the Institution of Electrical Engineers, Part III, 94(32):369-387. Gabor, D. 1974b. "Acoustical Quanta and the Theory of Hearing," Nature 159(1044):591-594. Gogins, M. 1991. "Iterated Functions Systems Music," Computer Music Journal 15(l):40-48. Gogins, M. 1992. "How I Became Obsessed with Finding a Mandelbrot Set for Sounds," News of Music 13:129-139. Mandelbrot, B. 1983 [1977]. The Fractal Geometry of Nature (Updated andAugmented). New York: W. H.L Freeman and Company. Peitgen, H.-O, I Juirgens, and D. Saupe, 1992. Chaos and Fractals: New Frontiers of Science. New York: Springer-Verlag. 350 0ICMC PROCEEDINGS 1995