Page  487 ï~~COMPOSITION WITH TIME-SHIFTED ENVIRONMENTAL SOUND USING A REAL-TIME GRANULATION TECHNIQUE Barry Truax Dept. of Communication & School for the Contemporary Arts Simon Fraser University, Bumaby, B.C., Canada VSA 1S6 The author describes the compositional process involved in using sampled environmental sounds as the sole source material for his recent work Pacific (1990). All sounds are heard at their original pitch but are processed using a time stretching technique implemented in his software for real-time granular synthesis. In terms of timbral composition, the technique increases the perceived volume or magnitude of the sound through multiple overlays, phase independent voices, and time extension such that spectra that are brief instants now occupy virtually any duration. The time-stretching, however, poses the compositional problem of how to re-structure the time flow of the material. I. Timbral Design Using Environmental Sound Sampled environmental sounds are the sole source material for the author's recent work Pacific (1990). These materials, one sequence for each of four movements, are recordings of Canadian West Coast environmental sound, namely ocean waves on the west coast of Vancouver Island, boathorns in Vancouver harbour on New Year's Eve, Vancouver harbour ambience with seagulls, and the Dragon Dance in Vancouver's Chinatown celebrating the Chinese New Year. All sounds are heard at their original pitch in the final work but are processed using a time stretching technique implemented in software for real-time granular synthesis developed by the composer (Truax, 1988, 1990a). Briefly, the technique works by controlling the rate at which new samples enter the signal processor's memory from which the grains are taken. The lack of any pitch change stems from the fact that the samples in each grain appear in exactly the same sequence as in the original sample; however, successive or overlapping grains may be repetitions of the same material, hence the time extension. In this work, sound densities ranging up to 2000 events/second per stereo pair of channels were recorded on 8-track tape and mixed down in the Sonic Research Studio at SFU. From the point of view of timbral composition (Truax, 1990b), the technique increases the perceived volume or magnitude of the sound without necessarily altering its pitch or loudness. First, there is the increase of spectral richness by the superposition of 12 versions of the source per stereo pair of tracks. Such overlays intensify bands of spectral energy whether those of formants or noise elements. Secondly, the simultaneous voices are normally not phase coherent with respect to each other because of the randomness of synchronization created by the variable duration of the grains and the delay between them. This temporal independence of voices (in the range of phasing and reverberation effects, viz. less than 50 ms) also results in the composite sound seeming to have greater volume. Finally, the time-stretching technique adds a third dimension to the perceived magnitude, namely spectra that are normally brief instants in time can now occupy virtually any duration. The result is that even noise-like spectra (such as water or percussion sounds) are perceived as having larger-than-life vocal characteristics. This discovery of magnified 'inner voices' in environmental sounds suggested a compositional strategy in which these voices could be revealed as elements in a life cycle of death and rebirth through sounds common to the Pacific Rim. The time-stretching, however, also posed the compositional problem of how to structure the time flow of each movement. The stretched material, though timbrally rich and evocative, is generated at the expense of its temporal complexity - attack transients and amplitude variations are largely smoothed out into a steady state continuum. Though mesmerizing, these sounds can cause aural fatigue if experienced for long periods. An equally serious problem is that, because most environmental sounds are ICMC 487

Page  488 ï~~broad-band, multiple tracks of this material are difficult to mix without extensive masking and loss of clarity. The major compositional tasks in dealing with stretched sounds are to create a temporal structure and a spectral balance. II. Image and Metaphor in Environmental Timbral Design The compositional concern for timbre and its design is central to contemporary composition. Timbre evades description in conventional music notation, hence its exploration suggests a different musical domain from the rationalized two-dimensional field of pitch-time music. Timbrally based composition is less abstract than that music and more situated within the world of lived experience. In other words, an emphasis on timbre involves the listener in the real world of gender, environment and cultural symbols (Shepherd, 1987). The technique of granular time-stretching which I have described provides a unique way to experience the inner structure of timbre, hence to reveal its deeper imagery. Each movement of Pacific is based on the imagery inherent in the environmental sound used as its material, and in the composition of each movement, a metaphor is established that connects the sound to a deeper sense of cultural symbolism. In the first movement, Ocean, the literal title refers to the source material which is a recording of powerful ocean waves on a coarse sandy beach. The movement's global structure is determined by a simple trajectory which begins with the waves at normal speed, gradually stretches them in time to reveal their inner voices, then rather abruptly returns the listener to the original version at the end. To make the 'voices' more apparent in the middle section, the high frequencies are filtered out and vocal formant regions are emphasized through equalization. This process gives the added impression of a gradual submersion beneath the waves as their inner timbre is revealed, concluded by a rapid surfacing at the end. The process of submersion is accompanied by the loss of the rhythmic pattern of the waves which further heightens the illusion of entering a normally hidden realm. In the program notes, the voices are referred to as those of the 'unborn'. The imagery and symbolism is traditional - water, life, and consciousness. Mythology throughout the world refers to the ocean as the 'mother' of all life and the locus of the collective unconscious. We begin life as an emergence from the amniotic fluid and later experiences of oneness of the self with the universe are often described as 'oceanic'. The listener to this movement is encouraged by its symbolism to identify the aural image of submersion beneath the waves as a descent into the unconscious, the repository of all symbolism, the aural manifestation of which is the blur of imaginary singing voices which emerge from the granular transformation of the waves. These voices are indistinct, yet strangely human enough, to evoke whatever images the listener chooses - hence the movement depends on the listener's participation for its completion. The listener, however, may not be aware that another 'trick' is being played out in the movement. The first is to hear voices in the sound of the ocean, and the second is to identify the rhythm of the waves with the human breath. Again, the symbolism is traditional and ancient, but its implementation here is facilitated by the granular approach. The duration of grains used in granular synthesis is 10-50 ms or less, but there is nothing to prevent the composer from using longer durations that can be identified as macro level events. In this case, the rhythm of the waves heard at the start is not their own but rather a superimposed 'long grain' pattern with an average duration of 3.4 sec (with up to 0.7 sec shorter or longer). The illusion is created that these 'grains' are the envelopes of waves breaking; however, as the movement unfolds, these durations are progressively shortened. The process continues even while these tracks drop out of the mix during the 'submersion' sequence, but they appear later as short, energetic breaths of increasing speed and vigour that seem to trigger the final surfacing. The fact that the content of the short grains is from the original waves gives it the broad-band character of breath. Hence the movement can be understood as a process of giving birth, of air replacing water. Such a separation is not without trauma, hence the unsatisfying sense of abrupt cadence. Also, in the background the careful listener may detect a raven at the beginning and end. In aboriginal West Coast imagery, Raven is simultaneously a creator and trickster. ICMC 488

Page  489 ï~~The second movement, Fog, has a metaphoric title in that it attempts to recreate an environmental experience often found on the Canadian West Coast, a blanket of fog in a harbour surrounded by mountains. However, it does so by using sonic material unrelated to fog, namely boathorns recorded on New Year's eve. The sustained boathorn sounds are stretched even further to create a texture of seemingly timeless proportions. The movement starts with the assertive E-flat minor triad of the Canadian Pacific ferry horn, and after it dies away more distant horns are revealed. The reverberation of these horns in Vancouver harbour as the sound reflects across the water and bounces off the mountains blended these sounds in the original recording into a smooth texture which is enhanced by the time extension. Some of the lower pitched horns begin to resemble voices, while the higher pitched sounds come from the actual voices of young people who were welcoming the New Year. Time extended, these voices transform into long musical notes, and throughout the movement there is an amorous interplay between all of these tones. Hence in the program notes, the 'inner voices' are referred to as those of 'lovers'. Unlike the other movements, Fog uses 12 component tracks which, after the initial CPR fanfare, are designed to blend seamlessly like the visual effect of fog in an environment. However, even within this 11 minute suspension of breath, some underlying temporal structure seemed to be needed, apart from the natural progression and eventual cessation of the horns in the original recording. This structure is subtly present in two forms. The first is a series of percussive gestures, modelled after the boathorns themselves but which are in fact 'frozen' moments of their timbres. This technique is achieved in the so-called 'continuous' model of granulation where the material is re-synthesized at the original speed, but the signal processor's memory may be frozen at any moment by the composer and its contents used to supply a series of grains. Further, the composer may specify a rate of decay to this material and the amount of feedback (if any) of this material into the next 'frozen' moment. In the course of the movement, these percussive elements appear in the first half and recede in the second. The other spatial/temporal element in this movement is a quadraphonic panning of four channels of the source material in the clockwise direction simultaneously with another four channels counter-clockwise. Of course, this is heard only in the four-channel version of the piece, though an analogous panning is attempted in the stereo mix. Because the rate of rotation is relatively slow, it is not expected that the listener will be very aware of it. However, it adds to the constant interplay of seeming stasis in the sound with its gradual metamorphosis; the environment seems stable, yet it's always changing. The image of dislocation by being shrouded in fog becomes a metaphor for the merging of selves experiencing love. With the third movement, Harbour, we once again have a literal title and the sound of water. However, here the water is the seemingly domesticated wavelets that lap invitingly on the shore in a protected environment. These sounds are background to the raucous cries of seagulls as they fight for food, in contrast to the solitary raven in the background of the stronger waves of Ocean. As the movement progresses, so does the amount of time extension given these cries until they become long lyrical songs of striking complexity - a metaphor for the inner wisdom of age that often lies behind a weathered exterior. In this case, the idea of 'inner voice' is a different kind of transformation - the actual voice of the bird, which most listeners find irritating, becomes musical through the the time extension, which results in the sound being heard up to 75 times slower than normal. The structure of the movement loosely follows that of Ocean, with which it shares other similarities, namely that it begins with apparent naturalism and progressively introduces increasing levels of transformation, aided by equalization to emphasize formants. The fll range of transformations possible with both 'continuous' and 'variable rate' modes of granulation are used to impose phase modulation, gating, feedback, and stretching of the material. The use of envelopes of 'frozen' moments of the material (as described above) becomes increasingly insistent in the second half of the movement, ending with events with long attacks, reminiscent of the 'breath' motif of the first movement. These sounds reinforce the ICMC 489

Page  490 ï~~increasingly prominent stretched sounds of the bird cries and the movement ends with an upward octave leap found in one of them. In terms of the related imagery, the movement takes the sounds of nature, now domesticated and familiar within the protective environment of the city harbour, and delves into them with obviously imposed compositional techniques. In the result, it is hoped that the listener discovers a hidden beauty that is metaphorical to the wisdom of age which discovers essential truths in life experiences. The final movement with its metaphorical title, Dragon, is the culmination of the work but it was also the starting point for the composition. Its materials were developed first, and in many ways the entire work was composed in reverse sequence of movements in order to arrive at it. The source material is the spectacular percussion sounds of the Dragon Dance celebration of the Chinese New Year, including drums, bells and cymbals, and strings of small firecrackers that are set off when the Dragon, seeking to be appeased, approaches one of the storefronts. These sounds are time-extended to the point where they suggest the fury of the mythical Dragon. In contrast to the sounds of the Canadian New Year in Fog which were compared with the voices of lovers, the inner voices from the Chinese New Year suggested those of the dead. However, as in many celebrations of cyclical phenomena and in the image of the dragon devouring its own tail, this ritual reminds us that we must confront death in order to welcome the promise of new life symbolized by the New Year. The structure of this 14-minute movement is a microcosm of the work as a whole: four sections alternating energetic and sustained moods. The sustained sections use the same quadraphonic rotation system as the Fog movement, and particularly towards the end of the piece, the image of inner voices is similar to those heard in Ocean, hence the sense of life cycle. The rhythms found in the two energetic sections are modelled after the two principal component sounds of the original recording, the drums and firecrackers, respectively. In the first section, the repeated rhythmic pattern of the drums was imitated and accompanied by a short loop of the firecrackers that advanced approximately 0.2 sec with each repetition to produce complex syncopated rhythms. This material was processed with a digital comb filter to add a rising and falling sense of pitch. In the second energetic section, the sound of the firecrackers is processed in a variety of ways to suggest the buildup of tension and excitement to a climax marked by the recording of the string of explosions (the only untransformed sound in the entire work). In the sustained sections, the stretched sounds of the drum formants provide a deep bass presence, the cymbals a sizzling, broad-band fiery sound, and the cries of the dancers some kind of netherworldly voices (those in Purgatory?). Together, these sounds create an image of the Dragon or Phoenix and suggest the metaphor of death and renewal. In conclusion, the compositional thinking found in Pacific shows the difference between signal processing as 'post production' or 'composing with sound' and an approach that may be termed 'composing through sound'. With the latter, processing techniques are used to reveal the inner levels of meaning and symbolism contained within the timbres of familiar sounds. REFERENCES: Shepherd, J. (1987) "Music and Male Hegemony," in Music and Society, R. Leppert & S. McClary, eds. Cambridge University Press. Truax, B. (1988) "Real-Time Granular Synthesis with a Digital Signal Processor," Computer Music Journal, 12(2), 14-26. Truax, B. (1990a) "Time Shifting of Sampled Sound with a Real-Time Granulation Technique," in Proceedings of the 1990 International Computer Music Conference, Computer Music Association. Truax, B. (1990b) "Composing with Real-Time Granular Sound," Perspectives of New Music, 28(2), 120-134. ICMC 490