Page  00000580 An Acousmatic Composition Environment Morten Breinbjerg*, Ole Capranit, Rasmus B. Lunding,t,Line Kramhoft, *Multimedia Studies, Aarhus University, Denmark tComputer Science, Aarhus University, Denmark, Abstract In this paper we describe the intentions, the design and functionality of an Acousmatic Composition Environment that allows children or musical novices to educate their auditory curiosity by recording, manipulating and mixing sounds of everyday life. The environment consists of three stands: A stand for sound recording with a soundproof box that ensure good recording facilities in a noisy environment; a stand for sound manipulation with five simple, tangible interfaces, a stand for sound mixing with a graphical computer interface presented on two touch screens. 1 Introduction Since February 2005, an Acousmatic Composition Environment has been part of the permanent exhibition of the Danish science center "Experimentarium" located nearby Copenhagen, [1]. The Acousmatic Composition Environment allows for the exploration of everyday sounds and for the creation of musical objects and structures by the use of recorded and manipulated sounds. In the design of the environment, we have been greatly inspired by the French composer and theoretician Pierre Schaeffer. Especially his concept of "Acousmatique", his accentuation of audibility, and his compositional approach of sound recording and mixing have been influential to us. In everyday life sound is indexical to the source and the environment of which it belongs, [2]. Listening to the sound of everyday life as a musical object one need to disregard the source of the sound and focus on its inner qualities. Pierre Schaeffer used the concept of "Acousmatique" to designate this last way of listening and called for a reduced listening process (l'ecoute reduite) in which the sound object was isolated, studied and experienced out of context for the sake of its own timbre and shape, [3]. He achieved this by means of sound recording and techniques of looping and tape speed variation among others. Thus the loop became not only a technical term for the repeating of sound, but his fundament for a new way of listening. In our project we have created an environment in which a reduced listening process can be performed and in which children/novices are able to manipulate and compose with sounds of everyday life. That is why we have chosen the name: "Acousmatic Composition Environment". 2 An Acousmatic Composition Environment The Acousmatic Composition Environment consists of three stands: a stand for sound recording; a stand for sound manipulation and a stand for sound mixing, Figure 1. Each stand enables sound experiments and exploration and in every stand the sound is played back on stereo loudspeakers integrated into the physical design of the stands. Figure 1: An Acousmatic Composition Environment at the Danish science center "Experimentarium". The sounds available in each stand are either fixed, prerecorded sounds or sounds created and recorded by a visitor. The pre-recorded sounds at each stand secure that visitors can start manipulating and/or mixing sounds without first having to record sounds of their own. This is important in the context of the science center where visitors can freely enter the stands in their own order. The visitor's sounds are saved to and retrieved from a sound database. From all of the three stands the visitor can access his/her sounds in the database by means of a unique barcode for each visitor. This enables sounds recorded in the recording stand to be manipulated in the manipulation stand and later mixed into a visitor created sound composition in the mixing stand. 580

Page  00000581 Access to the sounds and sound compositions of each by using one or two hands. Every handle include sensors visitor is also available as a net service. A visitor can connected to a Teleo module, [4]. The sensor data that download his/her sound composition as an mp3 sound and captures the gesture from one handle is send to a Max/MSP e.g. use it as a ring tone in a mobile phone. patch [5], as input to one sound transformation algorithm 2.1 The Sound Recording Stand where it controls one or more sound parameters. The stand for sound recording includes a Sound Box, Figure 2. The Sound Box is made of Plexiglas and designed as a standard laboratory box with a microphone inside. The purpose of the box is to isolate the sound as it is produced by the visitor. Placed inside the box is a variety of small everyday objects made of solid materials like metal, wood, plastic, glass, etc. The sounds that can be produced by the use of these sound objects covers a wide range of different sound types in both time and frequency domain e.g. short and long attacks, harmonic and inharmonic spectra. The signal from the microphone inside the box is connected to a computer so that the sounds produced inside the box can be listened to immediately through the speakers inside the stand. A simple graphical interface on a touch screen placed within the box specifies fundamental recording functionalities like record, stop, play, store and save. When played, a recorded sound is automatically looped as a first step towards a reduced listening process, e.g. rhythmic patterns emerge, especially when short sounds are looped. Figure 3: The 5 Manipulators. From top left to right bottom they are: a wheel, a ball, a steer, a gear lever and a roll. As shown in Figure 4 the five manipulators can be operated at the same time and the sound transformation algorithms work on the sound source in parallel. In this way the transformation algorithms work independently on the recorded sound. Consequently you can produce one or more sounds that are perceived e.g. as low frequency background sounds together with high frequency melodic structures when a single sound source is manipulated. input 1 input 5 Source F 2Mieduleo..I.............................. I I sstbFigure 2: The Sound Box Mixe 2.2 The Sound Manipulation Stand The stand for sound manipulation holds five Sound Manipulators that interface to sound transformation Figure 4: Flow chart of input signals algorithms and control transformations of a recorded digital sound in real time, Figure 3. Sound Manipulators are physical palpable objects with different kinds of handles. Each manipulator has only one handle that can be controlled 581

Page  00000582 2.2.1 The Sound Manipulators The Sound Manipulators are small physical interfaces that vary in shape, material, and functionality and each allows a specific sound manipulation. Each manipulator can be used on already recorded sounds that are looped while the manipulation takes place. In accordance with the context (the science center) each manipulator is designed to be robust and simple to use. Furthermore, the output of the sound transformation is one dimensional in the sense that the visitors will experience continuous sound transformations in both the time and frequency domain i.e. from dark to bright, noisy to harmonic and fast to slowly evolving sounds. Every manipulator has a neutral position in which no transformation takes place; hence the original, recorded everyday sound is heard. Only by continuously pushing, pulling, squeezing or turning the handles, the sound transformation will be applied. If the visitor loosens his/her grip on the handle it moves back to its neutral position like a pitch bender on a keyboard. Five different manipulators have been developed for the Acousmatic Composition Environment: The wheel, the ball, the steer, the gear lever, and the roll. Below we will describe three of them in greater detail. 2.2.2. The Wheel The wheel is like a turntable. It is made of steel with small sticks sticking out from the side at regular intervals, Figure 3. This makes it easier to get a grip on the wheel. As the name indicates the wheel can be turned one way or the other and can be brought to spin. The speed of the wheel is measured by the use of a potentiometer and the speed is used as input for a Max/MSP patch. When the wheel is not turned it slowly decelerates. When it stops the sound is not manipulated. The sound transformation uses granular synthesis techniques to cut up the sound in grains. The speed of the wheel controls the grain lengths, how often the grains are activated and the pitch of the grains. The more speed the smaller the grains, the more often they are played, the more variation in pitch. 2.2.3. The Ball The Ball is a hemisphere made of soft rubber and filled with air, Figure 3. The ball can be squeezed and the more pressure the greater impact it has on the sound being manipulated. The amount of air squeezed out of the ball is measured in a pressure gauge and the amount is sent to a Max/MSP patch. When the visitor lets go of the ball, the ball fills with air and returns to its initial state. The sound transformation uses a three band low pass filter that cuts off the high frequencies. The cut off frequencies move from 18 kHz to 75 Hz in accordance with the amount of pressure that is put on the ball when squeezed. Also an FFT is applied to the sound to approximate the fundamental frequency. This is used to modulate the filter cut-off points. 2.2.4. The Gear Lever The gear lever can be pulled from the top end towards the visitor, Figure 3. When placed in the top end no sound transformation takes place. The more the gear lever is pulled the greater impact it will have on the sound. The handle of the gear lever connects to a mechanical pump. The resistance in the pump builds up the more the handle is pulled. As a consequence the visitor has to use more power the more the gear lever is pulled downward. The position of the handle is measured by a potentiometer and the position data is send through a Teleo module to a patch in Max/MSP that performs the sound transformation. The sound transformation uses the "bong-" and "paf-" objects developed by Miller Puckette, [6], [7]. It keeps the rhythmic and dynamic attributes of the original sound, but gives the timbre a more and more synthetic like character as the handle is pulled downward. 2. 3 The Sound Mixing Stand The stand for sound mixing consists of two touch screens with a graphical interface designed as a ten track sequencer. Each track loop's a manipulated sound. The sound can be muted, pitched in a pentatonic scale and its volume controlled by a staircase envelope with a time resolution of 250 msec, Figure 5. A number of preset buttons on the screen can be used to select different staircase envelopes for all the tracks. This gives immediately access to different ways of mixing the sounds. Figure 5: Graphical interfaces for sound mixing, presented at two touch screens. 582

Page  00000583 3 Discussion We believe the most important aspects of our work are: 1) The construction of an environment that allows children/novices to work with everyday sounds as musical material. 2) An environment that allows for recording, manipulation and mixing of sound objects. As a result a child/novice can perform the basic steps in the creation of a sound composition and e.g. use it as a unique ring tone. The "Acousmatic Composition Environment" deliberately and continually pursues an aesthetic of music that connects to the acousmatic music of Pierre Schaeffer and to sound art experiments, [8]. By this we wish to underline the importance of listening and to strengthen the ability of children/novices to listen in a qualified manner. The use of everyday sounds as opposed to e.g. instrumental sounds, confronts children/novices with a musical material that they are normally not ask to listen to. As such we emphasize computer technology as an extension of audibility and regard the Sound Manipulators and the Sound Box as media technologies that allow children/novices to hear the world differently and to immerse into a world of sounds that we expect is not normally listened to. 3.1 The Sound Box The Sound Box is a fairly simple, but yet very efficient way to record sound in public space. Normally sound recording in public space suffers form the noise of the surroundings, but in the Sound Box, which is coated on the inside, the sound producing event is isolated. When the prototype of the Sound Box was tested,[8], it was obvious that children/novices engaged in listening to the sounds that everyday objects produced. 3.2 The Sound Manipulators As David Wessel and Matthew Wright suggest, [9], the major advantage of computer based instruments is the possibility of "immense timbral freedom". They believe it should be relative easy to start playing a computer based instrument but points to the fact that making an instrument easy to play often is contradictory to the sounding complexity of that same instrument. As a consequence a "simple-to-use" computer based instrument quickly gets a "toy-like" character. The instrument - the audio output - is not complex enough to encourage a continuing exploration. We find this argument to be a strong one and most relevantin our context. Our response to this argument is twofold. First we try to secure the ease-of-use by making the physical interfaces quite simple and well-known although not necessarily in a musical context. In the design of the five manipulators we have (as so many before us) been greatly inspired by Gibson's term affordance, [10], which has been introduced into the interaction design community by Donald Norman, [11l]. It basically states that form giving should invite effective action since affordance concerns the relation between appearance and action [12]. We believe the shape of each manipulator expresses the kind of action necessary to operate it. We find this to be crucial in the present context and therefore we have chosen to use well known interfaces like a wheel, a steer, a gear lever, a ball, and a roll. As such the familiarity of the interfaces chosen is meant to ease the understanding of performance: A wheel is for spinning, a gear lever for pulling and a soft ball for squeezing. Hence a visitor can easily figure out what to do with each of the five manipulators. In other words, the way to operate the Sound Manipulators is not very advanced or exotic, but it opens a vast space of aesthetic experience with the sound objects produced and recorded in the Sound Box. Furthermore, the sound transformations react immediately to even the smallest change of the interface. Not in the sense that the sound changes suddenly in all its parameters by the least touch of the manipulator, but in the sense that even a small gesture is audible, letting you feel immediate control over the sound transformation. To strengthen the feel of control we have designed the interfaces to demand continues physical input. Otherwise the manipulator will settle at its initial position where no transformation is taking place. By continuously having to use the power of his/her muscles to control the interface and thereby the sound transformation, the visitor constantly receives audible as well as physical feedback and most important of all a feedback of the correspondence between the sound and gesture performed. As such our interfaces match the interfaces developed by Dominic Robson in the sense that they "all incorporate a physical change in the sound along the physical continuum", [13]. This we believe is very important; mapping the gesture to the audible result lets child/novice feel that he/she is in charge of the audible sound. Second, we have provided algorithms that can produce rich timbral variations. Despite the simple one handle interfaces the complexity is secured by letting the input control several parameters in the sound transformation. Also, the variability of the sounds produced is not only dependent on the complexity of the sound transformations, but also on the recorded sound which varies according to the way the visitor play with the small everyday objects within the Sound Box. 3.3 The Sound Mixer The Sound Mixer does not reflect our initial proposal. We proposed to make the sequencer as a table with holes, into which small containers holding the manipulated sounds (by the use of RFID tags) could be plugged. Because of the context of the Science Center the idea were refused, since the containers can be moved away or even stolen. As the Sound Mixer Stand in the current setup does not reflect our wishes, we choose not to comment on the design further in this paper. 583

Page  00000584 3.4 Related work Our use and design of tangible interfaces clearly points in the direction of previous research carried out by Gil Weinberg and Seum-Lim Gan, [14]. Like their project on"The Squeezables" our project stresses the physical design of the interface as an important way to encourage new ways of interaction. Other projects like Blok Jam, [15], or Agroove,[16], have inspired us in some aspects of our work, e.g. in using tangible interfaces to interact with prerecorded sound segments. Like Blok Jam our original idea of the mixer board uses physical objects as "sound containers" and like Agroove our system allows for live manipulation and mixing of musical structures. However, the aesthetic approach of the "Acousmatic Composition Environment" is fundamentally different. Pursuing the aesthetics of making music by recording, manipulating and mixing sounds of everyday life, we encourage children/novices to reveal the musical quality of the sound object it self. As such our goal is to cross the boundary between everyday listening and music listening. Instead of making music with MIDI notes mapping the control information of the Sound Manipulators to the MIDI standard as in Squeezables and thereby remaining within the pitch paradigm of music, we offer children/novices the opportunity to seek the music in everyday sounds. 4 Future Work As mentioned in the beginning of this paper The Acousmatic Listening Environment is part of the permanent exhibition of the Danish Science Center "Experimentarium". In the future we will perform a thorough investigation on how the children/novices use the environment. We had hoped to be able to monitor the installation by video in order to gather the information necessary to validate the human-computer interaction, but we have not yet received the permission to do so, since the science center is a public space. One further development that we would like to develop is a physical mixing table. As mentioned earlier we are not entirely satisfied with the touch screen solution. We believe a physical sequencer and physical objects as containers for recorded and manipulated sounds will be in the line of the overall concept in a much more satisfying manner. 5 Conclusion The goal of our project is to design and develop an Acousmatic Composition Environment in which children/novices can perform a reduced listening process, record, manipulate and mix sounds of everyday life. We are happy to see the environment in a context like a science center and hope in the long run to get valuable experience of the actually strength of the idea and the robustness of the system. Hopefully, the children/novices will get a playful exploration of digital sound and an emotional experience of fun, involvement and beauty. We encourage them to listen in a qualified manner. 6 Acknowledgment Thanks to Jesper Rasted, Christina Fromberg and Niels Hornstrup from the Danish science center "Experimentarium" and ISIS Katrinebjerg for financial support. Please note that this paper is a slightly extended version of a paper previously presented as a poster at NIME 2006, IRCAM, Paris and printed in the proceedings of NIME 2006. REFERENCES [1] The website of the Experimentarium can be reached at [2] Truax, Barry. 2001. Acoustic Communication 2. Edition, ABLEX Publishing, Westport, Connecticut. [3] Schaeffer, Pierre. 1966. Traite des objets Musicaux. Paris, Edition du Seuil. [4] Making things, [5] Cycling74, [6] Puckette, M.,Apel, T., Zicarelli, D. 1998. Real-time audio analysis tools for Pd and MSP, Proceedings, ICMC 98 [7] Puckette, M. 1996. Formant-based audio synthesis using nonlinear distortion, Audio Engineering Society. JAES 43/1, pp. 40-47. [8] Breinbjerg, M., Caprani, O., Lunding, R. 2002. Touch the Sound - the concept of the "object Sonore" as a design metaphor. Proceedings of the International Workshop of "Interaction Design and Children", Eindhoven. [9] Wessel, D., Wright, M. 2002. Problems and Prospects for Intimate Musical Control of Computers, Computer Music Journal, 26:3, pp. 11-22. [10] Gibson, James J. 1986. The Ecological Approach to Visual Perception, Lawrence Erlbaum Associates, Publishers, London. [11] Norman, Donald A. 1998. The design of every day things, MIT press, London. [12] Djajadiningrat T.,Wensveen S., Kees Overbeeke J. F. 2004. Tangible products: redressing the balance between appearance and action. Springer-verlag London Limited. [13] Robson, Dominic. 2002. PLAY!: Sound Toys for NonMusicians, Computer Music Journal, 26:3, pp. 50-61. [14] Weinberg, Gil and Gan, Seum-Lim. 2001. "The Squeezables: Toward an Expressive and Interdependent Multi-player Musical Instrument". Computer Music Journal, MIT Press. [15] Newton-Dunn, H., Nakano, H., Gibson, J. 2003. Block Jam: A Tangible Interface for Interactive Music, Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada [16] Poupyrev, I. 2000. "Augmented Groove: Collaborative Jamming in Augmented Reality" in SIGGRAPH 2000 Conference Abstracts and Applications, ACM Press, NY, p.77 584