/ The AIMS Project: Creative Experiments in Musical Sonification
THE AIMS PROJECT: CREATIVE EXPERIMENTS IN MUSICAL SONIFICATION Reginald Bain University of South Carolina Experimental Music Studio (xMUSE) rbain@mozart.sc.edu Abstract The Applications in Musical Sonification (AIMS) project is dedicated to creative experiments in musical sonification. It studies sonification in the context of algorithmic composition, focusing on the design and implementation of these experiments in software. An initial distribution of cross-platform, open-source software is now freely available that implements some basic mappings, including historically significant compositional formalisms and mathematical models that are widely discussed throughout the literature. These applications can be used for instructional purposes or for experimentation with the use of sonifications as compositional determinants. 1 Introduction In the field of auditory display, sonification is defined as "the use of non-speech audio to convey information." (Kramer et al. 1997) In the context of algorithmic composition, however, the term sonification is sometimes used to refer to "the process of turning non-aural information into sound" for essentially artistic purposes. (Burke et al. 2005) The Applications in Musical Sonification (AIMS) project studies the latter, focusing on the design and implementation of creative experiments in musical sonification in software. The AIMS project also attempts to study the so-called mapping problem and to identify issues, concerns and strategies common to the fields of auditory display and algorithmic composition. Whereas previous studies such as Bargar (1994) and Childs (2003) have attempted to link compositional techniques and thought processes with the design of improved, more intuitive methods of auditory display, the AIMS project attempts to do just the opposite. We use the term musical sonification here to refer to the composer's creative engagement and exploration of the multiplicity of musical ideas that are the potential resultants of a given sonification. Using Cycling 74's MaxMSP and other tools, we create custom software applications to implement these experiments. Each application explores a different mapping of an extra-musical idea, mathematical model, data set, and so forth, into the audio domain, allowing us to study different aspects of the mapping problem. A model for our work is Ben-Tal and Berger (2004), whose study of the creative aspects of sonification focuses on the listener's ability to track changes occurring in variables associated with complex multidimensional data. As the model for their work is creative music listening, they also use the term musical sonification to describe their approach to the study of nonverbal categorical perception in the context of auditory scene analysis. (Bregman 1990) They propose that such abstractions and characterizations are "simplified models of creative engagement with sound." What is more, Ben-Tal and Berger are composers who have used sonifications as compositional determinants in their music. Though we do share some of Ben-Tal and Berger's research goals, it should again be noted that we are not concerned with the design of improved methods of auditory display. Rather, the primary goal of our research is to use the wealth of knowledge emerging from the field of auditory display to suggest new approaches to sonification in the context of algorithmic composition. 2 The Mapping Problem Polansky (2002) succinctly describes the mapping problem in the following way: "an idea in one domain is manifested in [another]." Exactly how should one go about mapping extra-musical data to musical parameters such as pitch, amplitude and timbre? Given the artistic nature of the problem, it is perhaps not surprising that there is no easy answer to this question. In addition to describing the problem, Polansky also discusses the types of distinctions that should be made when determining what should rightly be called sonification and what should more appropriately be called algorithmic composition. Furthermore, he makes a distinction between scientific sonification and artistic sonification, proposing the term manifestation to refer to the latter when in his words, "the intent is clearly to use a formal or mathematical/formal process to create a new musical idea as a form of sonification." (Polansky 2002) We hope to shed light on the mapping problem by methodically 656
Top of page Top of page