Page  1 ï~~INTERACTIVE SWARM ORCHESTRA AN ARTIFICIAL LIFE APPROACH TO COMPUTER MUSIC Daniel Bisig Martin Neukom John Flury Zurich University of the Arts Institute for Computer Music and Sound Technology Baslerstrasse 30 CH-8048 Zuirich Switzerland ABSTRACT The project Interactive Swarm Orchestra (ISO) employs swarm algorithms to create computer music. The project tries to highlight some of the potentials that Artificial Life provides for computer music. The project is motivated by the assumption that Artificial Life and computer music are related disciplines that benefit from a practical and conceptual exchange. In particular, the authors believe that simulations of life-like systems can help to address some of the fundamental challenges of musical creation and performance. This paper elucidates the rationale behind these assumptions. A first practical result of the ISO project consists of a series of programming libraries that are intended to aid in the development of swarm-based computer music. This paper outlines some of the libraries' design considerations with respect to the intended goal of supporting the creation of interesting forms of swarm-based computer music. 1. INTRODUCTION Artificial Life (ALife) assumes a synthetic approach to natural science in that it tries to understand universal principles of life by abstracting and modelling properties of biological systems [1]. Since ALife has emerged as a discrete scientific field in 1987, it has been a source of great inspiration for computer-based art [2, 3, 4]. As a result, many artists have created works that were inspired by some of the concepts and employed techniques from this new and emerging field [5]. As it stands now, much of the initial fascination and motivation that inspired these early artworks and that created a fertile environment for conceptual and practical exchange between artists and scientists seems to have faded away. ALife has matured into a more clearly defined field that declares its own set of grand challenges [6], none of which mention art. Since Ars Electronica has dedicated its thematic focus to Artificial Life in 1993, much of computer-based art has moved towards other issues such as privacy and ownership. Despite the fact, that this initial flurry of activity at the intersection of ALife and art has led to the creation of fascinating artworks, we are convinced that these examples have barely scratched the surface of the underlying potential that ALife possesses for art. Since both ALife and computer-based art have left behind their childhood sandpits, a more systematic engagement and discussion about the relationship of these two fields seems to be necessary. Such an engagement will hopefully give rise to conceptual and practical foundations that pave the way for a new generation of ALife-based art. We believe that the field of computer music can play an important role in reaching this goal. After all, computer music has always had strong connections to science and engineering and has always been one of the early adopters of new technologies in an artistic context. In addition, computer music in itself spans the entire range of a systematic science and a highly creative form of art that is driven by the endless curiosity towards the unheard. 2. ARTIFICIAL LIFE AND COMPUTER MUSIC The following section tries to develop the conceptual rationale that forms the basis for the ISO project. Several claims that are likely to raise objections are part of this rationale. Therefore, the following explanations serve not only to elucidate the motivation behind the ISO project but will hopefully contribute to a general discussion about the relationship between computer music and ALife and about the potential impact of ALife concepts and methods on musical creation. 2.1. Kinship between Artificial Life and Computer Music ALife and Computer Music share a synthetic approach that combines abstraction and creation as part of their research. Both fields blur the distinction between science and engineering and cover the entire range from basic research to application-oriented development. ALife and computer music constitute novel approaches to the huge and traditional fields of biology and music, respectively. By combining know-how from mathematics and computer science, these novel approaches try to distinguish between essential and coincidental constraints that shape the currently known existing specimens of animals and music pieces in order to study life and music as it could be. ALife and computer music heavily rely on computerbased technology as conceptual and practical tools. These tools create the means and spaces within which particular forms of exploration and experimentation become possible; forms that rely on algorithmic abstraction to link speculation with reality and that allow a seamless integration of faithful representations of reality with free creative thought. 2.2. Openness Breeds Challenge Computer technology has fundamentally reshaped the conceptual and practical aspects of musical creation. Computer music in particular provides a vast territory for artistic experimentation and expression. The acoustic qualities of synthetic sounds are not dependent on the physical properties of music instruments nor is the creation of these sounds tied to predefined gestures or conventional interfaces. Furthermore, computer music is unbiased towards particular forms of performance and

Page  2 ï~~can freely shift between improvisational and compositional as well as presentational and participatory styles. This inherent openness and flexibility constitutes the main strength of computer music and challenges preconceived notions of music. But for the very same reasons, computer musicians tend to find themselves in a conceptual and practical void that offers little guidance through the endless space of choices, a void that tends to oppose musical intuitions and that renders the distinction between arbitrary and meaningful choices very difficult. This paper suggests that concepts and techniques from ALife can inspire new approaches to some of these challenges. 2.3. Life-Like Systems Benefit Musical Intuition In the field of ALife, there is agreement that the properties of autonomy, adaptation and diversity constitute essential aspects of life-like systems and therefore form important research topics [7]. For the composition and performance of computer music, issues of structural and temporal organisation, parameter selection and correlation, interaction and control play important roles [8]. We would like to point out, that there exists a high degree of analogy between these issues and the very properties of life-like systems. Computer simulations play an important role in ALife research and serve as models that exhibit some of the properties of life-like systems. By adding means for interaction to simulations of life-like systems, they can become flexible and powerful tools for computer musicians that offer an intuitive and natural approach to handle the complexity of computer music creation. Prior to the description of the ISO project, this paper discusses the applicability of interactive life-like simulations with regard to two topics that are of relevance for computer music: the selection of parameter values for musical algorithms (referred to as the issue of choice) and the topic of gestural control of computer-generated music (referred to as the issue of control) [9, 10]. 2.3.1. The Issue of Choice Computer music offers a large variety of algorithms that create or affect musical structure on a micro and macro scale. The effects of these algorithms typically depend on a multitude of parameter values. The combination of all possible parameter values gives rise to a huge and complicated search space that can't possibly be traversed exhaustively in order to find aesthetically interesting regions. ALife research has derived algorithmic abstractions of biological principles that deal with finding optima within huge search spaces [11]. Famous examples include evolutionary methods such as Genetic Algorithms or Genetic Programming, learning algorithms for Neural Networks such as classical Back-Propagation or Hebbian Learning, and coordination strategies for group activities such as swarm behaviour or ant navigation. These methods have in common that they balance randomised and deterministic aspects for a structured explo ration of an otherwise intractable search space. Throughout the search process, these methods tend to extract statistical patterns from available data and therefore create correlations among parameter values that are meaningful with regard to the problem at hand. These algorithms are well suited to deal with musical search spaces that defy intuition due to their sheer size, high dimensionality or complicated topology. Furthermore, the algorithms' inherent tendency to couple parameter values in meaningful ways reduces the dimension of search spaces and therefore simplifies interactive and real time exploration of musical diversity as well as algorithmic composition [12, 13]. 2.3.2. The Issue of Control Computer music has caused a total separation between gestural activity and musical result that manifests as a dissociation in their respective temporal and physical structure. This dissociation is accompanied by a lack of widely accepted and familiar physical interfaces whose visual and haptic feedback would provide important guidance cues for both the musical performer and the audience [14]. Furthermore, most commonly employed techniques and algorithms that link gesture to music are based on very abstract mechanisms for which it is difficult to gain an intuitive understanding. Accordingly, it is very hard for musicians to quickly make informed aesthetical decisions. We propose that ALife simulations can act as powerful linkage between gesture and music and thereby provide natural and intuitive forms of control. In such a setup, the autonomy, adaptivity and flexibility of the life-like system isolates the musician from algorithmic details and direct control but rather creates an interaction environment that favours improvisation and exploration. This approach tries to balance complexity and intuition by abandoning the requirement for a detailed understanding of musical algorithms in favour of more intuitive and natural forms of musical practice. 3. INTERACTIVE SWARM ORCHESTRA The project ISO [15] attempts to develop conceptual ideas and practical tools that promote research and artistic creation at the intersection of ALife and computer music. The project focuses on swarm simulations as a prototypical example of an ALife-based approach to computer music. Swarm simulations form an important part of ALife research and explore principles of selforganisation and emergence in the appearance of group behaviour [16, 17]. The project ISO provides software tools that are intended to aid researchers and musicians to create diversified types of swarm simulations which are specifically tailored towards the realisation of musical applications. This capability is achieved by a highly generic and versatile implementation of a swarm simulation library (ISO Flock). This simulation can be flexibly coupled with functionality for sound synthesis and video tracking that is either provided by additional ISO libraries or any third party software that possess an interface for OSC or Midi based communication. A description of the technical aspects of ISO is provided in [18, 19]. The latter reference also introduces the programming concepts underlying ISO and contains example source code. While the ISO libraries can be used to create noninteractive and offline versions of swarm-based music applications, we suggest that their most promising potential lies in their capability to deal with interactive musical performance situations. The software's responsiveness and flexibility should encourage spontaneous musical experiments and acoustic explorations. The following

Page  3 ï~~sections describe these situations in more detail and point out those capabilities of the ISO tools that are meant to promote these approaches. 3.1. Swarm-Based Autonomy The ISO tools provide the possibility to transfer some of the musician's autonomy and deliberation to a swarm simulation. The swarm assumes an intermediary position between the musician and the sound generating system. Accordingly, some or even all aspects of musical creation become subject to the swarm's autonomy. The degree of swarm autonomy and the diversity and complexity of its behaviours greatly affect the characteristics of the musical result. The dynamics of the music, its structural organisation and its diversity can all be linked to the swarm's capabilities for self-organisation and emergence. ISO Flock allows the creation of a wide diversity of swarms that range in their complexity from simple reactive systems to highly autonomous and adaptive organisations. Implementations of purely physical simulations such as particles moving in an interactively generated force field constitute an example of a mostly reactive system that responds directly and with little latency to a musician's input. On the other hand, simulations of ecological interactions among different populations of agents such as predator-prey relationships give rise to highly autonomous and complex behaviours. In such a situation, a musician's control might be very limited and indirect and for instance affect the survival rate of one agent population. In ISO Flock, the degree of complexity and autonomy that a swarm exhibits can be changed at any time. This can be achieved either by continuously modifying some of the behavioural parameters or by adding new behaviours or removing existing behaviours on the fly. Accordingly, the capabilities of a swarm and its effect on the generated music can change entirely over the course of a performance. 3.2. Swarm-Based Explorations Most sound synthesis and sound processing algorithms are controlled by a multitude of parameters. Often, only small ranges of parameter values give raise to musically interesting results and these parameter values correlate in non-trivial ways. Swarm simulations constitute a promising approach to deal with these challenges. Swarm agents can roam parameter spaces in a coordinated fashion. Agents may spread out in sparse parameter regions or cluster in promising spots. The dynamics of selforganised spatial distributions that emerge in swarms can be exploited to create correlated changes in parameter values. For example in simulations of ant foraging behaviour, agents establish trails that restrict and channel group movements. An additional example constitute simulations of social organisations. Depending on the intensity and frequency of aggressive actions among agents, hierarchical structures emerge that vary in steepness and stability. In ISO Flock, agent properties and behaviours are im plemented in a very generic fashion. Agent properties represent vector values of arbitrary dimension. In addition, the properties store information about their spatial distance to other properties and therefore provide the basis for proximity dependent agent behaviours. Agent behaviours specify dependencies among properties. The behaviours serve to correlate or constrain the values of agent properties. Because of these implementation principles, a wide variety of swarm simulations can be realised and the simulations can be adapted to deal with parameter spaces of arbitrary dimension. Finally, ISO allows the selective addition of spatial objects such as attractors or vector fields that affect a swarm's traversal of a parameter space. This approach combines swarm based self-organisation with a manual top-down structuring of a parameter space. For example, the musician might want to emphasise particular parameters regions by attracting agents towards these regions. Furthermore, a musician can gradually alter the balance between guided and autonomous agent movements as he becomes more familiar with a musical parameter search space. 3.3. Swarm-Based Interaction The interaction with natural swarms forms a very familiar experience of daily life. Children chasing birds or pedestrians moving in crowds are typical examples. Accordingly, the interaction with swarms has a very natural and intuitive appeal that can be exploited for swarmbased computer music. Forms of interaction that create reciprocal dependencies and inducements between swarm and musician seem particularly promising. In these situations, the direct causality between a musician's intentions and actions and the acoustic result is replaced by a blending of human and swarm behaviours that opens up the possibility for emergent and surprising results. The project ISO provides basic means to generate and exchange control data between interfaces (MIDI instruments and video cameras), swarm simulations and music patches. The swarm's response to this control data depends on the specifics of the swarm simulation's implementation. Examples of interaction-dependent effects on swarm simulations include the addition and removal of agents, the modifications of agent properties, and the manipulation of spatial objects such as vector fields. These effects could be employed to cause interactiondependent changes in a swarm's autonomy. This would allow to gradually alter the quality of swarm-based interaction from direct control to mutual adaptation. In case of direct control, a swarm performs the role of traditional mapping algorithms. Interaction with a highly autonomous swarm on the other hand resembles an improvisation situation where performer and flock engage in a musical dialogue. 4. CONCLUSIONS We are convinced that concepts and techniques from ALife possess great potential for research and artistic creation in computer music and that this potential has hardly been exploited. The creation of interactive lifelike simulations at the intersection between human performer and musical algorithms creates an interesting balance between complexity and intuition for the creation and performance of computer music. This balance abandons the requirement for a detailed understanding and control of musical algorithms in favour of more intuitive and natural forms of musical practice. The project ISO tries to contribute to conceptual discussions and practical creations that deal with the intersection of ALife

Page  4 ï~~and computer music. The development of a set of software tools that help in the creation of swarm-based computer music constitutes a small but hopefully useful step towards the establishment of practical and theoretical foundations for ALife-based computer music. 5. REFERENCES [1] Levy, S. Artificial life: A report from the Frontier Where Computers Meet Biology. Vintage Books: Random House, New York, 1992. [2] Sommerer, C. "ALife in Art, Design, Edutainment, Game and Research", Leonardo Journal, MIT Press, Cambridge, USA, 2001. [3] Stern, A. "Deeper Conversations with Interactive Art: Or Why Artists Must Program", Convergence: The International Journal of Research into New Media Technologies, 2001. [4] Bird, J. and Webster, A. "The Blurring of Art and Alife", Proceedings of Second Iteration, CEMA, Melbourne 2001. [5] Wilson, S. Information Arts.: Intersections of Art, Science, and Technology, MIT Press, Cambridge, USA, 2003. [6] Bedau M.A., McCaskill J.S., Norman H.P., Rasmussen S., Adami C., Green D.G., Ikegami T., Kaneko K., and Thomas S.R. "Open Problems in Artificial Life", Special issue on the Artificial Life VII.: looking backward, looking forward, MIT Press, Cambridge, USA, 2000. [7] Ruiz-Mirazo, K. Pereto, J. and Moreno, A. "A universal definition of life: autonomy and openended evolution", Origins of Life and Evolution of the Biosphere, Kluwer, Norwell, USA, 2004. [8] Bresson, J., Stroppa, M. and Agon, C. "Symbolic Control of Sound Synthesis in Computer Assisted Composition", Proceedings of the International Computer Music Conference, Barcelona, Spain, 2005. [9] Zhang, Q. and Miranda, E. R. "Evolving Expressive Music Performance through Interaction of Artificial Agent Performers", Proceedings of ECAL, Workshop on Music and Artificial Life, Lisbon, Portugal, 2007. [10] Beyls, P. "Interaction and Self-organisation in a Society of Musical Agents", Proceedings of ECAL, Workshop on Music and Artificial Life, Lisbon, Portugal, 2007. [11]Engelbrecht, A.P. Fundamentals of Computational Swarm Intelligence, Wiley, New York, USA, 2005. [12]Flury, J. "Celerina - Development and Implementation of an Interactive Realtime System for Music Composition", Diploma Thesis (german), University of Zurich, 2005 [13]Jacob, B. L. "Composing with Genetic Algorithms", Proceedings of the International Computer Music Conference, Banff Alberta, Canada, 1995. [14] Kimura, M. "Performance Practice in Computer Music", Computer Music Journal, MIT Press, Cambridge, USA, 1995. [15] ISO project website: http://www.i-s-o.ch [16] Martinoli, A. "Swarm intelligence: emergence and self-organization in natural and artificial systems." Course notes, EPFL, 2005. [17] Eberhart, R., Shi, Y. and Kennedy, J. "Swarm Intelligence", Morgan Kaufmann, 2001. [18] Bisig, D., Neukom, M. and Flury, J. "Interactive Swarm Orchestra", Proceedings of the Generative Art Conference, Milano, Italy, 2007. [19] Bisig, D., Neukom, M. and Flury, J. "Interactive Swarm Orchestra, a Generic Programming Environment for Swarm Based Computer Music", International Computer Music Conference, Belfast, Ireland, 2008.