Page  00000001 Hypermusic and the Sighting of Sound - A Nomadic Studio Report Jonathan Impett*, Bert Bongers The Meta-Orchestra, *School of Music, University of East Anglia, UK email:, Abstract In this paper we describe the work of the "Meta-Orchestra " - a variably-constituted laboratory/performance group - in the project "Hypermusic and the Sighting of Sound". We consider aesthetic and practical issues in the networking and the networked practice of a group of interactive systems, and describe the solutions developed. 1 Introduction The present paper describes the work of the MetaOrchestra - a variably-constituted laboratory/performance group - in the project Hypermusic and the Sighting of Sound. This instantiation of the orchestra took place at the Dartington International Summer School in Devon, UK, in August 2000. The project was supported by the EC under the Connect 1999 scheme. The directors Bert Bongers and Jonathan Impett invited three further workshop leaders Richard Barrett, Nicola Bemardini and Ludger Brtimmer, and were joined by a group of nine apprentices and in one concert by saxophonist Evan Parker. A report of 90 pages in full colour including an audio and a video CD describes the group in more detail (Bongers and Impett, 2001), and is available for download from the Meta-Orchestra web site. The primary aim of this project was to investigate the ways in which the use of electronic digital media influences the collaboration between artists in a performance environment. New technologies enable a conflation of the previously discrete activities of composition and performance - Hypermusic - and provide a means of structuring the relationship between music and other art forms such as the visual arts, dance and architecture - the Sighting of Sound. In this new situation, where new technology is used by a multi-disciplinary group, individual roles have to be established as well as the mechanisms for interaction. Conventional divisions and roles can be further challenged by means of electronic networks through which audio, performance information and image material can be exchanged in real time. Of primary interest is the emergence of theory concerning the changing modes of production and relationships of music and other arts in their new technological environment through the situated practice and working together of individual artists. In developing an approach to deal with this new situation this project explicitly places an emphasis on practical aspects. The multidisciplinary team finds its common ground in working together on performance issues. The group of artists together constitutes an 'orchestra', developing its own repertoire and conventions, as well as working on several individual and group projects. This group became known as the Meta-Orchestra. The long-term goal is to establish a 'travelling circus' of knowledge and practice, instantiated in different locations to create, to learn and to teach. In bringing together the individual approaches of different artists, it is hoped to encourage the emergence of common terms, issues and techniques, and to explore these with students confronting the challenge of developing an individual practice of technologically-embodied creativity. 2 Background The wider availability of new technologies - particularly those of computation and sensing - has made possible a deep and irrevocable transformation of the practice of music. With certain well-publicised exceptions, this has remained largely unclear to wider cultural observation for two reasons. Firstly, the use of these technologies in music has been

Page  00000002 developed largely for the commodification of conventional music practice (digital recording and transfer, for example) - the fetishising and marketing of certain cultural phenomena. Projects such as Hypermusic demonstrate that these tools afford the subversion of such a purpose: that they can bring new life to musical practice rather than fixing it at an imaginary point in cultural history. Secondly, the creative endeavour involved in this process has by definition been largely individual - idiosyncratic, even. Above all, these technologies offer the musician the opportunity to re-invent a personal mode of musical activity, plotting new trajectories through the artificially polarised spaces of composition and performance, determinism and improvisation, "serious" and "popular" cultures. Whilst there has been a good deal of sharing of techniques and technologies, the visions, problem-spaces and implementations do not necessarily have shared terms. For this reason, the individual systems which emerge do not always offer obvious modes of interaction with each other. A wide range of emerging musical practices have their roots in these technologies. These approaches have in common the bringing into an informational space of aspects of the physical world (environmental, performance, sound) which could not previously have been structured or structural as "material". Furthermore, common virtual spaces allow the construction of relationships between disciplines which were previously separated by their modes of addressing the human senses. Not only can the cause and effect relations previously implicit in music-making be reconfigured, they can be separated in place and time, raising issues which artists have not hitherto had to confront explicitly. At the same time, the understanding that a music is inseparable from the technologies in which it is embedded is central to the "authenticity" of the early music movement, to modern dance music and to many other cultures. The theoretical tools of western "serious" music are predicated largely on the ignoring of this property. This can lead to artists developing unfounded senses of both alienation and individual radicalism, which militate against the emergence of any common practice - the cultural scaffolding necessary for the development of more complex phenomena. The project brought together a culturally diverse group of fourteen artists from seven countries, each with an individual experience, approach, and technical solution to the possibilities afforded by this new environment. As well as musicians, a video artist and a dancer were invited to participate. The intention was to seek common issues, terms, techniques and technologies which would facilitate the working together of such a heterogeneous assemblage. 3 Technologies This is not the place to describe each individual artist's technological environment. It is relevant to the present discussion, however, to consider the technological world which they represent, and the extent to which they overlap or have aspects in common. Several artists make use of sensing techniques, either in extending conventional instruments or in the design of new performance interfaces (Bongers 2000). Events in the physical world (gestural or environmental) become data that can be manipulated by the computer. It thus exists in a virtual space and is commensurable with other types of material - sound, image or formal process, for example. Whether such data is regarded as material for manipulation or a means of control is likewise open for dynamic reinterpretation. The lingua franca of interactive music is Max/MSP, used by several members of the orchestra. Most live sound processing was performed in MSP or in Miller Puckette's Pd. Studio STEIM's LiSa was also used, which adopts the more specific strategy of using performance data to control the live recording and manipulation of sound. An interesting aspect of recent developments in music software is that the distinction between "real-" and "deferred-time" modes of working has become one of paradigm or functional model, rather than time-frame. Many of the metaphors through which musicians address the abstraction of digital art refer to previous working practices: modular synthesis, acoustic instruments, the tape recorder and the score. The functionality now overlaps to a greater extent however, as evinced by the composition/performance environments of several members of the orchestra. Pd was used in some contexts for the graphical implementation of models conceived in Csound, which itself was used as a real-time sound generator controlled graphically or via MIDI. The Max environment was also used for the real-time manipulation of video material, as described in a separate paper (Harris and Bongers, 2001) One of the orchestra members - Jonathan Impett - used the Swarm complex systems simulation package (Santa Fe Institute), which is an example of the increasing transference of paradigms from other disciplines to music. This ALifebased approach allows the modelling of dynamic populations of interacting local musical activities to encourage the emergence of higher-level structure. As each exists in its own virtual space, this can be made up equally of real-world or computational factors, such that human intention or environmental circumstance can be instrumental at any level. This model thus mirrors the dynamics of the orchestra as a whole. Such dissolving distinctions are characteristic of the situation of a project such as this. Without there being any "barrier-crossing" dogma in play, the emergent, situated and individual redefining of roles, materials and working practices became central to the research of the orchestra.

Page  00000003 4 The Networks A central principle of the project was to connect all the electronic instruments / systems of the orchestra members together in a network. It was originally intended to use the fast Ethernet network for all of communication between the orchestra members, including audio, video and performance data (see The Video-Organ, elsewhere in these proceedings). This challenge was severely enhanced by the multi-platform nature of the network, and in practice this proved to be a hurdle that could only be overcome pragmatically by creating some added layers in the network. Several experiments and developments took place however to write multi-platform audio server software, further discussed in Nicola Bernardini's section in the report. the gesturality of one performer and the software or material of another; some constructed, some serendipitous. By this process, "higher" levels of structuring gesture fell to the most temporally abstract activity. Orchestra members shaped, processed and scheduled sounds "outside time" but during the performance, to provide the formal pillars of the work, generated and then performed these sounds in their own context - real time composition in the truest sense. These were the only sounds to be distributed uniformly throughout the space; the single layer of common "meaning", when all other activity focused on reinterpretation and state change, on the use of performance as material. Visual signals could in future be incorporated as a dynamic score using the network to communicate and shape the overall performance (composition) in real time. 5 Communication Within the Orchestra 6 Experience and Invention in the Lab The placement of artists and their working environments in the physical space was conditioned by issues similar to those described in the previous sections. As the internal structure and dynamics of the orchestra evolved, it became clear which groups should work at the same spatial location, how the performance as a whole could be structured by the physical distance between others, and where a degree of visual or acoustic separation was necessary. Instrumental gesture is a vital element of communication between musicians during performance, and players become very adept at recognising the gestural "language" of other instruments. The wide range of physical control represented within the orchestra reflects the temporal relationships of different artists with their material: * the immediacy of acoustic instruments * computer-extended instruments projecting events forward in time * gestural control of real-time processing of sound and image material (present or stored) * real-time synthesis with some degree of performance control * computer composition and reproduction (not in real time; but this concept refers to the time-base of the operation, which may in fact be faster than real time) It was felt that a vital level of communication required addressing, dealing with a broader range of activity than is the case in conventional performance. Not only issues of synchronisation, balancing and shaping, but also structuring activity; compositional gestures, in effect - suggestions, instructions and indications which cannot be read from a representation of performance parameters. In the new technological context, musicians can aspire to perform compositionally - to structure, recall and project forwards in time. "Universal" solutions did not present themselves as being appropriate to these contextual and critical purposes. Rather, relationships, affordances and fits emerged between The sharing of performance data became a central topic of discussion and experiment. Common access to performance information and its intentional "communication" are clearly related situations, yet in some ways they demand to be treated quite differently. Both depend on some degree of agreement or openness to re-interpretation. Both construct an informational mapping which may produce some sort of phase transition in the transformation of the system in question - the information cannot "mean" the same when placed in another context. Paradoxically, the most precise informational representation of activity affords the highest level of abstraction or re-use, for example, the continuous output of a physical sensing device on an extended instrument. More general instructions or indications of intention rely to a greater extent on an agreed contextual significance; their internal dynamics are less likely to be of interest or reveal emergent structures. A universalist approach suggested making available to all artists of the orchestra every parameter of each individual system which is represented numerically, expressed as MIDI data. In an extreme and simplistic case, this would enable the construction of virtual avatars for each performer, or hybrid characters. Experiments showed that the computational burden, replicated in each system, made such a scenario unworkable. Agreement emerged that a better solution would be for a server to transparently collect performance data from each system, and forward particular streams "on demand", the request triggered by whatever mechanism is appropriate to the requesting system. This would enable an artist to select a specific aspect of another musician's performance and to incorporate it into their own, as material or as an external controlling parameter. A pragmatic solution was found in a software object developed for Max. OSC, Open Sound Control, (Wright and Freed 1997) was intended as a means for sharing packets of performance data across a network. A specific channel of

Page  00000004 communication is established, through which packets of MIDI messages can be sent. In the present context, this mechanism raised two issues: how to implement this so that the shared data represents events as they occur, rather than accumulating lists of past events; and how to open such a channel as a receiver, rather than a sender. The first was addressed by the effective but inefficient means of sending each message as soon as it takes place. The use of this protocol thus becomes an active gesture on the part of the sender, not the transparent layer that would be ideal. As OSC insists on its reception, the opening of channels requires a degree of negotiation and co-ordination. Shigeto Wada developed an elegant common tool in the shape of a pair of stand-alone Max patches. They allow the user to select destinations and sources from a menu by name, without having to deal with the mechanics of networking. There remains a level of inter-dependence, in that the loss of transmission in either direction can cause instability. The common technologies, programs and protocols belie hard-wrought and finely-characterised differences between individual approaches which, as working environments, not theoretical or public constructs, are often not explicitly defined. To design them into a network on the basis of apparent commonalities would therefore do violence to such personal processes. The dynamic, situated construction of a common environment which encourages and facilitates the emergence of relationships seems more fruitful. 7 Lessons The combination of a "laboratory" environment with regular performances was felt to have been very conducive to productive work. The networking of ideas and approaches was as important as that of performance data and material, but required more complex structuring; an implicit property of the design of many "interactive" systems seems to be that they do not afford interaction with those of other artists. A further result of this environment was the networking of musical empowerment. Musicians who had considered themselves primarily performers had been taking compositional decisions and structuring works. Composers found themselves practicing their craft in real-time, in performance, and modifying their actions in interaction with other musicians. Members from a technical background were fundamentally involved in the design of specific aesthetic activities and their inter-relating. For some members, this was the first involvement in live performance. All therefore felt that the scope of their own creative activity had been substantially broadened. We refer here to the participants section in the project report for further details and opinions. It is clear that the chosen structure, of bringing together a heterogeneous group of artists in a master-apprentice like situation rather than in a naturally more fixed institutional situation was successful. The concept of an 'orchestra' proved to work as a model to facilitate collaboration between people with different backgrounds, approaches, disciplines and media. We are encouraged to continue with this model with a more diverse group, including disciplines such as theatre, architecture and design. From a technical point of view, the chance to experiment with a large group of networked computer-based artists was unique and invaluable. Such a network is a rare circumstance, usually heavily time-constrained, requiring a great deal of co-ordination, and often being far from robust. The network becomes an important instrument, now in its first phase of development, with enormous potential for further development. We aim to incorporate a wider diversity of cultures and disciplines, brought together by the common grounds of performance and the use of electronic media. By constantly changing the appearance of the organism that the project is, rather than its essence, it is hoped that development will rapidly evolve towards solutions. 8 Further Work A concert of the orchestra took place in Amsterdam at Felix Meritis in February 2001, and several smaller "metaensembles" are active at the moment. Throughout the project a presence on the web was created and maintained in order to describe the project and announce performances. The web site will remain active as a hub for exchanging information and communication, between instantiations of the orchestra in the real world. 9 Acknowledgments This part of the project was made possible by a grant of the European Commission CONNECT 1999 programme. It was further supported and administrated by the Dartington International Summer School (Gavin Henderson, director, and Lisa Warren, administrator). The Institute of Sonology at the Royal Conservatory hosted a session and supplied a substantial part of the equipment. The IJsbreker co-organised the presentation and concert in Amsterdam. References Bongers, A. J. 2000. "Physical Interaction in the Electronic Arts: Interaction Theory and Interfacing Techniques for Real-Time Performance". In: M. Wanderley and M. Battier (eds.)Trends in Gestural Control of Music, IRCAM Bongers, A. J. and J. F. Impett. 2001. "Hypermusic and the Sigthing of Sound". Project report for the EC, 91 pages Harris, Y. C. and A. J. Bongers. 2001. "The Video-Organ". Proceedings of the International Computer Music Conference. Havana, Cuba. ICMA Wright, M., and A. Freed. 1997. "OpenSound Control: A New Protocol for Communicating with Sound Synthesizers," pp. 101-104, Proceedings of the International Computer Music Conference Thessaloniki, Hellas. ICMA.