Page  00000001 - A Quintet on the Internet Georg Hajdu Hamburg Institute for Microtonal Music and Electronic Media (HIMEM) Hochschule fuir Musik und Theater Hamburg email: Abstract is an interactive Internet performance environment enabling up to five performers to play music in real-time over the Internet under the control of a "conductor." The environment consists of four components: a server, a client, a conductor and a listener. The players interact over the Internet by sending musical (control) streams to the server. In addition, the conductor can log in and control the musical outcome by changing settings remotely, either manually (improvisation) or by executing a score (composition). has additional video and graphical properties, which allow for better interaction and control on a symbolical level: The performers see the music they produce on screen in "space" notation. The conductor, acting as video DJ, can mix films or live video streams to create a truly medial Gesamtkunstwerk. Several compositions have been specifically written for or used it as an integral component, such as the Munich Biennale opera Orpheus Kristall. Orpheus Kristall was conceived by a team consisting of composer Manfred Stahnke, librettist Simone de Mello, director Bettina Wackernagel, literary manager Peter Staatsmann, set designer Stephanie Wilhelm and web designer Bettina Westerheide under the aegis of Tilmann Broszat (Munich Biennale for Contemporary Opera) and Jens Cording (Siemens Arts Program). My task primarily consisted of the development and preparation of, an interactive, networked, real-time performance environment that enables interaction between musicians in and via the Internet. During a performance with, the audience follows the Internet events projected on a screen. On May 3rd, 2002, in the opera Orpheus Kristall, for the first time Orpheus journeyed into the Internet instead of into the Underworld. This occurrence symbolized the increasingly profound relationship which opera is entering into with the new media. The new media make it possible to simultaneously present both real stages and virtual ones and to extend the space of operatic events to a global dimension through the inclusion of the Internet. consists of 4 program components that are connected in a local network or over the Internet. As the name implies, up to five participants can be involved in an Internet performance; however, "participant" does not indicate one individual performer in this case, but rather a location in which several performers can be present. The program consists of four components and runs on the Apple Macintosh platform. It was programmed with the graphical software MAX, with the addition of the audio and video extensions MSP and Jitter. The core of is the server, to which all of the

Page  00000002 computer screen in graphic notation. The notation occurs on five grand staves that correspond to the music played by the performers in the five locations. During a performance the conductor monitors the course of the musical events. He/she can manipulate the sounds, as well as the effects, by means of remote control over the client and server computers, and is in contact with the musicians through verbal directions that appear in a text window visible to the client. A chat window facilitates the exchange of more complicated issues by the participants (including the listeners). The conductor can also send short scores, which are to be performed as precisely as possible, and determines the form of the background sounds (granular synthesis performed on the sounds stored in the sampler) which occur simultaneously to the clients' performances. In certain performances, he/she is additionally in contact with the listeners, who can influence the course of the music through voting. The screen content of the client program users first log in with their computers. In addition to managing the data streams, the server also assumes further musical tasks, by which it manipulates the incoming music similar to an effects machine. The term "users" comprises up to five performers, the conductor, and the Internet audience, each of which have the corresponding client, conductor or listener program components loaded. At login, the server is informed of the IP number as well as the name and location of each user. Now the musicians can play their lines into the computer and send them on their journeys to cyberspace. Playing the music into the computer, which is only possible with the client program component, occurs through a microphone, a MIDI controller or simply the computer keyboard. If the music is entered with a microphone, a pitch tracker integrated into the program (via Miller Puckette's fiddle-) provides for the recognition of the pitches that are played and their transformation into corresponding MIDI signals. Spectral information can also be sent; however, this option is of lesser significance for making music via the Internet. The client uses a modified MIDI protocol for the transmission of musical events. It employs 53-tone equal temperament instead of the 12-tone standard tuning, thus allowing greater melodic and harmonic richness and flexiblity. Having arrived at the server, the data packets are multiplied and sent to all clients (including the initiating computer) and listeners. Before this, however, the option exists of modifying the data with effects algorithms. These include filters, harmonizers and transformation effects that can alter the character of the music dramatically and allow the players to also perform duets with themselves. Once the data packets have arrived, they are evaluated; during this process the phrases are liberated from their numerical cyber-existence and transformed once again into audible pitches by means of an integrated sampler. At the same time, the notated form of the music appears on the 2 The conductor monitors the output of the performance and is in contact with the performers and the cyber-audience. Finally, the listeners operate a program component that is identical to the client program except for the option of entering music into the computer. Instead, this program component possesses a further window with a questionnaire concerning important aspects of the performance which can be filled out and sent off by the listeners at any time. This allows the cyber audience to offer feedback, which is usually perceived by performers in the form of "vibrations" in the concert hall setting, and which can have considerable influence on the course of an interpretation. This form of feedback, which has a great similarity to political-democratic processes, utilizes the interactivity that the Internet makes available to the online-audience through its up- and down-stream capabilities. In general, Internet performances give rise to the question as to how the aesthetical experience depends on the pres

Page  00000003 * Polls....................................:D i............. The cyber public has the option to influence the outcome of a performance by submitting questionnaires. ence of musicians. In a performance of the composition Mind Trip, in which was implemented for the first time and took place both in cyberspace as well as on a real stage, it became clear how difficult it was for both the musicians and the audience to orient themselves. The musicians had to become accustomed to being present in both spaces simultaneously, while the audience desired a clearer orientation in the sonic space with specific identification of the acoustic sources and their creators. was subsequently reworked and now also uses QuickTime broadcasting as a further level; this enables all musicians - as long as they are performing in front of a video camera - to experience each other in their actions. If the monitor is projected onto a screen, the audience can also share this experience. now makes extensive use of Jitter (the MAX matrix processing objects) by mixing and processing the live video streams as well pre-produced streamed video. Thus, the conductor can perform as a video DJ, too. Through the use of a multi-channel sound system, each player can be associated with a specific loudspeaker. The lack of physical presence in the Internet and the anonymity that arises from this can, however, have a desired effect: through masquerade and the adoption of artificial identities. This concept is promoted to its limits by the company m9ndfukc, the creators of the real-time video processing software nato.0+ 55. In Internet mailing lists, multiple identities trade under the name of nn or Netoshka Nezvanova; at seminars, a different person appears each time - all under the same name; few know the true identity of this person/these persons. allows for this form of masquerade: Precisely because of the pitch tracker's transformation of the music into abstract information, which is then retransformed into pitches on the user's computer, a sound which is that of a flute in its origin can become the sound of a violin. The performer can also change his/her instrument unobserved, or switch with another musician without this being noticed by the others. By this means, a shortcoming of Internet communication becomes aestheticized and serves as a springboard for the invention of new forms of communication. A further essential problem in Internet performances is that of the synchronicity of musical events, the solution of which highly resembles an attempt to square the circle - at least in the Internet's current structure. When a musician plays a phrase into the computer, it cannot be exactly determined when this phrase will arrive at the computers of the other performers. Typical delays span from 200 milliseconds to 1000 milliseconds, depending on the distance between the performers. Since the ear perceives deviances of as little as 50msec as being "out of beat", the synchronous performance of exactly notated scores is out of the question from the start: The Internet thus gives birth to its own aesthetic. There are two strategies in compensating for this synchronization problem, one that is event-based, and one that is sequencebased. In the event-based strategy, all of the events (for example the beginning and ending of a tone) are given a time-stamp before they are sent from the client and subsequently to all of the other participants. When they have arrived at their destinations, the time-stamps are evaluated and - presupposing a certain average time delay, played back with this delay in comparison to the time-stamp. By this means, events can be synchronized to some degree, since they are, so-to-speak, pushed into the future. Eventually, this strategy was adopted for "Phrase sampling" is another strategy, by which groups of events with fixed time structures - sequences - are sent instead of individual events. These sequences have the advantage that the duration of a given note is always known. Should events get lost in the Internet, it is still possible to avoid the infamous "hangs", in the case of which long "stuck" notes can only be turned off by a "panic" command (the communication of the participants is based on the faster but also less reliable User Datagram Protocol [udp] - using Matt Wright's OpenSoundControl and otudp objects for MAX/MSP). Due to the use of phrase sampling, several seconds occasionally go by (the length of the phrase plus the time delay) until the phrase can be made audible on all of the computers; however, the real-time notation offers the opportunity to compensate for this condition to a certain degree, since the phrase is visible in its entirety and thus enables the musicians, who find themselves in a mixed form consisting of composed and improvised music, to react to the notated music immediately. The disadvantage of this strategy is that, while the time structure of each individual phrase is preserved, the phrases can't be synchronized in respect to each other.

Page  00000004 Thus, synchronous performance in real-time is not possible and can only be simulated, if achieved at all. Consequently, only specific contemporary musical forms are suitable for performing via the Internet (Intemet2 promises a higher degree of synchronicity through greater speeds and bandwidths though); but who really wants to perform Mozart or Beethoven in this manner? Long before the invention of the Internet, the 20th Century brought about forms which can serve as ideal models for making music a la The composer John Cage (1912-1992), who was involved in the discovery of new means of expression his life long, wrote the so-called number pieces in a later phase of his life, in which musical events are organized through time windows. In ensemble performance, a tone is thus allowed to be played before, simultaneous to or after another tone, as long as it is within the same time window. Interestingly, this form of asynchronicity hardly influences the audible musical result. The Polish composer Witold Lutoslawski (1913-1994) also developed a compositional technique, influenced by Cage, in which chronological approximation leads to precisely calculated musical results, the so-called aleatoric counterpoint (the word aleatoric refers to the random components in the ensemble performance). Several pieces (by John Cage, Anne La Berge, Manfred Stahnke and myself) have either been transcribed or originally composed for A performance of a composition with generally utilizes a score, which is saved as a timeline in the conductor program component. The score consists of a chronologically defined series of instructions, control commands, and short, notated passages that are passed to the musicians. Each performance with thus creates one global as well as up to five local versions. The global version can be followed by the listeners in cyberspace, while the performers create a local variant, particularly due to the use of individual acoustic instruments (which are not heard as such in the Net). Further information as well as samples of the compositions are available on Appendix The six criteria of network music The American composer and MIT Media Lab professor Tod Machover writes concerning his technology opera: "The Brain Opera is an attempt to bring expression and creativity to everyone, in public or at home, by combining an exceptionally large number of interactive modes into a single, coherent experience. The project connects a series of hyperinstruments designed for the general public with a performance and a series of real-time music activities on the Internet. Audiences explore the hands-on instruments as preparation for the performance, creating personal music that makes each performance unique. The project is attempting to redefine the nature of collective interaction in public places, as well as to explore the possibilities of expressive objects and environments for the workplace and home". Six categories relevant to making music in the Internet can be extracted from this text: 1. Network ("activities in the Internet") 2. Interactivity ("the tangible instruments") 3. Real-time ("series of real-time activities") 4. Algorithmic composition ("series of hyperinstruments"; the term hyperinstrument refers both to gestural controllers - sensors which quantify gestural actions - as well as to the transferal and development of these actions by means of compositional algorithms). 5. Open forms ("music that makes each performance unique") 6. Sociological dimension ("attempts to redefine the nature of collective interactions") All six of these criteria are also taken into account in The network consists of up to 5 players, one conductor and an arbitrary number of listeners that are connected to each other through the Internet. The MIDI interface and the integrated pitch tracker allow a man-machine interaction with immediate reaction from the system; the music that is played into the computer can thus be modified and further developed through pre-programmed effectsalgorithms. Such interaction-modi demand the use of open forms with compositional and improvisational elements. The modification of forms of musical behavior is of far-reach ing significance; questions arise concerning the role of the audience, the characteristics of the performance spaces, the identity of the performers and the concept of musical public.............. 1 pl MarkerTr... I elonl Smres markers Setionl I 3 [] ploy..o IIPooolootpo13 I Excerpt from a score