Page  1 ï~~SCORING THE COLOR OF WAITING Margaret Schedel Stony Brook University Music ABSTRACT The Color of Waiting is an interactive theater work with music, dance, and video developed at STEIM in Amsterdam and further refined at CMMAS in Morelia Mexico with funding from Meet the Composer. Using Max/MSP/ Jitter a cellist is able to control sound and video during the performance while performing a structured improvisation in response to the dancer's movement. In order to ensure. repeated performances of The Color of Waiting, Kinesthetech Sense created the score excerpted in this paper. Performance is essential to the practice of time-based art as a living form, but has been complicated by the unique challenges in interpretation and re-creation posed by works incorporating technology. Creating a detailed score which incorporates all of the art forms being used is one of the ways artists working with technology can combat obsolescence. 1. INTRODUCTION This score with descriptions of the electronic sounds, video compositing, choreography, cello tracking, music notation, lighting, costume, and stage diagram will enable performance of this work well into the future. A DVD with the Max/MSP/Jitter patch saved as text, the sound and video files used in the performance and video clips from various performances is included with the score. By including screen shots of relevant sections of the max patch in the score, we show which part of the interaction is most important for the artistic success of the work. 2. CONTEXT The following text and figures show excerpts from the score. The entirety of the introductory materials (figures 1-3) is included while due to space restrictions only excerpts from the timeline of the performance are included (figures 4-6). The introduction serves to document all elements of the piece including the set, the lighting and the costumes. The timeline contains sketches of the choreography, performance instructions for the cellist and dancer, musical notation for the cellist including light and dance cues, and stills from the video showing brightness, and placement of elements. 3. THE SCORE The Color of Waiting is an interactive theater work with music, dance, and video which was developed at STEIM in Amsterdam and further refined at CMMAS in Morelia Mexico with funding from Meet the Composer. Using Max/MSP/Jitter a cellist is able to control sound and video during the performance while performing a Alison Rootberg Kinesthetech Sense Los Angeles, CA structured improvisation in response to the dancer's movement. This work uses animation, movement, music, lighting and video to portray various themes of expectation. The performers shift between textures of frustration and acceptance while exploring the way time unfolds when waiting for something or someone. The Color of Waiting is an experience and a mood, an abstraction exposing human interaction. After a performance at Stony Brook University Professor Daria Semegen called it a "visual haiku within timbres of stasis." The work is divided into five sections: I. Waiting for Love II. Waiting for Inspiration III. Waiting for Agreement IV. Waiting for Something You Need V. Waiting for the End Each section uses a different kind of interaction between the elements of the work thereby providing an ever-evolving framework for the performers. 3.1 Electronic Sounds There are only five sound files used in the piece which were created by processing recordings of rain falling on a skylight combined with a sample of a rainstick: llove.aiff 2inspire.aiff 3agree.aiff 4something.aiff 5end.aiff Sounds 1-4 fade in together. In the first section llove.aiff is the most prominent. In the second section llove.aiff fades out completely and 2inspire.aiff becomes the most prominent. Thus in section two, 2inspire is the loudest sound and sounds 3-4 are playing at a low volume, and in section 3, 3agree is the loudest and files 4 is playing at a low volume Soundfile 5, which is a version of 4 with more rests is triggered in the final section. In this way the layers emerge and disappear, until all of the sound is stripped away. 3.2 Video Compositing There are two video files used in the piece, and These movies are composited onto an opengl window, Stamping, with three white columns providing a framing element. This image is projected onto three sets of vertical blinds 2.5' wide x 6' long. These are opened slightly, giving a texture and

Page  2 ï~~dimensionality to the video which can be seen slightly behind the blinds. Vertical and horizontal placement of the eyes, waves, and white columns is adjustable within the patch, enabling the projection to fit precisely on the set (see Figure 1). The subpatch sw is a thirty inlet switch which takes the video signals a,b,c etc from the patch above and allows the computer to choose a column numerically. 3.3 Choreography The choreography is very clearly divided into five sections. In waiting for love the movement is sensual, slow and seductive, as the dancer wades into water and draws the light up. In the second section the dancer focuses the lights on the angles of the head and face while waiting for inspiration. The third section is the only time the dancer leaves the box; the explosive movement in the space contrasts and compliments the music and the dancer's own shadow in the projection while the performers wait for agreement. In section four, waiting for something you need, the dancer must make the position of the lights explicit so the cellist can track them with the projected eyes. The final section is waiting for the end-as if it is the end of life, the dancer lies down, exposed and open to space. Finally free of waiting, the end has come. 3.4 Tracking the Cello The Max/MSP/Jitter patch tracks three elements of the cello sound: volume, pitch and noise content. The volume of the cello controls the height of the waves: in a deliberate decision to set up an opposition, the louder the volume of the cello, the lower the projection of the waves......................................................... -- - - - - - - - - Figure 1: Video The cellist sits between two columns with the computer system, and the dancer performs most of the piece on a stool or other stable object placed, the other two columns. Throughout most of the pieces the audience sees only the far right, left and middle 4/30ths of the video. The file is divided into thirty columns and then put back together in three sets of four and projected. Figure 2: Waves In section four the video scans horizontally over the movie, creating a sense of lateral motion which contrasts with the rising and falling waves. The parameters change randomly, but the right edge of the left columns and the left edge of the right column bound the parameters for the middle column. Figure 4: Cello Tracking The noise content in the timbre of the cello controls how responsive the waves are to the cello's volume. This control was added to increase complexity of the interaction. By using Miller Puckette's fiddle~ object to track the pitch of the cello, the cellist is able to control the direction the projected eyes appear to look. On the open D string the eyeball is on the left edge of the column when seen from the audience. The D two octaves above corresponds to the eyeball on the right edge of the columns. Figure 3: Subpatch SW

Page  3 ï~~In section four the automatic playback of is stopped. Instead, the cello pitch controls which frame of the animation is chosen. The smooth subpatch creates a transition between frames, so the projected eyes move gradually as the cello changes pitch. This control only appears in section four. Until then the movie plays on a ping pong loop - scanning slowly back and forth. 3.5 Lighting The piece is performed in total darkness. The dancer holds a small light in each hand, using them to reveal areas of the body appropriate to the content of the section. Full illumination of the body occurs only when (s)he is in the beam of the projector. 3.6 Costume The dancer and cellist each wear a sweater with horizontal stripes of black and gray. The stripes are revealed when the lights come on at the end of the piece and the performers take their bow. The gray and black costumes and grayscale video are accented by a red tie on the dancer, and red fabric flowing across the cello and pooling on the floor. 3.7 Video The source files for the video were created by traditional frame drawn animation combined with digital video recordings. The eyeball movie is fully animated, while the waves use a real video of water masked by an animated wave. This combination of recorded and sketched imagery adds a layer of abstraction to the visuals. 3.8 Cello During most of the piece the sound of the cello is distinct from the pre-recorded sounds creating a deliberate sonic opposition. The long tones on the cello centered on D contrast with the unpitched, particulate, electronic sounds. As the piece comes to conclusion, the live cello playing col legno finally blends with and matches the timbre of the pre-recorded files. 3.9 Interdependence of Elements The Color of Waiting fuses disparate elements together into one piece that defies conventional boundaries. The cellist improvises elements of the music based on the dancer's movement. For example in section one, the cellist tracks the height of the dancer to determine the dynamic of the music. In this way it appears that the dancer is controlling the video. The choreography and music link the cellist and dancer; this relationship is reinforced by the identical costumes. The small lights in the dancer's hands are equivalent to the eyes in the video. The choreography anticipates the scanning pattern of the eyes in the video before they appear. Then, in section four, the relationship between the lights and the eyes becomes clear as the eyes, under the control of the pitch of the cello, follow the lights. The pre-recorded music was developed from the sound of water, referencing the waves in the video projection. This relationship is reinforced by the fabric pooled around the cello. The sound of the cello controls the waves and eyes in the projection. Thus sound and video are linked in two ways through content and control. By evoking windows, the Venetian blinds form a strong metaphor for waiting. The horizontal blades of the blinds visually balance the vertical of the columns of video, and the broad horizontal stripes on the performers' costumes echo the horizontal slats of the blinds. The set and the video form a cohesive whole, which the dancer can become part of by entering the beam of the projector and casting a shadow onto the video. The computer is the neutral interface where all of the components come together. The structure of the piece is found in the computer program which advances under the control of the cellist. Every element of the piece is linked to the others through multiple methodologies, creating a work which is beyond multimedia. It is intermedia in the fullest sense of the term where each medium is influenced and informed by the others. Stage Diga fo The Color Of Wa tting Figure 5: Stage Diagram

Page  4 ï~~A N; ' FS WAITNG FORILOVE 24 rai a~l ceistb:dva iisN?. hthe YiN ' t.o ns ' ps'dv.n pea (1 'd k i as ' a4it EN 1 3.. K I&a nd '. 3 in Ow lit i L i i tn A nl" A 1. columns of v tdio 1 iN NI il N 133. A: U ii n 'kA w et "Kr N jk s............................. Xxl Wait hfior N~u Eiiii to.N: hi i3 \\t.....i'.ni..:::: TI' (3.odbnns' in -n 'ns V a n' h;sa t (3:ddin; Aftmcv".,a c 4 m.. o ".--mu. i i.Â~* v:t:a Â~ a". Smuiidfie 0- 11% tdfrs Snnnd ilt 0$4bti21%lC sonntilt 2.30 1()% dtlnia.s Figure 6: Score Page 5