Page  00000001 Participation and Response in Movement-Sensing Installations Todd Winkler MacColl Studio for Electronic Music Brown University Todd Winkler@brown.edu Abstract Audio and video installations that require audience participation pose a unique set of problems. These works typically use various types of sensors so that natural movement and action can be used as the primary input to a responsive computer system. What is the psychology of participation? How can installation artists engage, prompt, and empower amateur "performers" to create a rich experience? This paper examines questions associated with presenting interactive installations from the viewpoint of the audience experience, with special attention to non-digital concerns. Observations are drawn from several of the author's installations, and from the works of others. 1 INTRODUCTION Movement-sensing installations offer audience members an opportunity to become actively involved in the creative process by influencing sound or image output from a computer based on their own activities, location, and gestures. These works are often presented as environments open for exploration, with each "performance" determined by individual action, curiosity, and play. What separates interactive installations from interactive concert works or other types of installations is that the work is only realized through a participant's actions, and those actions do not require special training or talent to perform. All of this suggests a new social and artistic dynamic that is unique to interactive installations, a paradigm where participants are required to use their mind and body to actively create their own experience. Gone is the traditionally passive viewing/listening of a single "masterwork" presented by "experts." Gone are the taboos against moving around during a concert, or touching things in a museum. However, artists must be willing to give up tight control and perfection of form and presentation they have with traditional media. "Rather than creating finished works, the interactive artist creates relationships" (Rokeby, 1995). With the audience's acceptance of their responsibility comes a greater acceptance and ownership of the results: participants seem to enjoy, accept and pay great attention to the results of their own creative actions. For example, many people who do not attend concerts of computer music might be more willing to listen to new sounds if they feel they are responsible for creating them. Interactive installations pose their own set of problems that can be addressed by examining the factors contributing to the user experience. Composers and artists creating such works usually focus on artistic concepts, digital content, and technical issues. However, many aspects of the work are only realized after the installation is taken out of the "laboratory" and set up where the public can try it out. These non-digital aspects of an installation are not trivial, but have a large impact on the audience's perception of a work. The term "audience" may be potentially confusing, since it may refer to anyone viewing or participating in an installation. To clarify those roles, person(s) activating an installation will be referred to here as the "player(s)," as in a musician playing music or someone playing a game. Audience members simply viewing or listening to the players will be called "spectators," implying a group's role in a live event. 2 FACTORS IN THE AUDIENCE EXPERIENCE While the artistic intentions and methods may be of great interest to some visitors, many viewers will not read descriptive narratives of a work. One challenge is to make instructions and prompting part of the work, so that the audience is educated and guided in a non-obtrusive manner. "Natural" interfaces requiring everyday movement or the manipulation of familiar objects may be so obvious as to not require further explanation. Many players will learn how to operate the installation simply by watching others. Most impressions will come from four interrelated factors. They are listed here in order of how much influence an artist has over each area: The Digital Factor includes the digital content, stored sequences or samples, generated material, processed material, software for interpreting movement data and other input, computers, and digital output.

Page  00000002 The Physical Factor involves the installation space and set, including items inside the space, such as sensors, props, projections, speakers, and furniture. One could view this active area as the total interface to the computer. The feeling of the venue and location also contribute to the audience's reception and reaction to a work. The Social Factor examines the relationships between people before, during, and after the installation experience. Humans are social animals, and part of the reason they go to an exhibition is to spend time with companions, watch other people, and feel the excitement of the social scene. A big factor in an audience's potential understanding and enjoyment of a work is the ability to have companions watch them play (in single player works), play together (in multi-player works), and talk about the work. The ability of a work to foster social interaction is an artistic decision that may be limited by a given space, or the capabilities of software and sensors. Four general models exist for social interactions: a single isolated player (no social interaction), multiple isolated players, one player at a time with spectators, and a group of players with spectators. Studies of audience reactions suggest that works encouraging social interactions among multiple players are often viewed as most enjoyable (Graham, 1997; Mitchell and Bicknell, 1994). Ironically, the presence of many players triggering the same system makes it more difficult for individuals to follow the results of their actions in the digital domain. Their engagement with the computer may decrease as their social interactions increase. The end of this paper will show several strategies devised for a group of players to facilitate both social interaction and computer interaction. The Personal Factor is the area most difficult to touch directly, although the three factors, above, all contribute to create the individual experience. Beyond the control of the installation artist are issues such as a person's mood, musical taste, interest in technology, or whether they have the knowledge and skills required to participate and understand the installation. Some people may be easily intimidated and unwilling to participate, or they just don't have the time. However, by knowing the audience, some potential problems may be solved in the design and layout of the installation. 3 CHALLENGES TO AUDIENCE PARTICIPATION One big challenge to the installation artist is to know the full range of audience members, and provide a multilayered work that will be engaging on many cognitive, physical, and emotional levels. Ideally, all of these factors add up to more than the sum of their parts, reinforcing the artist's intentions, and leaving the participants with transforming ideas or emotions. Potential problems can be identified in each stage of participation: waiting/watching, decision to play, playing, staying, and leaving. Although a crowd of spectators watching an installation reduces the chance of anyone playing, people will flock to a group out of the assumption that the installation must be good if so many other people are watching or waiting. Allowing spectators to view an installation engages both nonplayers and potential players, who carefully watch the performance to learn how it works, how long it will take to play, and how "safe" it is to enter. Many players enjoy the attention of the audience, and the rapport that comes from the collective experience. Other people might be intimidated by spectators, afraid of looking foolish, or being watched. On the other hand, installations that only allow one or more players inside a closed space can evoke a more intimate, reflective, and "whole" world, without the distractions of everyday life. Sound and light can be more carefully controlled within a closed space, as can the number of people entering and exiting the installation. The artist's perceived attitude towards the audience will ultimately shape their experience and willingness to participate (Graham, 1997). In what role are the players cast? Will they be separated from family and friends to be isolated in a dark room for an exciting and dangerous experience? Will they be asked to playfully cooperate with others in a well-lit open space? Are people forced to wait on line or read confusing instructions? What about the digital material? Is it insultingly simplistic, pleasantly aggressive, or impenetrable in its complexity? Once inside, players will need to know how to run the installation and the rules for engagement with the computer. Ideas for effective interaction strategies between performers and computers have been discussed extensively by the author and others (Winkler, 1998; Rokeby, 1995). The interface and digital content may include nonobtrusive training or instructions for use. Players should be clearly prompted to begin, continue and end. Having clear cues for leaving are especially crucial when many people are waiting to participate. Often times, an attendant is needed to take care of the "traffic flow," but simple written instructions or media cues are also effective. What are the minimum and maximum times needed to get a good understanding of the work? The scope of the installation may determine the necessary range of time needed for a full experience. Factors that influence the duration of use include: attraction to the material, intellectual and emotional interest, ability to understand and enjoy the content, the feeling of engagement with the computer, and social interactions (Graham, 1997). The pacing will also contribute to the perception of expected time as will physical objects in the room, such as couches or comfortable chairs (both spectators and players will stay longer if they have a chance to be seated). Most people avoid doing things in public that will draw the attention of strangers. Usually, people do not feel free to dance in front of an audience, and they know not to touch artwork in a museum. Artists need to give permission to players to do the activities required of the

Page  00000003 installation, even if it goes against the normal expected behavior in a museum or other public space. The permission may be liberating or intimidating, depending on social and personal factors, and the intention of the artist. 4 EXAMPLES OF PARTICIPATION IN THREE INSTALLATIONS The author's Light Around the Edges (1997) is a sound installation that uses a video camera to detect location and movement of people in a large public space. The sensing camera is placed high above the audience, pointed downward. Movement on the ground is transmitted as numbers into the Max programming environment via David Rokeby's Very Nervous System (Rokeby, 1995). There, software generates original music, or triggers samples, based on movement data. Except for four speakers, the installation is invisible. Any number of people may walk through the sensing area, often just passing by. Any number of spectators can watch players dance or move around the space. The players are not only the musicians playing the music, they are the dancers in their own dance. In other words, the players are the content of the work. To meet the challenge of accommodating an unknown quantity of players, the installation operates in three different modes, based on how many people are playing. For one to five players, the software is highly interactive, with speed and location perceived as having an immediate and obvious impact on the sound, generating music, processing sound, and controlling panning. With six to 15 players, the perception of immediate interaction is lessened, and the space transforms into a soundtrack of a train station, with players' locations triggering conversations in many languages, train doors opening, announcements, and the sounds of trains coming and going. Finally, with too many players to identify any individual's input, the third mode turns the space into a big party, with movement triggering canned laughter, sounds of people eating, conversations, glasses clinking, and other crowd sounds. The effect is a fortified social space, where the audience's movements alter their own social interactions in real time. All three modes are playful environments that encourage conversation, eye contact, and movement between players and spectators, companions and strangers. Don Ritter's installation, Skies (1998), not only enhances social interactions, it requires conscious cooperation between people to play. Any number of players walk around a video image, projected on the floor, to discover one of five black paths. Each player can only occupy one path, and various combinations of paths trigger specific video imagery ofnature, so that the collaboration of one to five people will have different results. When all five paths appear they form a star pattern on the floor and the video clip changes to a scene of the sun bursting through clouds. Ritter writes: The interactive component of this installation was designed to be meaningful, because the viewers--possibly strangers to each other--have to cooperate with each other in order to experience the entire work. When this work has been exhibited, viewers began speaking to each other as they attempted to discover all the levels and sequences of imagery. Their cooperation with the nature imagery--as detected by the interactive system--is the content of the work. In Maybe...1910 (1999), the author created a set that resembled a bedroom from the earlier part of the 20th century. Using the Icube system, the bedroom furniture was wired with light and pressure sensors to act as the interface to two computers, which played back three video channels and four audio channels. Fourteen sensors in all were imbedded into the drawers of a vanity and dresser, a bed, chair and rug. The material played back was taken from interviews of elderly people telling stories of their earlier years, and historical footage. All sensors, wires, and speakers were invisible, and the video sources were disguised as an old television set, a two-way vanity mirror, and as scenes projected on a curtained window. Each sensor played back one of six short video or audio clips, based on one topic. The complete collection of video totaled forty minutes, so that players could spend three to thirty minutes inside the space for a "full" experience, without too much repetition. Antique objects, placed in drawers and around the room, could be examined and helped to illustrate the stories. The installation was built as a small bedroom inside a much larger room, with boundaries indicated by a carpet, fumrniture placement and ropes. Spectators could gather around the bedroom on three sides, and peer through the "invisible walls" to watch players and view the computer playback. Only 2-3 players were allowed inside together, any more added confusion as to who triggered what clip, and encouraged interruptions. Social interactions between companions was very high inside the room, while much less so for strangers. People were respectful of the interviewees, quietly listening to their stories, but were eager to talk to one another about what they had just heard during the silence between segments. The pacing was intentionally slow. The digital material and the antique set encouraged people to reflect on their own pasts. Many of the elderly participants recalled their own stories of "the old days." Younger participants were reminded of elderly relatives and seemed to enjoy the freedom to explore the physical space. Eight hundred people saw the installation over a three-day period at the Rhode Island School of Design as part of a millennium celebration. On the final day, chairs were provided for the spectators, and this seemed to greatly increase the average viewing time and enjoyment. Although there were simple written instructions as to how the installation worked, most people didn't read them. A

Page  00000004 person to facilitate traffic control and to encourage participation was necessary at all times. Perhaps in a less crowded and more secure museum environment, the installation could be run on its own; the most difficult aspect would be how to limit players and playing time. One solution would be to use software to track the number of players and automate new video messages encouraging them to stay or leave. 5 CONCLUSION Composers and artists creating movement-sensing installations will enhance their vision by carefully considering how non-digital aspects of their work contribute to the audience's experience. Participation in interactive installations is not simply a novel way to view a finished composition, it is a collaboration with the artist where the audience becomes content in a digitally mediated work. As sensing technology becomes commonplace, artists will be forced to conceive of work where interaction is invisible yet vital. In his lecture for the Kwanju Biennale, David Rokeby (1995) writes: I am looking forward to a time when interaction in art becomes...banal and unremarkable... merely another tool in the artistic palette, to be used when appropriate. The idea that an artwork can interact with the audience is only interesting in the context of centuries of theory that celebrate 'inertness' as a desirable quality in 'Art.' For the next generation, interactive art will not be exciting and problematic, it will be necessary and obvious. Once the hype dies away, interaction in art can return to its natural role as a tool for exploring and critiquing relationship itself, an important role during a time when the nature of all our relationships...personal, economic, political, and with the media are in constant flux. 6 REFERENCES Graham, Beryl. 1997. A Study of Audience Relationships with Interactive Computer-Based Visual Artwork in Gallery Settings, through Observation, Art Practice, and Curation. Ph.D. thesis, University of Sunderland, England. Mitchell, Kate and Bicknell, Sandra 1994. Launch Pad: Summative Evaluation. Summary of Creative Research Reports. Unpub. paper, The National Museum of Science and Industry, London. (As cited in Graham, 1997). Ritter, Don. 1997. "My Finger's Getting Tired: Unencumbered Interactive Installations for the Entire Body." Lecture for the 1997 International Society of Electronic Arts (ISEA). Liverpool, England. http://users.rcn.com/ritter.interport/article.html. Rokeby, D. 1995. "Transforming Mirrors: Subjectivity and Control in Interactive Media" Simon Penny, (Ed.), Critical Issues in Electronic Media. SUNY Press (USA). Rokeby, D. (1995) Lecture for "Info Art." Biennale. Kwangju, http://www.interlog.com/drokeby/install.html Kwanju Korea. Winkler, Todd. 1998. Composing Interactive Music: Ideas and Techniques Using Max. Cambridge, Massachusetts: MIT Press. Winkler, T. 1998. "Motion-Sensing Music: Artistic and Technical Challenges in Two Works for Dance." Proceedings of the International Computer Music Conference. San Francisco: International Computer Music Association.