Page  00000001 The Shamanic Object as a Model for New Multimedia Computer Performance Interfaces Matthew Burtner VCCM, McIntire Department of Music, University of Virginia, Abstract Traditions of Shamanic cultures reveal a striking approach to the use of ceremonial objects in ritual. This paper proposes applications of shamanic object design for the construction of new computer music movement-based performance interfaces. These new computer music instruments have been created specifically for a series of multimedia compositions. The anthropological theory of liminality is combined with computer interface design research to achieve staged interactive media representations of shamanic rituals. 1 Introduction The design principles for the shamanic multimedia computer performance interfaces discussed here draw on a perceived mutability of physical objects in traditional shamanic cultures. The mythology of Alaska, for example, reveals a fluidity between the human, animal, natural and spiritual worlds. The shaman negotiates the relationships between these worlds by means of carefully orchestrated rituals. At the center of these rituals, characteristic objects such as masks and other adornment actuate the ceremonial joining between the metaphysical and physical worlds. In a series of compositional multimedia works dealing with Alaskan shamanism, environment and culture, the author has employed new computer music interfaces as a way of evoking the shamanic transmutational state. Using computer controllers and sound synthesis allows for a clear connection between the embodiment of a performer on the physical stage, and a disembodied counterpoint of forms in the virtual stage. Where previous work has looked at the nature of disembodiment and physical acoustic reality through the use of computer controllers and physical modeling synthesis (Burtner 2003, Burtner/Serafin 2001), this work with shamanic controllers crosses from strict acoustics into multimedia, employing dance and theater, video, sculpture and surround sound. 2 Liminality and HCI Design The theory of liminality, proposed by anthropologist Victor Turner (Turner 1974) refers to an ambiguity arising from everyday tasks being reinterpreted as symbolic activities. The transformation from common activity into ritual activity is paramount to an understanding of shamanism. Ethnographically, rituals are described as "the collectively patterned performance forms through which processes of cultural or sacred signification are integrated into consciousness and social practices." (Tomaselli, 1996, p50). Liminality describes this passing from common reality, into a symbolic understanding that enacts a change in an individual's personal relationship with herself and with society. The symbolization of common reality through orchestrated ritual creates the sense of transcendence experienced by the participants. Samanic traditions in Alaska employ the use of special objects to manifest spiritual understanding. Face and hand Masks, staffs, or clothing can act as the symbolic axis around which liminal transformation occurs. In the case of human-computer interaction (HCI), any common object can act as a computer controller. The mouse and keyboard are two simple HCI interfaces but the type of data communicated between the human and computer can be housed in a variety of instruments. This has been discussed in depth in a previous paper (Burtner 2004). This author's study of traditional Alaskan shamanism has led to multimedia work that takes advantage of expressive interfaces to construct virtual reality worlds evoking the transformative rituals of shamanic exploration. The controllers themselves create a real-time interface between the artist and the media, much as the shamanic ritual object allowed the shaman to interface with the spiritual world. Here technology is used as a means of representing the magic observed in mythology. Proceedings ICMC 2004

Page  00000002 2 Ritual Masks and Shaman Staff in Ukiuq Tulugaq The ceremonial performance tradition of the Inuit people of Alaska was an elaborate theater supported by song, dance, drumming, costume and masks (Kaplan 1982). As the primary interpreter of the spirit world the shaman would prepare the people for a festival by guiding them through a variety of ceremonies. Through these activities the cosmological views of the people were expressed. The creation of special masks was an important part of the ceremonial preparation. Ukiuq Tulugaq (Winter Raven) is a large scale multimedia work for instrumental ensemble, surround sound electronics, interactive video, dance and theater (Burtner 2001). For the piece, a special set of masks and a shaman staff were constructed to be used by the dancer to control interactive video. In Ukiuq Tulugaq, a dancer portraying the Shaman character personifies the natural forces of sun, ice and wind. A desire to evoke the shamanic relationship between human and nature on the stage inspired the creation of these special masks and a shaman staff. The shaman staff allows the dancer to capture the detailed changing movement of the masks she wears. A computer receives and processes the video signal, projecting it onto a stage screen. Video tracking technology is used as an analogy of the shaman's ability to effect reality by entering a dream-like spirit world. Figure 1. diagram of the shaman mask/shaman staff concert setup for interactive video Figure 1 shows the performance setup for the Shaman Masks and Shaman Staff. Affixed to the top of the wooden staff formed of interwoven dried vines, is a light and wireless camera. The staff is held by the dancer and she performs a "mask video choreography," moving the camera and mask simultaneously to change the image. The signal is received and interpreted in Isadora software running on a computer. The processed image is then projected onto a screen behind the performer, The Shaman character uses three different masks in Ukiuq Tulugaq, representing Sun, Ice and Wind. Each mask is distinguished by different choreography, music and video processing. An interface written with Isadora, processes the incoming live video and layers it with prerecorded video. and mask dancer (below). The rows from left to right show: kaaq) and Wind (Anugi Unipkaaq). of these performances. In HCI, controllers that extend the human hands are possibly the most common of new musical interfaces. Michel Waisvisz's Hands, Laetitia Sonoma and Bert Bonger's Lady Glove, Butch Rovan's SoundGlove, the Reality Quest N64 NGlove (Figure 3 left), the P5 3D Virtual Glove (Figure 3 right), and the 5DT Data Glove mn above snows the processed video (above) 1) Sun (Siknik Unipkaaq)Ice (Siku Unip 3 Shaman Hands Controller Hand-based costume objects are commonly used in traditional Alaskan dance forms. From the women's "dance fans" to the giant shaman hands, the accentuation of hand movements contributes to the unique choreographic forms Proceedings ICMC 2004

Page  00000003 (Figure 3 center) are just a few of the many "glove" controllers in use. Figure 3. Three of the many glove controllers in use 1) Reality Quest N64 NGlove (left), 2) P5 3D Virtual Glove (center), and 3) the 5DT Data Glove (right) The Shaman Hands controllers are based on the author's contact with a pair of the giant shaman hands in southwest Alaska during a research trip. These particular hands, in use approximately 100 years ago (Figure 4) are heavy, dramatically oversized, and visually striking. It occurred that the legends of giant hands with magical properties wouldcwork well as a model for a multimedia controller Figure 5. ca.1906 picture of a Togiak shaman, Bristol Bay, Alaska, with a sick boy. photo by John B. Thwaites. The Shaman Hands controllers differ from other handbased computer interfaces in that they make no attempt to be ergonomic. On the contrary, they are unwieldy objects, heavy and crude as a control interface. These hands are not designed for subtle movements but rather for the stage. They are dramatic when worn by a dancer and viewed from the back row of a concert hall. They imply large-scale theatrics. The cumbersome design of the hands influences the approach to mapping in important ways. Microlevel control mappings are not idiomatic for these hands in the way they are for most hand-based controllers. These hands would not be idiomatically employed as a video game controller for example. Rather, the use of broadly articulated gestures suggests macrolevel control mapping strategies. As a result, the video and audio source material is rich in timbre and largely composed before performance. The performer of the Shaman Hands works with this material by controlling layers of material while other layers act independently. In addition, the Shaman Hands occupy and traverse a great deal of physical space. For this reason it has made sense to use spatial processing mappings. Following the traditional principles of ritual object design, special wood was gathered to construct the Shaman Hands. Three fitted slats of a barrel were used for each hand. The slats were attached together with leather joinings and glue. Following the pattern from the photographs, the boards were then cut into the shape of the hands, sanded and the wood treated. This process yielded a thin curved hand shape. Leather arm straps and handles on the back of the hands allow the dancer to strap the hands to her arms. Despite their weight and size, the hands feel surprisingly freeing due to the dramatic exaggeration of arm movements they provide. Figure 4. Completed Shaman Hands outside view. The white dots are finger prints as seen on the orignal hands. The horizontal lines are marks left from the barrel straps that held the original wooden barrels together. The controller is designed to capture touch, turn and bend information from the performer. Two Memsic 2125 dual axis accelerometer chips, mounted on small circuit boards, are embedded in the hand. The accelerometers measure movement in two dimensions independently for each hand. Earlier work in tilt sensing used an Analog Devices ADXL 202-series accelerometer employing a mass and spring technique to sense tilt. (Burtner 2002). By contrast, the Memsic 2125 senses tilt by heating a pocket of air that passes by thermopiles and is detected as it moves inside the sensor (Williams 2002). Bend sensors attached to the wrists of the hands measure the elbow joint movement of the dancer. The bend sensors are connected through an RC circuit design to convert analog voltages to digital signals from each pin. Force sensing resistors on the fingers allow the performer to the ability to touch other performers or objects, sensing the force of impact. The Shaman Hands software runs on a Proceedings ICMC 2004

Page  00000004 Parallax BIISX microprocessor converting the signal to a MIDI message. Figure 7 illustrates several distinct motions that are idiomatically captured by the Shaman Hands. composer/performer controller development, the Shaman Hands are idiosyncratic in the sense that they are intended for a specific composition rather than as an adaptable instrument interface. Shamanic objects suggest such a specialized approach to musical interface design. In shamanic ritual, performance using unique, hand made objects transform daily activities into ceremony. It is in fact the specificity of use that forms meaning for these objects. By creating similar objects and turning them into computer controllers, an attempt is made to project the mythological power of the shaman onto the concert stage in theatrical multimedia performance, evoking the power shaman conveyed in their own cultures. The theory of liminality forms a basis for this approach to musical controllers. In technological media it is the connection between physical gesture and signal processing that creates the liminal dialectic. The ambiguity arises between the real and the virtual in the dramatic concert setting. References [1] Burtner, Matthew. Shamanic Controller Design Based on a Theory of Modulated Objects. Proceedings of the 2004 Conference on New Interfaces for Musical Expression (NIME-04), Hamamatsu, Japan. 2004. [2] Burtner, Matthew. Composing for the (dis)Embodied Ensemble: Notational Systems in (dis)Appearances. Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada. 2003.. [3] Burtner, Matthew. Ukiuq Tulugaq. Doctorate Thesis, Stanford University, Stanford, CA. 2001. [4] Burtner, Matthew. The Metasaxophone: Concept, Implementation, and Mapping Strategies for a New Computer Music Instrument. Organized Sound, Cambridge University Press, Cambridge, UK,. 2002. [5] Burtner, Matthew and Serafin, Stefania. The Exbow Metasax. Journal of New Music Research (JNMR). Swetz and Zeitlinger, Netherlands, 2001. [6] Sam Fox Museum Special Collection. Wooden Shaman 23 Hands. Dillingham, Alaska. 2003 [7] Turner, Victor. The Ritual Process: Structure and Antistructure. Chicago: Aldine Publishing Co. 1969. [8] Turner, Victor. Dramas, Fields and Metaphors: Symbolic Action in Human Society. Ithaca, NY: Cornell University Press. 1974. [9] Turner, Victor. From Ritual to Theater: The Human Seriousness of Play. New York: PAJ Publications. 1982. [10] Kaplan, Susan A. Inua: Spirit World of the Bering Sea Eskimo. Ed. Fitzhugh, William W; Smithsonian Institution Press: Washington DC, 1982. [11] Williams, Jon. It's All About Angles Nuts and Volts. Parallax Inc. 2002. Figure 8. Shaman Hands block diagram showing sensor and microprocessor circuit diagram 4 Conclusion The Shaman Hands, and the Shaman Masks and Staff are highly specialized instruments. Even in the rich area of Proceedings ICMC 2004