Page  190 ï~~NUNTIUS: a transmedial communication system for computer music and for computer choreography, Tamas Ungvary, Peter Rajka, Peter Lunden and Magnus Lundin Royal Institute of Technology (KTH), Inst. of Speech Communication and Music Acoustics Box 70014, S-100 44 Stocholm, Sweden e-mail ":ungvary@ Introduction. Publications by M. Clynes [ 1-41, 1. Fonagy [5], I.Bengtsson & A. Gabrielsson [6], and U.Kronman & J.Sunberg [7] lend support to the hypothesis that there is an archetypical correlation between body movement patterns and sound patterns. Clynes considers the correlation to be of neurophysiological origin and to reside in DNA as so-called "sentic forms". He has studied the relationship between tactile movements and music and musical interpretation. Fonagy has demonstrated a correlation between speech and music and archetypical movement patterns. A. Friberg and J. Sundberg. at the Royal Institute of Technology, Stockholm studies the relationship between movement and music. Their hypothesis is that there are analogies between locomotion and certain rhytmical patterns in music.. The above mentioned researchers have in common that they relate movements to traditional music, i.e. to pitch sequences played on conventional musical instruments. Our experiments and comparisons on the other hand, involve the use of SOUNDS, both natural and artificial ones used in contemporary and electro-acoustic music. Through the history of dance, theatre and opera, it always has existed a natural relationship between music and dance, between sounds and movements. We faced the question: how to describe and analyse this relationship in different contexts and what kind of tools we need for the exploration of this relationship? The creation of both movement and sonic structures is a hierarchical process. Different notational systems and tools are necessary for different levels of the description, manipulation and analysis, such as scripts, graphs, symbols, complex symbols, animated sonic and kinetic movements. Hypertext based programmes enable us today to connect graphic notation with text, pictures, sounds, animation, or video sequences. We can also develope different ways and levels to find and connect information. However, the communication between the different media is limited to the access of various materials and their separate or simultaneous display. Changes in one media does not affect the data in another one. It is our belief that further development of multimedia requires facilities enabling experimentation within a certain media with a resulting effect upon another one. Accordingly not only the communication channels need to be established, but also common analytical mechanisms for interpretation and manipulation of common data types should be concieved, established. Consequently, transmedial integration and a multiple presentation system with both analytical and manipulative facilities became the basic idea for the development of the system NUNTIUS.[8- 101 Technical solutions of transmedial communication. While the described transmedial communication above is the foundation of the system NUNTIUS, its assisting tools are implemented on several levels and for various purposes, based on one common data type, on equidistant normalized function files and on MIDI messages.: a.) With the help of the notation editor (MotoSymb) it is possible to notate a movement sequence and to translate that information into movement data. b.) The animation program (ANIMA) reads the movement data immediately (in real time) and displayes the result as animated dance. In this way the user receives immediate feedback of his symbolic movement notation The animation may be controlled by external MIDI Time Code, thus synchronized with other programs. User defined MIDI messages may trigger previously defined movement patterns of any complexity and length. c.) A movement data language (MOVE) allowes to create low level movement data (ASCII format) by external programs or within a text editor. The MOVE language ailowes also to create high level procedures, movement libraries. d.) The movement data can also be presented as parameter curves, edited and tranformed by a curve editor (MotoFunc). These curves may be stored as equidistant, normalized function files and exportede.g. to a sound synthesis program (CCPL or others). Since MotoFunc also reads normalized function files, it is possible to import data from any other source which is capable to Interactive Performance 190 ICMC Proceedings 1994

Page  191 ï~~produce them. e.) The Movement Procedure Editor (MotoProc) allowes to treat complex movement patterns as symbols, adjust their duration and place them on a time line f.) Normalized function files, created by MotoFunc, may be imported to e.g. Common Composers Programing Language (CCPL) and used to create sonic events. CCPL [ 11 ] is an interactive, object-oriented system implemented in Common Lisp for applications such as the controlling of MIDI-devices, Digital Signal Processing and Algorithmic Composition. g.) As the animation reads MIDI Clock, movement and sonic events may be synchronized when a program like Studio Vision is used to play the music. They can be started, stopped at the same time. h.) Sonic events may also be visualized and simultaneously displayed by using the program MacSonogram [ 121 1) Abstract gestures may be conveyed via MIDI Controller data paralell to ANIMA and to a MIDI interpreter (MAX patch- MS) MS has been developed to interpret the signals of the 3D touch sensitive input device, the Sentograph. It is possible to control both usual synthetizers and human motion parameters to be animated on the using the ANIMA - MS program combination together with the Sentograph. Artistic experiment During the spring of 1994, for the first time, an extensive choreographic work, titled "MELOS 3" has been created with NUNTIUS and has been performed at "Fylkingen" in Stockholm. MELOS no 3 is a 9' 46" long composition by Tamas Ungvary, for Computer Generated Tape and Solo Violin. A solo dancer and the violin player acted on the stage, before four animatated "dancers", projected on a huge background screen. The choreographic material is based on a few movement themes, notated in details with the Symbolic Notation Editor (MotoSymb). These themes has been mapped into higher level representation (movement procedures or macros) and manipulated with the Movement Procedure Editor (MotoProc), untill the variations and the final structuring has been satisfactory realized. The choreographer has been able both to see and to listen at every stage of the creative process, to associate the animation with the synchronized music. The compositional technique of the choreography is based on the alterations of oppositions, displaced imitations, correspondences and convergencies. Visual and sonic elements has been related to each other as well as the visual elements of the four animated dancers. This complexity of relationships is hard to realize with traditional choreographic methods. We have successfully solved the major part of the choreographic compositional problems by using NUNTIUS, which leads the way towards a new multimedial composition technique. References [1] Clynes, M.: SENTICS, The Touch of Emotions Â~ Anchor Press/Doubleday, 1977; ISBN 0-385-08605-9 [2] Clynes, M.: "The communication of emotion; Theory of sentics." in: Emotion, Theory, Research and experience. Volum 1: Theories of Emotion, RPlutchik and H. Kellerman eds.;New York, Academic Press 1980 [3] Clynes,M.: "Expressive microstructure in music" in Sundberg,J.(ed) Studies of Music performance, Stockholm: Royal Swedish Academy of Music, No.39, 1983 [4] Clynes, M., S.Jurisevic and M.Ryan:"Inherent cognitive substracts of specific emotions: Love is blocked by lying but not anger" in Perceptual and Motor Skills,1990,70,195-206 [5] Fonagy, I.:" Emotions, Voice and Music" in Research Aspects on Singing, pp 51- 79.Publication issued by the Royal Swedish Academy of Music, No33, 1981 [6] Bengtsson I. & Gabrielsson A. "Rhytm research in Uppsala" in Music, Room, and Acoustics, Publications issued by the Royal Swedish Academy of Music,1977, No. 17, 19-56 (7] Kronman,U.,Sundberg,J.:"Is the musical ritard an allusion to physical motion" in Gabrielsson,A.(ed.), Action and perception in rhythm and music. Stockholm: Royal Swedish Academy of Music, No.55, [987 (8] Ungvary, T. and Rajka, P.: "Nuntius " in International Conference on Coordination Method. Dance Notation and Application. Digest of Papers (Nanjing Institute of Technology, China, 1988) [9] Ungvary, T., Waters, S. and Rajka, P. "Nuntius: A Computer System for the Interactive Composition and Analysis of Music and Dance." Leonardo vol. 25 no. 1. Pergamnon Press, England 1992,. (10] Ungvary,T. and Rajka, P.: Data Link Between Computer Music and Computer Choreography. In the Proceedings of The Arts and Technology 3., 1991, Connecticut College [ 11] Lunden P.:"Sound Models: The Representation of Knowledge about Sound-Synthesis in the CPL EnvironmenL" in the Proceedings of the 1991 Int. Computer Music Conference, Montreal. [12] Lunden P., Ungvary T.: "MacSonogram: a Programme to Produce LargeScale Sonograms for Musical Purposes." in the Proceedings of the 1991 Int. Computer Music Conference, Montreal. ICMC Proceedings 1994 191 Interactive Performance