Page  302 ï~~Studio report from the KACOR multimedia lab Peter Lunden, Peter Rajka and Tamas Ungvary Dept. of Speech Communication and Music Acoustics Royal Institute of Technology (KTH) Box 700 14 S-100 44 Stockholm Sweden email: ludde@kacor.kth.se ABSTRACT This report describes the current state of the KACOR multimedia lab, the hardware and the software. The lab have facilities for computer based DSP and computer graphics with the emphasis on computer music and computer aided choreography. Nuntius is currently our main project The goal of this project is the integration of two compositional systems, one for computer music and one for computer aided choreography. 1. NUNTIUS Nuntius is a multimedial communication and creation system [Ungvary, Waters, Rajka 921. The ultimate goal of the multimedial communication is to allow both transmission of data between the medial modalities and autonomous evaluation and utilisation of that data. This goal requires the existence of two or more work stations. We have currently one work station for music and one for choreography. In its present form Nuntius is concerned primarily with communication at either the parametric or structural level. 1.1. Background It is common knowledge that human motion is related to sound as dance is related to music. The execution of both dance and music are highly complex motor activities and they are both rooted in utterance and gesture as a function of time. Over the centuries;music has developed a whole vocabulary that deals with its-time dependent aspect, as well as complex notational and extensive theoretical.systems. The musical notation system provided the foundation for an increased complexity of structure, which contributed crucially to western music's movement towards the harmonic and rhythmic complexity of the twentieth century. Oral and physical transmission of information has been more important for dance, as a medium with less developed notational traditions, than for western music. Choreographies still dependent to a large extent on the dancer's internalised storage of kinetic and motor functions, and the practicalities of structuring and crossreferencing within a complex dance composition grow organically out of the dancer's abilities to store such instructions accurately, and to provide feedback to the choreographer, not just of the movements, but of the ideas and situations out of which they arose. 1.2. Motographicon: a multimedia system for human motion Simulation of human body motion has long been a subject of computer graphics. Different concepts have been successfully implemented, but computer scientists have always encountered difficulties in describing animation and its dynamic control in an easy and efficient way. Our approach is based on a symbolic notation system developed by Peter Rajka and used interactively on a workstation. 1.2.1. Symbolic movement notation editor The Symbolic Notation (SN), which formalizes the motor knowledge and expresses it in symbols, provides the tools for structure building and manipulation of movement materials. 302

Page  303 ï~~The choreographic score is vertically divided into six columns. Each column contains the motion definition of a group of body parts (e.g. the right arm) and consists of a angular scale system. The placement of a symbol on the scale system indicates the angular relationships between body parts. The trajectories of a movement occur between consecutive body positions placed in time. The time flow upwards on the vertical axis and is scaled in arbitrary time units. File Edit Pelettes MouementOeto Macro Score The score editor of Motographicon con- PleeUntitled-I sists of the Symbol Palette, the Combi- x L!ft arm Left t. C-l.i:,,.Zt;.,aAm ned Symbol Palette and the Score Win- ".. dow. Symbols are placed into the score....... 1o.j. E. 18..........................,..................... by selecting the appropriate symbol or...1It H. {. Â~.. combination of symbols from one of the"r4*i{i{ d { {t pallets and clicking with the mouse in......................... the Score W indow at the desired posi- s Â~.................{.{..............].]........ I., tion. M ovem ent data files m ay be inser-.................I.................... ted into the score at any selected tim ez...........I....... 1 I..,!......... step and during the loading process the T fII II I. I..!! I I data may be treatedby treatments selec-Cl ted from a menu such as augmentation, o, mirroring, shifting body parts, etc. Mo- F J J F I vement data files can be assigned to a macro menu for repeated use in the score. Fig. 1. The Score editor of Motographicon. 1.2.2. Meta symbol editor With the Meta Symbol Editor the system developer may create his own graphical symbols for the SN Editor. The Meta Symbol Editor includes facilities for designing the shapes of the symbols and for the definition of the syntax and semantics of the notation, i.e. rules for allowed placement of each symbol and rules for the generation of the movement data. 1.2.3. Function notation editor As described above, the trajectories of a movement is defined by the SN. In the Function Notation (FN) a sequence of motion may be represented by a set of consecutive trajectories, i.e. by a set of functions. This concept allows the transformation of SN into FN and vice versa. The FN Editor offers visual compatibility between the two kinds of movement representation by usingthe same time representation and angular scale system as in the SN editor. The FN Editor provides an effective visual record both of the time-space structures and of the evolution shapes of the parameters. 1.2.4. Anima Anima is an application for visualisation of human movements. It is not intended for rendering purposes, but as a tool for choreographers. Users work with scenes consisting of dancers and a simple floor and a background. It is possible to view several scenes where each scene may be viewed from an arbitrary position in space. The execution of a choreography is controlled by a dialog-window similar to the controls on a tape recorder, with controls for forward, backward and stop. Tempo and position in the score is controlled by sliders. The Macintosh MIDI Manager facilitates the synchronise Anima with various musical devices and applications, either as a slave or a master. ANIMA is based on a support package for human movement animation, called MOVELIB developed by Magnus Lundin. 1.3. The computer music system The computer music system consists of several software packages integrated into a working unity. The kernel in this integration is a collection of file-format conversion routines, a "software network" developed by Tamas 303

Page  304 ï~~Ungvary. The most important music software is: Common Composer's Programming Language (CCPL) developed by Peter Lunden, CSound, MAX, StudioVision, DECK and SoundTools. 1.3.1. CCPL Common Composer's Programming Language (CCPL) is a computer programming environment aiming at composers and researchers in the field of electroacoustic music [Lunden 89]. It is an interactive, object-oriented system implemented in Common Lisp for applications such as the controlling of MIDI-synthesizers, DSP and algorithmic composition. The current version is emphasised on digital sound synthesis where the most fundamental concept is Sound Models. It is used to represent knowledge about sounds and sonic-structures [Lunden 91]. The knowledge is distributed in a hierarchy of classes. An instance of a Sound-Model class can be thought of as an "instrument". It describes a sound with a set of parameters which are appropriate for that type of sound, not with arbitrary parameters enforced by the synthesis method. I.e. a Sound-Models that behaves like a drum could have parameters like drumstick hardness, point of impact and hitting force. Dynamic objects are a special kind of objects that can be used to build complex dynamic controlling structures to control Sound-Model objects (e.g. envelops and random generators). An interface to CSound is also included to facilitate the creation of CSound files directly by CCPL. 1.3.2. MacSonogram The lack of a common notation system is a severe problem in electroacoustic music and other types of music without such system [Cogan 84]. Our solution to this problem is the use of large scale sonograms [Waters, Ungvary 90]. Sonograms are well known in speech research but are not widely used for musical purposes. The traditional techniques to produce sonograms are not directly transferable to the musical field. We have solved the problems by developing the MacSonogram programme [Lunden, Ungvary 91]. It is an interactive tool designed to be used by musician, composer, musicologists and other, for visualise play and analyse music. 2 HARDWARE 2.1. Computer systems Two computer systems are in useat our lab. They are interconnected via an ethernet network. Either system can be uses for both music and/or dance. 2.1.1. Macintosh IHfx The first computer is a Macintosh IIfx with 16 Mb memory, 1.2 Gb fast SCSI-2 disks, a 19 inch b/w monitor and a 13 inch colour monitor. The system is equipped with one DigiDesign's SoundAccelerator and one MacProteus syntesizer card. The system can handle two 16 bit audio channels at 48 kHz sample rate. Development software on the system are: MPW, Think C, Think Pascal and Macintosh Common Lisp. 050 020.0 0'25.0 000 Fig. 2. Example of a sonogram created by MacSonogram (Basic Barrier by Tamas Ungvary). 304

Page  305 ï~~2.1.2. Iris Indigo Recently we bought a second computer, a Silicon Graphic's Iris Indigo with 32 Mb memory 1,5 Gb disk space and a 50MHz R4000 processor which should give a floating point performance of more than 16 Linpack MFLOPS. This together with the native audio system makes it an ideal musical workstation. The software running on this system so far are: CCPL, Allegro Common Lisp, CSound and TAE+ graphical interface builder. 2.2. Audio subsystem. The audio subsystem is shared between the two computers. It consists of: two Genelec S30NF high quality studio monitors, one Tascam DA-30 DATrecorder, one Allen & Heath Scepter analogue audio mixer used for monitoring and one Lexicon 300 digital sound effect unit. The audio signals can be kept in the digital domain duringthe whole working process. The digital signal from either of the two computer systems can be stored on and retrieved from DAT-tape. Digital signals can also be retrieved from the CD-ROM player on the Iris. 2.3. Sentograph The sentograph is a pressure sensitive input device [Clynes 80]. The original construction which measured the horizontal and vertical components of a finger pressure (2D) has been improved by the Dept. of Psychology, Uppsala University, Sweden. The new construction of the sentograph allow independent measuringof the finger pressure in all three dimensions (3D). This is accomplished by using a linkarm system with ball-bearings to divide the finger pressure into there distinctly separated coordinates. The sensitivity of the total system is estimated to 1-2 grams. 2.3.1. MIDI-Faders Fadermaster is a MIDI Control Device with eight sliders. The device transmits MIDI controller data when the sliders are moved. The first there sliders has been modified to become ether voltage to MIDI converters or keep there old functionality. This is used to convert the there voltage outputs of the Sentograph to MIDI. Acknowledgments Our project is supported by The Bank of Sweden Tercentenary Foundation (Riksbankens Jubileumsfond) and The Swedish Council for Planning and Coordination of Research (ForskningsrAdsnamden). References [Clynes 80). Clynes, M. "The communication of emotion; Theory of sentics." in: Emotion,Theory, Research and experience. Volume 1: Theories of Emotion, R.Plutchik and H. Kellerman eds. New York, Academic Press. [Cogan 84]. Cogan, R. "New Images of Musical Sound.", Havard University Press 1984. [Lunden 89]. Lunden, P. "CPL: a Composers View of Computer Programming". KACOR report 13/89. Royal Institute of Technology, Dept. of Speech Communication and Musical Acoustics. [Lunden 91]. Lunden, P. "Sound-Models: the Representation of Knowledge about Sound-Synthesis in the CPL Environment.", Proceedings of the 1991 International Computer Music Conference, San Francisco: Computer Music Association. [Lunden, Ungvary 91]. Lunden, P. and Ungvary, T."MacSonogram:a Programme to Produce Large Scale Sonograms for Musical Purposes", Proceedings of the 1991 International Computer Music Conference, San Francisco: Computer Music Association. [Ungvary, Waters, Rajka 92]. Ungvary, T., Waters, S. and Rajka, P. "Nquntius: A Computer System for the Interactive Composition and Analysis of Music and Dance." Leonardo vol. 25 no. 1. Pergamon Press, England 1992. [Waters, Ungvary 90]. Waters, S. and Ungvary, T. "The Sonogram: A Tool for Visual Documentation of Musical Structure."Proceedings of the 1990 International Computer Music Conference, San Francisco: Computer Music Association. 305