Page  308 ï~~Towards a Musician's Cockpit: Transducers, Feedback and Musical Function Roel Vertegaal Tamas Ungvary, Michael Kieslinger Department of Ergonomics Dept. of Speech Music and Hearing University of Twente Royal Institute of Technology P.O. Box 217 Box 700 14 7500 AE Enschede, The Netherlands 100 44 Stockholm, Sweden R.Vertegaal@wmw.utwente.nl ungvary@kacor.kth.se mkies@magnet.at ABSTRACT: This paper describes our on-going theoretical research into the design of computer music instruments. Traditionally, an acoustical instrument is used as an extension of the body. We feel the importance of this tight relationship between musician and instrument has traditionally been underestimated in the design of computer music controllers. In this paper, we will show the benefits of a cybernetic approach to the man-instrument system, in which we specify relationships between body-parts, different types of transducers, feedback, and the musical function performed. Matching these different requirements is a first step into bringing computer music instruments into reach of the skilled musician. Introduction A key aspect in the ergonomics of computer music instrument design is that musical instruments are intended for skilled performance with limited functionality (i.e., one does not need to be able to edit text with it). According to [Shackel, 1990], the usability of systems can be measured using four criteria: (O Learnability: the amount of learning necessary to achieve certain tasks; O Ease of Use: the efficiency and effectiveness with which one can achieve these tasks; O Flexibility: the extent to which a system can adapt to new task and environment requirements; (4) Attitude: the positive or negative attitude of the user towards the system. The usability of computer musical instruments is not so much defined in terms of learnability or, for example, safety considerations, as it is in terms of ease of use: the effectiveness and efficiency in achieving the desired result. We also feel it is this opportunity for the musician to effectively and efficiently express emotions that most affect the musician's attitude towards a computer music instrument system. According to [Polfreman and Sapsford-Francis, 1995], no single such system is likely to fulfil a user's demands. Systems should therefore be customizable to other users and uses (criterion 3). However, this does not mean to say computer music instruments should be continuously flexible. Continuous flexibility in the instrument demands constant adaptation and memorization from the musician. The mental load of dealing with a constantly changing system will never allow a musician to internalize the system and achieve efficiency and effectiveness (criterion 2) [Keele, 1968]. According to Waisvisz [Krefeld, 1990], the ability to allow a freeze of functionality is important: "We decided not to do any further developments on [the instrument] in order to start the musical phase [...]. It was time to learn to play [...], to forget about technology [...], to enter the musical domain." In order to meet the ease of use criteria, acoustic instruments have traditionally been designed and used as an extension of the musician's body. In the design of computer music instruments, however, the importance of this tight relationship between musician and instrument has so far been underestimated. The ability to discouple control input from the acoustical generator was seen as an important opportunity to reach new results, not possible previously with acoustic instruments. The thought that control input would not be hinderered by the practicalities and idiosyncrasies of the acoustical generators has sprouted a wealth of new instrument designs, each with its own controllers and control mappings. The discoupling has given us the possibility to think of control input and musical output as two separate processes, each with their own degrees of freedom. However, this line of thought also made us loose certain properties. For example, feeling the vibrations of the audible result on the actual transducer is a form of tactile feedback usually not found in computer music instruments. When we take a cybernetic approach and look at the man-instrument system as a whole of constituting parts (emotional tension, body parts, different types of transducers, feedback and musical function), we see that the relationship between these different entities are not merely simple input-output relationships [Pressing, 1990]. For example, the tension in the muscle itself as conveyed by muscular receptor feedback is utilized in skilled performance as a representation (a form of output) of the audible result for which this same tension provided the Vertegaal et al. 308 ICMC Proceedings 1996

Page  309 ï~~input [Vertegaal and Ungvary, 1995]. This of course only holds when the audible result has some causal relationship with the actual physical effort. Physical effort is an important musical parameter for both the artist and the audience: 0 Artists need to feel a piece as it is being created and performed. When one is writing a piece for instruments, the composer needs to consciously or subconsciously take into account the various physical aspects of the particular instrument. According to Waisvisz [Krefeld, 1990]: "One cannot compose the musical tension structure uniquely by formal rules; one can only compose for it. [...] One has to suffer a bit while playing." (2) The audience perceives the physical effort as the cause and manifestation of the musical tension of the piece [Krefeld, 1990]. We feel that the problems of using physical effort as a computer music parameter have two underlying causes: the above-mentioned lack of a causal relationship between the musician's effort and the audible result [Winkler, 1995]; and very little physical effort is needed to play some of the transducers used in new instruments. With regard to 0, when we design the physical interface of a computer music instrument, we need to carefully match transducers with the musical function they perform, taking feedback requirements into account. A good software mapping, however, is crucial too. For example, physical modeling in sound synthesis is important because it may provide musicians with a causal relationship between input and output again [Cadoz et al., 1984; Gibet and Florens, 1988]. As for Â~, candidate solutions comprise the enlargement of motion by projecting it, and the expression of musical tension through the musician's whole body language. Even if pushing a button does not need a complex movement, like hitting a key on the piano it can be done in an expressive way. Educating instrumentalists for computer music instruments is a part of this solution. Transducers, Feedback and Musical Function When building a computer music instrument system, transducers should be selected according to the musical function they are to perform. Important criteria for determining the suitability of a transducer for a particular musical function are parameters such as: movement type sensed (position, movement, force); resolution (continuous or discrete); agent of control (hand, fingers, lungs); and the type of feedback (tactile, kinesthetic, visual) [Pressing, 1990; Vertegaal and Ungvary, 1995]. For reasons of clarity, we focus on the role of feedback in this selection process. When we refer to input devices as transducers, we explicitly incorporate their inherent output qualities in terms of the tactile, kinesthetic and visual feedback they provide. We refer to this feedback provided directly by the input device as primary feedback. Highly dynamical and expressive musical functions require a large """Â~" Fnction amount of physicality in control. This very much relates to the ability for the musician to build up tension in the body when controlling these functions [Clynes, 1970]. We feel that tactile and kinesthetic feedback play an important role in Primary k constituting this ability. By the Fdbsck same token, less dynamical musical functions require less physicality in control. Figure 1 shows an elementary model in which we attempt to match musical function with different transducers, indicating the eTran..ducer. importance of the different types of r. primary feedback provided by each type of transducer. The height of the squares in Figure 1 indicates the effectiveness of each transducer type for operating a musical function, and the importance of each type of primary feedback in operating the transducer. Figure 1. Matching function, transducer and primary feedback. ICMC Proceedings 1996 309 Vertegaal et al.

Page  310 ï~~For musical function, we restricted ourselves to a very simple categorization in which we define three types: o Absolute Dynamical Functions e.g. absolute selection of pitch, amplitude or timbre; ( Relative Dynamical Functions e.g. modulation of a given pitch, amplitude or timbre; 0 Static Functions e.g. selecting pitch range, duration range, scale or transposition. When we look at bowed instruments, control of pitch by finger position and control of timbre by bow position are examples of function T. Relative control of pitch by vibrato and relative control of timbre by bow velocity and bow pressure are examples of function Â~. Selecting different tunings, or putting a mute on or off are examples of function Â~. Table 1 shows our categorization of transducers, which is largely based on a model by [Mackinlay et al., 1990]. In their model, input devices are decomposed into units which sense a particular physical property in a certain direction with a certain resolution. Physical position rotary velocity rotary isometric isotonic isometric isotonic Property position velocity force force rotary force rotary force discrete key rotary switch button 0t bend sensor pitch-bend fader rotary pot aftertouch wheel tablet abs. joystick mouse dial accelero- isometric mod. wheel trackpad trackball pressure pad meter joystick springmounted infinite tracker joystick Table 1. A categorization of transducers. In this categorization we look at input devices from a human control perspective. We therefore make a distinction between isotonic and isometric force transducers. With isotonic force transducers, motion is needed to operate the sensor. With isometric force transducers, no motion is needed. This distinction is important when it comes to feedback properties. We categorize spring-mounted devices as isotonic force transducers since with these devices, the force exerted is directly proportional to the position sensed. In figure 1, we only included the resolution parameter for position transducers. As for primary feedback, we distinguish three types: tactile (sensed by the surface of the skin), kinesthetic (sensed internally by muscle and other receptors) and visual feedback. We excluded primary auditory feedback since this is usually masked by the secondary auditory feedback produced by the musical result. We must stress that the relative importance of kinesthetic, tactile and visual feedback very much depends on the learning phase. For a musician, visual feedback of his movements is more important during the learning phase (governed by learnability), than during the expert phase (governed by ease of use). There is evidence which suggests that the inverse holds for kinesthetic feedback [Keele, 1973]. In the expert phase, tactile and kinesthetic feedback are important to allow a high level of precision for certain musical functions. Conclusions In this paper, we stressed the importance of the tight relationship between musician and instrument. We have shown how the design of computer music instrument systems may benefit from a cybernetic approach in which we specify relationships between different types of transducers, feedback, and the musical function performed. The development of such theoretical models is of key importance in the future development of computer music composition, improvisation and performance systems which meet the usability requirements of learnability, ease of use, flexibility and attitude. We hope the ideas put forward in this paper may bring about the paradigm shift necessary to finally bring interactive computer music control into reach of the skilled musician. Acknowledgements We would like to thank Bert Bongers, Axel Mulder and Bert Lenting for their genuinely helpful comments. Vertegaal et al. 310 ICMC Proceedings 1996

Page  311 ï~~References [Cadoz et al., 1984] C. Cadoz, A. Luciani and J. Florence. Responsive Input Devices and Sound Synthesis by Simulation of Instrumental Mechanisms: The Cordis System. Computer Music Journal. 8 (3) 1984. [Clynes, 1970] Manfred Clynes. Toward a View of Man. In: Clynes, M. and J. Milsum (Ed.), Biomedical Engineering Systems. McGraw-Hill, pp.272-358, 1970. [Gibet and Florens, 1988] Sylvie Gibet and Jean-Loup Florens. Instrumental gesture modeling by identification with time-varying mechanical models. Proceedings of the 1988 ICMC, Cologne, Germany, International Computer Music Association. pp.28-40. 1988. [Keele, 1968] S. W. Keele. Movement Control in Skilled Motor Performance. Psychological Bulletin. 70 pp.387 -402, 1968. [Keele, 19731 S. W. Keele. Attention and Human Performance. Goodyear Publishing Company, 1973. [Krefeld, 1990] Volker Krefeld. The Hand in the Web: An Interview with Michel Waisvisz. Computer Music Journal. 14 (2) 1990. [Mackinlay et al., 1990] J. D. Mackinlay, S. K. Card and G. G. Robertson. A Semantic Analysis of the Design Space of Input Devices. Human-Computer Interaction. 5 pp.145-190, 1990. [Polfreman and Sapsford-Francis, 1995] Richard Polfreman and John Sapsford-Francis. A Human-Factors Approach to Computer Music Systems User-Interface Design. Proceedings of the 1995 ICMC, Banff, Canada, International.. Computer Music Association. 1995. [Pressing, 1990] Jeff Pressing. Cybernetic Issues in Interactive Performance Systems. Computer Music Journal. 14 (1) pp.12-25, 1990. [Shackel, 1990] B. Shackel. Human Factors and Usability. In: Preece, J. and L. Keller (Ed.), Human-Computer Interaction: Selected Readings. Prentice Hall, 1990. [Vertegaal and Ungvary, 1995] Roel Vertegaal and Tamas Ungvary. The Sentograph: Input Devices and the Communication of Bodily Expression. Proceedings of the 1995 ICMC, Banff, Canada, International Computer Music Association. 1995. [Winkler, 1995] Todd Winkler. Making Motion Musical: Gesture Mapping Strategies for Interactive Computer Music. Proceedings of the 1995 ICMC, Banff, Canada, International Computer Music Association. 1995. ICMC Proceedings 1996 311 Vertegaal et al.