Page  00000375 Gestural Control of a Real-Time Physical Model of a Bowed String Instrument Stefania Serafin* Richard Dudasý, Marcelo M. Wanderley and Xavier Rodet Ircam - Centre Georges Pompidou 1, Place Igor Stravinsky 75004 - Paris - France ICNMAT, Department of Music, University of California Berkeley, 1750 Arch Street, Berkeley, CA 94709 USA serafin @ircamrn.fr, dudas@cnmat.berkeley.edu, mwanderley@acm.org, rod@ircam.fr Abstract In this paper we discuss the real-time simulation of a physical model of a bowed string instrument by means of alternative gestural controllers. We focus on the model's behaviour when submitted to different bow strokes performed on a graphical tablet using different types transducers. The tablet was chosen because it allows a performer to reproduce most of the basic physical gestures of bow strokes in a natural way. 1 Introduction The development of real-time modeling of musical instruments has exposed the problems of their gestural control. When a suitable device is attached to the physical model, the natural interaction that exists between a musician and his instrument is preserved, enabling the parameters of the model to evolve following the gestures of the performer. The choice of controller is intimately tied not only to both technological and musical requirements, but also to availability. Before focusing on the controller, we briefly describe the structure of the bowed string model used in this work. 2 Description of the Model The model we have built is based on previous work by McIntyre and others [MSW83]. The propagation of the transversal waves along the string as well as the energy loss along the string and at the bridge and nut are modeled following the digital waveguide implementation first proposed by Smith [Smi82], whose elegant formalisation is suitable for an efficient realtime implementation. Furthermore, the friction curve that describes the bow-string interaction is represented by a hyperbola, ' Current Address:.CCRMA, Stanford University, CA, USA - serafinOccrma.stanford.edu which is used to analitically solve the coupling between the curve itself and the linear vibrational behaviour of the string in order to speed up the computation in the context of a real-time implementation. The model is mainly driven, as displayed in figure 1, by the input parameters bow force fb, bow velocity Vb and bow position Pb. Physically, vb allows the player to control the sound level, pb is responsible for changes in tone quality and fb influences both sound level and timbre. The various relationships between these parameters allow to maximize the playability of the model, as shown in [SSW99]. pb Figure 1: Simplified structure of a bowed string model, that shows the main input parameters related to the player's control of the bow which are bow force fb, velocity vs and position pb. ICMC Proceedings 1999 -375 -

Page  00000376 3 Bow Strokes The aim of our project is the real-time simulation of highly skilled bowed string instrument performance. In particular, we are interested in studying the model's behaviour, when submitted to different bow strokes, such as detachd, balzato, staccato, flying staccato.... The basic gestural parameters which can be used to define bow strokes are: a specific precision grip in which all five fingers are in contact with the bow, a variable length linear displacement in the axis of the bow, a rotational movement relative to the strings' axes in order to choose the string to be played,the pressure of the bow against the string and the bow velocity. Other important variables include the bow position relative to the string and the amount of bow hair in contact with the string, i.e., a rotation with respect to the bow's axis. One must also be aware that the technical demands required to preform many of these bow strokes are typically obtained by musicians only after years of practice. Furthermore, in addition to the performer's skilled motor behaviour, these bow strokes heavily rely upon physical properties of both the string and the bow, such as the elasticity of the bow hair, tension of the string, etc.... 4 Choice of the Input Controller In order to obtain an accurate simulation of the bow strokes mentioned above, it is necessary to have a flexible input controller at one's disposition. We have therefore decided to use an input device that provides both the means to reproduce the fundamental characteristics of the performer's gestural control and which is sufficiently generic in order to be able to extrapolate the typical violin technique. Other important considerations for our choice of input device were availability, accuracy, precision, resolution and affordability. Among the commonly available standard input devices that match the above requirements, the one that appeared to best suit our needs was a WACOM graphic tablet equipped with a stylus transducer. One of the main factors in our decision was the number of control parameters simultaneously available (WWF97]. More specifically, the stylus can provide control for five variables: horizontal and vertical position in a plane, pressure perpendicular to the plane, and angle relative to both plane axes. The stylus is then used to control bow force, bow velocity, distance from the bridge and inclination of the bow. One can notice from figure 2 [CMR91] and 3 that the stylus provides roughly the same control possibilities as those of a real bow, and that there is also 0 Styhu ( extmra snoa del I 1 I z I 1, dA P T Uxr Roz r X Z 100r X rWf 1010 rZ 1 1 0 of the types o variables sensed and resolution. 2 dP dT 1!lOlCOif I lIOlOi of 1 Iaf i I 10101 l f' I 01f001.1 1 lOlOOief Figure 2: Stylus, puck and extra sensors depicted in terms of the types of variables sensed and resolution. a direct correspondance between the physical parameters of the stylus and those of the bow (e.g. position in x and y axis, force in the z axis and angle in both x and y). In addition, the stylus' attributes can be considered as integral, and thus match the perceived manipulation task (bow manipulation) [JSMM94]. 4.1 Extending the Tablets Capabilities The ability to use two devices on the same tablet simultaneously seems optimal for simulating both hands of a bowed string instrument's player. This allows the performer to control bowing with one hand, while controlling pitch changes, vibrati and glissandi with the other, either using a second stylus or a puck transducer. After experiments with both devices, we noticed that the fact that they do not provide the physical interaction that exists between the finger and the string restrains optimal control. We coped with this problem by fitting the tablet with sensors that can measure position and force simultaneously. These are shown in figure 4. Although positioning these sensors on the tablet does not provide the same tactile feedback as the fingerboard of a violin or other stringed instrument, the left hand finger position and pressure on a string of a real instrument may nonetheless be simulated. The main advantage in using these extra sensors is that, when compared to a stylus or a puck, they are operated using similar motor skills. 5 Use of the tablet We extended the musical programming environment Max/MSP [Zic98] by writing a DSP code resource, called violin, which implements a physical model of a - 376 - ICMC Proceedings 1999

Page  00000377 STYLUS-TABLET BOW-VIOLIN 1 Y POSITION ON THE BOW POSITION TABLET 2 DERIVATIVE OF THE BOW VELOCITY X POSITION 3 Z POSITION BOW PRESSURE 4 ANGLE IN THE X AXIS STRING PLAYED 5 ANGLE IN THE Y AXIS AMOUNT OF BOW HAIR Figure 4: WACOM tablet fitted with additional pressure and position sensors (on the left side of the tablet). simply needs to rub the tablet with the pen backward and forward and then release it. The evolution of the resulting parameters, shown in figure 5, corresponds to measurements made by Askenfelt ([Ask86]) on an actual violin. Figure 3: Comparison between the degrees of freedom of the bow-string and the ones of the pen-tablet interaction. bowed string instrument..Although the object's name is "violin", it can be nonetheless be modified to model other stringed instruments. A second external code resource called wacom (written initally in 1996 by one of the authors, but updated for this project in order to accomodate the latest features of the current series of WACOM graphic tablets), is used to detect and output the parameters of the transducer(s) on a tablet. The values provided by the stylus-tablet interaction are detected by the wacom object which sends them as input parameters to the violin object. 6 Simulation of Bow Strokes Using the stylus, we are able to reproduce most of the bow strokes (such as staccato, balzato, martellato...) without resorting to any special non-linear mapping of stylus output parameters to model input parameters. In particular, even bow strokes obtained by skilled musicians can be reproduced immediately and intuitively from the use of the stylus in place of a bow. For example, to obtain a balzato the player rubs the string quickly with the bow backward and forward, "jumping" on the string using both his wrist and his forearm, and taking advantage of the elasticity of both the string and the bow ([SD99]) In order to examine the behavior of the model when submitted to fast repeated balzatos, the player 0 0.5 | 1 Time (sec) Figure 5: Top: pressure of the stylus on the tablet during balzato. Bottom: velocity of the pen. Note that in this bow stroke, while the hand that holds the stylus reproduces with fidelity the movement performed using a bow, the behaviour of the controller is quite different. An important characteristic of the balzato stroke is the fact that the player takes advantage of the elasticity of both the string and the bow hair to facilitate the bouncing of the bow. Since this elasticity is absent in both the tablet and stylus, the performer must use a slightly modified gesture in order to furnish all the energy necessary for the stylus to rebound. A similar situation is observed in other bow strokes that are completely based on physical properties of the instrument, the more remarkable example of which is the gettato. In it the player simply allows the bow to fall and freely rebound against the string. The rigid surfaces of the tablet and of the stylus do not ICMC Proceedings 1999 - 377 -

Page  00000378 provide the same elastic feedback felt by the violinist. The bow strokes shown so far have the common characteristic that the bow is not in constant contact with the string, which is not always the case with other bow strokes. For example, to play detachd the stylus simply moves back and forth along the horizontal axis of the tablet, at an almost constant velocity and pressure, as shown in figure 6. Another example would be staccato, in which the performer exerts a high initial force and velocity and then stops the stylus almost immediately, as can be seen in figure 7. It is remarkable that, while learning to hold a bow and perform skilled bow strokes requires a considerable amount of time, the familiarity with a device like a pen allows the performer to obtain immediately a confidentiality with the controller. 0o5 o I S. ', / ' i 0.2 5; - ',, ' o 2 3 Tm(Mc) 0.1, also like to thank Patrice Pierrot for his dedicated work on the additional "left hand" sensor. References [Ask86] A. Askenfelt. Measurement of bow motion and bow force in violin playing. JASA, 80(4):1007 -1015, october 1986. [CMR91] S. K. Card, J. D. Mackinlay, and G. G. Robertson. A morphological analysis of the design space of input devices. In ACM Trans. Inf. Syst, volume 9, apr. 1991. [JSMM94] R. J. K. Jacob, L. E. Sibert, D. C. Mcfarlane, and M. P. Mullen. Integrability and separability of input devices. ACM Transactions on Human-Computer Interaction, 1(1):3 -26, March 1994. [MSW83] M. E. McIntyre, R. T. Schumacher, and J. Woodhouse. On the oscillations of musical instruments. JASA, 74:1325-1345, 1983. [SD99] S. Serafin and R. Dudas. An Alternative Controller for a Virtual Bowed String Instrument. 1999. in Trends in Gestural Control of Music, M. Wanderley, M. Battier and J. Rovan, eds., IRCAM, 1999. [Smi82] J. 0. Smith. Synthesis of bowed strings. In Proceedings of the International Computer Music Conference (ICMC), Venice, Italy, 1982. Computer Music Association. [SSW99] S. Serafin, J. O. Smith, and J. Woodhouse. An investigation of the impact of torsion waves and friction characteristics on the playability of virtual bowed strings. In to appear in Proceedings of the 1999 IEEE WASPAA, 1999. [WWF97] M. Wright, D. Wessel, and A. Freed. New musical structures from standard gestural controllers. In Proceedings of the International Computer Music Conference (ICMC), Thessaloniki, 1997. Computer Music Association. [Zic98] D. D. Zicarelli. An extensible real-time signal processing environment for Max. In Proceedings of the International Computer Music Conference (ICMC), Ann Arbor, Michigan, 1998. Computer Music Association. m.( (em) Figure 6: Top: pressure of the stylus on the tablet during detache. Bottom: velocity of the stylus. I ~T ", ',, - T, - - ...-'.... o.s 0.25 o - i "? ' i 0.5 1 Time (sec) Figure 7: Top: pressure of the stylus on the tablet during staccato. Bottom: velocity of the stylus. 7 Acknowledgments The authors would like to thank David Wessel and the France Berkeley Fund (Project Title: Gestural Control of Musical Sound Synthesis), for providing the tablet used for the simulations. The authors would - 378 - ICMC Proceedings 1999