Page  00000001 Bowed string synthesis with force feedback gesture interaction Jean-Loup Florens, Cyrille Henry ACROE - ICA. INPG - 46av. Felix Viallet 38000 - Grenoble - France httpI:// 5) User graphical interfaces that include modeling, Abstract compositional and analysis tools. The CORDIS ANIMA formalism allows to model physical objects according to a modular methodology which guaranties, at each step of modeling, the energetic consistency of the behavior of the model. Maintening this energetic consistency is a crucial point in the use of interactive simulation by means of Physical Modeling and Force Feedback Gesture Devices. This paper presents a CORDIS-ANIMA model of bowed string, which closely links the properties of the produced sounds to the gesture and energetic investment of the player. A pertinent feature of real bowed instruments is their high sensitivity to the gesture dynamic. The proposed model restitutes this sensitivity providing high musical quality and nuances in the synthetic sounds. In addition, the use of the consistent physically-based modular designing presented here, allows the designer to lead towards a minimal physical model able to restitute this so pertinent feature. 1 Introduction In sustained oscillation instruments (like bow string instruments, reed instruments...), the excitation gesture quality is known to widely influence sound timbre. For example, in bowed string instrument a correct bow force is known to be essential for the quality of the tone. In this paper, we are especially interested in the bowed string: it suppose a continuous gesture interaction between hand and string, which is difficult to simulate. Our general goal is to extract from the study of real instruments some minimal significant properties, so that we could model a large variety of new computer real-time playable instruments. The context of this work is the research program that aims generally at creating an 'instrumental relation' between musicians and computers. The main research axis are the followings: 1) The physical model formalism definition and conceptualization: Based on the mass/interaction physics, it provides a very general mean of designing instrument models. 2) Experimental and theoretical works on physical models among which bowed string, reed instrument plucked string and percussion instruments. 3 ) Computer architecture and software for interactive real-time synthesis. 4) Force feedback gesture interface devices for the gesture interaction. In this context it is possible to make interactive physical model synthesis that present interesting gesture sensitivity in a similar way than real instruments. In the sustained oscillation instrument the sound evolution is closely linked to gesture and even in its short time determination, it depends on the action or behaving of the instrumentalist. It is well known that the player can widely induce the timbre properties of a sustained instrument sound. In the following we present a bowed string interactive PM synthesis that present such properties. Our goal was not to reproduce the exact playability of violins but to extract from the knowledge on these instruments some minimal significant properties to make new computer instruments. In a first part we present some similar works in the field of real-time synthesis. In the second and third part we present the bowed string model and its implementation on a real workstation. 2 Related Works Several real-time implementations of sustained sound instruments have been made since 1982 (Smith, 1982, Smith 1986, Cook 1992). They are based on wave guide techniques, which provide an interesting efficiency for real-time implementation. To our knowledge, they have anyway never been used with a real-time gesture force feedback interaction. On the contrary, J Guerard and X Boutillon's interesting works on hybrid wind instrument simulations provide a real interactive gesture control (Boutillonl995)(Guerard 1998). Technically, these works improved the knowledge on hard real-time interaction between real world and a simulation. In 1985, we introduced a first real-time bowed string simulation based on the particle-interaction system we had designed. It was not retroactive but allowed a gesture control of pressure and sliding. A device had been designed especially with displacement captor, and, perpendicular with it, a force sensor (Florens &al. 1986) The force-feedback gesture interaction was introduced in the same mass-interaction context in 1990 with a force feedback keyboard (Cadoz & al., 1990). This interactive simulation demonstrated that it is possible to obtain a very sensitive virtual instrument even with an elementary model (Florens, 1990).

Page  00000002 3 Physical Model context In our physical modelling context objects are designed as physical system whose internal evolution law can be computed in an explicit and deterministic way. The ability of these objects to be composed with each others is obtained by providing them with the connection points that are dual physical signal inputoutput pairs. The main difference with other physical model approaches is that, at the modelling level, the user is not concerned by the force/position signals manipulation but only by structural operations as defined. The minimal elementary components from which a general physical model can be built are: 1) the <MAT> element, that receive forces and calculate its position (typically the punctual inertia). 2) the <LIA> element, or interaction element, that calculate a force according to the positions of the two <MAT> it is connected with. <LIA> elements model linear visco-elastic interaction or non linear interaction. All these components are provided with sizing constants that are usually called physical parameters. In addition to these basic components special modules may be necessary to model specific properties in a non expensive way. One of these is the parameter control function that we will use in the bow/string friction model.(cf.4.3). Finally, the coupling with a gestural interface are made through special <MAT> elements. The displacement space may be limited to one dimension (<MAT> have one degree of freedom) in the context of sound synthesis. That implies that neither shape nor other geometrical properties can be represented. However in this ID system the significant and essential dynamics properties of instruments can be modeled and synthesized with a great efficiency. 4 The bowed string model Using this modeling language our bowed string is made of 3 main modules: The bow module, the bow/string interaction and the string module (figure 1). In addition and apart to the computed parts of the model we must firstly consider the gesture interface whose configuration determines the gesture morphology of the instrument. i2D interface / Bow N.L joystick I Acturator ' Module interaction; String Gesture intreface I Figure 1. The bowed string model. 4.1 Gesture interface. The gesture interface provides the force feedback coupling with the virtual bow. It is made of two parts: 1) A special electro-dynamic multi-axis actuator that has been specially designed for the gesture force feedback coupling. 2) An adaptable passive mechanism that determines the manipulation morphology. It is shaped as a two axis joystick, that can be used to drive the two mains minimal motions of a bow that are the transversal sliding and the pressure control one. Compared to real bowing we have neglected many other motion axis, like ability to vary the hair width, or the distance of the bowing point from the bridge. 4.2 The bow The part so called "bow" is an intermediate mass between the gesture interface and the string. It is linked on one side by a visco-elastic link to the interface and on the other side to the link by the non linear friction module. Because of the ID representation, the two axis of this part that correspond to each of the two dimensions of the gesture, are made of two distinct <MAT> / <LIA> links (figure 1). The tunable parameters of this element are: 1) The stiffness and damping of the link between these inertia and the gesture interface. 2) The scaling parameters of the link with the force feedback gestural interface. These concern independently the two axis forces and the two axis displacements. 4.3 The string The basic model of the string is a chaplet composed of 25 to 60 masses linked by visco-elastic elements. Its ends are attached to high inertia damped oscillators that behave as bridges. These bridges can serve as sound output points or can be linked to other parts of the instrument. Because of its non harmonic natural spectrum the discrete homogeneous chaplet produces slower attacks than an harmonic string (figure 2). It may be useful to compensate this non-harmonicity by introducing a non homogeneous repartition of the stiffness and inertia. ( a )... i..... (b) Figure 2. An attack transition of an homogeneous non-harmonic chaplet (a) and of an harmonic (b) one. These re-tuning techniques are based on modal analysis and it has been shown in (Incerti 1996) that it is possible to obtain for a chaplet, any natural spectrum by an adequate repartition of stiffness and inertia. We can also use in place of the chaplet, its equivalent modal model. The modeling formalism CORDIS-ANIMA allows to build a standard modal structure (Cadoz 1993, Djoharian 1993) as made of a set of mass-spring cells and special coupling modules that provide the equivalent <MAT> points of the chaplet. On one hand the modal model is more interesting to directly control and observe the dynamic properties of the system. On the other the chaplet is more interesting with a structural approach of the model making.

Page  00000003 4.3 Bow-string interaction The specificity of the bowed string instruments is mainly due to the particular properties of the bow-string interaction: sharpness of a specific dry friction and wide sensitivity of friction effect to bow pressure. Recent related works on bowed string synthesis take into account some hysteretic properties of dry friction (Serafin &al 1999), in the aim of getting a more accurate representation of the rosin effect. In the past we have also used such a memory module (Florens 1990) in order to increase the sticking effect but in our case it was necessary because of the low computing power of the system whose computing rate was limited to 18 kHz. In the present simulation we use only the simplest memory-less model of rosin friction: This model is sufficient to provide the stick-slip effect and the usual behaviors of bowed string (like Helmholtz motions, subharmonic oscillation, and even some flattening effects). Within it, sliding forces are linked 1) to the sliding velocity by a non-linear function and 2) to the pressure by a proportional law. Friction link <LIA> Pressure control Buffer link <LIA> (a) (b) Figure 3. The friction model f(v) curve (a). The friction physical module is a double "quadripolar" <LIA> (b). This model has been implemented in a special double <LIA> module which also takes into account the "vertical" component of the interaction. In this module this vertical interaction model is an elastic buffer whose force is used as the pressure control parameter of the friction part. The use of such a "quadripolar" element avoids explicit signal flow manipulation and is thus compatible with the physical basic formalism. We can check that this module is strictly dissipative. and can then be used as an independant physical component. 5 Implementation The bowed string model has been implemented on a SGI workstation specially adapted for physical model interactive simulation. It is equipped with specific hardware mainly the gesture interface system, an audio interface system, and hardware concerning clock generation and synchronization. The software environment is a simulation engine that consists in an open library of physical modules and a kernel that assumes all real-time synchronization and communication functions. This software takes advantage of the multi-processor architecture by dedicating one processor to system management whereas all of the others perform the real time computations in a special reserved mode (Florens 1998) The specificity of the model that are the sharpness of the non linear part and the need to provide an efficient gesture coupling have led us to use various computing rates for the different parts of the model. The critical parts that are 1) the bow-string non linear interaction module 2) the last mass element of the bow module are computed at the higher frequency (typically 48kHz). The link to the force feedback gesture interface is computed at the maximum rate of 3kHz that is compatible with the reactivity of the system. This multi-rate mode is supported by the simulation engine in which several rate level are pre-defined, each elementary component being affected to one of these levels. We must also precise that all the synchronization constraints are driven trough the data-flow conservation rules, from a unique master clock. 6 Experimentation / Results Generally, this bowing model reinforces the idea that one can obtain musical timber variation and phrasing even on simple model, provided the use of a gesture interaction. (a) (b) Figure 4. The string motion (a),bowing force(b) and velocity(c) at the bowing point in a real time interactive playing. Compared to our previous works on the same topics, the main results are: 1) The ability to use a 'sharper' bow - string interaction model, while preserving the stability of the computation. This allows higher bow pressures, higher pitches and string damping. 2) Some general rules that concern the gesture interface coupling and the use of multi rate simulation have been obtained and will be useful in the future developments. 3) The model presents other interests that could not be found in real instruments. For example, by tuning the displacement and force scaling factors of the device, we provide a way to enhance some sensitive and effective gesture effects without changing the virtual model. This means that we can focus our research on the perception of the musician gesture and then explore furthermore the gesture/sound relation. 4) A more precise restitution of this bowing force has been obtained. We can observe in particular that its first pulsed component amplitude directly depends on the vibration energy that is dissipated in the string (figure 5). We can also observe some oscillations during the sticking phase (figure 4). These are produced by the reflexions of the Helmholtz corner against the bow as explained in (McIntyre &al. 1981). We have observed that they were playing an

Page  00000004 important role in triggering some secondary Hemoltz motions. 5) In the simulation, thanks to the bandwidth improvement of the force feedback interface these main pulsed components of the bowing force are also driven to the instrumentalist's hand, as it does in the case of a real bow (Askenfelt &al. 1992). By this mean the hand is closely linked to the vibrating system............................... (a)...... / I i i....:i........:.::: i i ii:::I:: i.... (b) Figure 5. The bowing force observed for two different values of the damping coefficient in the string: high damping (a) and low damping (b) low damping. The left figures are the corresponding force versus string displacement diagrams. 7 Sound Examples The following sounds are made from a simple string model, bowed by a beginner playing a variety of gestures without any training nor musical goal. Pitch: 110 Hz 220 Hz 440 Hz 880 Hz Sound 1 Sound 2 Sound 5 Sound 6 Sound 3 Sound 4 8 Conclusion This model and its implementation allow investigations concerning bowed string sound synthesis or other sustained oscillation models that use a sharp non-linearity. Its high efficiency enlarges the field of possible complexity and pitches for the real time interactive simulations. The high bandwidth of the gesture interface channels has provided in the case of force feedback coupling a more precise and sensitive interaction between the instrumentalist's hand and the vibrating string. This is interesting in the case of live synthesis but also for the musical studio creation. It also provides a powerful experimentation tool on the instrumental gesture that concerns especially the vibration coupling. Indeed, thanks to the scale factors, the research on gesture action not yet quantified (stiffness of the arm...) can be enhanced. All these results open a new way in for the musical creation and technical research in the context of virtual instruments. References ASKENFELT(A)& JANSSON(E V),1992 < On Vibration Sensation and Finger Touch in Stringed Instrument Playing. > Music Perception. Spring 1992, Vol 9 N~3 pp. 311-350 BOUTILLON (X) & GUERARD 1995. < Hybrid Synthesis >, Proc. International Symposium on Musical Acoustics.Socie6t Francaise d'Acoustique, Paris. pp. 439-445. CADOZ(C),LISOWSKI(L),FLORENS(JL) - "Modular Feedback Keyboard", Computer Music Journal, MIT Press Vol. 14 N~2, 1990. CADOZ (C), LUCIANI (A) & FLORENS (JL), "CORDISANIMA: "a Modeling and Simulation System for Sound and Image Synthesis- The General Formalism", Computer Music Journal - MIT press - Vol 17-1 spring 1993. CAUSSE (R) 1992- "Mise au point des archets num6riques. Utilisation pour 1'6tude de la corde frotte". These d'Etat. Universite du Maine. 1992. COOK (P) 1992 -"A meta-wind-instrument physical model", Proc. International Computer Music Conference, San Jose, pp. 273-276. DJOHARIAN (P) - 1993."Modeles pour la synthese modale "- Colloque International sur les Modeles Physiques. Grenoble Septembre 1990. Ed. Maison des Sciences de l'Homme. Paris 1993. FLORENS(JL),CADOZ(C) & LUCIANI(A), 1986 < Optimized Real Time simulation of objects for music synthesis and animated images synthesis > Proc. Of the International Music Conference. International Computer Music Association.. pp.65-70. FLORENS (JL), CADOZ(C) 1990- "Modele et simulation en temps reel de corde frottee". ler Congres Francais d'Acoustique. Editions de Physique Paris 1990. C2-1990 pp C2-873-876. FLORENS(JL) 1998 < A real-Time Workstation for Physical model of multi-sensorial and gesturally controlled instrument. > Proc. Of the International Music Conference. International Computer Music Association.. GIBET(S),1987. "Codage, representation et traitement du geste instrumental". These INPG. INPG Grenoble,1987. GUERARD(J) 1998 - "Modelisation et Simulation experimentale de systemes acoustiques. Application aux instruments de musique. " These de doctorat. Universite de Paris VI. 1998. INCERTI (E) "Synthesis and Analysis tools with Physical Modelling: An Environment for Musical Sounds Production" - ICMC 96. INCERTI(E),1996. "Synthese de sons par modelisation physique de structures vibrantes. Application h la creation musicale par ordinateur". These INPG Grenoble 1996. McINTYRE(,SCHUMACHER&WOODHOUSE 1981. ~ Aperiodicity in bowed string motion ~ Acustica Vol. 49 (1981) pp. 13 -32. SERAFIN(S),VERGEZ(C)&RODET(X),1999-"Friction And Application to Real-time Physical Modeling of a Violin" - ICMC 99 Proc.. International Computer Music Association, pp. 216-219. SMITH (J.O.) 1986 < Efficient Simulation of the Reed-Bore and Bow-String Mechanisms. Proc. Of the International Music Conference. International Computer Music Association.. pp.275-280. SMITH(JO) 1982 < Synthesis of bowed strings >. Proc. Of the International Music Conference. International Computer Music Association..