Page  00000382 A PLAYABLE SYNTHESIS MODEL AND CONTROLLER FOR BOWED STRING INSTRUMENTS TapioTakala Helsinki University of Technology Department of Computer Science ABSTRACT A synthesis model has been built for bowed string sound, with musical playability as the main goal. The sound is generated with a physical string model based on masses and springs (equivalent to waveguide models), excited with a conventional friction model. The control gestures for bowing are produced with a real bow acting on force sensors attached to a violin body. The pitch is controlled with MIDI signals taken either from a keyboard, or (more ergonomically) from a fretbased sensing device built into the fingerboard of a violin. The latter utilizes real string as part of the controller, thus giving normal haptic feel to the player. Due to careful experimental tuning of synthesis and control parameters, the model follows playing gestures in a natural way and produces very convincing violin sound. It can easily be modified for other bowed strings, and the controller can be used to play other synthesizers as well. 1. INTRODUCTION Bowed strings have long been a tough challenge to digital synthesis. Traditional approaches (FM or other abstract algorithms) do not produce the right temporal evolution of sound spectrum to match real playing. Sampling has been utilized recently with much better results (e.g. the Garritan Stradivari), but still lacking full flexibility, especially for changing the timbre dynamically while playing a note. Physical modelling, despite of intensive research, still has not reached a real success. Although offering high flexibility, physical models are notorious for being hard to control and thus producing inadequate sound quality. Physical modelling of violin and other bowed strings has been studied a long time. An excellent review is given by Woodhouse and Galluzzo [17], referring to both important physical research before digital synthesis and to recent approaches, particularly concerning the bow-string interaction. One of the first digital models presented by Smith [13] was commuted synthesis based on a waveguide string excited by a pulse train. Since then, more accurate string models have been built using waveguides divided by a scattering junction to represent the bow interaction. Specific aspects of the violin family have been studied in a number of recent doctoral theses (reviewed below), each of which is based on a number of more focused individual articles. Guettler [3] is concerned with playability measured as the time interval in which the string starts to make Helmholz motion. This is mapped in the parameter coordinates of bow velocity vs. pressure into areas where most natural playing happens. He also explains the physical phenomenon of subharmonics, used to bring special sounding effects by playing with very high pressure and slow movement of the bow, as often performed by Kimura [6]. Serafin [12] concentrates on detailed models of friction between bow hairs and the violin string, but also suggests gestural mechanisms for controlling the model [11], such as scratching with a graphical tablet. Later she also has extended her research to frictional sounds other than from the violin family. Many of the studies deal with controllers of string instrument models. Poepel and Overholt [9] present a review of recent developments. Trueman [15], together with Cook [16] presents a number of ingenious devices for sensing the bow (R-Bow) and fingers (Fangerboard) utilizing accelerometers and force sensitive resistors (FSR) among other sensors. They also experiment with the sound reproduction through an instrument body containing loudspeakers (BOSSA). These developments have lead to playing techniques and music styles clearly different from traditional violin. Special controllers aiming at traditional playing are for example the vBow by Nichols [7][8] and Hyperbow by Young [18]. Both utilize special sensors attached to the bow. Holm [4] has built a controlling system based on motion tracking in immersive virtual environment. Rasamimanana [10] and Carrillo [1] analyze the bowing gestures, in order to gain general understanding, and to use them for controlling synthetic instruments. Producing bowing gestures procedurally from a special notation, instead of physical controllers, has been proposed by Takala et al. [14]. Despite of intensive research, we still lack a usable control mechanism for bowed synthetic strings. Personally I haven't yet heard a satisfactory synthesizer for string sound, and it is not uncommon to hear researchers say that "physical models sound bad". In my opinion this is mainly due to the lack of adequate controllers. The chracteristic sound of a bowed string instrument comes more from its typical use than from its detailed physical properties. As a violinist myself, I would like to have a synthesizer that can be played like a real violin. This paper attempts to get a step forward towards that goal. 2. PHYSICAL MODEL OF STRING AND BOWING In this study, the vibrating string is modelled as a chain of point masses connected by springs, and with viscous 1 382

Page  00000383 damping for each mass. The transversal force acting on each mass (longitudinal movement not considered here) is the difference of spring tensions on each side of the mass (proportional to bending of the string). Integrating the induced acceleration in time gives instantaneous velocity and position, which in turn determines the spring tensions. Normalization and discretization for unit masses and time steps results in the same formulas as used in waveguide models [20]. In addition, there is a damping force proportional to the velocity of the mass relative to its neighbors. This is also in principle equivalent to a waveguide with all damping lumped to a single filter, the mechanism mostly used for modelling string instruments. Although computationally not as efficient as the waveguide, this model is simpler to implement and gives immediate data for visualization of the string shape, thus making it particularly useful for interactive experimentation. The friction model used here is similar to previous ones [12]: the dual action of bow hairs alternatingly sticking and sliding on the string follows a hyperbolic relation for bow pressure and the instantaneous velocity difference between bow and string. The model has a large number of different parameters governing its behaviour. Unlike most previous studies, the purpose here was not to identify the parameters from physical measurements or acoustic data from real instruments in order to make exact replicas of existing instruments or recorded sounds. Instead, the system was tuned experimentally to result in a sound audibly similar to real violin, based on my subjective assessment. As the parameters are strongly interrelated, much trial-and-error was needed. A complete analysis has not been done and would be out of the scope of this paper. Some considerations are given below. The modelling parameters most affecting the sound of the string as such are material and finger damping (determining the spectral decay rates, and that way brightness), violin body resonances (formants), and the coupling of string to the bridge (sympathetic vibration). However, more important than string and body, is the bow and its interaction with the string. The control values needed are bow pressure and velocity, and (to a lesser extent) distance from the bridge. The relative values of pressure and velocity roughly determine what kind of sound is played (harsh for high pressure, whistling for high velocity), and there is an optimal relation range for best playability [19]. The dynamic shape and synchronization of these values (the gesture) determines a bowing style. gradually die away while the amplitude grows. Often there is considerable aperiodic (chaotic) noise in the motion during the first ca. 50 ms time. Another typical feature is the continuous variability of both envelope and spectrum, due to the variability of bow force and velocity. This is hard to achieve artificially, but comes naturally with a real-time gestural controller. 3. TANGIBLE CONTROLLER The violin bow is a gentle device allowing very different gestures to be used for producing various bowing styles, such as legato, detache, spiccato, staccato, martele, etc. Thus simply turning the bow excitation on/off for a synthesis model is unacceptable, but we need more flexible, continuous control. In this study my aim was to keep the playing experience as close to real violin as possible. For that reason I had to abandon options where the instrument is radically reconfigured, such as BOSSA. Also in order to avoid disturbing mechanical attachments and additional weight in the bow, I decided to leave the bow as it is, and to put all sensory instrumentation into the violin body, deviating that way from vBow and Hyperbow. The arrangement consists of two parts, a thin rod simulating the string at the place where it is bowed, and a real but non-vibrating string to sense fingering on the fingerboard. 3.1. Friction controller The bow controller consists of an aluminium rod with two strain gages attached in orthogonal directions, as depicted in figure 2. Material flexibility is utilized to measure force against the rod. One strain sensor measures the vertical force caused by bow pressure. The other sensor measures the transversal force caused by the bow's friction when moving over the rod. P V horizontal force (v, bo vlocity) vertical force (p bow pressure) Figure 2. The two strain gage sensors for measuring bow movement against a flexible rod. The device is not measuring exactly the bow pressure and velocity, but the vertical and horizontal forces measured are closely related to these quantities. The main difference is that the horizontal force is based on friction and is thus dependent on both pressure and velocity. This nonlinear correlation was partly compensated in software in order to make these control quantities more independent. This was not based on exact measurements, and needs further research. Figure 1. Evolution of the sound after an attack. Qualitatively an important feature that makes a distinguishable violin (or bowed instrument) sound, is the waveform evolution from first attack to the sawtooth wave of full Helmholz motion. Figure 1 shows a typical such progression. The sound starts with a double frequency due to "slips" in the waveform, which 383

Page  00000384 Another difference to real bowing is that for a tilted bow (rotated about the rod) the pressure and velocity map to both vertical and horizontal sensors, again reducing their independency. However, this is acceptable because normally the bow is in a fairly stable orientation against the string - there is not much room for tilting without switching to another string. By suitable scaling, the two sensor values give a fairly large parameter space for variable bowing styles. Particularly, the dynamic proportions of the parameters (the bowing gesture) determine the necessary attack and release transitions idiosyncratic to each style, e.g. spiccato [2]. stringr finger 3.3. Measurement devices Signals from both bow and finger sensors are forwarded to the computer through a Phidgets' Interface Kit, which performs 10 bit A/D conversion and connects directly to a USB socket. As the strain gage's (Kiowa KFG-5-120-C1) resistance changes very little in normal conditions, each of them is measured in a Wheatstone bridge and an analog preamplifier (standard 741 op amp with ca. 100x gain) is used. Changes in the resistor chain on the fingerboard are large enough to be detected directly by Phidgets. The sample rate is about 100 Hz, which is acceptable, but does not allow fine temporal analysis of gestures. There is some noise, but it appears to be dominated by the natural randomness of hand motion. However, more research is needed to assess the role of this. 4. RESULTS AND DISCUSSION At the time of writing, I have a functioning prototype (figure 4) consisting of four bow sensors and a single fingerboard with one octave (12 semitones) of frets. The next on-going step is to augment it into four strings and some more frets to get a fully functional violin model. I have tested the synthesis model and controller in practical use and found them to be functional. In particular, the most common bowing styles (legato, detache, spiccato) perform well. An experienced violin player can produce them after only a little adaptation, which indicates that the bowing behaves very similarly to real violin. S I ) " " I k Figure 3. Frets with a resistor chain for detecting finger position on the fingerboard. 3.2. Detection of finger position There are different options to define the string length, which in turn defines the played pitch of a physical model. A MIDI keyboard was used in the early prototype as a technically easy solution. However, it is not very natural to use bow with one (right) hand and keyboard with the other (left) hand. Instead I wanted a controller attached to the neck of the violin to detect finger positions, and preferred to even have a real string to give natural haptic touch to the fingers. Various options were considered, for example long FSR sensors along the fingerboard as by Kessous et al. [5], but no suitable component was available in the market. The final solution is based on metallic frets attached to the fingerboard, connected in chain by fixed resistors (figure 3). The string touching a fret makes a shortcut and yields a well-defined resistance to be measured. Frets are not a natural arrangement for the violin, as they do not allow fine tuning of the pitch, but the solution works well technically. In order to get finer control, I also tried resistive material on the fingerboard to detect where the string touches the board, but the contact resistance appeared to be too variable to give a reliable signal. Figure 4. The prototype with controllers for bowing four strings but sharing a single fingerboard sensor. Those bowing styles where a note starts with the bow not moving but only pressing the string (martele, staccato) are more difficult to perform, because an unintended horizontal force causes the model to play even if the bow is not moving. On a real instrument this force just pulls the string off-centre without making any sound. A partial remedy is a mapping that scales horizontal force down when the vertical force increases, before interpreting the forces as pressure and velocity. This also helps to play subharmonics, i.e. tones lower I 384

Page  00000385 than the natural string resonance, which appear when applying high bow pressure and very slow velocity. The fret controller for left hand is easy to use, and helps to play correct pitch easily. On the other hand, it prevents fine tuning and glissando effects, and forces vibrato to be made artificially. It also does not measure the finger pressure, which would be needed to play flageolettes (harmonics), which the synthesis model is able to produce simply by applying only light damping instead of fixation in the position of the finger. The mass-spring model is not optimal in view of computational efficiency. Currently it has been tested in a PowerBook G4 with 1 GHz processor and 22050 Hz audio sample rate. Under these conditions, the software is able to handle in real time about 80 point masses, which means two or three violin strings simultaneously, or a single lower string of the viola. An obvious future enhancement is to convert the model into a waveguide by moving damping from individual nodes into a lumped filter. Potential extensions are also other bowed string instruments, or control of virtually any synthesis software with the same device. Important future work is a careful analysis of typical bowing gestures [1] [2], and their effects on the controller, as well as development of the friction model based on physical measurements of the string under the bow. 5. CONCLUSIONS A sound model and physical controller have been presented for real time performance with bowed string instruments. The work combines previously developed modelling techniques with a novel controller, and presents a usable instrument prototype. The various parameters governing a physical model's behaviour have been tuned experimentally, such that the model is playable within a wide range of controller input values. As a consequence, it enables the temporal variation necessary for lively musical performance with different bowing styles, which has not been possible thus far. 6. REFERENCES [1] Carrillo A. Gesture based synthesis of bowed string instruments, Doctoral thesis preproposal, Universitat Pompeu Fabra, Barcelona, 2006. [2] Guettler K, Askenfelt A. "On the kinematics of spiccato bowing", Report TMH-QPSR 2-3/1997, Royal Institute of Technology, Stockholm 1997 [3] Guettler K. The Bowed String - On the Development of Helmholtz Motion and On the Creation of Anomalous Low Frequencies. Doctorate thesis, Royal Institute of Technology, Stockholm. 2002. [4] Holm J-M. Virtual violin in the digital domain. Doctoral thesis, University of Jyviskyld, 2004. [5] Kessous L, Castet J, Arfib D. "GXtar, an interface using guitar techniques". Proc. NIME'06, Paris, 2006, pp.192-195. [6] Kimura M. "Subharmonics: A Revolutionary Technique For The Violin". Proc. ASVA '97 - Intl. Symp. on Simulation, Visualization and Auralization for Acoustic Research and Education, Tokyo, 1997. [7] Nichols C. "The vBow: Development of a Virtual Violin Bow Haptic Human-Computer Interface". Proc. NIME'02, Dublin, 2002, pp.29-32. [8] Nichols C. The Vbow, An Expressive Musical Controller Haptic Human-Computer Interface. Doctoral thesis, Stanford University, 2003. [9] Poepel C, Overholt D. "Recent Developments in Violin-related Digital Musical Instruments: Where Are We and Where Are We Going?", Proc. NIME'06, Paris, 2006, pp.390-395. [10] Rasamimanana N. Gesture Analysis of Bow Strokes Using an Augmented Violin. Memoire de stage de DEA ATIAM, Universite Paris VI, 2004. [11] Serafin S, Burtner M, Nichols C, O'Modhrain S. "Expressive Controllers For Bowed String Physical Models", Proc. DAFx'01, Limerick, 2001. [12] Serafin S. The Sound Of Friction: Real-Time Models, Playability And Musical Applications. Doctoral thesis, Stanford University, 2004. [13] Smith J O. "Nonlinear commuted synthesis of bowed strings", Proc. ICMC'97, Thessaloniki, 1997, pp.264-267. [14] Takala T, Laurson M, Vilimiki V, Hiipakka J. "An Expressive Synthesis Model for Bowed String Instruments". Proc. ICMC'2000, Berlin, 2000, pp.70-73. [15] Trueman D. Reinventing the Violin. Doctoral thesis, Princeton University, 1999. [16] Trueman D, Cook P. "BoSSA: The Deconstructed Violin Reconstructed", Proc. ICMC'99, Beijing, 1999, pp.232-239. [17] Woodhouse J, Galluzzo P. "The Bowed String As We Know It Today". Acta Acustica / Acustica, Vol. 90, 2004, pp. 579-589. [18] Young D. "The Hyperbow Controller: RealTime Dynamics Measurement of Violin Performance", Proc. NIME'02, Dublin, 2002. [19] Young D, Serafin S. "Playability Evaluation of a Virtual Bowed String Instrument". Proc. NIME'03, Montreal, 2003, pp. 104-108. [20] Van Duyne S, Smith J. "Physical. Modelling with the 2-D Digital Waveguide Mesh", Proc. ICMC'93, Tokyo, 1993, pp. 40-47. 385