Page  00000394 INVESTIGATING THE PERFORMANCE OF A VIOLIN PHYSICAL MODEL: RECENT REAL PLAYER STUDIES Diana Young MIT Media Laboratory ABSTRACT This paper presents the results of two recent pilot experiments conducted to investigate the performance of a realtime physical violin model using bowing parameters generated by a real player. In each of these two experiments, the calibrated bowing parameters of bow force, bow speed, and bow position (bow-bridge distance) are mapped to the calibrated inputs of the physical model in order to examine model's response to realistic bowing control. In the first experiment, previously recorded bowing parameters generated by a player are used to drive the model. In the second experiment, the player is given live control of the model (while the player monitors the synthesized sound through headphones). This work is enabled by a playable measurement system for violin bowing recently developed. 1. INTRODUCTION Physical models of violins, which take as inputs physical bowing parameters, have now reached the level of sophistication at which the sound they produce compares well to that of their real instrument counterparts. However, true and complete validation of a particular model has not yet occurred, as this requires demonstration of the same behavior as a real violin in response to realistically varying physical input parameters, and large sets of real input data have as of yet been unavailable. In this paper, the results of two simple virtual violin validation experiments are presented. In both of these experiments, the calibrated bowing parameters of bow force, bow speed, and bow position (bow-bridge distance) generated by a real violinist were mapped to the calibrated inputs of the physical model in order to examine model's response to realistic bowing control. In the first experiment, previously recorded bowing parameters generated by a player were used to drive the model. In the second experiment, the_ player is given live control of the model (while the player monitors the synthesized sound through headphones). The main motivation in conducting these experiments was to begin to explore whether the coupling between a calibrated bow controller and violin physical model is now sufficient, as discussed in [7]. Included below are details of the experiments and the results obtained. Stefania Serafin Aalborg University Medialogy 2.BACKGROUND The physical model used in these experiments was first implemented as a Max/IMSP external [1, 2, 3] but was ported to the Pd environment for use in these studies. This model incorporates the computationally efficient waveguide synthesis first developed by Julius Orion Smith III [6], with recent discoveries in the acoustics of bowed string instruments [8]. In particular, the friction interaction between the bow and the string is modeled taking into account the thermodynamic properties of rosin discovered by Jim Woodhouse and Jonathan Smith [5]. The model is calibrated from physical measurements on real violins and takes the physical parameters of bow force normal to the string, bow speed, and bow position (bow-bridge distance) as inputs. The basic operation of this implementation of the model was verified in previous experiments [10, 4]. Two measurement systems were employed in the experiments presented in this paper. In the first experiment, the archived performance study, a Vicon motion capture system was used to obtain bow speed and bow position, while a custom designed playable measurement system [9] was used to provide the bow force measurements. In the second experiment, that involving live control by a player, the custom system was used to provide the measurements of all three bowing parameters. 3. EXPERIMENTS In these virtual violin performance studies, two different cases were explored, one in which the violin physical model was driven with bowing parameters recorded from a previous performance by a real player, and one in which the player was given live control of the model. The measurement of bowing parameters was achieved using a custom designed system, evolved from earlier Hyperbow controller designs [9], as well as a Vicon motion capture system. 3.1. Archived Performance Study The first virtual violin experiment used the data recorded from a previous performance on the test violin. This simple performance featured one full down-bow bowstroke, followed immediately by one full up-bow bowstroke. Both 394

Page  00000395 of these bowstrokes were performed on the open G string and eachohad a duration of approximately three seconds. The audio and gesture data were recorded using the custom measurement system described in [9]. In addition, the gesture was also recorded using the Vicon motion capture system. The position data and speed data from the Vicon system were then combined with the force data from the custom measurement system. Using these bowing gesture data as inputs, the virtual violin was played. That is, the three recorded primary bowing parameters were used to drive the violin physical model. The resulting synthetic sound was recorded to a file, and its corresponding waveform was then compared to that corresponding to the original acoustic sound (produced by the recorded gesture data) from the test violin. 3.2. Live Performance Study In the virtual violin experiment, a violinist was asked to play the violin model in realtime using the custom bowing measurement system as a physical controller for the model. In these trials, the player listened to the resultant real-time synthesis through headphones. The player was first asked to imitate as closely as possible the sound of the real-player performance featured in the archived performance study described above. After several trials, the task requested was modified and the player was asked instead to achieve the best quality of sound possible in two bowstrokes, one down-bow and one up-bow, of equal duration. In this experiment, the outputs of a real-time Kalman filter (described in [9]) were input to the violin model (running in Pd) using the OpenSound Control (OSC) protocol. Though it is possible to produce the bowing parameters relative to the violin with the hardware sensing system that was built, for this preliminary experiment only the bow movement was used. Because of the simplicity of the performance task and the fact that the experiment was conducted while the player was seated, this was determined to be a reasonable approximation (though future studies will take full advantage of the hardware measurement system). 4. RESULTS 4.1. Archived Performance Results As described above, this virtual violin experiment relied upon the use of the Vicon motion capture system to provide input bowing parameters corresponding to an actual performance by a real player. The example performance used for this experiment was that of one simple, broad downbow stroke followed immediately by a similar upbow stroke, played on the open G string. The position data provided by the Vicon motion capture system and the force data from the bowing measurement system were input to a Kalman filter. This produced a data file containing estimates of the three primary bowing parameters for every time step corresponding to the gestural C75; -5 0.2 --0 I I---I C6 0.4 - 0.2 - _0.2 -401"Now 0 -E -0.4 -co L L 0 1 2 3 4 0.5 0 0 1 2 3 4 5 6 7 0.4 0 1 2 3 4 5 6 7 time (s) Figure 1. This figure shows the waveforms produced by the real electric violin and the virtual violin, and the three primary bowing parameters that produced them both. performance previously recorded. (The Kalman filter updates the parameters at 200 Hz, at the same rate the hardware sensor system produces the raw sensor values.) This data file was then input via Python and OSC to the physical violin model implementation in Pd, so that the model would receive a continuous data stream of the three primary bowing parameters, just as if the model was being played by the violinist in realtime. The resulting audio produced by the model was then recorded. To the ear, one of the most striking features of the synthesized violin sound was that there was no distinct attack for the second part of the sound file corresponding to the original upbow stroke recorded by the real player. This observation was verified by plotting the virtual violin waveform, which is displayed in Figure 1, along with the same visualization of the real electric violin sound for reference and the plots of the primary bowing parameters (that produced both real and virtual sound). As can easily be seen here, the violin model fails to produce a distinction between the two bowstrokes. 4.2. Live Performance Results The second virtual violin experiment was conducted by driving the physical violin model with gesture data produced by a violinist in realtime. The purpose of this investigation was to address the question of how well a real player is able to play the model using the measurement system as a controller. The performance given by this violinist was quite similar to that used in the previous virtual violin performance, which consisted of one downbow and one upbow, played on the open G string. In fact, as previously discussed, the violinist participating in this study was asked to listen to the recording of the real electric violin sound from the archived 395

Page  00000396 _5 3 c, S 2.5 3 3.5 4 4.5 5 measurement system to easily produce a better sonic result than was produced using the archived gesture data, even when the model produced unrealistic damping behavior. 5.1. Archived Performance I I I I lal I I I I 0.25 0.2 r S0.1 r/ So.i5 I 0.05 1.5 2 2.5 3 0.5 E 0.4 0.3 0.2 -0.12 o0.1 1.5 2 v ~ 2.5 2.5 3 1.5 2 2.5 3 time (s) A. The results from the first virtual violin performance experiment, in which the physical model was driven with the ges3.5 4 4.5 3.5ture data from an archived performance on a real instru\ ment, revealed one very significant point concerning the behavior of the physical model. Though the gestural per3.5 4 4.5 formance used to drive the model was that of one downbow 1 stroke followed by one upbow stroke, the synthesized violin sound produced failed to exhibit the distinction between the two strokes that was obvious to the listener in the origi35 4 4.5 5 nal real audio. As seen in the waveform of the virtual violin audio displayed in Figure 1, the absence of the second attack (of reforms produced by the the upbow stroke) is clear. This analysis indicated that rimary bowing parame- the physical model implementation had an improper rey the player. In order to sponse when the speed of the bow was zero (as was the o bowstrokes, the player case at the moment that the bow changed directions bene second between the tween the downbow and upbow strokes) and the bow force was nonzero. Figure 2. This figure shows the wav real electric violin, and the three p ters that were input to the model b) obtain a distinction between the tw( made a significant pause of about downbow and the upbow. performance and then asked to try to imitate it. In the event that this task was too difficult, the player was asked to produce the highest quality sound possible with the same downbow-upbow bowing as in the example. After listening to the example, the violinist attempted to imitate both the sound quality and timing of the recording. However, after several attempts, the violinist abandoned this approach and continued according to the second instruction, trying to coax the best sound possible out of the model with bowstrokes of different lengths and speeds. 5. DISCUSSION The virtual violin study, though limited in scope, enabled two discoveries by considering two different cases for driving the physical model with real player gesture data. The first experiment, designed using data from an archived gestural performance, addressed the question of whether or not the virtual violin model responds realistically to calibrated data extracted from an actual performance by a real player. Conducting this experiment made it possible to test the response of the model to calibrated bowing parameters and thereby enabled a significant observation concerning the damping behavior of the violin model, discussed below. The second experiment was designed to use the bowing parameters generated in realtime by a violinist (monitoring the virtual violin sound with earphones) to drive the violin model, in order to address the question of whether or not the virtual violin model responds realistically to live control by a real player. Though this experiment was conducted with only one violinist participant, it was very instructive, as the violinist was able to control the model with the playable In order to validate this assertion, a simple test was constructed. The model was once again driven with archived gesture data, but this time the data were altered so that the values of the three bowing parameters were held constant for a duration of one second from the moment of zero bow speed between the two bowstrokes. The results of this investigation are shown in Figure 3. As can be seen, a considerable length of time passes (well over half a second) before the synthesized audio dissipates. Therefore, it appears that this implementation of the physical violin model does not exhibit proper damping to emulate that which is typical to its real instrument counterpart (as seen in the waveform for the real electric violin in Figure 1). This may possibly be explained by the fact that the loss filters included in the model were designed according to decay times measured from plucked strings rather than bowed strings. Therefore, the bowing action applied by the player was not properly represented when estimating decay time. (This issue will be addressed in future versions of the violin model.) 5.2. Live Performance The second virtual violin experiment, in which a violinist was given live control of the model while monitoring the synthesized sound through earphones, was quite informative. As shown and discussed above, the violinist playing the model in realtime was able to achieve a more convincing performance of the two-bowstroke playing task, due to the production of a distinct attack for the second stroke not present in the sound produced using the archived gesture data. The success of this live performance serves as a clear illustration of the ability of the violinist to quickly adapt to the constraints of the virtual violin (as any musician would 396

Page  00000397 -c. 0.5 -o0 0- -0.5 0 1 2 3 4 7 8 9 0 1 2 3 4 5 6 7 8 9 S 1 2 3 4 5 6 7 8 9 0.2 o 0 1 2 3 4 5 6 7 8 time (s) Figure 3. This figure shows the effect of holding the bowing parameters constant for a duration of 1 second, at the point when the speed is zero at the moment of the change in bow direction. As can be seen in the audio waveform produced, it takes almost the entire added second for the amplitude of the sound to decrease to zero, indicating that the violin physical model does not produce realistic damping behavior. with a new real instrument) in order to obtain better performance than created in the offline comparisons. Because of the innate ability of skilled musicians to compensate for limitations in their instruments, it is not surprising that the virtual violin performances produced in these early experiments are more favorable in real-time playing demonstrations than in demonstrations relying on archived data. 6. CONCLUSION Because the custom bowing system used for this work is not only a measurement system but is also highly playable, it can be also used as a real-time controller for a calibrated physical model of a violin. This application was clearly demonstrated in the experiment involving live performance of the model by a real player, in which the player was able to produce articulated bowstrokes, and, in doing so, was able to compensate for an apparent inaccuracy in the model. Clearly much future work lies ahead, as the pilot experiments discussed in this paper represent preliminary steps toward the validation of both the physical model as well as the bowing measurement system. Further investigation will focus on the unrealistic damping behavior observed while testing the response of the model to real bowing data from an archived performance. Specifically, the loss filters in the model will be revised to include more precise decay parameters for bowed violin strings. Other future work will include additional studies to explore the ability of the model and the measurement system to reflect nuances of bowing technique will be conducted. The measures used in these studies will include perceptual evaluations (listening tests) in addition to quantitative assessments. Additionally, the possibility of integrating both the measurement system as well as the implementation of the model in one controller will be explored. 7. ACKNOWLEDGEMENTS The authors would like to thank Julius Orion Smith III for porting the implementation of the violin model to Pd for use in these studies and for his guidance throughout this work. 8. REFERENCES [1] S. Serafin and J. O. Smith. A multirate, finitewidth, bow-string interaction model. In Proceedings of the COST G-6 Conference on Digital Audio Effects (DAFX-00), Verona, December 2000. [2] S. Serafin and J. O. Smith. Impact of string stiffness on digital waveguide models of bowed strings. Catgut Acoustical Society Journal, Series II, 4(4):49 -52, November 2001. [3] S. Serafin, J. O. Smith, and J. Woodhouse. An Investigation of the impact of torsion waves and friction characteristics on the playability of virtual bowed strings. Proceedings of 1999 Workshop on Applications of Signal Processing to Audio and Acoustics, pages 87-90, 1999. [4] Stefania Serafin and Diana Young. Bowed string physical model validation through use of a bow controller and examination of bow strokes. In Proceedings of the Stockholm Music Acoustics Conference (SMAC 03), Stockholm, August 2003. [5] J. H. Smith and J. Woodhouse. The Tribology of rosin. Journal of the Mechanics and Physics of Solids, 48(8):1633-81, August 2000. [6] J. O. Smith. Physical modeling using digital waveguides. Computer Music Journal, 16(4):74-91, 1992. [7] J. O. Smith. Virtual acoustic musical instruments: Review and update. Journal of New Music Research, 33(3), 2004. [8] J. Woodhouse and P. M. Galluzzo. The bowed string as we know it today. Acustica - Acta Acustica, 90(4), Aug 2004. [9] Diana Young. A Methodology for Investigation of Bowed String Performance Through Measurement of Violin Bowing Technique. PhD thesis, M.I.T., 2007. [10] Diana Young and Stefania Serafin. Playability evaluation of a virtual bowed string instrument. In Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, 2003. 397