Page  00000001 NBody: INTERACTIVE MULTIDIRECTIONAL MUSICAL INSTRUMENT BODY RADITION SIMULATORS, AND A DATABASE OF MEASURED IMIPULSE RESPONSES Perry R. Cook"'2 and Dan Trueman2 'Departmnent of Comzputer Science, Princetonz University, Princeton, NJ 08544 2Departmzent of Music, Prinzceton University, Princeton, NJ 08544 prc ecs.princetonz.edu dan emusic.princeton. edu~ Abstract: Directional impulse responses were collected from stringed instruments, including guitar, mandolin, violin, and hardanger (Norwegian folk) fiddle. Impulse responses were recorded simultaneously from 12 microphones spaced uniformly at the vertices of an icosahedron. Software tools were created which allow a user to position him/herself at points around the virtual instrument, with signal sources including plucked and bowed string physical models, or any external sound source. Two multi-channel spherical audio display devices have been constructed, allowing sound to be projected in a performance space using the measured (or modified) transfer functions. 1.0 INTRODUCTION Musical instruments radiate sound in directional, frequency-dependent spatial patterns. For some instruments such as brass, the patterns are fairly predictable fr-om the known properties of horns. For other instruments such as woodwinds, the patterns are more complex due to a number of toneholes which can radiate sound, and the configurations of these tonehole radiation sources vary with different fingerings [Causs6 etal. 1992]. For stringed instruments, the radiators are wooden boxes whose shapes, materials, and techniques of construction vary greatly between families, and from instrument to instrument within a sub-family. Players of electric stringed instruments are aware of the benefits of solid bodied instruments with magnetic pickups, such as increased sustain times, decreased problems with feedback when amplifying the instrument, and the ability to process the sound of the instrument without the natural sound being heard. However, performers using solid body electric stringed instruments often find that these instruments lack the "warmth" associated with acoustic instruments, and using traditional loudspeakers to amplify an electronic instrument does not provide a satisfactory dispersion of sound in performance spaces. In recent years, synthesis by physical modeling has become more possible and popular [CMJ 1992/3]. To synthesize stringed instrument sounds using physical modeling, models of the bodies of these instruments are required which are efficient, realistic, and parametrically controllable. The latter is important to composers and interactive performers wishing to exploit the flexibility of parametric body models, allowing dynamic changes in the parameters to be used as compositional and performance gestures. Another application area is virtual reality and 3D sound, which has brought a need for data and algorithms for implementing the directional radiation properties of musical instruments, the human voice, and other sound sources [Hiipakka etal. 1997]. In the project described in this paper (dubbed the "NBody Project"), directional impulse responses were collected for six stringed instruments, including three guitars, a mandolin, a violin, and a Hardanger (Norwegian folk) fiddle. Various researchers have investigated the radiation properties of the violin [Weinreich 1997] [Bissinger 1995] [Bissinger & Bailey 1997] [Bailey & Bissinger 1997], but the primary purpose of the NBody project is to obtain a set of useable filters for implementing realistic spatial radiation patterns of a variety of stringed instruments for simulation and performance. The data will be made publicly available in both raw and processed forms, allowing researchers to use it for various purposes, including verification of theories about the radiation properties of instruments. A previously published paper [Cook & Trueman 1998], discusses in depth the data collection methods, the instruments which were investigated, and some preliminary results from analyzing the collected data. This paper will focus on applications which were constructed to use the collected data for synthesis and live performance.

Page  00000002 2.0 INSTRUMENTS STUDIED Three guitars were investigated, a Sam Dunlap 1988 classical guitar, a Sergio Abreu (Brazil, 1997) classical guitar, and a Fender Elite (d'Aquisto 1987) arch-top acoustic/electric jazz guitar. For the arch-top guitar, an extra channel was recorded from the electric pickup, with the tone control set to maximum brightness. Other instruments investigated include a 1987 Kentucky KM1605 F-hole mandolin, a David Folland 1989 violin, and a Hauk Buen (Norway, 1993) hardanger fiddle. 3.0 DATA COLLECTION An icosahedral (20 faces, 12 vertices) grid 4' in diameter was constructed of 1/2" dowel rods, with a microphone mounting flange located at each vertex. Figure 1 shows a photograph of the microphone array, with a researcher outside, a mandolin suspended inside the array, and the twelve microphone positions labeled. The instruments were excited by striking the bridge with a calibrated Modal Shop Model 086C80 miniature force hammer. Sets of "good" impulse responses were selected based on the excitation signal as recorded from the force hammer, and other factors. The multi-channel recordings were transferred digitally to computer for editing analysis. 4.0 SOME OBSERVATIONS FROM THE DATA Figure 2 shows the magnitude spectra of the 12 microphone signals for the mandolin, held by the player, after normalizing the impulse responses by deconvolving the force hammer signal (division in the magnitude spectrum domain). Clear differences can be noted, with the most striking being the deviation from front to back of the player/instrument. Further spectral observations and analyses can be found in [Cook & Trueman 1998]. Principle components analysis was performed on 72 player-held mandolin signals (collected 12 at a time by rotating the instrument a few degrees for each excitation). Analysis was done both on magnitude spectra, and log magnitude spectra. As shown in Figure 3, the results were not as promising as was hoped. For the magnitude spectrum case, ten principal components explained only 84% of the variation, and twenty explained 93%. The performance was slightly better for fewer principal components in the log magnitude spectrum space, but even if magnitude could be reconstructed easily, there is still a prickly question of how to reconstruct phase. Further investigation is required to arrive at suitable interpolation methods that are both efficient and accurate. Mn(idolin normalized (hammer function ivmovcd) mngnitude spcctrn Mild Mi2 _Mic3 FaWing Plaiyerlinsl. Above Plaver Froni iPlayer Lell Up Mic4 fic5 Pver. arlioni Mis6 Pwlay F rnit JkIZ 5.5. 11 J 5.-klTh 1 klTh 5.51Tr. 111CHr suspended inside and researcher holding force hammer, deconvolved) mandolin magnitude spectra for all 12 microphones, player held case......................I l~f lll~~ IIH.............i ~~cpcrit ) Ti Ipp~t~ 3.......................ln ~ye rr uw~ Kih K r ~''''''''''''''''''''...........''..........................................................de on olv d)ma doinma nitdesp cta oral................................

Page  00000003 5. NBODY GUI APPLICATIONS A software workbench as shown in Figure 4 has been created which allows a listening point to be chosen at any point around a virtual instrument. Signal sources for the basic NBody applications include plucked and bowed string physical models, or any external sound source. Physical models of all six instruments, implemented in the Synthesis ToolKit in C++ (Cook 1996), sit beneath the basic Inventor interface. The physical models can also be controlled from MIDI, scorefiles, or GUIs in TCL/TK. The body transfer characteristics can be parametrically edited, adjusting individual filter resonances, and so on. Mandolin Avg. 9% Variation explained Magnitude Spectrumn Spectrum 0 Log vaiu Setm Spectrum Magnitude 60 20 # Principal components S 2.75kHz 5.5kHz 0 5 10 15 20 Hz 2.75klHz 5.5kdHz PrincipalPrincipal component #1 Principal component #4 Im 2.75kr~z 5.5kH 5.5 k1_1Z S.lrz: FIGURE 3. Principal components analysis for mandolin FIGURE 4 Nbody application interface 6.0 MULTI-SPEAKER DISPLAY DEVICES Two multi-speaker display devices (nicknamed "the Boulder" and "the Bomb") were constructed, and are shown in Figures 5 and 6. In these devices, 12 speakers are arranged in an evenly-spaced array, facing outward. The Boulder and Bomb are essentially the dodecahedral dual display devices for the icosahedral microphone data collection array shown in Figure 1. Any sound that was incident on a given microphone in the microphone array can be played back on the matching speaker in the speaker array, resulting in a fairly faithful reconstruction of the original spherical wavefront emitted by the instrument. Fast, multi-channel convolution has been implemented to allow any sound source, such as a solid body electric violin signal, to be filtered by the directional radiation component #4 comimpulse responses measured in the NBody data collection project. 7.0 CONCLUSIONS AND FUTURE WORK The NBody project continues forward, emphasizing further analysis and parameterization of the measured impulse responses, enhancements of the applications, and the construction of additional multispeaker display systems. The Bomb and Boulder have been played in live performances a number of times, and lessons learned from those performances will inform the construction of future multi-speaker performance systems. Collection of additional impulse responses is planned, including double bass and Hz ',r.7ýM 5.5kH-rz 5.5k'~~7lrz S.lHz violincello 3. MoPrincipal component and parametric algorithms for implemmandolin. FIGURE 4. Nbody application, modification, andce. 6.0 MULTI- -SPEA KER DI SPLAY D EVICESE Two multi-speaker display devices (nicknamed "the Boulder" and "the Bomb") were constructed, and are spatialhown in Figuterpolation of the impuln these rdevices, 12 speakers are arrainvestigated, based on anced extendarray, facing outhe workd. Thof [Steiglitz & Lansky 1981] [Karjd Bomb are essentially the dodecahedral dual display devices for the icosahedral microphone data 1996]. collection array shown in Figure 1. Any sound that was incident on a given microphone in the microphone array can be played back on the matching speaker in the speaker array, resulting in a fairly faithful reconstruction of the original spherical wavefront emitted by the instrument. Fast, multi-channel convolution has been implemented to allow any sound source, such as a solid body electric violin signal, to be filtered by the directional radiation impulse responses measured in the NBody data collection project. 7.0 CONCLUSIONS AND FUTURE WORK The NBody project continues forward, emphasizing further analysis and parameterization of the measured impulse responses, enhancements of the applications, and the construction of additional multispeaker display systems. The Bomb and Boulder have been played in live performances a number of times, and lessons learned from those performances will inform the construction of future multi-speaker performance systems. Collection of additional impulse responses is planned, including double bass and

Page  00000004 12 speakerdisplays: FIGURE 5 "The Boulder"FIUE6"hBob FIGURE 6 "The Bomb" 8.0 ACKNOWLEDGEMENTS Thanks to Jim Moses, who provided planning and engineering for the recording sessions, the archtop guitar, and great advice throughout. Thanks to Monica Mugan for the use of the Abreu guitar. Thanks to Rob Jensen for constructing the Inventor model of the guitar, and expertise on graphics applications. 9.0 REFERENCES Bailey, M. and Bissinger, G., 1997, "Measurement of direct radiation fr-om violin excited by force hammer impact at bridge," Proceedings of the Acoustical Society of America Conference, State College, Pennsylvania, Paper 4aMU4, Abstract only. Bissinger, G., 1995, "Some mechanical and acoustical consequences of the violin soundpost," Journal of the Acoustical Society of America, 97:5, 3 154-3164. Bissinger, G. and Bailey, M., 1997, "V-R model predictions of averaged radiation from a violin compared with spatial average of bridge force hammer-excited direct radiation," Proceedings of the Acoustical Society of America Conference, State College, Pennsylvania, Paper 4aMU5, Abstract only. Causs6, R., Bresciani, J., and Warusfel, 0., 1992, "Radiation of musical instruments and control of reproduction with loudspeakers," Proc. of the International Symposium on Musical Acoustics, Tokyo. Computer Music Journal, 1992/3, Various authors, Special Issues on Physical Modeling, 16:4 & 17:1. Cook, P. and Trueman, D., 1998, "A Database of Measured Musical Instrument Body Radiation Impulse Responses, and Computer Applications for Exploring and Utilizing the Measured Filter Functions," Proceedings of the International Symposium on Musical Acoustics, Leavenworth, Washington. Cook, P. 1996, "Synthesis ToolKit in C++, Version 1.0," Course Notes from Creating and 1Manzipulating Sound to Enhanzce Computer Graphics, SIGGRAPH Courses 17 and 18. Hiipakka, J., H~inninen, R., Ilmonen, T., Napari, H. Lokki, T., Savioja, L,. Houpaniemi, J., Karjalainen, M., Tolonen, T., V~ilim~iki, S., and Takala, T, 1997. "Virtual orchestra performance," Visual Proceedings of SIGGRAPH, 81, 1997. Karjalainen, M. and Smith, J., 1996, "Body modeling techniques for string instrument synthesis," Proceedings of the International Computer Music Conference, Hong Kong, pp. 232-239. Steiglitz, K., and Lansky, P., 1981, "Synthesis of timbral families by Warped Linear Prediction," Computer Music Journal, 5:3, 45-49. Weinreich, G., 1997, "Directional tone color," Journal of the Acoustical Society of America, 101:4, 2338-2347.