Page  00000492 APPLICATIONS OF ULTRASONIC SOUND BEAMS IN PERFORMANCE AND SOUND ART Juan Pampin Center for Digital Arts and Experimental Media (DXARTS) University of Washington [email protected] Joel S. Kollin Center for Digital Arts and Experimental Media (DXARTS) University of Washington [email protected] Eunsu Kang Center for Digital Arts and Experimental Media (DXARTS) University of Washington [email protected] ABSTRACT Ultrasound technology has been used in many fields for measurement, imaging, and even therapeutic goals. Recently ultrasound has become available as a medium to create hyper-directional sound beams, enabling the ability to manipulate sound projection in a way analogous to light. This paper presents a brief overview of ultrasound audio technology, and introduces several lines of investigation related to applications of sound beams in performance and sound art currently under development at our center. 1. ULTRASOUND AUDIO 1.1. History While the implementation of ultrasound audio is relatively recent, we can trace some its basic principles of operation back to Helmholtz's experiments with combination tones in the XIX century [1]. Helmholtz introduced a theory describing how combination tones could be produced by non-linearities in the air, showing that a pair of loud tones played by a common acoustic source can generate an extra pair of tones in the air, with frequencies corresponding to the addition and subtraction of the frequencies of the performed tones. While his experiments empirically showed the presence of differential and additional tones in the air, Helmholtz's theory about their production wasn't sustainable in scientific terms, and it took the field of non-linear acoustics many years to find its solution, the first clear explanation of this phenomenon being the one published by Black in 1940 [2]. 1.2. Theory and Implementation Ultrasound audio technology is based on this non-linear property of air, creating interference patterns between tones in the ultrasonic frequency range (beyond 20 KHz) lower difference tones can be generated in the audio range (between 20 Hz and 20 KHz). For instance, two inaudible tones of 40 and 41 KHz played from the same acoustic source could produce an audible difference tone of 1 KHz (and another inaudible tone at 88 KHz). The great advantage of generating audio in such a way is that ultrasound wavelengths are extremely short (about 0.7" and shorter in the air), creating a narrow beam of sound. A device capable of producing such an acoustic output is called a parametric array. Combination tones can be produced by adding two independent signals, or by means of amplitude modulation (AM), generating upper and lower sidebands around a carrier frequency. The latter allows for a simple implementation of the parametric array, using an ultrasonic carrier modulated by audio-range signals. In this case the air acts as the demodulator, converting the amplitude envelope of the ultrasonic carrier into audible output. With the right modulation scheme, a parametric array with a powerful sound beam can be implemented by aggregation of off-the-shelf ultrasonic transducers. It should be noted that the audio output level of a parametric array is proportional to its diameter, this implies a large number or transducers would be required to achieve high SPL output, considering conventional ultrasonic transducers have a diameter between 0.5" to 1.7". 1.3. Commercial Products Currently, there are only two ultrasound audio products available in the market: ATC's "Hyper Sonic Sound" (http://www.atcsd.com/site/) and Holosonics Research Labs' "Audio Spotlight" (http://www.holosonics.com/). These products are not widely commercialized, being mainly marketed for applications in the corporate world. Both products use proprietary DSP for generating the required ultrasound signal from consumer-level audio input and come with a built-in power amplifier 2. After close evaluation of the characteristics of both products, we have decided to get a set of Audio Spotlight units to run some of our early experiments. One of the applications presented in Section 3 has been developed using this technology. 2. ART-DRIVEN RESEARCH The Center for Digital Arts and Experimental Media (DXARTS) is a creative research convergence zone for intrepid artists and scholars who are pioneers of an This technique was first developed in the 1960s in the field of underwater acoustics, finding its first application in sonar systems; it was in this field where the term parametric acoustic array was coined [3] [4]. The first demonstrations of a parametric array in air were carried in the mid-1970s [5]. For further discussion about this topic see [6] pages 6-8. 2 The product by Holosonics Research Labs has the actual transducer separated from the DSP/amplifier box, it is possible to order transducers of different diameters. 492

Page  00000493 unfolding new era in the arts. One of our main goals is to develop new lines of artistic inquiry intersecting with novel emerging technologies. One of these fields of research has been the use of ultrasonic sound beams in the domains of performance and sound art, exploring the possibilities of available commercial products and custom-designed devices. 2.1. Open Ultrasonic System While our explorations in this field started with a product available in the market obtaining fairly good results, it was clear from early on in our research that a more flexible and open system would be required for further investigation. One of the main limitations of commercially available devices is that they cannot be used as ultrasonic transducers, providing a limited interface that only allows for audio-range input. For the aforementioned reasons, we have decided to develop our own customized parametric array using offthe-shelf components. Our system has completely open components that can be re-arranged in many different configurations. The DSP components are also modular and have been implemented in the Supercollider (SC3) real-time sound processing language (www.audiosynth.com/). Figure 1 shows the main components of the system, it can take professional level audio input (or synthesized sound); the input signal is conditioned for modulation by the Spectral Processing (SP) module; a Single Side Band (SSB) modulator produces the ultrasonic output signal, it can generate carriers at any frequency to match the resonance point of a variety of transducers. Finally, the ultrasonic signal is amplified by a Broadband Linear Amplifier (BLA) that drives a custom ultrasonic Transducer Array (TA). Being modular, the system allows for partial use of its components, for instance, one could use it to generate ultrasonic signals only, without using the SP and SSB modules (something needed for some of our experiments). Audio SP SSB BLA TA Input Carrier Figure 1. Open Ultrasonic System diagram, the doted square encapsulates the software components of the system. SP: Spectral Processor; SSB: Single Side Band modulator; BLA: Broadband Linear Amplifier; TA: custom ultrasonic Transducer Array. As it will be seen in the following section, such an open system allows for many different transducer It should be also noted that these devices use proprietary DSP which cannot be bypassed by the user, making it impossible to experiment with the different components of the system. configurations, portable and wearable devices, massive arrangements made out of smaller arrays, etc. Also, the use of professional audio equipment the makes it possible to work with high quality audio up to sample rates of 192 KHz, which allows for wide ultrasonic bandwidth 2 Finally, It should be noted that moving most of the complexity to the software side allowed us to use a very simple modular scheme on the hardware side, giving lots of flexibility for experimentation, and providing the capability of interfacing with many other processing tools in Supercollider. It is our intention to make the software components of our system open-source in the near future. 2.2. Custom Ultrasonic Transducer Array One key component of the system is our custom ultrasonic transducer array. After experimenting with both piezoelectric and electrostatic transducers we have decided to build our prototype array with piezoelectric components. There are two main reasons for our choice: first, piezoelectric transducers are cheaper and easier to find in electronic stores; second, they are smaller and don't require high DC voltage polarization as the electrostatic ones do, this being very important for our wearable sound projects (see Section 3.3). Figure 2 shows a picture of one of our array prototypes, it uses 7 piezoelectric transducers in a hexagonal topology, transducers are mounted on a Plexiglas substrate. These devices can be easily aggregated to create larger arrays for higher SPL output. Figure 2. Custom ultrasonic transducer array using piezoelectric transducers. These devices can be easily aggregated to create larger arrays for higher SPL output. 3. PERFORMANCE AND SOUND ART APPLICATIONS This section briefly describes some of our current research projects using ultrasonic sound beams. 2Most of our research has been done using a RME Fireface 800 audio interface (http://www.rme-audio.com) 493

Page  00000494 3.1. Ultrasonic Waveguides Acoustic waveguides can be implemented taking advantage of the focused nature of ultrasound audio and of its reduced acoustic losses (compared to those of normal speakers). Our first experiments have been done using a hyper-directional speaker (Audio Spotlight) and hyper-cardioid microphone facing each other at a distance to create focused acoustic delay. By adding a feedback loop connecting the input of the microphone back to the transducer output, a basic acoustic system was devised having similar properties to a digital waveguide [8], but using the air both as a demodulator and a delay line. While such a system could allow for the creation of more complex physical modelling topologies, we have decided to follow a different path, consistent with our concern for creating new bodily interactive interfaces and inspired by our collaboration with dancers. In the following iteration of our system development, the feedback loop was used to create constant audible feedback which was controlled by adaptive filtering, keeping the output level always within a reasonable dynamic range. As the system was calibrated for being always at the edge of feedback, minimal interaction of the dancer's body with the sound beam would interfere with the acoustic waveguide, allowing for a wide repertoire of bodily interactions with the system which result in a wide range of complex acoustic output. When the dancer's body completely blocks the waveguide the feedback loop is broken, bringing the system down to silence. Based on this principle, the system can detect when a beam has been broken not only completely but also partially (doing sound edge detection) and react accordingly. Figure 3 shows a representation of this system used in combination with an ambisonics 3D sound system, many parameters of the 3D sound projection (such as rotation, image zooming, etc.) are controlled by the dancer's interaction with the waveguides. The system uses 2 waveguides converging into one point in space and it is calibrated to allow the dancer's body to block both waveguides if located at the convergence area. We have experimented with similar systems using up to four waveguides, doing edge detection on each of the beams to triangulate the location of the interacting body. In this case, a scattering matrix has been used to tune how much energy is re-distributed to each of the acoustic paths, allowing for further calibration in terms of feedback stability. This device can be a useful to adapt the system to the body size of the dancer in a flexible way. One of the most interesting properties of ultrasonic sound beams is that they can be reflected off any surface creating the illusion of virtual sources. In this project we have been exploring this property by adding reflecting panels behind the receivers to reflect the sound beams. Orienting the panels to deflect sound in other directions can extend the acoustics of the system beyond the limits defined by the waveguides, presenting some interesting possibilities for acoustic interaction with a surrounding audience. 3.2. Acoustic Mirror The main goal of this project is to produce the dissolution of static acoustic space, turning it into a bodily negotiated acoustic experience. People interacting with the Acoustic Mirror can dynamically change a sound field using different parts of their body. A massive aggregate of customized transducer arrays is used in this installation, each component of it being an output and input channel of the system. Figure 3 illustrates the installation of the system. Figure 3 Acoustic Mirror a massive aggregation of customized transducer arrays creates a dynamic sound pressure field reactive to the participant's body. As mentioned previously, an important property of ultrasound is its ability to reflect from objects. This property has been exploited industrially to create distance measuring devices and proximity sensors. The Acoustic Mirror combines this property with the hyperdirectivity of sound beams both to project sound to focalized points of the human body and to measure their distance from the device. Distance measurements are used to change parameters of a dynamic pressure field created by a multitude of ultrasonic transducers. Acoustically, the interaction between the human body Figure 3. Dance performance installation using a pair of ultrasonic waveguides in combination with an ambisonics 3D sound system. 494

Page  00000495 and the surface device results in an interference pattern between the generated sound field and the reflections off the body. Minimal changes in the body position can be detected in a grid of points which is mapped to the matrix of transducers in a variety of ways. For instance, a direct acoustic interaction could make a linear mapping of loudness with distance, sending softer sounds to points of the body closer to the surface and louder sounds to more distant ones. 3.3. Wearable Sound One area of our interest is that of wearable devices which could enhance performance practice, in particular in the domains of dance and music. An advantage of our custom ultrasonic transducer array is that it doesn't require much power to generate reasonable SPL output. This is due to the particular quality of the piezoelectric transducers we use which are mostly voltage driven, with minimal consumption of current. This means that transducers can be easily attached to gloves and cloths without risk, and that portable DC power supplies can be used. Figure 4 shows our Sound Glove, which uses an array of 16 transducers. Transducers can work all together as a group, creating a larger parametric array, or can be split to have some of them work as receivers as well. They are connected to a portable device that can send and receive sound and control data. With the Sound Glove, the performer can steer sound with her hand and interact with other performers wearing similar devices. In one of our experiments testing sound sending and receiving capabilities of the glove, performers could create ultrasonic waveguides across the stage by pointing at each other with their hands, changing the acoustic result of their interaction with the beam incidence angles and the sender-receiver distance. While the Sound Glove has been our first wearable device to experiment with, we have plans to extend this concept to devices which can be worn all over the body, allowing for many more performance interactions. Also, some other materials, such as piezoelectric e-textiles, are being investigated [9]. 4. CONCLUSIONS AND FUTURE DIRECTIONS Several novel applications of the use of ultrasonic sound beams have been presented, as well an open system used for their development. Many other applications of this system are currently under development, always keeping focus on sound/body interaction. We are also exploring the use of ultrasonic sound beams in combination with other sound projection systems such as ambisonics, and wavefield synthesis. Some future research directions include the extension of the system to the fields of mechatronics and robotics to create autonomous sound beam steering and focusing devices. 5. REFERENCES [1] H. L. F. Helmholtz, On The Sensations of Tone. Dover Publications, New York, 1954. [2] L. J. Black, "A Physical Analysis of Distortion Produced by the Non-Linearity of the Medium", The Journal of the Acoustical Society ofAmerica, 12:266, 1940. [3] P. Westervelt, "Parametric Acoustic Array", The Journal of the Acoustical Society of America, 35 (4):535-537, 1963. [4] H.O. Berktay, " Possible Exploitation of NonLinear Acoustics in Underwater Transmitting Applications", Journal of Sound and Vibration, 2 (4):435-461, 1965. [5] M. B. Bennett and D. T. Blackstock, "Parametric Acoustic Array", The Journal of the Acoustical Society of America, 57 (3):562 -568, 1975. [6] J. J. Croft and J. O. Norris, "Theory, History, and the Advancement of Parametric Loudspeaker. White paper of American Technology Corporation", http://atcsd. com/pdf/HSSWHTPAPERRevE.pdf, 2001. [7] F. Joseph Pompei, "The Use of Airborne Ultrasonics for Generating Audible Sound Beams", Journal of the Audio Engineering. Society, vol. 47, pp. 726-731, 1999. [8] J. O. Smith, "Physical modeling using digital waveguides," Computer Music Journal, vol. 16, pp. 74-91, Winter 1992. [9] J. Edmison, M. Jones, Z. Nakad, and T. Martin, "Using Piezoelectric Materials for Wearable Electronic Textiles," Proceedings of the Sixth International Symposium on Wearable Computers, 2002. Figure 3. Sound Glove: transducers can work all together as a group creating a larger parametric array or can be split into a sender/receiver configuration. 495