Page  00000270 The Bluetooth Radio Ball Interface (BRBI): A Wireless Interface for Music/Sound Control And Motion SoniU cation Woon Seung Yeo CCRMA, Department of Music, Stanford University woony@ccrma.stanford.edu Abstract The Bluetooth Radio Ball Interface (BRBI) is a wireless interface for motion tracking and sonil cation. The device is embedded in a palm sized foam ball. A 3-dimensional sensor with a Bluetooth module seated in the center of the ball transmits acceleration/tilt measurement data to a computer Data is then converted into Open Sound Control messages for use with other applications. This paper presents the design concept and implementation of BRBL Details of its hardware and data handling software are also discussed. Applications include gesture control in music performance as well as sonil cation of athletics. 1 Introduction A ball is among the simplest, and most ubiquitous and familiar recreational devices. It has been used for millennia and across virtually all cultures in a wide variety of games and sports. However, with the exception of some idiophones, the use of handheld balls for creating music is relatively rare. More recently, studies in motion tracked controllers for creating or shaping music have been the subject of increasing interest. Wireless technology has eliminated one of the main barriers in developing useful hand held gesture controllers. In this paper, we describe the Bluetooth Radio Ball Interface (BRBI, pronounced "Barbie") - a novel wireless ball interface for sound control and motion sonil cation. Inside a soft, easily grasped palm-size foam ball is a circuit board containing a 3-dimensional accelerometer/tilt sensor and a Bluetooth transmitter to send out measurement data to a Macintosh computer. Data is processed by the WiTilt to OSC (W20) - a program to decode binary inputs and re-format them as Open Sound Control (OSC) messages, and then transmitted over a UDP connection to any OSC-compatible application. 1.1 Review of Comparable Works Sound controlled by spatial hand coordinates has been of interest since the theremin. With the development of motionand gesture-tracking systems, numerous composers have explored the movement of a dancer as a means of creating, triggering, or processing musical sound. In addition, recent work in sonil cation of motion has been inspired by the expressive and gestural control produced in the physical-auditory feedback loop in traditional musical instrument performance including string instrument bowing and conducting. In this paper we pay special attention to spherical ball shaped interfaces. Examples include the StressBall (Verplank 2001) by Heidema et. al., a rubber ball that contains an accelerometer inside and force sensors underneath its surface, thereby taking squeezing and shaking gestures as its control inputs. Sensor data from the StressBall is transmitted to a computer over a wired connection. In (Hermann, Krause, and Ritter 2002), Hermann et. al. presented the Audio-Haptic Ball. This device consists of various sensors (i.e., force sensor, accelerometer, piezo sensor) and buttons/switches as well to provide higher dimension of control. In addition, it contains an actuator to generate haptic feedback. All these parts are integrated in a ball-shaped housing which 0 ts into a human hand. Nevertheless, like StressBall, it is wired to a data processor, and therefore fails to give the freedom of motion. In contrast, Muggle (Verplank 2002), is an RF-based wireless controller incorporating accelerometers and LEDs (for visual feedback) housed in a translucent plastic ball. Being wireless, Muggle offers much more freedom of control. However, its fragile translucent housing prevents it from being handleable as an ordinary ball for sonil cation of various ball movements. In addition, an extra RF receiver is required for a computer to communicate with it. Prior work on the use of Bluetooth in wireless control includes Soundstone (Bowen 2005) which senses 3-dimensional acceleration and provides both visual and haptic feedback. 270

Page  00000271 Figure 2: Wireless Accelerometer/Tilt Controller (version 2.0) by Sparkfun Electronics. Figure 1: System diagram for BRBI. 1.2 Features of BRBI BRBI has a number of advantages over the aforementioned examples. These include: * Connection with data receiver is wireless based on Bluetooth: in addition to providing tether-free freedom of motion offered by wireless control, Bluetooth is compatible with most computers, easy to conf gure, and does not require any extra sensing device. * Being housed in a small soft ball which is neither fragile nor heavy, BRBI is easy to grab and control. It is also quite robust to endure shocks caused by common ball-handling motions, and shows solid performance when transmitting sensor data. Therefore, BRBI can serve as a sound/music control interface and as a device for sonifying various ball-handling gestures and ball movements as well. It should be emphasized that BRBI was designed for not only sound control with "active" ball-handling gestures, but also "passive" sonil cation of ball movements in mind. 2 System Overview Figure 1 illustrates the data 0 ow for BRBI and its supporting system. 3-dimensional tilt/acceleration data measured by BRBI is received by the data processor (W20). This processor then decodes and transmits the data as OSC messages to any compatible application: connection can be made within the same machine, or to a different machine on either LAN or WAN. Figure 3: Sensor conl guration using terminal. 2.1 Sensor To measure ball movements, a Wireless Accelerometer/Tilt Controller (version 2) (Sparkfun Electronics 2005) (0 gure 2) is used. This device contains an MMA7260Q - a 3-axis, lowg accelerometer (Freescale Semiconductor ), and a class I Bluetooth module, on a single PCB board. In addition, it offers a built-in command line conf guration utility that allows easy setup of sensor parameters, as shown in 0 gure 3. 2.2 Ball The sensor, together with batteries, is housed in a foam ball which is about 4 inches in diameter: components are shown in H gure 4. Based on the type of movement to be soniH ed, balls with different size and/or degree of H rmness might be desirable. 271

Page  00000272 Figure 4: Components of the BRBI. A WiTilt sensor and a battery pack are inserted in the center of the foam ball. 2.3 Data Processor: W20 W20 is a Mac OS X-based data processor, written by the author of this paper, for use with the Wireless Accelerometer/Tilt Controller. More speciD cally, W20 consists of a Bluetooth data server, a binary-to-OSC data converter, and an OSC client. Binary data transmitted from the sensor is processed by W20 to provide a) raw measurement values from 10-bit ADC (0 1023), b) tilt angles (-90-90 degrees), and c) ratio of acceleration magnitude to the acceleration of gravity (per cent), as OSC messages with customizable target information and OSC address. Figure 5 shows the user interface of W20, in which OSC output options can be determined. Messages from W20 can be utilized by any OSC supporting applications, such as Max/MSP and Pd. An example of a Max/MSP patch that receives sensor data from W20 is depicted in 6. W20 is compatible not only with BRBI but also with any other interface incorporating the same Wireless Accelerometer/Tilt Controller. Moreover, the program can be easily customized to work with other Bluetooth devices. Detailed information on W20 can be found in (Yeo 2006). 3 Gesture Mappings and Motion Soni0 cation Any movement of BRBI can be considered as combinations of linear and angular components. Based on the output of W20, although not perfectly distinguishable from each other, linear and angular motions can be characterized by relatively bigger changes in overall acceleration magnitude and Figure 5: User interface of W2 0. W. imt Raw Data: Index # / X / Y/ Z Figure 6: A Max/MSP patch to receive sensor measurement data from W20 as OSC messages. 272

Page  00000273 tilt-angle values, respectively. 4Conclusion 3.1 Gesture Mappings for Sound/Music Control Different ball-handling gestures show different patterns of variation in measurement data. Examples include: * Rotation. By slowly rotating BRBI, it is possible to control three tilt angles with high precision while keeping the acceleration magnitude very small. I used this to control the parameters of an FM instrument, which proved to be an excellent example of timbre control by 0 ne hand motions. * Spin. Like any ball, BRBI can be spun. Compared to rotation, this makes the tilt angle(s) change more rapidly, and the acceleration magnitude somewhat high. * Shake. This involves fast, repetitive gestures that introduce frequent high jumps in acceleration with less tilt angle variations. As an emulation of real percussion instrument, acceleration magnitude could be mapped to the "shake energy" parameter of a Physically Informed Stochastic Event Model (PhISEM), such as the Shakers class (Cook and Scavone 2005) of the Synthesis Tool Kit (STK), to synthesize a shaker sound. * Toss/throw. These are mostly linear motions, resulting in changes of acceleration both at the beginning and at the end. Angular motion, however, can be introduced together to make them more complex. Since BRBI is a sphere, there's no absolute external reference in terms of direction, whereas the sensor inside the ball has its own coordinate system. Therefore, depending on the mapping and the initial tilt angles of the sensor, similar movements of BRBI could produce quite different results. Although BRBI's performance is robust, it currently is prone to instability when exposed to excessive shocks: for example, the sensor "freezes" in case of collision against a hard object. In fact, this limits its use for acquiring bouncing gestures. 3.2 SoniU cation of Ball Movements BRBI can also be used for sonil cations of ball movements in a passive way: instead of being held and controlled by hand, it could be put into an environment in which only indirect access (or no access) to the ball is possible (i.e., a big container, or the cargo space of a moving van) to convert its movements into sound. I have proposed BRBI as a wireless ball interface. Together with W20, it provides a convenient and 0 exible method for sound control by gestures and sonil cation of ball movements. Future works will include: * Enhanced physical implementation. This means not only using better material for the housing, but also adopting a new sensor which is more robust and precise. * Software upgrade. W20 will be able to 0 Iter sensor data, and support more 0 exible network conf gurations (i.e., multiple target, TCP connection, etc.) * Various gesture and sonil cation mappings. Especially, more reliable physical implementation of BRBI will allow it to be used for sonif cation of ball-sports activities. References Bowen, A. (2005, June). Soundstone: A 3-d wireless music controller. In Proceedings of New Interfaces for Musical Expression, Vancouver, Canada. Cook, P. and G. Scavone (2005). Shakers class reference in the synthesis toolkit in C++ (STK). http://ccrma.stanford. edu/software/stk/classShake rs. html. Freescale Semiconductor. MMA7260Q product summary page. http://www.freescale.com/webapp/sps/site/prod-summary.jsp? code =MMA7260Q&nodeld= 01126911184209. Hermann, T., J. Krause, and H. Ritter (2002, July). Real-time control of sonil cation models with a haptic interface. In Proceedings of International Conference on Auditory Display, Kyoto, Japan. Sparkfun Electronics (2005). Wireless accelerometer / tilt controller version 2.0. http://www.sparkfun.com. Verplank, B. (2001). Music 250a 0 nal projects. http://ccrma.stanford.edul--verplank/25Oa/2OO.html. Verplank, B. (2002). Music 250a 0 nal projects. http://ccrma.stanford.edulr-verplank/25Oa/2OO2/2OO2.html. Yeo, W. S. (2006). W20 - Witilt to OSC. http://ccrma.stanford.edu/r-woony/software/w2o/. 273