Page  00000092 A Wireless, Network-based Biosensor Interface for Music Gilles Dubost email: Atau Tanaka Sony Computer Science Laboratories Abstract We present a new hardware system using biosignals for musical applications. Architecture and design decisions made in its realization are described as well as initial experiments that have been conducted. The device is modular, wireless, and network based, permitting a wide range of applications. It is a reasonable cost, open platform for research in performance, musical gesture recognition, and wearable music systems. Keywords Wireless, wearable, eletrcomyogram, sensor instrument, gesture recognition 1 Introduction In this paper we present a biosignal based musical interface. The interface is based on active EMG electrodes, whose signal is transmitted over a small area digital wireless network to its basestation. The basestation then translates the signal into a variety of formats, including MIDI, RS232, and Ethernet to transmission to a synthesizer, host computer, or network of recipients. The interface is intended for musical applications, including use as a performance instrument, as a gesture analysis device, and wearable musical computing. We describe the design of the interface, and discuss issues confronted in its realization. 2 Background Use of biosignals for music has precedent dating to the 60's [Rosenboom]. Biosensors as human interface devices came into their own in the early 90's [Knapp-Lusted]. While the first efforts to harness biosignals were in the analog domain, this second wave of biosignal use was digital. This had the benefit of allowing a shift of focus from biofeedback to bio-control. Musical practice based on this techniques has been reported [Tanaka1993, Tanaka2000], and has been expanded to go beyond a unidirectional control paradigm [Knapp-Tanaka]. While the first evolution in the field were enabled by the arrival of specialized digital signal processing hardware, this paper presents a further evolution of treating biosignals now made possible by recent developments in general purpose microprocessors and widely accessible network infrastructures. These changes allow the development of a system that is at once more portable (for concert use) and more open (for gesture analysis research). The portable and network orientation of the device allow investigation of new areas in wearable musical instruments [Nishimoto] Prior systems can be divided into two categories: special purpose, and general data acquisition. General purpose data acquisition systems have been used for musical applications with biosignals [Marrin]. However, as these systems are designed to accept any kind of input signal, they may have features not needed at the same time lack special needs particular to biosensors. Furthermore, these data acquisition systems tend to be ungainly for live concert use. Specialized hardware is able to address the specificities of the biosignal: a relatlively weak voltage, with with high signal/noise ratio and nontrivial grounding issues between body and circuit [Cram]. However, until now, the signal processing tasks necessary to extract a useful information from the biosignal has required specialized analog circuitry and dedicated digital signal processors (DSP). This has resulted in systems that were highly capable of certain defined tasks, but not easily modified. 3 Motivations Recent developments have permitted us to merge the advantages of an open generalized system with those of bio-signal specific hardware. Analog amplification and filtering circuitry has been optimized and miniaturized to allow the development of active dry electrode systems. Increased power in recent microprocessors allows the use of general purpose processors for basic signal treatment. Such a system can be quickly reconfigured in software written in C to allow different application types. 92

Page  00000093 In the present project, we exploit these recent devepments to create a high performance, open system at reasonable cost. We define a device architecture that leverages these advantages to create a portable re-configurable system. The system needs to be compact yet robust and be wireless, for performance applications, yet provide high quality signal acquisition for data analysis and gesture recognition studies. 4 Biosignals The electromyogram (EMG) signal is an electrical voltage generated by the neural activity commanding muscle activity. Surface electrodes pick up this neural activity by making electrical contact through the skin. Typically increased muscle tension results in higher energy in the biosignal. The EMG signal is in the millivolt range and has as frequency range from DC to 2KHz, where the signal is down 50dB [Putnam]. There is a high variance of biosignal from person to person and in one person from muscle group to muscle group. In addition, the human body is a floating conductive electrical system. As soon as it is connected to a fixed system, it seeks ground through the fixed system, and serves as an antenna for electrical artifacts. Any EMG measurement system must account for these problems. In addition a musical system must display consistent behavior across a wide range of different environmental contexts and conditions. 1In.. i ---- - microprocessor 1 (beltpack) - UART/Bluetooth circuit - Microprocessor 2 (basestation) - Host system or network This section describes each of these modules in detail. ITo Py To To NDI PC S M Local,rea ETran Jfi T *1 Tfnmibstm r 12 tAntu n 80 [ l l l, i,, - 60 0 CL a4 fig. 2. System architecture Active Electrodes The electrodes are dry active surface electromyogram electrodes. They are differential electrical circuits: there are three points of contact with the body, 1. A ground reference, then two poles at a close spacing. The biosignal is defined to be the difference in electrical potential across the two poles with respect to the reference. As the differential signal is symmetric (like a microphone cable), it exhibits common mode rejection and is less susceptible to interference. Previous systems have been passive wet-gel electrodes. The gel was required to assure electrical conductivity with the body. The passive nature of previous electrodes meant that the unamplified biosignal traveled up to 5 meters before being treated. With the active electrode, amplification and filtering circuitry is on the electrode itself, minimizing the distance traveled by the signal, and therefore minimizing the accumulation of noise artifacts. This makes possible to forego the conductive gel, and implement a dry contact with the body. There are various manufacturers of dry electrode systems on the market, generally requiring 5V power for 1000x amplification and filtering at 4,000Hz. This is an off the shelf component that we have integrated into our system. Modules The system is divided into two units, a battery operated wireless beltpack and a basestation. The electrodes enter the beltpack, are digitized and pretreated, and sent out a Bluetooth connection to the 20 ilkJy jA-il 0 100 200 300 400 50 Frequency (Hz) Figure 1. Frequency spectrum of the EMG signal detected from the Tibialis Anterior muscle during a constant force isometric contraction at 50% of voluntary maximum. 0 5 Architecture The architecture of the system can be divided into several distinct modules: - active electrodes - analog stage 93

Page  00000094 basestation. The basestation further treats the signal and sends the data to a host computer, synthesizer, or network. Unit 1 Analog stage The analog stage can be divided into three subsections - multiplexer, programmable amplifier, and analog/digital convertor (ADC). The multiplexer is a digitally controlled 8 channel analog signal multiplexer. This makes possible the use of up to 8 electrodes simultaneously. The multiplexer is controlled by the microprocessor, sequentially switching inputs in a synchronized fashion. The output of the multiplexer, representing the signal of between 1 and 8 electrodes, enters the programmable amplifier. The amplifier is a 12 bit digital/analog convertor (DAC) and analog multiplier. A control signal from the microprocessor becomes the analog gain. This complements the fixed amplification onboard the electrode with a programmable gain stage. Finally, the multiplexed, amplified signal enters a 16 bit ADC. The total sampling rate of the ADC is 40KHz, giving an effective sampling rate per electrode of 5KHz when using eight electrodes. (Aliasing????) The synchronous signal then enters the microprocessor for signal treatment and routing. CPU A Two microprocessors are used in the system at different stages, both of them 8 bit microprocessors running at 22.1MHz. These processors are programmed in C, with the software stored in onboard Flash-ROM. The first processor is used at the output of the analog stage as a signal pretreatment processor. This processor controls the analog components - the multiplexer, amplifier, and ADC. It works in a synchronous fashion with the muliplexer to set the number of inputs and sequentially sample the currently active input. A 12 bit word from the processor serves as gain setting for the programmable amplifier. Finally the signal is preconditioned and prepared for serial transmission to the basestation. Bluetooth The output of the first processor enters a Universal Aynchronous Receive/Transmitter (UART) compoment. The 16 bit electrode data is nibblized into 8 bit bytes and is transmitted at 115.2kbps. A serial/Bluetooth component follows, creating the wireless link to the basestation. The Bluetooth standard allows for maximum datarate of 723kbps and range of 10meters. Unit 2 CPU B A second Rabbit processor receives the biosignal over the Bluetooth link. It can perform data processing on the signal (such as RMS power calculations, filtering). It's primary role is to format the signal for various outputs and then route the signals to these outputs. Output The final output stage consists of multiple parallel outputs in standard formats: RS232, MIDI, and Ethernet. The RS232 port (UART) provides direct serial connections to a host computer. The MIDI port allows direct connection to commercial synthesizers. The Ethernet port allows the biosignal to be published to a local area network, to be exploited by any other device on the network. The basestation has an IP address on the network and sends data over various Ethernet protocols, either to a specific destination IP address, or in broadcast mode to all nodes on a network. fig. 3. The completed system Discussion The system was conceived as an open architecture with modules that can be used in a variety of different configurations. Unit 1 can be used alone in a direct serial link to a host computer, or over Bluetooth to a host. When MIDI or Ethernet are needed, Unit 1 and Unit 2 are used together. They can be connected directly via RS232. In a performance setting, the Bluetooth link between the two units allows the performer to be wireless onstage with the back end offstage connected to the basestation. In a distributed system, Ethernet transmission can be used to deliver the same biosignal to multiple computational or analysis nodes. As both processors in the system are identical and programmable in C, a coherent development environment is easily established. The C code is compiled on an outboard computer, and downloaded to the nonvolatile memory of the microprocessors. A decision was made at the conception of the device to stay with simple microprocessors. Although this is 94

Page  00000095 has less calculation power than a dedicated DSP, we felt that the cost benefits and programming simplicity justified the decision. The use of simple micoprocessors led us to conceive of a distributed processing architecture. There are two identical microprocessors in the system - one in the mobile beltpack and the other in the basestation. Each processor can be used for tasks specific to the role of the unit in which it is found - for the beltpack this means data pre-processing. For the basestation this means data formatting for the various output ports. The data is delivered to the host system in a format where further signal processing can be carried out at the host or network level, completing the distributed processing architecture. The choice of microprocessor does not, however, preclude realtime operation, something typically associated with DSP systems. The microprocessors we selected have multitasking libraries, and can handle tasks at interrupt level, allowing a real-time scheduler to be written. The data formats and data rates were chosen to at once fit within standards norms yet still allow real time transmission of data from multiple electrodes. Computer based RS-232 ports are typically limited by the operating system to 115.2kbps Post-processing on the basestation allows simultaneous transmission of Runtime parameter modifications (of multiplexer or amplifier, MIDI or destination IP address) can be user configured by commands over the RS232, MIDI, or Ethernet ports. This makes possible parallel multiple musical usage of a single control signal. Conversely, the inputs are generalized, allowing other sensor types to be utilized alongside the EMG signal. This opens possibilities for exploiting the biosignal in contexts of multimodal interation [Knapp-Tanaka]. The wireless and network orientation of the device bring a new context to sensor-based musical instruments. A user could change spaces while wearing the device and have his signal be picked up by a new local area network. The Ethernet output of the system makes the biosignal available to whole networks, to be used by various host machines or to share data with similar peers on the same network. On the user side, EMG is a signal that is very close to the body, as it is electricity of the body itself. There are issues in user-user variation, as well as context and environment specific variation, that we address in the architecture of our system. The software controlled amplifier allows the system to adapt to these different situations - whether they be at the muscle level, user level, or context level. Different muscle groups have widely ranging output amplitudes. At the other extreme, the same person might send quite different biosignals depending on whether they are in a relaxed situation or in a stressful situation (such as stage performance). There are safety considerations not to be neglected. As the user's body becomes part of the electrical system, proper electrical isolation is critical. For this, it is not envisioned to use Unit 1 in direct serial connection with a host computer. The Bluetooth subsystem allows not only the performance benefits of wireless, but the safety benefits of electrical isolation of the user from the rest of the system - host computer or network. The system has been successfully interfaced with host side signal processing environments such as Matlab, and music performance software such as MaxMSP. References Cram JR, Clinical EMG for Surface Recordings: Volume 1, J&J Engineering, Poulsbo, WA, 1986. R. Benjamin Knapp and Hugh S. Lusted, "A Bioelectric Controller for Computer Music Applications," Computer Music Journal, MIT Press, Vol. 14, No. 1, pp. 42-47, Spring 1990. Knapp, R.B. and Tanaka, A. Multimodal Interaction in Music Using the Electromyogram and Relative Position Sensing. In press. Marrin, T. and Picard, R. 1998. "The Couductor's Jacket: a Device for Recording Expressive Musical Gestures." In Proceesings of the ICMC, San Francisco: ICMA, pp. 215-219. Nishimoto, K., et al. 2001. Networked Wearable Musical Instruments Will Bring a New Musical Culture. In Proceedings of the 5th International Symposium on Wearable Computers. Los Alamitos, California: IEEE Computer Society. William L. Putnam and R. Benjamin Knapp, "Real-Time Computer Control Using Pattern Recognition of the Electromyogram," Proc. of the IEEE International Conf on Biomedical Eng., San Diego, CA, pp. 1236 -1237, October 27-29, 1993. Rosenboom, D. 1990. Extended Musical Interface Human Nervous System: Assessment and Prospectus Cambridge, Massachusetts: MIT Press. Tanaka, A. 1993. "Musical Technical Issues in Using Interactive Instrument Technology." In Proceedings of the ICMC, San Francisco: ICMA, pp. 124-126. Tanaka, Atau "Musical Performance Practice on Sensorbased Instruments," In M. Wanderley and M. Battier (eds.) Trends in Gestural Control ofMusic. IRCAM, p. 389-405, 2000. 95