Page  00000001 JACK-ON: A LOW-COST WIRELESS SENSOR INTERFACE Dr. Barry Moon Sacha Atkinson Bath Spa University College School of Music and the Performing Arts ABSTRACT JACK-ON is a wireless sensor interface using sine tone oscillators. Sensor resistances alter oscillator frequencies or amplitudes, and the audio output of oscillators is sent through a generic wireless audio transceiver. The wireless audio receiver is plugged into the computer's sound card, where oscillator frequencies and amplitudes are analyzedXi in software. Thus,tanalog to digital conversion is achieved through the sound card in the computer receiving the audio signal. It is hoped that this method provides a cheap and robust wireless sensor transmission solution. In this paper, we look at some design issues and early results gathered from the JACK-ON system. 1. CURRENT OPTIONS FOR WIRELESS SENSOR INTERFACING Figure 1. SM2 Circuit Capturing large-scale gestures can be inexpensively achieved through camera based motion-tracking systems. Small-scale gestures can also be captured using these systems if the camera is focused on a small area, as in Barry Moon's 'Postcard' (2003) for chair with live audio/video processing [1]. However, this approach implies that the performance space is small and focused. The most popular method for capturing small-scale gestural information involves using physical sensors manipulated by the performer. Capturing gestures in a performance such as dance, where the overriding concern is freedom of movement, a wireless sensor interface is needed. This generally involves digitizing voltages (sensor resistances) into MIDI, or some other proprietary format, before transmitting them wirelessly. There are currently very few commercially available sensor interfaces dedicated to wireless communication. The Kroonde Gamma [2] is one very expensive example, costing $1495. There is also the option of using one of several interfaces that convert sensor voltages to MIDI, such as the I-Cube [3], and sending MIDI data wirelessly using something like the Midistream. Other, noncommercial, interfaces have been developed by small research labs, or individuals, for use in their own creative work, such as the Eos Pods [4] developed at NYU. Sacha Atkinson developed his SM2 by hacking a wireless games controller. This is a very cheap solution, but requires careful conditioning of sensor voltages, and is susceptible to changes in response due to battery charge. Another downside to hacking games controllers is that they are in a constant cycle of obsolescence, meaning the one you hack today may not be available to hack tomorrow. The SM2 circuit is shown in Figure 1. Hacking games controllers has interested many other researchers in the field, evidenced by the Oik project at Steim [5]. 2. JACK-ON 2.1. Requirements Questions we considered in the design of JACK-ON included cost, number of sensor inputs, resolution, sensor sampling rate, and latency. When considering the cost of JACK-ON, we looked at the price of available wireless audio systems. While there are low quality systems available new for around $50, there is a huge second-hand market, where high specification systems can be bought for less than $100. In JACK-ON, each oscillator, complete with housing, only costs around $10 in parts, while the labor costs are potentially very high. Mass production would significantly reduce all costs. The question of the number of sensor inputs to allow for JACK-ON comes down to the particular application. We considered 8 sensor inputs to be sufficient for most performance applications. We found in our tests that the number of sensor inputs impacts upon resolution. The resolution of JACK-ON is dictated by the stability of the oscillator circuit and the accuracy of the software analysis. Since sensor information remains in analog form until being converted by the ADC of the computer, the sample-rate is dictated by the speed of the analysis. All analysis is done by FFT, and therefore, the rate will be dictated by the FFT hop-size. A hop-size of 64 samples provides a very satisfactory sampling period of 1.45 ms. As there is negligible latency inherent in the analog system, latency is governed by the speed of the computer's ADC, and the size of the FFT window.

Page  00000002 2.2. Choosing an Oscillator Initially, we experimented with a square wave oscillator This was unsatisfactory, as our goal was to feed multiple oscillators through one audio channel, and separate them out using audio analysis in the computer. In the case of a square wave, all oscillators would have to lie within one octave to avoid interference from harmonics. For JACK-ON, it was therefore crucial that the oscillator produce a signal as close to a sine wave as possible. Size was a key issue in the choice of an oscillator, as the interface is designed to be worn on the body. Many performers find it difficult enough to work with technology, and unless the presence of the technology is important to the work, as it is in Tomie Han's and Curtis Bahn's Pikapika [6], the less obtrusive the better. We considered using pre-existing wavetable synthesisers, such as those found in the cheapest soundcards, but they are driven by MIDI, and suffer from its limitations. New incarnations of classic sine oscillator circuits using op-amps were tested, but the circuits are too unstable, changing radically according to minor component differences, and too large due to the number of components involved. We then tested the circuit shown in Figure 2 that uses a 74C04 hex inverter. -.---------........... (D) 3.8. KHz Figure 3. Waveform Outputs of 74C04 Circuit (a) 152 Hz ----\. it 4 i Figure 2. 74C04 Oscillator [7] The variable resistor labelled RI can be used to control oscillator frequency, and R2 used to control output amplitude. These are the points in the circuit where one would connect a sensor to alter either frequency or amplitude according to sensor resistance. The maximum frequency in the configuration shown in Figure 2 is 3.88 kHz, and the maximum output amplitude 0-dBm into a 600-ohm load. The "sine wave" produced by the circuit in Figure 2 is remarkably clean, but still contains several harmonics. Waveforms and spectral analysis of the circuit output can be seen in Figures 3 and 4. (b) 3.88 kHz Figure 4. Spectral Analysis of 74C04 Output We were very happy to see such a consistently shaped spectrum across the frequency range. In particular, that the strengths of partials relative to the fundamental remained consistent. The usable frequency range was found to be between 148 and 95601 Hz, which we considered a suitable range. The oscillator in Figure 2 demonstrated high frequency stability in temperatures ranging from 25 to 1 9560 Hz is the maximum frequency with a 6.8nF capacitor between pins 5 and 6 on the 74C04. We later replaced this with a 10nF capacitor, because it was less susceptible to temperature changes. This gave us a maximum frequency of 3880 Hz.

Page  00000003 -100 Celsius, with battery charge creating much greater fluctuation (+/- 10%). It is imagined that JACK-ON users would use fresh batteries, and not in such extreme temperatures as to effect battery output. A single 74C04 circuit draws 13.78 mA, meaning a 9V alkaline battery will last 43 hours, or if the maximum of 8 oscillators are used the battery will last more than 5 hours. 2.3. Frequency or Amplitude? In a system converting sensor resistances to audio, the question is whether to use frequency or amplitude. We thought frequency was less likely affected by inconsistencies in the wireless transceiver. In particular, if an audio dropout occurred, our frequency analysis would not report anything, whereas the amplitude analysis would report a sudden drop. 2.4. Software Our first approach to software analysis was to use Tristan Jehan's analyzer- object in Max/MSP [8]. The only output used was the sinusoidal decomposition, which provides lists of frequency/amplitude pairs. Miller Puckette's fiddle- object also provides this spectral decomposition, and is available for Pd (making the software side of things free). We chose to use analyzer- because fiddle- (in its current state) doesn't allow for specification of FFT hop-size, meaning that its analysis period is set by the FFT window size. 2.5. Tracking Oscillator Pitch When analyzing the frequency output of oscillators using analyzer-, we gated frequencies according to their amplitude. If the amplitude of any pair of partials did not exceed that of any fundamental frequency, frequency output could be simply gated according to the amplitude of fundamentals. Otherwise, further data filtering would be required, adding to the latency of the system. We were able to satisfactorily track the frequencies of three 74C04 circuits connected together, providing the spectrum shown in Figure 5. amplitude was all that was required to track these peaks. The frequency response of oscillators is logarithmic, therefore, frequency to MIDI conversion provided the best way to convert values into a linear range. This range was found to be between MIDI pitches 50.1 and 122.3. Given that the system seems stable within 10 cents, this provides a usable range of 722 10-cent intervals: a resolution of around 9 bits. We found that oscillators could be set as close as three 10-cent intervals apart at the high end, and nine 10 -cent intervals at the low end without causing serious crosstalk. This distance will affect the maximum range of each oscillator. For the simplicity of software programming, it is best to make sure frequencies do not overlap. At this point we haven't tested frequency tracking of more than three circuits running simultaneously. The more sensors added to the system, the narrower each oscillator's frequency range must be. If we break up the 722 10-cent intervals available to us into 8, we will have less than 90 steps for each sensor. This is not a satisfactory resolution for most applications, and this inspired us to investigate amplitude tracking. 2.6. Tracking Oscillator Amplitude By fixing the frequency of oscillators and analyzing amplitude output we were able to get a range from analyzer- decomposition of between 0. and 1.07. The amplitude was stable at a scale of 0.0001, giving us a range of up to 10700 steps: a resolution of over 13 bits. These measurements were made with the output of the oscillator plugged directly into the computer's sound card. Wireless audio systems were found to introduce noise that limits the useable amplitude range. Figure 6 compares amplitude values between 0.46 and 0.48 captured with an oscillator plugged into two different wireless systems. The noise introduced by the AKG system lowered resolution by a factor of 5 (maximum of 2140 steps), whereas noise introduced by the Nady system lowered resolution by a factor of 100 (maximum of 107 steps). (a) AKG SR60/PT60 (b) Nady DKW-1 Figure 6. Noise introduced by wireless audio systems The noise introduced by wireless audio systems did not cause any problems in the tracking of oscillator frequencies. Another approach to the measurement of amplitude was made using FFT magnitudes, with oscillators tuned to frequencies corresponding to the center of FFT bins. One advantage to this approach is that the analysis data obtained is already in the form of an audio signal, thus minimizing latency when data is applied to signal manipulation. Figure 7 shows the magnitude of an oscillator running at frequencies corresponding to bins 3 and 10 of a 512-point, 8-overlap FFT. It should be also. 1 I Figure 5. Spectral analysis of three circuits running at 1027, 3214, and 5103 Hz It can be seen from Figure 5 that the three peaks from the fundamental frequencies of the oscillators easily outweigh the collective partials, and gating according to

Page  00000004 noted that this approach uses about twice as much CPU as the analyzer- or fiddle- options. (a) bin 3 (b) bin 10 Figure 7. FFT magnitude of 74C04 circuit Complications of tracking FF T magnitudes arise because of the linear frequency scale of the FFT. As can be seen in Figure 5, higher frequency oscillators leave larger gaps in the FF T because their harmonics are further apart on a linear scale. The problem with setting oscillator frequencies to match higher bin numbers is that the tolerance for tuning becomes more critical. The advantage of implementing an FFT magnitude-tracking algorithm is that the same algorithm could be used for various pieces of software. 2.7. Future Work Further testing and development needs to be done to resolve several issues regarding JACK-ON performance. Frequencies that work with FFT magnitude tracking need to be found, and other partial-tracking algorithms investigated. We are also in need of a low-noise. summing amplifier for combing the output of multiple oscillators. For the tracking of multiple amplitudes we have thus far been running oscillators through separate channels on a mixing desk. For the physical design of JACK-ON, we envisage one unit housing a battery that jacks into the audio of the wireless transmitter. Up to 8 units for connecting sensors will be daisy-chained off this unit. Each unit is small enough to fit into small compartments on a belt to be worn around the waist or torso. The housings currently being considered are RJ 11 splitters, where one socket can be used to connect multiple units together, as shown in Figure 8, and the other to connect sensors. either the frequency or amplitude. A potentiometer could then be connected to the unused input to set frequency or amplitude. 3. CONCLUSION Although early in its development, the JACK-ON system appears to be a viable option for low-cost wireless sensor interfacing. If manufactured in large numbers it should be very cheap to produce. The user will be able to determine how much to invest in a wireless audio system, based on their requirements for accuracy and reliability. The hardware system is completely analog, making it relatively easy to configure. The resolution, sample-rate, and latency are comparable to other more expensive systems. We hope to develop the system further and market it. 4. REFERENCES [1] [2] [3] [4] Bianciardi, D. Igoe, T. Singer, E. "Eos Pods: Wireless Devices for Interactive Musical Performance" Proceedings of the Fifth International Conference on Ubiquitous Computing, Seattle, USA, 2003. Hahn, T. and Bahn, C. "Pikapika - the Collaborative Composition of an Interactive Sonic Character." Organised Sound. An International Journal of Music Technology. vol. 7 no. 3, 2003. [5] [6] [7] Mondragon, A. "Gates provide low-cost sinewave generator." EDN Access, Reed Elsevier Inc. August 15, 1997: di_03.htm [8] analyzer- is based on Miller Puckette's fiddle-: Puckette, M. and Apel, T. "Real-time audio analysis tools for Pd and MSP". Proceedings of the International Computer Music Conference, San Francisco, USA, 1998. Puckette's fiddlewas named jack~ when originally developed for Max-fts. Figure 8. RJ-11 Splitters We only need 3 conductors on every cable in the throughput to the wireless transmitter: battery voltage, ground, and audio signal. We are considering various ways to allow the user to connect sensors to control