Page  00000324 Extended Applications of the Wireless Sensor Array (WISEAR) David Topper University of Virginia Virginia Center for Computer Music 112 Old Cabell Hall Charlottesville, VA 22903 US +1 434 924-7355 topper@virginia.edu ABSTRACT WISEAR (Wireless Sensor Array), provides a robust and scalable platform for virtually limitless types of data input to software synthesis engines. It is essentially a Linux based SBC (Single Board Computer) with 802.11a/b/g wireless capability. The device, with batteries, only weighs a few pounds and can be worn by a dancer or other live performer. Past work has focused on connecting "conventional" sensors (eg., bend sensors, accelerometers, FSRs, etc...) to the board and using it as a data relay, sending the data as real time control messages to synthesis engines like Max/MSP1 and RTcmix2. Current research has extended the abilities of the device to take real-time audio and video data from USB cameras and audio devices, as well as running synthesis engines on board the device itself. Given its generic network ability (eg., being an 802.11 a/b/g device) there is theoretically no limit to the number of WISEAR boxes that can be used simultaneously in a performance, facilitating multiperformer compositions. This paper will present the basic design philosophy behind WISEAR, explain some of the basic concepts and methods, as well as provide a live demonstration of the running device, worn by the author. Keywords Wireless, sensors, embedded devices, linux, real-time audio, realtime video 1. INTRODUCTION Previous work on the WISEAR box has been an attempt to build upon conventional sensor applications using BAISC stamps and PIC chips. While both versatile and relatively inexpensive, the are also are prone to forcing projects into a unique one-of-akind category. Typically projects end up being very custom made and difficult to reproduce. Additionally, the inherent limitations of the BASIC language are often found to be a nuisance. WISEAR is essentially a TS-7200 Single Board Computer (SBC) made by Technologic Systems3. The first generation WISEAR used an x86 based chip which was much larger and power hungry than the current version, which uses a 200 MHz ARM9 processor with power as low as 1/2 Watt. The device is essentially a complete Linux based computer that weighs less than lb and measures approximately 4" x 6". Hard drive space is provided via a flash memory card. A standard configuration includes two USB ports, a serial port, RJ11 (ethernet), twenty channels of digital Daughter boards are inputs. 10 and eight channels of analog input. available to increase the total number of The only thing that needs to change from application to application when using WISEAR is some basic soldering to the inputs. Software loaded onto the board automatically scans each input device for a signal, and if received sends data out via wireless TCP packets to any number of software synthesis engines such as Max/MSP and RTcmix. Data is transmitted in generic pairs (controller number / data value) and hence is routed on the receiving end. The device itself remains largely unchanged. Using 802.11a/b/g instead of FM radio overcomes a variety of issues. Range is increased. Tuning the respective transmitter and receiver are no longer an issue. The frequency range for 802.11 devices is also far less prone to interference from other devices, since it is such a popular standard for wireless internet access. Bandwidth is in the 5 megabyte per second range. Figure 1. WISEAR using TS-7200 SBC with standard ports. Being a general purpose device, it becomes very simple to add off-the-shelf (OTS) components such as USB audio and video input. There is no need to design a special interface to read audio or video data. Similarly, 802.11 a/b/g hubs and routers are readily available. Since the unit is also a fully functional personal 324

Page  00000325 computer running the Linux operating system, it is capable of running software synthesis engines on board. Speed is obviously an issue, being a 200mhz processor, but simple synthesis and effects processing algorithms are easily realized. WISEAR moves far beyond the limitations of MIDI. Data for conventional sensors is limited only to the data range of the sensor itself, as opposed to seven bit data MIDI offers. Data speed is only limited by the sensor latency and the processor speed of the device. Latency and speed measurements are currently in progress, but initial results suggest orders of magnitude improvement over the 31.2kbs data rate used by conventional MIDI. 2. AUDIO INPUT I OUTPUT WISEAR does not come standard with an audio input device. It does, however, have two built in USB ports. As a result, using the Linux AL~SA driver, it is possible to connect most generic USB audio devices. Our research has used the USBgear USB microphone and speakter dongle4. The device is both lightweight and small, which allows the WISEAR device to remain both portable and wearable. Audio data is read using standard ALSA5 read/write calls from with RTcmix running on board the device. It is then sent out in a manner different from other WIISEAR sensor data. Sensors are numbered 0-27. The first eight are analog inputs. When written via TCP write calls, data is sent using two sixteen bit integers. The first is the controller number. The second is the data itself. Audio data, while being read at a higher rate, is written out via TCP in regular interleaved audio frame format. Each channel is sent sequentially, first left then right. A special purpose RTcmix instrument has been created to perform the task called TCPMIX. WISEAR supports fu~ll duplex audio via the USB audio device. So data can also be received via the TCPMIX instrument and sent out through the USB audio device, then routed to a pair of wearable speakers or headphones. The generic nature of the device supports virtually limitless applications. 3. VIDEO INPUT Video input is also achieved using OTC hardware. Our research has used Logitech Quickcam for Notebookts Pro6. Other webcams can also be used, but the Logitec device is already designed to be portable and lightweight, so fits this application. Video data is read using the Video4Linux7 API. Pixels are sent as RGB data (uncompressed) in a manner somewhat different from other audio and sensor data. There is no real need for controller/value pairs... as the data type qualifier is only necessary at the outset: video. Currently, video can only be sent to the GAlA8 (graphical audio interface application) program. But work is being done to facilitate sending network video data to Max/MSP using Jitter'. Image information is sent line by line via TCP writes. So, for example, once six hundred and forty rgb triplets have been sent, the next line of the image is written, another four hundred and eighty times. This is not the most optimal method. But using USB video with WISEAR is still very much in the development stage. Work needs to be done on compressing the video image. Currently, we are averaging approximately five frames per second of video. 4. EMVBEDDING REAL TInME AUDIO APPLICATION S INT O WI SEAR Persistent memory space comes at a high premium on the TS7200 and most SBCs. So putting a complete Linux distribution on the chip, complete with libraries, header files, compilers and various other utilities isn't really feasible. Thankfu~lly, Technologic Systems provides a development kit which facilitates compiling binaries on your Linux PC and copy them over to your Linux ARM-based SBC. The kit involves installing a custom kernel and using special compilers which will build binaries for ARM processors under an x86 architecture. But development is still relatively seamless. Code that runs on a L~inux PC, even USB video code, can be recompiled and copied to WIISEAR. Getting RTcmix running on the WISEAR was simply a matter of recompiling the basic program using the development kit. Speed remains somewhat of an issue for complex processing. Also, the board only ships standard with 32 mb of SDRAM. So memory space is also limited for storing audio buffers. 5. FiUTURE DIRECTIONS Despite limitations of speed and built in memory, WISEAR remains an effective platform for gathering sensor, audio and video data. It's small form factor and wireless data transmission method make it an ideal candidate for dancers and performers. More work needs to be done optimizing video data compression. Benchmarks for latency on digital and analog sensors also needs strict testing. Additionally, more work is needed using multiple WISEAR boxes simultaneously. Current testing has only looked at two. There is theoretically no limit, other than wireless bandwidth, on how many can run simultaneously in a performance space. There are many implications for using audio and video feedback, using the work described in this paper. Similarly, special sensors such as sonar can be used in novel ways by synchronizing transmit / receive times across a WIISEAR based network. 6. REFiERENCES [1] http://www.cycling74.com (n.d.) Retrieved 1/31/06 [2] http://www.rtcmix.org (n.d.) Retrieved 1/31/06 [3] Technologic Systems, http://www. embeddedarm.com (n.d.) Retrieved 1/31/06. [4] http://www.usbgear. com (n.d.) Retrieved 1/31/06 [5] http://www.alsa-proi ect. org (n.d.) Retrieved 1/31/06 [6] http://www.log-itech.com (n.d.) Retrieved 1/31/06 [7] http://www.exploits. org-/v4l/ (n. d.) Retrieved 1/31/06 [8] Topper, D. "GAlA: Graphical Audio Application'', Proceedings of the 2005 NIM~E Conference, Vancouver, Canada. 325