Page  00000196 ICAST: TRIALS & TRIBULATIONS OF DEPLOYING LARGE SCALE COMPUTER-CONTROLLED SPEAKER ARRAYS Stephen David Beck, Ph.D. Louisiana State University Center for Computation & Technology Laboratory for Creative Arts & Technologies Brian Willkie Louisiana State University Center for Computation & Technology Laboratory for Creative Arts & Technologies Joseph Patrick Louisiana State University Center for Computation & Technology Laboratory for Creative Arts & Technologies ABSTRACT The Immersive Computer-controlled Audio Sound Theater (ICAST, pronounced "eye-cast") is an on-going research project that addresses performance issues in electroacoustic music. By exploring a variety of differing performance modes through a computer-controlled loudspeaker environment that is malleable, adaptable and intuitive, we are able to explore new and novel metaphors for sound control using both conventional and unconventional interfaces. ICAST is a hardware and software solution, based on a client-server computer control system, commodity audio interface hardware, and high-quality audio amplifiers and loudspeakers. Our system has been deployed for concerts and conferences throughout the past year. Through this initial deployment phase, we have identified critical issues with respect to our client/server systems design, our client software control, and our audio server. This paper identifies both the successes and the struggles we have had, and outlines the solutions we have taken to address these issues. What we have learned from this research has applications that go well beyond concert hall performances, and include virtual reality environments, cinema, and video games. 1. INTRODUCTION Motivation for this project grew from our experiences hosting regional and national electroacoustic conferences (SEAMUS and Electric LaTex) as well as our own concert series, High Voltage, which presents 3 or 4 concerts throughout the academic year. Our response was the creation of the ICAST system, a portable 24 -channel loudspeaker array, controlled by multiple computers in a client-server architecture. Our client software manages sound mixing and user interaction with the system, and is written using Max. Java code written to run within Max constructs OSC control messages that are sent over a wireless network to our server software, which is written in SuperCollider. Having developed a sustainable and fairly robust system, we decided to deploy the system for our 2006 -2007 concert season, which consisted of 3 major concerts and our regional LaTex conference (3 concerts). We were also invited to deploy ICAST at ICMC 2006 at Tulane University, where 6 afternoon concerts of fixed media and live interactive media works were presented. In preparation for ICMC, we used the first two concerts in 2006 as "warm-up" concerts, where we would anticipate the kinds of performance situations we would face in New Orleans, identify specific technical and structural problems, and quickly develop robust and transparent solutions for those problems. We would then use the robust code from ICMC as a base of development for new features we proposed in our previous work [Beck 2006]. 2. REFINEMENT AND DEVELOPMENT THROUGH CONCERT EXPERIENCE In June 2006, our lab was invited to present ICAST as the speaker array for use in concerts during ICMC 2006 at Tulane University. The system was used for the afternoon concerts held in McAllister Hall. Concerts held both before and after ICMC 2006 were used to debug and improve ICAST by introducing new features and exposing technical problems within the system. With each concert, we incrementally added new features to the ICAST system. The majority of problems we faced in the development of ICAST were related to performance, and were compounded by the lack of a dedicated rehearsal space in which to deploy the full system for testing. This led to feature additions that worked on a small scale, but ran into processor limitations during full deployment in technical rehearsals and concerts. 2.1. September 2006 The first concert of the season was in September 2006 at the Shaw Center for the Arts in Baton Rouge, with Elainie Lillios as the guest composer. This was the first concert produced on ICAST outside of the LSU School of Music, and showed the malleability of the system. The concert included 24 speakers plus a subwoofer. With this deployment of the full system, we noticed that client performance problems created messaging bottlenecks crashing the client and severing communication with the audio server. The server continued to process the audio at the levels set prior to the crash, but the diffusionist was unable to adjust those levels until we restarted the client. On some occasions, these bottlenecks became so severe that we were forced to reinstall firmware on the TASCAM US-2400 fader controller, which we use as our primary user interface. We were able to improve client performance, stability and response by reducing the amount of messaging between Max and the controller through gating, and by using Java instead of JavaScript to construct control messages. 196

Page  00000197 We also found that our initial audio bussing design created complications when adding new features and signal processing to the audio server. By routing the audio output of each channel to a single bus rather than a unique bus for each channel, the audio aggregated along the bus, building in amplitude as each channel wrote its output. This process was exacerbated when trying to implement new additions such as high-pass filters. This issue did not appear until technical rehearsals, as it was masked in small scale studio testing. After this first concert, a new routing scheme was developed for use in future concerts, allowing for independent processing. During the technical rehearsal, it became apparent that file playback directly from SuperCollider was not working correctly. The playback would simply stop for no apparent reason in the middle of performance at random points within the sound file. Whether this behavior was connected to using pre-compiled binaries on a dualprocessor G5, problems with our particular G5, some combination, or none of the above is unclear. Our solution thus far has been to play the audio files from a third computer and feed its output to the physical inputs of the system. In the future we will attempt to stream the digital audio to the server in order to reduce the conversion to and from analog audio. 2.2. October 2006 The second concert of the season was in the LSU School of Music Recital Hall. For this concert, all of the system's JavaScript was ported to Java, resulting in a dramatic performance improvement. Using Java, we began to dynamically allocate node IDs, allowing better management of the client-server communication. Additionally, gates triggered by the touch sensors of the TASCAM fader controller were added to prevent messaging bottlenecks. Four band parametric equalizers and simple delays were also added to each channel strip. In adding support for the TASCAM's dials, we enabled the user to select and control parameters (e.g. delay-time) from the fader board rather than from the computer screen. As with the September concert, the primary playback mechanism was Logic Pro on a third computer. This concert employed a new feature of Macintosh OS X 10.4, MIDI over Network. This allowed time-code from Logic to be transmitted for displayed within MAX, while also allowing the TASCAM's playback controls to directly control Logic, creating a friendlier user interface. Due to hardware issues with our PowerMac G5 audio server, we ran the concert from a single iMac (single 1.5GHz G5). By using another new OS X 10.4 feature, the aggregate audio device, SuperCollider was able to address multiple FireWire audio interfaces without the specialized AudioWire cards our audio server normally requires, while retaining the use of all 24 speakers and the subwoofer. The issues with the audio server arose only with the full system in use, and did not appear in small scale testing. Our ability to switch out the server only hours before a concert demonstrated the flexibility of the ICAST system. 2.3. ICMC 2006 The only major addition to the system for ICMC 2006 was the ability to solo individual channels for live level setting. We incurred only two technical issues related to ICAST during ICMC 2006 in the McAllister venue. First, the computer used as a playback device failed on the last day of the conference, however a backup computer was available. Second, we discovered a problem with the high-pass filters native to SuperCollider. For some electroacousticaly-generated sounds, these filters introduce a subtle but clearly audible distortion. At the time of this writing we are still seeking a clear diagnosis and solution to this problem. Additionally, we addressed a number of performance issues with the server by significantly increasing the values for some of the command line options such as the number of audio bus channels (-a, 2048 instead of 128), the max number of nodes (-n 2048 instead of 1024), and the real-time memory size (-in 1048576 instead of 8 192). At ICMC, the array was configured with 20 fixed speakers, four variable position speakers, and two stage Li~~ Figure 2. Speaker Configuration for the Shaw Center. 197

Page  00000198 monitors. Many composers did not take advantage of the speaker array for diffusion, in part because they were unfamiliar with either (a) the specific layout of the diffusion array, or (b) the general concept of multichannel diffusion. In either case, a 15-30 minute rehearsal window was insufficient for many composers to develop a diffusion plan for their work, so they set a simple static mix that would suffice for the entire piece. N----. --- --,:\,. /,,,\/\,.................. 7<.I~ fader that scales the preset levels of up to 24 audio channels, making it possible to mix between groups of speakers. Currently, a limit of 24 vectors ("audio scenes") can be programmed. Some issues arose with Vector Fading. In order to allow for multiple users to have individual Vector programs, a method of storage is needed. The second issue is performance. Vectors are computationally expensive, and could not maintain real-time performance under a heavy load. During the concert, Max crashed as a result of the expensive vector computations. Because of the client/server architecture model of ICAST, the piece continued with the levels set at the time of the crash, albeit without user control. We intend to explore the vector calculations in Jitter as a possible solution to this problem. S-eakers oln staase at various hights Less than Ba week after EiCMC, LSU hosted Electric La---- Tex, a small regional festival comprised of graduate student works from LSU, Tulane, Rice, UT-Austin and University of North TE DEC S concert employed 26 FouIr 12" 8agE.?.. TA2.Oa.1? "\ speakers plus a o Wed this is the largest. Figure 3. Speaker Configuration for ICMC 2006 2.4. Electric LaTex 2006 Less than a week after ICMC, LSU hosted Electric LaTex, a small regional festival comprised of graduate student works from LSU, Tulane, Rice, UT-Austin and University of North Texas. This concert employed 26 speakers plus a subwoofer. To date, this is the largest array driven by ICAST. The limit on ICAST speaker control is 44 speakers (if external Digital to Analog converters arudience added, this number rises to 52 at 96kHz). 2.5. March 2007 - Vector Fading Jeff Stolet was the guest composer forot of the March 2007 concert held at the Shaw Center. This concert used 24 speakers with a subwoofer, and mimicked the September 2006 concert in speaker placement, with two exceptions; the front center speaker was turned to face away from the audience and the subwoofer was placed in a corner of the room rather than at the foot of the stage. More of the dynamic object-creation code was ported from Max scripts to Java. The reduction in Max objects decreased the startup time of the client by 9.1%. A new type of diffusion mode was introduced during this concert: vector fading. Programmed in Java, vector fading is based on theatrical light board 'scenes', where one fader can control multiple instruments at various preset levels. Thus an "audio scene" consists of a single I Ajjiffi tia:! ^_ ^_ I \ _ f is I:i _ II'ui ^ ^ ^-rije8-- I:jf: Figure 4. Speaker Configuration LSU Recital Hall 3. FUTURE DIRECTIONS One question that commonly arises when discussing ICAST is "why not just use a digital mixer?" Good question. At the time, most digital mixers lacked the flexibility for which we were searching. We also wanted to eliminate as many cables as possible to the mixing console. ICAST's design separates controllers from servers, and modularizes the component parts for easy access, repair and upgrade. By building the application on a general-use CPU, we can construct our own system in order to explore new interfaces, new DSP algorithms, and new performance practices. 3.1. Alternative Controllers In the quest to break from the fader-board paradigm, we are exploring new control devices, such as the Lemur. The client software currently uses the built-in MIDI capabilities of the TASCAM-2400 and Max. Expansion of these to other objects creates a new level of complex 198

Page  00000199 ity for both the programmer and the end user. A technique to utilize multiple controller devices needs to be able to handle simultaneous controller commands without conflicts. Any performer using a non-standard controller needs to have time to practice and understand the response of the controller, making it difficult for guests to use ICAST alternatives. A user might have a device that he/she wishes to use, but without knowing the proper messaging commands, even standard MIDI devices would take time to reprogram for use with ICAST. To facilitate differing control devices, an API is being developed to allow performers to bring in a controller foreign to ICAST and connect with minimal difficulty. 3.2. Ambisonics ICAST can accommodate pieces in many forms without changing the position of the speaker array. ICAST already has two-dimensional controllers in the form of XY joysticks, and while these have not been employed as of yet, they are available for use in Ambisonic performance. Code has been written but not yet fully tested for real-time encoding of mono and stereo inputs into firstorder Ambisonic fields. Accompanying this is a firstorder decoder for an arbitrary speaker array. This code is currently being tested for accuracy. Progressing to higher order encoder-decoder fields and being able to manipulate multiple fields in performance is a priority. 3.3. Simplification of Use We are in a constant process of improving the ICAST audio server, and as a result, we have not focused much on "ease of use." Launching ICAST requires Max/MSP (Client), Audio/MIDI Setup (playback synchronization and control with Logic), and Terminal (to control the server) to be running. End users must also have a strong knowledge of SuperCollider in order to run and monitor scsynth properly on the server. Our goal is to reduce these end user requirements to a single launch command that is called from within Max/MSP. There are other aspects of ICAST that need optimization. For example, storage of user information is cumbersome, and the assignment of operational controls is tedious. One solution is to create user templates for the common venues where the system is deployed. 3.4. Intel Processors and Apple Computers As Apple has moved all of its new computer lines to Intel processors, testing must be done to ensure continued operation on the different processor. The MacBook Pro line currently has multi-core processor, and a faster internal bus, which may alleviate some issues with Vectors and other client-side as well as server-side performance issues. 4. CONCLUSION Sound diffusion is the performance practice of electroacoustic music, and ICAST is one of a growing collection of performance instruments in the United States. ICAST is an attempt to create a class of "diffusion instruments" that facilitate complex sound mixing through traditional and novel interfaces. As others begin to develop similar systems, we look forward to sharing our experiences and collaborating on developing similar approaches with the hope that a collective approach will spawn more and improved instruments. We gratefully acknowledge the support of the Center for Computation & Technology at LSU and its Laboratory for Creative Arts & Technologies for supporting this project. 5. BIBLIOGRAPHY Austin, L. (2001). "Sound diffusion in composition and performance practice II: An interview with Ambrose Field." Computer Music Journal 21(4), 21-30. Beck, S.D., J. Patrick, K. Malveaux, B. Willkie (2006). "The Immersive Computer-controlled Audio Sound Theater: Experiments in multi-mode sound diffusion systems for electroacoustic music performance." Proceedings of the 2006 International Computer Music Conference, New Orleans, LA Chadabe, J. (1997). Electric sound: the past and promise of electronic music. Prentice Hall. Clozier, C. (2001). "The gmebaphone concept and the cybernephone instrument." Computer Music Journal 21(4), 81-90. Gerzon, M. (1983). "Decoders for Feeding Irregular Loudspearker Arrays." United States Patent 4,414,430. Harrison, J. (1999, 9). "Diffusion: theories and practices, with particular reference to the beast system." eContact 2.4. Malham, D. G. and A. Myatt (1995). "3D sound spatialization using ambisonic techniques." Computer Music Journal 19(4), 58-70. McCartney, J. (1996). "SuperCollider: a new real time synthesis language." In Proceedings of the International Computer Music Conference. Moore, F. R. (1989). "A general model for spatial processing of sounds." In C. Roads (Ed.), The Music Machine. MIT Press. Roads, C. and J. Strawn (1985). Foundations of Computer Music. MIT Press. Stampfl, P. and D. Dobler (2004). "Enhancing threedimensional vision with three-dimensional sound." In SIGGRAPH Proceedings. Ullmer, B., S. D. Beck, E. Seidel, and S. lyengar (2005). "Development of "viz tangibles" and "viznet": Implementation for interactive visualization, simulation and collaboration." NSF MRI Award CNS-0521559. Wright, M. and A. Freed (1997). "OpenSoundControl: A new protocol for communicating with sound synthesizers." In Proceedings of the International Computer Music Conference. Zicarelli, D. (1997). Max/MSP Software. San Francisco. 199