Page  1 ï~~STUDIO REPORT: UC BERKELEY CENTER FOR NEW MUSIC AND AUDIO TECHNOLOGIES (CNMAT) Richard Andrews Center for New Music and Audio Technologies (CNMAT), University of California, Berkeley richard@cnmat.berkeley.edu ABSTRACT The UC Berkeley Center for New Music and Audio Technologies (CNMAT) houses programs in research, pedagogy and public performance that are focused on the creative interaction between music and technology. CNMAT's pedagogy program is highly integrated with the Department of Music's graduate program in composition, while the research program is linked with other disciplines and departments on campus such as architecture, mathematics, statistics, mechanical engineering, computer science, electrical engineering, psychology, cognitive science, physics, space sciences, the Center for New Media, and the Department of Theater, Dance, and Performance Studies. This Studio Report describes a number of new or continuing projects and initiatives we are pursuing, including the presentation of noteworthy composers, performers, researchers, and other individuals of interest to the campus community and general public. 1. INTRODUCTION CNMAT has a long-standing goal of establishing a genuine connection between research and composer training - a link that is meaningful, productive and stable. We have addressed this challenge by developing and implementing a new model for bridging our research and pedagogy programs. Our initiative, which has several components and touches on many aspects of our work, makes extensive use of our new web site for collecting, organizing and disseminating content for our graduate student composers. This includes software, tutorials, history, examples of applied techniques, and more. By making this collection available to the entire world in an interactive, community-driven format, we hope to create a valuable repository that supports and inspires contemporary music education. 2. PERSONNEL CNMAT has been organizationally restructured and now has two faculty co-Directors: one, a technology specialist with musical expertise (David Wessel, CNMAT Co-Director of Music Science), and a composer with technical expertise (Edmund Campion, CNMAT Co-Director of Pedagogy). Other CNMAT staff include Adrian Freed (Research Director), Richard Andrews (Associate Director), Michael Zbyszynski (Music Composition and Pedagogy), Rimas Avizienis (Researcher), and Andy Schmeder (Researcher). CNMAT supports students in the graduate composition program (Jean Ahn, Mason Bates, Nils Bultmann, David Coll, Daniel Cullen, Aaron Einbond, Robin Estrada, Heather Frasch, Jeremy Hunt, Jason Levis, Jimmie Lopez, John MacCallum, Amadeus Regucera, Damon Waitkus, Jennifer Wang, and Robert Yamasato), students in the graduate ethnomusicology program (Miki Kaneda, Robert Beahrs), graduate students from other disciplines (Eric Battenberg from EECS, Cynthia Maxwell from Mechanical Engineering), and undergraduates working on CNMAT projects (Yotam Mann). Other faculty from the Department of Music (Myra Melford, Cindy Cox, David Milnes) are also supported in their work at CNMAT. Recent visiting scholars include composer and radio producer Michal Rataj from the Czech Republic, composer Martin Matalon from France, sound artist, researcher and ecoacoustic composer David Monacchi from Italy, graduate student researcher Franz Zotter from Austria, and graduate student composer Marc Codron from France. 3. FACILITIES 3.1. Recording Studio, Media Studio, Main Room Providing a professional-grade recording/production environment is essential for our composing, training and research needs. CNMAT's Recording Studio has been completely refurbished, with acoustic analysis and treatments, new hardware (mixing console, computers, monitors, speakers, hard drives) and software (ProTools, Waves, etc). Other spaces such as the Media Studio (for video projects) and Main Room (for recording, production, rehearsal and performance) are being upgraded and organized for increased use by students, faculty and researchers. The result has been a substantial increase in productivity, from mixing and mastering projects in our studios to experiments and events in our Main Room. 3.2 Sensor Library Many of our composers, instrumentalists, and researchers are using recent sensor technology to inform their work. For example, cellist Frances-Marie Uitti has been collaborating with CNMAT Research Director Adrian Freed on a project adding sensors to the cello, while our faculty and student composers have been exploring new applications of sensors in their work. To support these efforts, a Sensor Library has been developed, providing a curated assortment of off-theshelf and experimental sensing devices for use by CNMAT composers and performers. Included are game controllers such as Wii, Wacom digitizing tablets,

Page  2 ï~~single and multi-touch sensors, inertial sensors, midi keyboards, etc. Each device is stored with a reference code that connects to the CNMAT web portal, which tracks inventory and provides documentation, driver and applications software for each sensor. 3.3 Information Infrastructure The previous CNMAT web site has been replaced by a completely new Information Infrastructure [6] that features enhanced functionality, opportunities for user interaction, and a repository of software and related content that is presented in an aesthetic and pedagogical context. Developed using the Drupal content management system, the new CNMAT web site (www.cnmat.berkeley.edu) provides content and interactive components that support all of our programs. Some of the new site's many features are outlined below under Education. 4. EDUCATION 4.1. Pedagogical Software Resources The Max/MSP/Jitter (MMJ) Depot [9] is an expanding collection of patches for composers, programmers, performers and music educators. Comprised of software modules and applications, demos, guidelines, tutorials and other material, the MMJ Depot is designed to assist students as they explore the territory of computer music composition with the assistance of guided lessons, advice and examples. The Max/MSP Downloads Page makes the latest versions of our software (managed with Subversion/SVN) available, while the Music Information Center (MIC) presents a selected list of topics for students to explore and experiment. The site also contains a number of other useful features, such as the CNMAT Equipment Inventory (a searchable database with detailed information on each item, including software support, links to manufacturers' info, and comments from previous users), a bibliography module that both organizes CNMAT's published works and connects to outside collections via OpenURL, personal pages for individual users and groups for collections of users with common interests, needs, or access issues, and an events page that posts upcoming events and collects past events into an archive. The site will soon contain audio examples, video documentation, musical works with all of their components (scores, software, samples, recordings, publication and performance history), an online reservation system for studios, performance spaces and equipment, and opportunities for users from outside the immediate UC Berkeley community to participate in content development. Courses such as Music 201/Introduction to Computer Music Composition, Music 207/Advanced Projects in Computer Music, and Music 158/Musical Applications of Computers and Related Technologies are already making extensive use of the new site, as are many independent study students and researchers. 4.2. French-American Exchange CNMAT has a long tradition of collaboration and exchange with partners in France. For student composers in particular, the attraction has been the opportunities to study with established artists, create new works and participate in concerts in a foreign environment. Under a three-year grant from FACE/French-American Cultural Exchange, CNMAT has engaged in an exchange project with three educational partners in Nice, France: CIRM, the Conservatoire Regional de Nice, and the Universite Nice Sophia Antipolis. The project includes international residencies for graduate student composers, faculty visits for teaching and collaboration, the creation of new musical works, and concert performances in the U.S. and France. We hope to see the exchange continue and expand over time, with additional opportunities for student involvement and participation by performing ensembles, presenting organizations, festivals and venues at home and abroad. 4.3. Summer Workshops As part of our mission to serve the general public, CNMAT offers workshops during the summer that are open to students and non-students alike. Recent topics have included Max/MSP Day School (for beginners to intermediate users), Max/MSP Night School (for intermediate to advanced users), a Sensor Workshop for Performers and Artists, and Jitter Night School. 5. RESEARCH CNMAT's research program is guided by our ongoing interest in the intersections between science, engineering, music composition and live performance. Project results are rapidly employed in new creative works, allowing real-world experience to inform our research agenda. Our interests range from input devices (sensors, tablets, touchpads, augmented instruments), connectivity, parallel computing, sound analysis, synthesis, software tools and machine learning to music cognition, hearing augmentation, speaker and microphone arrays, and the development of protocols and formats. Note that publications for much of the recent work described below can be found on the CNMAT web site on the Research page. 5.1. Gestural control, interfaces 5.1.1. Multi-touch array We have developed a new musical interface that consists of an array of force and position sensitive touchpads [7]. The data from these pads is interpreted to create a three dimensional representation of musical material: two spatial dimensions plus a force measurement we typically use to control dynamics. The touchpads are sampled at a sub multiple of the audio sample rate and the gestural data is brought into the host application as audio signals.

Page  3 ï~~5.1.2. Wacom tablets CNMAT has over a decade of musical projects and research employing Wacom digitizing tablets as musical controllers [10]. Developments include implementation schemes using Max/MSP and OpenSoundControl, specific implementations in musical improvisation, interactive sound installation, interactive multimedia performance, and use of the tablet as a compositional assistant. 5.1.3. New fiber and malleable materials We have investigated new fiber and malleable materials (including piezoresistive fabric and conductive heatshrink tubing) and developed techniques and examples of how they may be used for rapid prototyping and agile development of musical instrument controllers [2]. 5.2. uOSC: The Open Sound Control Reference Platform for Embedded Devices uOSC is general-purpose firmware for a low-cost microcontroller that employs the Open Sound Control protocol over USB [5]. The software is designed with considerations for integration in new musical interfaces and embedded devices. Features of note include stateless design, efficient floating-point support, temporally correct data handling, and protocol completeness. 5.3. Modal Parameter Tracking for Shape-Changing Geometric Objects We have developed a method to estimate resonant frequencies of an instrument by combining information about the modes of two similar instruments [4]. By using modes computed for one geometry to estimate the frequencies for nearby geometries, we can hear much more quickly how changing the instrument shape changes the sound. 5.4. Optimizing Hearing Aids for Music Listening Our work focuses on a novel approach to hearing aid fitting applied to both individual differences in hearing impairment and differences among musical styles [8]. The method uses a subjective space approach to reduce the dimensionality of the fitting problem and a nonlinear regression technology to interpolate among hearing aid parameter settings. This listener-driven method provides not only a technique for optimal aid fitting, but also information on individual differences and the effects of gain compensation on different musical styles. 5.5. Harmonic Expectation and Affect in Western Music: Effects of Attention and Training We have investigated the effects of selective attention and musical training on the processing of harmonic expectations [3]. Our experiments highlight the importance of attentional focus in musical training, especially as it relates to the processing of harmonic expectations. 5.6. Room acoustics measurements Our approximately spherical source of 120 individually controlled drivers [1] has been employed to perform impulse response measurements in a room with a 1.4 second reverberation time and a distinct echo. The signal to the drivers was processed to produce both omni directional and unidirectional patterns. The omni directional pattern was compared with measurements made with a traditional 12 sided source. The unidirectional patterns were measured both pointed towards and away from the listener position. Intelligibility metrics for the different directionalities and orientations were measured. The unidirectional pattern was aimed in different directions to minimally and maximally excite the distinct echo in the room, and locate its origin. 5.7. Sensor fusion Working in collaboration with Sennheiser researchers, we have developed a compact hybrid 3-axis inertial measurement and magnetometer sensing platform to investigate issues in sensor fusion for head-tracking applications such as HRTF presentation, auditory display and other effects. A robotic servo-based testing apparatus was created for the purposes of automatic calibration and performance verification under a variety of configurations. Our platform is unique in its use of a high-speed timestamped Open Sound Control interface for digital communication. 5.8. Parallel computing In coordination with the Department of Electrical Engineering and Computer Sciences, CNMAT is participating in a multi-year collaborative research project known as the Parallel Computing Research Center (PAR lab) to investigate the implications of the parallel revolution for applications, programming models, architecture, and operating systems. The music and hearing aspects of the project are organized around the following themes: high performance signal processing in the right form factor to create much improved hearing aids; sound delivery systems for concert halls, conference calls, and home sound systems; musical information retrieval; music composition; and gesture-driven liveperformance systems that exploit new sensor technologies. 6. PERFORMANCES CNMAT continues to present a wide variety of composers, performers, researchers and other invited guests in our multi-channel Sound Spatialization Theatre: Frances-Marie Uitti (cello), Myra Melford (piano), and David Wessel (live electronics) Earl Howard (live electronics and saxophone), with guest David Wessel (live electronics) Composer/electronic musician Guillermo Galindo and experimental violist Charlotte Hug, with live video processing by Jen Cohen Nathaniel Bartlett: Solo marimba with threedimensional, high-definition, computer-generated sound projection

Page  4 ï~~The Berkeley New Music Project, featuring the Berkeley Contemporary Chamber Players and the Worn Chamber Ensemble Frank Gratkowski (alto saxophone and clarinets), Myra Melford (piano), David Wessel (live electronics), Nils Bultman (viola) Eric Lyon - lecture and concert Almut Kiihne (soprano), Georg Graewe (piano), David Wessel (live electronics) Michal Rataj lecture and concert Martin Matalon: series of lectures on his compositions as part of his invitation to the campus as a Regents Lecturer Liza Lim lecture The Timeless Pulse Quintet: Thomas Buckner (voice), George Marsh (percussion), Pauline Oliveros (accordion), David Wessel (electronics), Jennifer Wilsey (percussion) Composer Ronald Bruce Smith and guitarist David Tannenbaum, performing a new work ("Five Pieces for Guitar and Electronics") commissioned by CNMAT The Circle Trio: India Cooke (violin), Pauline Oliveros (accordion), Karolyn Van Putten (voice) 2007 MANCA Festival in Nice, France and Monaco, featuring music by Edmund Campion, Aaron Einbond, John MacCallum, and Gael Tissot Field Recording and Eco-Acoustic Composition: a colloquium + concert with Bernie Krause and David Monacchi The San Francisco Contemporary Music Players, featuring music by Edmund Campion duo pantoMorf: Palle Dahlstedt and Per Anders Nilsson, electronics Keynote+: Kate Ryder (piano) and Jane Chapman (harpsichord) FPR Trio: Frank Gratkowski, Phillip Greenlief, and Jon Raskin 7. ACKNOWLEDGEMENTS CNMAT gratefully acknowledges support from the UC Berkeley Department of Music and College of Letters and Sciences, Cycling 74, Digidesign, FACE/FrenchAmerican Cultural Exchange, Meyer Sound Laboratories, Native Instruments, Starkey Laboratories, Inc., Sennheiser, the University of California's IndustryUniversity Cooperative Research Program, Wacom Technology Corporation, and Waves Audio Ltd. 8. REFERENCES [1] Avizienis, R., A. Freed, et al. "A Compact 120 Independent Element Spherical Loudspeaker Array with Programmable Radiation Patterns", Audio Engineering Society 120th Convention, Paris, France, 2006. [2] Freed, A. "Application of new Fiber and Malleable Materials for Agile Development of Augmented Instruments and Controllers", New Interfaces for Musical Expression (NIME), Genova, Italy, 2008. [3] Loui, P. and D. Wessel. "Harmonic Expectation and Affect in Western Music: Effects of Attention and Training." Perception & Psychophysics 69(7): 1084-1092, 2007. [4] Maxwell, C. B. and D. Bindel. "Modal Parameter Tracking for Shape-Changing Geometric Objects", Proc. of the 10th Int. Conference on Digital Audio Effects (DAFx07), Bordeaux, France, 2007. [5] Schmeder, A. and A. Freed. "uOSC: The Open Sound Control Reference Platform for Embedded Devices", New Interfaces for Musical Expression (NIME), Genova, Italy, 2008. [6] Schmeder, A., M. Wright, et Information Infrastructure", Computer Music Conference, Denmark, 2007. al. "CNMAT International Copenhagen, [7] Wessel, D., R. Avizienis, et al. "A Force Sensitive Multi-touch Array Supporting Multiple 2-D Musical Control Structures", New Interfaces for Musical Expression (NIME), New York, 2007. [8] Wessel, D., K. Fitz, et al. "Optimizing Hearing Aids for Music Listening, 19th International Congress on Acoustics, Madrid, Spain, 2007. [9] Zbyszynski, M., M. Wright, et al. "Design and Implementation of CNMAT's Pedagogical Software", International Computer Music Conference, Copenhagen, Denmark, 2007. [10] Zbyszynski, M., M. Wright, et al. "Ten Years of Tablet Musical Interfaces at CNMAT", New Interfaces for Musical Expression (NIME), New York, 2007.