Page  00000440 Center for New Music and Audio Technologies (CNMAT): Studio Report Richard Andrews Center for New Music and Audio Technologies (CNMAT), University of California, Berkeley email:, url: Abstract The Center for New Music and Audio Technologies (CNMAT) is a satellite of the Department of Music atthe University of California, Berkeley. CNMAT promotes the creative interaction between music andtechnology via its programs in research, education,and public performance. Established in 1993, theCenter attracts scholars from campus departmentssuch as computer science, engineering, music,psychology, and mathematics. The general publicengages with CNMAT by attending lectures, concerts, seminars and workshops. 1 Introduction In addition to our ongoing concert presentations featuring new work by invited artists, CNMAT produced a new Music and Technology seminar series. Our education program continued to expand to support new projects by students and visiting scholars, and our research program produced results in many projects while launching several new initiatives. 2 Facilities CNMAT's central Machine Room was completely remodeled this year, making room for new switches, hubs, network connections, computers, and other hardware. Our Main Room was also reorganized to allow for flexibility in positioning our Yamaha 02R mixing board and associated outboard gear. New workstations were added to our labs and offices, including new Macintosh G4's, a Titanium Powerbook, and several new computers for Linux development. Our hardware lab was outfitted with high quality measurement and testing equipment including a 100MHz mixed signal oscilloscope and power supplies. Under a grant from the central campus, CNMAT's interior spaces were completely retrofitted for seismic safety. 3 Education 3.1 Music department courses CNMAT continues to offer a popular set of courses under the Department of Music, including Musical Applications of Computers and Related Technologies, Advanced Topics in Computer Music, Introduction to Computer Music Composition, Music Perception and Cognition, directed research and independent study. 3.2 Max/MSP Nightschool Our intensive summer workshop on Max/MSP programming attracted more students than ever and featured a new daytime lab session to address student requests for additional support. 3.3 Student-taught Courses Graduate student composer Ali Momeni taught a course entitled "Environmental and Ambient Sound in Electronic Music." The course, offered to undergraduate and graduate students, introduced students to signal processing techniques and their application to sound material gathered during field recording sessions. 3.4 Murota project Professor Edmund Campion, choreographer Carol Murota, and graduate student composer Ali Momeni produced a 25-minute collaborative theatre project with live interactive electronics, dance and actors. 3.5 CNMAT's Music and Technology Seminar A new seminar on Music and Technology was launched this year and featured a number of thoughtprovoking presentations by people such as Bob Ostertag (Why Can't Computer Music Be Better?), Bernard Mont-Reynaud (Steps Towards an Intelligent Editor of Musical Audio: What Has Changed in Twenty Years?), James A. Moorer (New Directions in Spatial Audio), Laetitia Sonami (The Lady's Glove and Recent Works), Roger Linn (new electronic 440

Page  00000441 music products), Xavier Rodet (Sound Analysis/Synthesis: latest results from IRCAM), inventor Don Buchla, composer Georg Hajdu, graduate student composer Ali Momeni, and Professor John Wawrzynek and Research Specialist John Lazzaro from UC Berkeley's computer science department. 3.6 Other Other presentations included composer James Dashow discussing his work on The Dyad System and representatives from Ircam presenting some of their recent work. 4 Performance and Composition 4.1 Edmund Campion Professor and CNMAT composer-in-residence Edmund Campion had a number of works performed during the year, including the Earplay Ensemble's presentation of "Mathematica III" for flute and live electronics with Tod Brody, flute; "Natural Selection" with Mei Fang Lin, piano, at Ball State University College of Fine Arts; "What Goes Up..." for ensemble and live electronics performed by the American Composer's Orchestra in New York City; and "Corail" for saxophone and live electronics performed at the AGORA festival in Paris. 4.2 Student concerts New works featuring electronics were realized by graduate student composers Mei-Fang Lin, Dmitri Tymoczko, Alan Tormey, Roberto Morales, Ali Momeni, and David Bithell. 4.3 Other concerts Other concerts included Pauline Oliveros with George Marsh (percussion), Jennifer Wilsey (percussion), and David Wessel (electroacoustics); pianist W. A. Mathieu and percussionist George Marsh presenting "Improvisations"; electro-acoustic works by Stanford composer Mark Applebaum; new works by a group comprised of Louis Sclavis (clarinet), Cecile Daroux (flute), Nicolas Verin (electronics), and David Wessel (electronics); an evening with Shafqat Ali Khan (Pakistani Khyal vocals), Matthew Wright (live electronics), and Salar Nadir Khan (tabla); the Barre Phillips String Trio featuring Barre Phillips (bass), Hans Burgener (violin), and Martin Schiiltz (cello); and Ben Goldberg's Brainchild, featuring Scott Amendola, John Schott, Will Bernard, Ashley Adams, Matthew Wright, Matt Brubeck, Graham Connah, Dan Plonsey, and others. CNMAT composer Ronald Bruce Smith had his piece "Return to Breath" premiered in the U.S. by the San Francisco Contemporary Music Players as part of their series Zeroes and Ones. Guitarist/composer John Schott, an integral part of CNMAT's guitar-related research, presented his NEW WORK series that featured New Music for New Quartet featuring New Technology in collaboration with Matthew Wright. 5 Research 5.1 Guitar project CNMAT's guitar-oriented research, supported by the UC Digital Media Innovation Program (DiMI) and Gibson Guitar Corporation, produced a number of results this year, including a new, modular version of our programmable connectivity processor that has a Gigabit ethernet port, 8-channels of D/A conversion and support for two daughter boards for analog input and gestural transduction input (a family of daughter boards is being developed for hex guitar input, a high performance keyboard, and general purpose transducers); the integration of the above scalable connectivity processor into composition and performance sub-projects, including a new software interface for setting analog preamp gains and a revised scheme for Max/MSP patches that handle MIDI to make them work with mididecode~; "The Listener"": pattern recognition of Axon MIDI output with finite state machines (related to score-following in that the computer is "listening" for certain precomposed material, but rather than a single score there are dozens of specific pitch and rhythmic patterns that are being matched); "Rhythmic Guitar Samples": a sample library of rhythmic events played on a variety of guitars; the reorganization of our effects software into a library of reusable components; "The Great 48": live sampling of the guitar based on the Axon AX-100 guitar-to-midi converter's estimate of when notes start and stop (features a buffer associated with each of the 48 pitches that can be played on the guitar -- each time a note is played that is long enough and loud enough, that note is recorded into the appropriate buffer, allowing one to play any arbitrary pitched material and each note played will be a sample of the mostrecently- played instance of that note on the guitar; transfer of musical material from Finale notation software to Max/MSP for live playback via "Great 48" samples or "rhythmic guitar samples"; inferring tempo (e.g., to determine how fast to play sequences) from a series of rhythmically even notes played on the guitar; and using the Tactex MTC Express to draw breakpoint functions of time to control the evolution of parameters such as fuzz amount, delay time, etc., over time. 5.2 Loudspeaker Arrays With the assistance of computer science graduate student Peter Kassakian and in collaboration with Meyer Sound Laboratories, we have examined the behavior of approximations to a spherical array of transducers with a dodecahedron array and studied 441

Page  00000442 the behavior of straight and curved line arrays. In both cases fine grain measurements of radiation patterns in an anechoic chamber play and essential role.EThis preliminary work has led to a new Digital Media Innovation (DiMI) grant for a collaboration with Meyer Sound Laboratories. This new work will concentrate on the development of design principles for large arrays of transducers. The emphasis is on adaptive techniques to control radiation patterns and interactions with the room. The adaptive techniques include LMS and convex optimization. 5.3 Assisted Acoustics Kent Nagano, conductor and musical director of the Berkeley Symphony Orchestra, asked CNMAT if we could liven up the dry acoustics in Berkeley's new Roda Theatre. In collaboration with Level Control Systems and Meyer Sound Laboratories, a Variable Room Acoustics System (VRAS) based on research by Marc Poletti was installed and tuned for the theater. The goal of the assisted acoustics system was twofold: to enhance the experience for the audience and to provide the players with an electronic shell that would enable them to hear themselves and each other more effectively. 5.4 Scalable Reactive Computing Cluster for Music Performance We are developing and tuning a low-cost, loosely coupled computing cluster for musical applications. Communications, audio, and gestural I/O is handled by a new programmable connectivity processor and a Gigabit ethernet switch. Hardened small-form-factor PC's are used for each node. 5.5 AudioIcon 2002 Supported by a grant from the UC Digital Media Innovation Program (DiMI), CNMAT produced Audiolcon 2002, a research-oriented workshop on representations for digital audio. The new workshop featured a keynote address by Gerry Kearby of Liquid Audio and included experts from several UC campuses and key figures from the audio industry. The topics focused on representational issues for the audio medium, including media authoring and editing in a world of competing formats, spatial audio, sound analysis and synthesis, and audio meta-data databases, asset management and music discovery. The goal of Audiolcon is to stimulate audio-related research collaborations that cut across campus and industry boundaries. 5.6 1394 (firewire) Wireless Standard CNMAT participated in the Wireless working group session of IEEE1394 (firewire) 2002 first quarterly meeting and hosted an interim meeting. We are contributing towards a wireless standard that would allow for synchronizations of low-jitter audio clocks across wireless bridge 1394 networks. 5.7 Musical Applications of Machine Learning Guitar Pitch Prediction: Encouraging results have been obtained from preliminary experiments concerning a pitch prediction system for the guitar using Support Vector Machine regression. Here the goal is to use only a fragment of the initial attack of the guitar note to predict the eventual pitch of the note. This work is being carried out by mathematics student Andrew Schmeder under the direction of David Wessel. Another issue is the combination of pitch detection and segmentation into one optimization. 5.8 Applications of Factorial Hidden Markov Models Computer science graduate student Brian Vogel is applying Factorial Hidden Markov Models to the problems of polyphonic pitch extraction and source separation. Such models pose are not tractable with the standard EM algorithms. Vogel is applying approximate inference techniques using Monte Carlo methods such as the Particle Filter. 5.9 Model Inversion Following the earlier work at CNMAT of Wessel and Drame we have continued to explore analysis synthesis techniques that analyze musical signals to obtain a control structure for a signal model such as sinusoidal tracks. Given the success of the preliminary experiments with feed-forward neural networks, we are exploring the use of a mixture of experts models and Support Vector Machine regression. 5.10 Internet 2 project With Pauline Oliveros at Rensselaer Polytechnic Institute we exploring the use of Internet 2 for master classes and performance. 6 Personnel Richard Felciano, Founder; David Wessel, Director; Adrian Freed, Research Director; Matthew Wright, Musical Systems Designer; Edmund Campion, CNMAT Composer-in-Residence and Assistant Professor, Music; Richard Andrews, Associate Director. CNMAT's list of researchers includes Rimas Avizienis, Richard Dudas, Alexander Jensenius, Peter Kassakian, Ahm Lee, Dominique Richard, Ron Smith, Takahiko Suzuki, Andrew Schmeder, and Brian Vogel. Our roster of graduate student composers includes Mei-Fang Lin, Dmitri Tymoczko, Alan Tormey, Roberto Morales, Ali Momeni, Keeril Makan, and David Bithell. 442

Page  00000443 7 Acknowledgements CNMAT gratefully acknowledges the support of our corporate sponsors and Industrial Affiliates Program members, including Gibson Musical Instruments, Grace Design, Yamaha, Silicon Graphics, Apple Computer, Meyer Sound Laboratories, Wacom Technology Corporation, Digidesign, Earthworks, Kurzweil/Young Chang, Octiv, and Tom Austin/Sherman Clay. We would also like to acknowledge the support of the UC Berkeley Consortium for the Arts, the Cultural Service of the French Consulate in San Francisco, and Cycling 74. 8 References R. Avizienis, A. Freed, T.Suzuki, and D. Wessel (2000),"Scalable Connectivity Processor for Computer Music Performance Systems," proceedings of the International Computer Music Conference, Berlin, Germany. J. Baird, J. Meyer, P. Meyer, P. Kassakian, and D. Wessel (2001),"Controlling Loudspeaker Array Radiation in Three Dimensions," 11Ith Audio Engineering Society Convention,New York,NY. A. Freed and R. Avizienis (2000),"A New Music Keyboard featuring Continuous Key-position Sensing and Highspeed Communication Options," proceedings of the International Computer Music Conference,Berlin, Germany. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola,and L. K. Saul (1999),"An Introduction to Variational Methods for Graphical Models," in Learning in Graphical Models,M. I. Jordan, Ed. Cambridge:MIT Press,pp.105-161. M. Poletti "Variable Room Acoustic System," theory. htm. B. Sch6lkopf and A. J. Smola (2002), Learning with Kernels.Cambridge:MIT Press. D. Wessel, C. Drame, and M. Wright (1998), "Removing the Time Axis from Spectral Model Analysis-Based Additive Synthesis:Neural Networks versus MemoryBased Machine Learning," proceedings of the International Computer Music Conference, Ann Arbor, Michigan. 443