Page  00000001 Georgia Tech Music Department - Studio Report Gil Weinberg, Frank Clark Music Department, Georgia Tech gil.weinberg@music.gatech.edu, fclark@music.gatech.edu Abstract The renewed Music Department at Georgia Tech, its new research center, and the proposed graduate program in music technology are presented. Considerations in defining the research focus and curriculum of the program are discussed and a number of current research projects are described. 1 Introduction The Music Department at Georgia Tech has recently undergone a major expansion process both in content and facilities. The department, uniquely housed in the College of Architecture, has a newly renovated facility with new computer and recording labs, several new tenure track faculty positions, and a new research focus on technology for performance and education. The department is also in the process of starting a graduate degree program in Music Technology (planned to open fall of 2005). The College of Architecture and the Georgia Institute of Technology have been supportive in allocating funding and resources for the development of the music technology program in an effort to position it as a leading research and academic center. 2. Facility The Couch Building, home for the department, has undergone a major renovation - a new computer lab has been installed for undergraduate classes in music technology, computer music, and multimedia (See Figure 1). The fifteen G4 Macintosh-based stations are fully equipped with mixers, keyboards, MIDI and audio interfaces, sound modules, and a package of commercial software applications. A new professional Pro-Tools recording studio is being constructed (See Figure 2.) The studio includes a closedcircuit video system, which supports full orchestra recordings from the adjutant rehearsal hall. rigure 1. iine new computer music tao nosts cilasses in music technology, computer music, and multimedia. Figure 2. The new Pro-Tools recording studio 3. Research 3.1 Considerations in defining research focus In order to define the nature and directions of research for the new center, the faculty examined a number of internal and external factors. These included the positioning of the department in the College of Architecture, nationally and internationally recognized peer academic departments Proceedings ICMC 2004

Page  00000002 for comparison, and recent trends in the industry. Based on these studies, the department has decided to focus on two main research areas: performance technology and educational technology. Below are some of the considerations leading to this decision: Internal Institutional Factors - As part of the College of Architecture, the Music Department has access to a number of unique resources that can help form a distinctive program. Currently, the department is collaborating with programs in the College of Architecture such as industrial design, space design and acoustics, advanced wood manufacturing, and assistive technologies. These collaborations support our focus on technology for musical performance. Examples of this include the development of new instruments and the design of new technology-rich performance spaces. Additional collaborations with other acclaimed programs within the institute, such as the nationally recognized programs in electrical engineering, computer science, and the center for graphic utilization and visualization, will serve as additional assets. To this point in time, the main focus of the Music Department has been cocurricular; providing creative outlets such as ensemble performance and classroom opportunities to enrich the musical education of the general student body. These important strategic goals, which are strongly supported by the college and the university, will be significantly enhanced by our focus on technologies for performance and education. We fully expect new developments from the research center to be implemented in classrooms and concert halls on a regular basis. Peer institutions comparison - As models for comparison, we investigated a number of successful music technology programs in peer institutions such as Stanford, Berkeley, University of Miami, McGill University, IRCAM and the University of Padova. We identified a number of research directions and foci pursued by these institutions. They include: composition, digital signal processing, recording, synthesis and performance. We did not identify a research center and a graduate academic program specializing in performance and educational technologies. This suggests that the Music Department at Georgia Tech can offer a unique and attractive program for students and researchers. Industry Factors - one of the first questions asked by any administration when approving a new program concerns employment opportunities in the field for graduates. In surveying the industry and interviewing a number of corporate leaders we found that there is a growing interest in music technology experts in fields such as performance technology and musical software and hardware for novices and the general public. Developments in the performance industry extend the role of music technology to a wide variety of genres, which require technologically literate musicians and artistically creative engineers. Additionally, the professional audio industry has undergone a major reorganization process in recent years, as large multimedia, computing, and consumer product corporations merged with or acquired professional high-end music technology companies. Examples of this include Creative Technologies acquisition of E-mu and Ensoniq, the Avid merger with Digidesign, and Apple's acquisition of E-magic. These developments suggest that industry seeks to develop markets in a variety of areas: the general public, novices, multimedia, and education. We believe that these areas do not receive enough academic attention (which in many cases is focused on developing technology for experts and high art) and that an exciting opportunity exists to establish a distinctive research center in these areas. 3.2 Ongoing Projects A number of faculty members in the department are engaged in ongoing research projects supporting the overarching focus on performance and education. Below is a brief summary of some of these projects. "iltur" - new instruments for collaboration and improvisation. "iltur" is a series of compositions by Gil Weinberg for a Jazz ensemble and a set of new electronic instruments called Beatbugs. Beatbugs players can record live input from acoustic and MIDI instruments and respond by transforming the recorded material in real time, creating motif-and-variation call-and-response routines. "iltur" allows players to respond and modify their peers' musical ideas in real time. Players gain direct control over their peers' music and personalize it by manipulating elements such as rhythm, pitch, and expression. The music that is created is a crossbred hybrid product of a unique shared collaboration and improvisation Figure 3. "iltur" studio session In "iltur 1" the first composition in the series, two Beatbug players control two different algorithms for rhythmic ornamentation and transposition of musical material played by a MIDI piano. The composition begins with a melody played by the piano and repeated in transformation by the Beatbugs players who can control its expressive envelope (through hitting the instrument and bending its two bendsensor antennae). The piece then moves to a section where the Beatbug players record short segments of an improvised piano solo (by tapping the instrument while the antennae are bent) and can then trigger these captured phrases by striking the instrument again. Players can then further transform the Proceedings ICMC 2004

Page  00000003 phrases by bending the antennae during playback. The left antenna is mapped to a rhythmic ornamentation algorithm so the more it is bent, the denser and less stable the rhythms become. The right antenna is mapped to a transposition algorithm based on a set of preprogrammed scales. In "iltur 2", one of the Beatbugs transforms audio recordings from trumpet improvisation. The audio algorithm detects and segments the trumpet solo based on event and pitch analysis. It then organizes the audio segments so that the Beatbug player can retrigger them in different orders based on pitch, velocity and/or timing. The other Beatbug player interacts with midi piano solos using a more complex algorithm based on contour and rhythmic changes so that the more the antennae are bent, the less similar the improvisation is to the original input. "iltur" team includes Gil Weinberg, Alexander Powel, Scott Driscoll, Brian Galloway, Michael Young, Chris Moore and Ron Mendola. "A Sensation of Music" - Multimedia for chamber and orchestral performance. "A Sensation of Music: Works by Walter Anderson" is a multimedia composition by Frank Clark based on visual and textual elements from the Walter Anderson Museum of Art in Ocean Springs, Mississippi. The energy, power, and intensity radiated through Anderson's pen and ink drawings, watercolors, ceramics, sculptures, and murals inspired the incorporation of sounds, colors, and rhythms in the composition. A chamber version of the piece was performed in New York at Weil Recital Hall at Carnegie Hall and the orchestral version was premiered by the Meridian Symphony (Meridian, Mississippi) in honor of Anderson's centenary. The three-movement work was designed to lead the listener through a gently guided tour of Walter Anderson's different styles. The first movement, The Voluptuous Return, blends a series of still lifes with an ABA design. The graphic representations are conservative and simulate looking at a book or strolling down a museum gallery. A Rhythm of Frogs is an aleatoric composition marrying the everchanging Gulf Coast flora and fauna with field recordings of frogs and improvised synthesizer and percussion patterns. The third movement, Invisible Music, is drawn from a poem of the same name, which can be found in the book Walter Anderson: A Symphony of Animals. Using a variety of realtime software and hardware control systems, the digital video, animations, graphics, and audio elements were created in conjunction with acoustic and electronic resources. Conceptually, it was imperative that the aural and visual elements be mutually supportive and flexible. Forcing the musicians to act as "accompanists" to a canned video stream would have undermined the spirit of Anderson's aesthetic. The next steps in exploring art-intensive interactive multimedia performances are aimed at increasing performer and artwork interaction. Max/MSP and Jitter are experimented with as controllers for visual/video software (Bliss Paint and Arkoas VJ) and devices such as Edirol's DV7-PR, V-4 Video Mixer, Roland VLINK instruments, and the Mark IV series of Yamaha Disklaviers. Two pieces employing this extended system are already underway. The first, Approaching the Magic Hour, is a multi-movement piece based on Anderson's unique Horn Island experiences, his ventures into "public art," and selected music-related poems. Particularly emphasized in this piece are the interaction between musical elements and the seven basic shapes Anderson employed throughout his watercolors and pen and ink drawings. The second composition, to be performed at the Great Hall in Beijing for the opening of the Beijing Music Festival and School (for which Clark serves as Head of the Composition/Multimedia Composition program), is entitled Song Sook Mie (Line, Speed, Beauty). This piece explores the dynamic interaction between nature and man from combined Eastern and Western perspectives. "Brain Waves" - Sonficiaton of neuron activity. With new developments in biological research, scholars are gaining more accurate information about complex systems such as the brain. However, effective mechanisms for representing this complex data have yet to be developed. Using sound and music to sonify complex systems is a technique that is rapidly getting interest in a variety of research communities. Working with Steve Potter and Douglass Bakkum from the Laboratory for NeuroEngineering at Georgia Tech, students of "Introduction to Computer Music" class, directed by Gil Weinberg, sonified signals from cultured neurons recorded by Multielectrode Array (See Figure 4). Students addressed two research goals, which at first seemed contradictory: 1. To produce a generic and meaningful representation of the data helping researchers understand the activity in the culture. 2. To create a musical product that has an aesthetic value. While working on the project students realized that using techniques for pattern recognition could help address both goals since acoustic patterns can be the key for a representative as well as an aesthetic outcome. As a final project, signals from 60 electrodes (see Figure 5) were mapped to an installation of 8 speakers in an effort to add a spatial element to the representation of the activity in the culture. The project was presented in a number of open house exhibitions at Georgia Tech. Figure 4. Multielectrode Array in a neuron culture (Courtesy of Steve Potter at Georgia Tech) Proceedings ICMC 2004

Page  00000004 3.3 Preliminary prototypes Automated Choral Rehearsal. As part of the department's focus on technology for music education and instruction, development is underway on a system to automate choral rehearsals. Our goal is to create a paperless rehearsal environment based on technology for speech recognition (analyzing instructions from the conductor) and score following (analyzing audio form the choir.) The system would present and scroll through an overhead projected score according to input from the choir and conductor. The first prototype employs only monophonic music and a Pitch-to-MIDI recognition application written in Max/MSP. The application is designed to change the speed of a QuickTime video that scrolls through a simple monophonic score. Future versions are planned to combine speech recognition and more elaborate audio analysis algorithms in conjunction with vector-based notation files. Additionally, we plan to develop educational functions for ear training including voice soloing and muting. The Automated Choral Rehearsal team includes Jerry Ulrich, Gil Weinberg, and Nicole Harris. Wireless percussive instruments. Students in the "Percussion MIDI Ensemble" class, directed by Chris Moore, are currently developing a radio frequency ballshaped device that can sense hits (using an array of piezo electric sensors), convert them to MIDI, and send wireless information to a central system. This project is developed in collaboration with students in "Introduction to Computer Music" class, which are currently designing musical applications for the device. 4. The Proposed Graduate Program The graduate program in music technology, with concentrations on performance and education, is planned to accept its first students in the fall of 2005. The program will provide an interdisciplinary academic framework and stateof-the-art research facility for students, faculty, and researchers interested in applying their musical, technological and scientific creativity for the development of innovative artistic and technological products for musical performance and instruction. The academic program will seek students with interdisciplinary backgrounds in areas such as music, computer science, design, media arts, education, and electrical engineering. Fields of study will include interactive music composition, programming for music applications, technology for music education, voice technology, music cognition and perception, instrument design, performance space design, music and audio analysis for real time performance. Students will be actively engaged in research and will have the opportunity to customize their academic program based on personal backgrounds and interests. The graduate program and the research center will be actively linked to the musical life of the community through the Music Department's instrumental, vocal, and symphonic ensembles. Technologies for band, choir, orchestra, and chamber rehearsals and performance will be explored and embedded in concerts and public events on a regular basis. In addition, a special research emphasis will be placed on contribution to the community in areas such as educational applications, musical instruments for people with disabilities, assistive technologies, and technologies for collaborative musical activities. A number of graduate classes are currently being approved and will be taught as special topics starting the fall of 2004. These include "Computer Supported Interactive Music", "Technology for Music Education", and "The Musical Mind." 5 Acknowledgments We would like to thank Dean Thomas Galloway and Provost Jean-Lou Chameau for their support. We would also like to thank Steve Potter's group for their help. 6 Final Note Can computers be responsive, surprising, and sensitive musicians? Can musical instruments serve as expressive guides for music and creativity? Can we use music to better communicate and understand the world? We hope that the new research center and the proposed graduate program in music technology at Georgia Tech will be positioned to address these and similar research problems, helping to define the future of music performance, expression and education. Proceedings ICMC 2004