Page  00000413 Georgia Tech Music Technology Group - Studio Report Gil Weinberg, Jason Freeman, Parag Chordia, Frank Clark, Chris Moore, Scott Driscoll, Travis Thatcher Music Department Georgia Institute of Technology gilw, jason.freeman, parag.chordia, frank.clark, chris.moore, scott.driscoll, travis.thatcher@ Abstract The report presents new developments at Georgia Tech's Music Technology Group. We discuss our new facilities, faculty members, academic programs, and research initiatives, as well as a number of creative activities, workshops and concerts that have recently taken place in the department. We conclude with a brief discussion of several future initiatives. 1 Overview The Music Technology Group (MTG) at Georgia Tech, located within the Music Department in the College of Architecture, is rapidly expanding to form interdisciplinary research initiatives and new degree programs to explore the connections between technological developments and musical practices. 2 Personnel Over the last four years, the Music Department has hired four new faculty members to establish its new initiatives and academic degrees in music technology. Frank Clark was appointed in 2002 as Director of the Music Department, with practical experience in program development and expertise in blending the traditional with the technological. In the following year, Gil Weinberg was hired as Director of Music Technology with the goal of establishing new academic degrees. In 2005, Parag Chordia and Jason Freeman were appointed to bring expertise in additional areas such as music perception, music information retrieval, networked music, and composition. These new faculty members joined Chris Moore, who has been teaching music technology classes for non-majors since 1996. The group also secured funding for two graduate students assistants, Scott Driscoll and Travis Thatcher, who are currently enrolled in an undesignated masters program with concentration on music technology. The students are planning to transfer to the M.S in Music Technology when it begins in Fall 2006. 3 Facilities In 2002 and 2003, the Couch Building, home to the Music Department, received over $650,000 in physical upgrades and renovations. During that same time, a state-ofthe-art computer music teaching lab was provided through institutional funds for instructional technology. In 2005, further renovations to the building created additional office space and a research lab. The lab, funded as part of a $100,000 grant from the Provost's Office, was designed to facilitate MTG's focus on computer music and digital technology with interdisciplinary connections to electrical engineering, mechanical engineering, hardware design and development (See Figure 1). Figure 1. Georgia Tech's Music Technology Research Lab 4 Academic Program Music at Georgia Tech has traditionally occupied a cocurricular role; a music minor program does exist, but there are currently no undergraduate or graduate degree programs. In February 2006, Georgia Tech's Academic Senate has approved the first academic degree in the department - an M.S. in Music Technology. This is a two-year program whose objective is to provide students with the practical skills and theoretical understanding needed to be leaders in the design, development, and creative implementation of music technology products and services in the coming decades. The program, scheduled to begin in Fall 2006, is based on our belief that successful design and development of music technology systems must be supported by knowledge of music literature, theory, perception, composition, 413

Page  00000414 acoustics, and performance, as well as digital media, computer technology, electrical engineering, human behavior, and design. The program will offer two different concentrations: Computer Music Research and Engineering, and Music Production. The first concentration is scientific and technological in nature, focusing on the research in areas such as music perception, computational modeling, and the design and development of novel enabling music technologies. The second concentration is productionoriented and will focus on creative utilization of current music and media technologies with an emphasis on creative work in recording, multimedia, and production. The program's curriculum includes core courses focused on perception and cognition, history and analysis, interactive music, and performance, elective courses in both music and related disciplines, and ample opportunities to pursue individual and collaborative research and projects. 5. Research 5.1 Networked Music The MTG is focused on a variety of research areas within networked music, exploring technologies and techniques to connect trained musicians and non-musicians over both local and wide-area networks. Gil Weinberg is extending his previous work in local-area music networks for non-musicians. In addition to developing a theoretical framework for such systems (Weinberg 2005), he and student Travis Thatcher have built new hardware and software into the Beatbug instruments (Weinberg 2004) to detect an expanded repertoire of physical gestures. In his iltur series - a set of three musical compositions for a Jazz ensemble and Beatbugs - Weinberg introduces a novel method of interaction between acoustic and electronic instruments. Beatbug players can now record live input from acoustic and MIDI instruments and respond by transforming the recorded material in real time, creating motif-andvariation call-and-response routines on the fly. A central computer receives MIDI and audio data from both acoustic and electronic instruments, it computes the improvisation algorithms, and facilitates the interaction among players. In addition to piezo electric sensors, bendable antennae, speakers and white and colored LEDs, the new Beatbugs also contain accelerometers, providing players a new range of expressive gestures for sound manipulation and transformation. Jason Freeman continues to explore networked music systems that facilitate audience collaboration with professional musicians in live performance. Flock, a new evening-length performance piece under development for the Miami Performing Arts Center in collaboration with the Miami Saxophone Quartet, video artist Liubo Borrisov, and students Martin Robinson and Travis Thatcher, uses a wireless 802.11 positioning system to track the positions of 75 audience members relative to each of four saxophonists as both audience and musicians move around the performance space. The data is algorithmically transformed into performance instructions - a combination of traditional and graphical notation shown to the musicians on wireless handheld displays - and also used to drive realtime video animation shown on multiple video displays. location Computer L performance -data Software nstructions location data visual visual Ss visua visual \. " " - sound -" " " Figure 2. Conceptual structure of Jason Freeman's Flock Freeman is also interested in moving such musical networks out of the concert hall and out of real time. Graph Theory, a project being created with designer Patricia Reed and violinist Maja Cerar for New Radio and Performing Arts and their Turbulence web site, seeks a common ground between web-based collaboration, live concert performance, and composition. Web users navigate through a visual structure of connected musical fragments, structuring their own path through a musical composition for solo violin. Their decisions are not only reflected to future users of the site; they also influence future live concert performances of the work. Networked music initiatives involving cellular phones are also underway. In a joint effort with researchers from Georgia Tech's Information Design & Technology group, development has begun on a collaborative sequencing project entitled Sequencer404. The goal of the project is to create a multi-user environment over phone networks (both landline and cellular) where participants can compose musical structures by editing a constantly cycling sequencer that resides on a server. Networked music is also an increasingly important part of our curriculum. We offered a masters-level course in networked music for the first time in fall 2005, focusing on local-area improvisatory laptop networks for musicians, on Internet and telephone-based shared sonic environments for novices, and on techniques for network sonification. 5.2 Music Information Retrieval Our current research in music information retrieval approaches the field from unique perspectives. Parag Chordia's work focuses on automatic transcription of tabla, a ubiquitous North Indian percussion instrument associated with a sophisticated improvisation-based system focused on timbre and rhythm (Chordia 2004 and 2005). Chordia attempts to teach a machine to perceive this timbral and rhythmic structure, furthering research in automatic 414

Page  00000415 transcription, creating representations of tabla performances that can be used for analysis, and allowing the musical patterns of tabla music to form the basis for new creative works. Chordia is also collaborating with David Huron (Ohio State) on a statistical learning study, which seeks to account for some of the phenomenal experiences evoked by listeners of Hindustani music. In both Western and non-Western music, listeners report that tones variously evoke a sense of expectation, anticipation, surprise, instability, inappropriateness, poignancy, strength, energy, repose, etc. In different contexts the same pitch may evoke dramatically different qualia. Chordia and Huron's research asks what contextual properties contribute to the distinctive feelings evoked by a tone. Freeman has also incorporated ideas from MIR research into an Internet-based software project, iTunes Signature Maker (Freeman 2006). Drawing from MIR research in algorithmic playlist generation, music summarization, and feature-driven audio editing, the software stitches together small segments of songs to create a concise audio signature, using an algorithm driven by spectral features intrinsic to the audio files themselves and by environmental features describing how those files have been used. strengths of computational and mechanical operations with human creativity and expression, can lead to novel musical outcomes. Haile can, therefore, serve as a test-bed for novel forms of musical human-machine interaction, bringing perceptual aspects of computer music into the physical world both visually and acoustically. For the new twohanded version of Haile developed by Driscoll, Weinberg composed a new piece, titled Jam'aa, for a Middle Eastern percussion ensemble and a robotic percussionist. The piece was commissioned by Hamaabada Art Center and is scheduled to premiere on March 2006 in Jerusalem. 5.4 Sonification The MTG is extending its research into sonification of neural activity as demonstrated in the "Fish and Chips" and "BrainWaves" projects. We base our work on the assumption that sonification can serve as an effective technique for the representation of complex spatial information such as neural activity due to the auditory system's ability to perceive stimuli at a wide spatial cover and its inclination to perceive spatial patterns in sonic input. Our recent collaboration with the Bioengineering department at Georgia Tech provides us with highresolution spatial data of neural activity in a culture. For this system we developed an interactive application and improved our technique, artistic approach, and interaction design in comparison to the low-resolution graphical user interface in our previous efforts in this area. Figure 3. Jam'aa in rehearsal 5.3 Robotic Musicianship Weinberg, Driscoll and Thatcher continue to develop the concept of robotic musicianship, combining computer models for musical, perceptual, and interaction skills with the capacity to produce rich acoustic responses in a physical and visual manner. The first robotic percussionist prototype, named Haile (Weinberg and Driscoll 2005), can listen to live players, analyze perceptual aspects of their playing in real-time, and use the product of this analysis to play along in a collaborative and improvisatory manner. It is designed to combine the benefits of computational power, perceptual modeling, and algorithmic music with the richness, visual interactivity, and expression of acoustic playing. We believe that such human-machine collaboration, which combines the Figure 4. A player interacts with the "BrainWaves" controller, triggering sound patterns in space based on probabilistic occurrence of neural spike in the brain 5.5 Multimedia Performance Incorporating multimedia into performance continues to be addressed in both creative work and teaching. In January, Frank Clark performed a new composition, Images of Air and Light, at the 2006 Annual Hawaiian International Conference on Arts and Humanities in Honolulu. The work featured real-time control of video, animation, and graphics using a combination of Electronic Valve Instruments built 415

Page  00000416 by Nyle Steiner and custom control surfaces supported by Max/MSP. The composition explores the time-honored elements of air and light from a variety of cultural perspectives utilizing a broad range of world-music sounds, tools, and techniques. Greek, Indian, Chinese, Korean, and Medieval European symbols, metaphors, and aural references are combined with original images and music throughout the composition. Many of the interactive tools and techniques utilized in the work are being repurposed and/or extended by students in Clark's spring 2006 course, Multimedia Performance Systems. On January 26th the Georgia Tech Symphonic Band premiered a new work by Chris Moore at the Georgia Music Educator's Conference in Savannah, Georgia. Unyielding Dispersions is a soundscape depicting the three stages of water. The piece creates an aural/visual experience through the use of various tonal and rhythmic concepts. As the title describes, the music takes the listener from obstinate, stiff content to an unsettled form. The first section, Ice, uses harsh tonalities and rhythmic stability. Water is the second section with fluid and intertwined lines. Finally, Steam completes the transformation with transparent effects and harmonic tendencies. The electronic drum kit plays a prominent role as a multiple percussion instrument and triggering and affecting video and still images of the three elements. Moore was assisted by students in the Electronic Percussion Studio in creating the interactive graphics and live video for the composition. 6 Concerts and Workshops In the last couple of years, the music technology group has produced and presented a number of events to complement our research and curricular activities. One of the highlights was a sold-out concert as part of our "Listening Machines" series at the Eyedrum Music and Art Gallery in Atlanta in January 2005, presenting performances and demonstrations by music technology students and faculty. The underlying theme of the concert was the question of whether machines can be responsive, surprising, and sensitive musicians. The concert included a sonification installation, an interactive piece for a robot and a human drummer, an interactive piece for a saxophone and drums, a multimedia piece for a quartet of drum heads, and a two compositions for beatbugs and a Jazz ensemble. The "Listening Machines" series continued in March 2006 with a concert at Hamabada Art Center in Jerusalem, Israel. The concert was the culmination of a week of lectures, demonstrations, and workshops for professionals, children, and the general public. It included Jason Freeman's Glimmer, in which the audience uses light sticks to shape the music played by a chamber orchestra; Gil Weinberg's iltur, in which jazz musicians and Beatbug players interact; and the premiere of Weinberg's Jam'aa, which integrates a robotic drummer with a human drum circle. On campus, we have sponsored lectures and workshops by Laurie Anderson and Tristan Murail in connection with concerts of their music in Atlanta. The Music Department also sponsors the Atlanta chapter of dorkbot, an international forum dedicated to "people doing strange things with electricity" that brings informal talks about a variety of interdisciplinary arts and technology topics to a diverse audience each month. 7 Future Plans A number of new initiatives are currently under development. Academically, we are in the early stages of designing a joint undergraduate degree - B.S in Computational Music - with the College of Computing at Georgia Tech. The degree will offer a combination of computer science and computer music classes, preparing students for careers in the music software and hardware industry. In collaboration with the GVU center and the College of Computing, we also plan to initiate in Fall 2006 an ensemble in residence, performing chamber music that uses technology in innovative ways. The ensemble, named Sonic Generator, is comprised of a core group of classical musicians, many of whom are members of the Atlanta Symphony Orchestra. It will present a series of concerts that will include classic works of electronic and computer music, instrumental works whose presentation is enhanced by multimedia technology, and works developed specifically for the ensemble by Georgia Tech faculty. In 2006, we also expect to upgrade the equipment in our teaching lab through additional institutional funds for instructional technology. Renovation of our recording studio is also a top priority, which will allow us to offer the Music and Audio Production concentration of our graduate degree. References Chordia, Parag (2005). "Segmentation and Recognition of Tabla Strokes." In Proc. of the 6th International Conference on Music Information Retrieval (ISMIR), pages 107-114. Chordia, Parag (2004). "Automatic rag classification using spectrally derived tone profiles." In Proc. of the 2004 International Computer Music Conference (ICMC). Freeman, Jason (2006). "Fast Generation of Audio Signatures from iTunes Libraries." Journal of New Music Research, 35 (1) (in press). Weinberg, G. and Driscoll S. (2005) "Robot-Human Interaction with an Anthropomorphic Percussionist" In Proc. of the International Conference of Human Computer Interaction (CHI 06). Montreal Canada. Weinberg, G. 2005. Interconnected Musical Networks - toward a theoretical framework. Computer Music Journal 29(2). Weinberg G., Driscoll S. (2005) "Iltur - Connecting Novice and Experts Through Collaborative Improvisation" Proc. of Conference on New Interfaces for Musical Expression (NIME 2005) Vancouver, Canada, pp. 17-22 416