Page  00000089 GHI PROJECT: NEW APPROACH FOR MUSICAL INSTRUMENTS Yoichi Nagashima Shizuoka University of Art and Culture Faculty of Design, Department of Art and Science ABSTRACT This is a report of research project about developing musical instruments for interactive computer music. The project's name - "GHI project" means that "It might be interesting that musical instrument shines, isn't it?" in Japanese. I examined the essences of instruments in my composition again on proverb "Taking a lesson from the past." First of all, I examined the relation between the performer, musical instruments and the audiences in my composition. I confirmed two of the relations: (1) classic style and (2) standard computer music (sensor/interface) style. Afterwards, I proposed the relation thirdly from a new aspect. At the first step, my project targeted and chose "Kendang"- the traditional musical instrument of Indonesia. I produced the new instrument called "Cyber Kendang" which shone by itself with human performance. I composed a new piece featuring this instrument and performed at some concerts. Finally I discussed the new aspect of this project. 1. INTRODUCTION Up to now, I have developed new musical instruments by the approach of installing the sensors into many kinds of traditional musical instruments [1-7]. Figure 1 was called "Hyper-Pipa," which was a Chinese traditional instrument with 3-D acceleration sensors, gyro sensor and vibration sensor [4]. Figure 2 was a special sensor for SHO which is the Japanese traditional instrument [3]. Besides this, there were many interesting reports of developing new musical instruments in the past ICMC and NIME, etc. [8-20]. materials. However, there is a serious problem in the approach of installing the sensor into traditional musical instruments. There is a possibility of changing the condition of the sound even if it is a very small hole to install the sensor, because natural musical instrument generates the sound by the film/string vibration and the resonance vibration. There is also an acoustic problem when bonding to install the sensor and the electric circuit. Figure 2. SHO sensor 2. GHI PROJECT First of all, I examined the relations between the performer and musical instruments and the audiences in my past compositions. I confirmed two of the relations: [A] claasic style (Figure 3) and [B] standard computer music style (Figure 4). Performer SOUND Instrument Audience VISUAL Figure 3. Relation in "classic" music style In the music of "classic" style in my composition, both SOUND and VISUAL was radiated from musical instruments by performer's actions, and the audiences enjoyed both of them. In the standard "computer music" style in my composition, the instrument was used as Figure 1. Hyper-Pipa In the history of traditional musical instruments, I can find proven individuality and senses of existence. In working with this field, I found the different interests in the development of musical instruments by modern 89

Page  00000090 sensors of performer's actions. The output signals of the instrument were connected to the computer system as MIDI, LAN and Wireless. The outputs of the computer system were SOUND and VISUAL, and they were radiated from PA system and projectors, and the audiences enjoyed both of them. O rhaTdTE SOUND Audience " Compute r Grphics I ytmIVI SUAL~~V Figure 4. Relation in "computer music" style Afterwards, I proposed the relation thirdly from a new aspect in my composition (Figure 5). There were three policies with looking straight the essences of musical instruments in this new approach. (1) Musical instrument is a tool of music, and it is good partner for musical expression in the performance. (2) As for natural musical instruments, the sound is radiated, and not only the audience but also the player receives the sound from the whole in the space. (3) Player's expression mesmerizes an audience from not only the aural route but also the visual route. Performer SOUND InstrmentAdi Sond Syste Coqnute r Graphic s System Figure 5. Relation in the GHI project The idea of a new project arose here. The project's name - "GHI project" means that "It might be interesting that musical instrument shines, isn't it?" in Japanese. I added the function "visual information generator" to the traditional musical instrument as shown in this name, and decided not to install sensors at all. The progress of recent image processing technology is also important as the background in which the sensor is not physically installed into musical instruments. When the musical instrument shines as an expression of the music performance, CCDIVideo camera can detect the image by noncontact. The same result as past MIDI sensors can be achieved by the real-time image processing (Max/MSP/jitter). It can be said that the possibility of the expression of the media art will expand from the standpoint of the music aesthetics by this approach. The policy of not physically installing the sensor excludes the possibility that the hole and the adhesive damage the sound characteristics of the traditional musical instruments. 3. TARGET: KENDANG I chose a drum called "Kendang" as the first target. Kendang (Javanese: Kendhang) is the primary drum used in gamelan music. It usually is placed on stands horizontally and hit with both sides. One side is generally larger than the other, but I used the same size type. The skin is typically made of goat or buffalo, stretched on y-shaped leather or rattan strings, which can be tightened to change the pitch of the heads. In archaic gamelan ensembles, the Kendang was hit with a stick, but I played with my hands. I performed the multimedia music with having this instrument from the neck by using the strap. Figure 6 shows the arranged musical instrument "Cyber Kendang." Figure 6. "Cyber Kendang" Figure 7 shows the front of "Cyber Kendang." When I met this Kendang for the first time, I was inspired by the pattern of the skink at the center of the front. So, I made the pattern of this skink a leading part in the stage of the design. This instrument has 5 blocks (total 304) of 90

Page  00000091 LEDs. Eight lines in the center part are composed of high luminance blue LEDs, and they shine corresponding to the power by which Kendang is beaten. High luminance white LEDs of the remained four blocks (240) continue displaying automatically the geometrical patterns. The brightness changes corresponding to power by which Kendang is beaten. and performed the work at "Media Art Festival 2006" in SUAC. Title of the new work was "Cyber Kendang." The key concept was not to play the musical instrument according to the computer, but to drive the entire acoustics and graphics by playing musical instrument (sound and light). I composed the whole of the work with Max/MSP/jitter environment. The image of the performer on stage was captured with video camera, and was processed by jitter in real time. This live image became an important element of the graphics part projected from the projectors with pre-produced movies. Figure 7. The front of "Cyber Kendang" 4. SYSTEM An important point in the development step was "not to make any holes and not to bond anything to the musical instrument." As the result, I could return it to former musical instrument by always detaching the system. PCB (printed circuit board), used with general electric circuits, was not able to be used for this condition. I combined the wires plated with tin like the mesh, and made this the frame equipped with the electric circuits. All of the 320 high luminance LEDs were soldered directly onto this meshed frame. I connected the frame to the string that pulled the skin of Kendang with the wire. With this method, the hole and the adhesive are unnecessary, and detaching is easy. Forty LED driver/latch ICs (74HC574) were stacked by each other for high density assembly. The system was constructed with AKI-H8 (CPU: Toshiba 32bits, 8ch 8bits A/D, USART, I/O ports). The capacitor microphone was attached on the part on the edge of Kendang with the double-faced tape. This tape can be easily peeled off in the weak force. The sound of Kendang was detected by this microphone, and processed with the software of AKI-H8 as a parameter of the change in the brightness of LEDs. I dared not to adopt though it was technically easy to output this information as MIDI. A big problem remained about Power Supply. Finally I decided to connect a small power supply unit and dragged it. Because the important feature of this system was the expression by 320 LEDs, and when they lit at the same time, the supply of +5V 5A or more was necessary. 5. PERFORMANCE As a Computer Music researcher, developing new musical instruments is one part of my composition. I composed a new work featuring the "Cyber Kendang," Figure 8. Dual 3-D acceleration sensor At first, I planned - the result of image processing would trigger and control all parameters in music. However, I changed this plan because of the latency about image processing in jitter. I prepared additional sensor, called "Dual-3D acceleration sensor" which send MIDI information of [x, y, z] acceleration data of both hands. In the performance of "Cyber Kendang," live image was projected to three screens. The appearance of the luminescence of this instrument was captured with video camera and was processed and superimposed in real time. Moreover, the brightness information was used for slow-changing parameters in music. Figure 8 shows the performance of "Cyber Kendang" in the Media Art Festival 2006 on Sat. 23 December. This work was accepted for NIME07 (New York University), and was performed in the concert session of NIME07 on Wed. 9 June. 6. DISCUSSION This time, I reported only in the first step of the GHI project. A lot of musical instruments brought up and exist in each region and the tradition all over the world. I can say that these musical instruments in the world are new objects of development and the research from the aspect of the GHI project. I examined types of musical instruments of a suitable object for the GHI project. Musical instruments with continuous sound like the violins are not so suitable. Musical instruments with percussive sound like drum 91

Page  00000092 and guitar are suitable. The visual sensitivity to timevariant information is very low compared with the aural sensitivity of human. I think it would better to generate "long-decay effect" part in music. 7. CONCLUSION I reported the "GHI project" about developing novel musical instruments for interactive computer music. At the first step, my project targeted and chose "Kendang"the traditional musical instrument of Indonesia. I also reported about the composition and the performance featuring this instrument. I want to advance this research for new traditional musical instruments to develop the possibility of new Computer Music continuously in the future. 8. REFERENCES [1] Y.Nagashima, "Real-Time Interactive Performance with Computer Graphics and Computer Music", Proceedings of the 7th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Man-Machine Systems (IFAC), 1998 [2] Y.Nagashima, "BioSensorFusion: New Interfaces for Interactive Multimedia Art", Proceedings of the International Computer Music Conference, Michigan, USA, 1998. [3] Y.Nagashima, "'It's SHO time' --- An Interactive Environment for SHO(Sheng) Performance", Proceedings of the International Computer Music Conference, Beijing, China, 1999. [4] Y.Nagashima, "Sensors for Interactive Music Performance", Workshop of the International Computer Music Conference, Berlin, Geamany, 2000, http://nagasm. suac.net/ASL/workshop/icmc2000 /index.html, http://nagasm.suac.net/ASL/berlin/index.html [5] Y.Nagashima, "Interactive Multi-Media Performance with Bio-Sensing and BioFeedback", Proceedings of International Conference on Audible Display, Kyoto, Japan, 2002. [6] Y.Nagashima, "Interactive Multimedia Art with Biological Interfaces", Proceedings of 17th Congress of the International Association of Empirical Aesthetics, Kobe, Japan, 2002. [7] Y.Nagashima, "Bio-Sensing Systems and BioFeedback Systems for Interactive Media Arts", Proceedings of 3rd International Conference on New Interfaces for Musical Expression, Montreal, Canada, 2003. [8] Greg Schiemer, "Pocket gamelan: developing the instrumentarium for an extended harmonic universe", Proceedings of the International Computer Music Conference, Singapore, 2003. [9] Ajay Kapur, Ariel J.Lazier, Philip Davidson, R.Scott Wilson, Perry R.Cook, "The Electonic Sitar Controller", Proceedings of 4th International Conference on New Interfaces for Musical Expression, Hmamatsu, Japan, 2004. [10] Stephen Hughes, Cormac Cannon, Sile O Modhrain, "Epipe: A Novel Electronic Woodwind Controller", Proceedings of 4th International Conference on New Interfaces for Musical Expression, Hamamatsu, Japan, 2004. [11] Michael Eyal Sharon, "The Stranglophone: Enhancing Expressiveness in Live Electronic Music", Proceedings of 4th International Conference on New Interfaces for Musical Expression, Hamamatsu, Japan, 2004. [12] Dan Overholt, "The Overtone Violin: A New Computer Music Instrument", Proceedings of the International Computer Music Conference, Barcelona, Spain, 2005. [13] Staut Favilla, Janne Cannon, Garry Greenwood, "Evolution and Embodiment: Playable Instrument for Free Music", Proceedings of the International Computer Music Conference, Barcelona, Spain, 2005. [14] Dan Overholt, "The Overtone Violin", Proceedings of 5th International Conference on New Interfaces for Musical Expression, Vancouver, Canada, 2005. [15] Damien Lock, Greg Schiemer, "Orbophone: a new interface for radiating sound and image", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. [16] Yu Nishibori, Toshio Iwai, "Tenori-On", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. [17] Loic Kessous, Julian Castet, Daniel Afrib, "GXtar: an unterface using guitar techniques", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. [18] Teemu Maki-Patora, Perttu Hamalainen, Aki Kanerva, "The augmented Djembe Drum - Sculpting Rhythms", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. [19] Staut Favilla, Joanne Cannon, "Children of Grainger: Leather Instruments for Free Music", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. [20] Cornelius Poepel, Dan Overholt, "Recent Developments in Violin-related Digital Musical Instruments: Where Are We and Where Are We going?", Proceedings of 6th International Conference on New Interfaces for Musical Expression, Paris, France, 2006. 92