Page  00000357 RENCON: toward a new evaluation method for performance rendering systems Rumi Hiraga', Mitsuyo Hashida2, Keiji Hirata3, Haruhiro Katayose4, Kenzi Noike5 'Bunkyo University, 2Chiba University of Commerce, 3 NTT Communication Science Research Lab, 4Kansei Gakuin University/PRESTO, JST, 5PRESTO, JST Abstract Evaluation of computer music research from a system point of view has not been established because of the subjective nature of music. In order for such research areas to evolve and prosper, we believe that an evaluation method based on objective understanding is essential. We started and propel RENCON (performance rendering concours), a world-widelandmark-project to be, in order to seek and propose an evaluation method for music systems taking performance rendering system as the typical research of manipulating Kansei information. Musicians and musicologists also expect to see the result of the project in how to describe music analysis and cognition objectively. Furthermore by letting people be widely conscious of the project, we hope RENCON to enlighten people on the computer music research, encourage them to participate in it, and empower the research itself. In this paper we will report the activities of RENCON and its majestic goal. Keywords: RENCON, performance rendering, evaluation, Kansei, natural/artificial performance 1 Introduction A performance rendering system automatically generates musical performance with emotions and expression as in a performance by a human player. Since performance rendering handles the object with subjectivity and Kansei, widely used number-oriented evaluation methods are not suitable for the system. So far the selection of music generated in each system varies depending on the issues of each system or preference of researchers. In order for the performance rendering research to evolve, an appropriate evaluation method is indispensable. The method should make people consent with the evaluation result by objective reasons while at the same time satisfy them from the musical point of view. We propose RENCON, performance rendering concours, as a project in order to seek and propose an evaluation method for music systems taking performance rendering system as a good start [1 ]. By running Rencon a continuous worldwide project, more and more people will be conscious of the project, and consequently we hope Rencon to enlighten people on the computer music research, encourage them to participate it, and empower the research itself. Figure 1 shows a typical process of performance rendering. The first stage is the preparation stage, giving a system musical score and real performances as input, obtaining information for rendering, such as musical structure, analyzed information of performance, and rendering rules. At the second stage, giving a rendering engine the derived information in the first stage and music set (a set of unified musical pieces for comparison) as input, then the engine generates performance as output. In the first and the second stages, in some cases, analysis and rendering are not completely automated. Intervening manually in these stages means that human may correct or supplement the system's result. "Sequencer programming", input score data to a sequence software then modify properties of each note with its MIDI values, corresponds to the manual intervention in the third stage. As the figure shows, the research of performance rendering is related to music analysis, music cognition, music performance, artificial intelligence and computer science in general. Thus in order to attain an evaluation method, wide and long range discussion will be required. In section 2, we describe how Rencon project started and the activities of Rencon so far. In section 3, we describe the evaluation matters in Rencon. Finally in section 4, we show the plan of Rencon workshop for the coming few years and the grand target of performance rendering system research. 357

Page  00000358 I1st stage S2nd stage 3 stage rendered performance intervene manually performance intervene manually intervene manually Figure 1: A typical process performance rendering 2 The activities The research of performance rendering with musical expression started in 1980s. Because the research handles and realizes Kansei information in a computer system, it got the attention of people as well as researchers of the area. Since then we have not been able to find an appropriate method to evaluate such type of systems. In 1996, the idea of performance rendering concourse is proposed for the first time [2]. In 2000, five groups in Japan that worked on performance rendering research participated a demonstration of rendered performance and a panel discussion on the next step of performance rendering research [3]. At the panel discussion session, an upsurge of having performance rendering concourse appeared. In the same year, we got a workshop fund by Nissan Science Foundation, then formed a feasibility study group consisting of researchers of computer science, music, and musicians, and started periodical meetings for a year [4]. We decided Rencon's objective and mission as described in Section 1, also to have the long-range purpose as in Section 4.In order to actually steer Rencon as the international, continuous events, we formed a working group to handle from clerical to technical problems [5]. 2.1 Petit Rencon The year 2002 is the start of Rencon's century. We have two Rencons in 2002, the first Rencon as a satellite workshop of ICAD2002 (International Conference of Auditory Display). Dr. Bresin of KTH (Sweden) participated in the committee. Since it is the memorable start of Rencon, the purpose is to clarify the current situation of performance rendering research. We prepared a CD-ROM including performances from all over the world which are rendered so far besides the proceedings (CD-ROM is purchasable through the homepage of Rencon). The second Rencon is held as a special session of FIT (Forum of Information Technology), a big joint conference of two large societies in Japan - IPSJ (Information Processing Society Japan) and ISS of IEICE (Information and Systems Society, a society of the Institute of Electronics, Information and Communication Engineers). As a preparation to the next year, we will have a panel discussion session where we invite a panelist who judges at contests of sequencer programming held by DTM and instrument makers as well as representatives of computer science and music area. After the panel discussion, we will have an open-judgment including attendees from the floor and give awards to participants. Although researchers demonstrate their music at these two Rencons, these do not form a concourse yet: there are not a set music nor I/O interface for rendering engines. Thus we call these Rencons "Petit Rencon". Prior to Petit Rencon, we will have a special session as a "near future challenge" in the annual conference of JSAI (Japan Society of Artificial Intelligence). Here performance rendering is treated as a design and reusability matter of multimedia contents. 2.2 Gros Rencon We plan to let Rencon be held in a real concour style - giving participants a set music and presenting them with an I/O specification before the concours from 2003 (Gros Rencon). We will have the fourth Rencon in 2004. Though we will simulate the style of real music concours in Gros Rencon, Rencon will have its 358

Page  00000359 uniqueness making use of the fact that performance is rendered by computer systems: * Since automated performance does not need any practice, a set music will be proposed to participating systems the day of the Rencon. The systems are expected to generate performance at sight. * In order to prevent a system to generate too excellent performance by generating exactly the same performance as a case performance, participants are given newly composed music. At Rencon, systems generate and compete performances of the world premiere. * Since there are many musical issues to solve for systems to generate performances, pursuing a part of music will be a good practice. For example, a set music is selected to see how a system realizes Auftakt and rendered performances are judged from only the Auftakt point of view. 3 Evaluation matters at Rencon 3.1 Evaluation viewpoints Quantitative evaluation is acceptable to evaluate performance rendering systems as a computer system. Needless to say, the problem to apply this type of evaluation to such systems is the reliability concerning music points. As music systems, we believe evaluation by human introspection is indispensable. In order to involve such subjective evaluation to system evaluation, we should indicate concrete elements and the standards for evaluation. Through numbers of Rencon, we expect to find a point of agreement between the objective and subjective evaluations. Rencon working group is going to prepare files for describing information of scores, their structures, and their performance data by human players. In order for these files to be disclosed on WWW, the group is designing an interface referring XML notations for music proposed so far. The interface and music information are presented to rendering engines to apply the Rencon. 3.2 Rencon keywords The concourse reflects problems in the real world. The ideas for Gros Rencon as music concours for systems described in 2.2 are to prevent possible problems listed below. * system doping: If a system-generated performance is manually given betterment taking time (changing the offset time, etc.), it is not possible to evaluate the system's original ability. Manual betterment is the same as the sequencer programming in the third stage of the system (Figure 1). We call this artificial change in performance system doping. * leakage and piracy: When information of a set music is leaked before the concourse, rendering engines can be tuned for the set. If a system uses the same musical piece of a set music from learning performances or case performances, the generated performance can be more than the imitation precisely the same performance as a human performance. Though this is not the piracy itself it is not the desirable situation which should be avoided. * sight reading: At a concourse, sight reading to indicate a set music at the day, is a way to prevent system doping and leakage. 4 Future Plan After heated discussions for a year,we came to a Rencon's long-term objective shown in Figure 2 [6]. In a half century, performance rendered by a computer is improved to be the first winner of the real music concours, say Chopin concours. From the system point of view, there are many interesting issues to be solved. Though Rencon's set music starts from piano pieces, we are going to extend them to many types of instruments where expressions are controlled by various information mostly in the continuous values. We will challenge the research of humanoid robot and try to integrate discrete and continuous musical information. In order to play Concerto as a soloist, the robot must have eyes to follow and understand a conductor's move and ears to do orchestra performance. Musicians and musicologists show their expectations to Rencon as follows: * For composers to expressively perform their newly composed music to pass their intention in the music. * For musicologists to confirm their hypothesis how a music theory is reflected on performance. * For players to reflect how their performances are attractive. * For music teachers to give many different ways of telling students to improve their musical skills. 359

Page  00000360 A student of a rendering system 2100 becomes the winner of a concour 2052 The wh er at the performance rendered by a aikovsky c r system becomes the Is 2050 winner of the Cho 2025 concour Sight reading, performance by a robot, try the Chopin concour 3rd RENCON, 2010 the start of the concour 2yste rendered per ormance 2002in over sequenc ogramming 1st RENCCN as ICAD satellite workshop, 2nd RENCON as a special session of FIT Figure 2: Rencon's century Rencon project is now an officially recognized working group of SIGMUS (Special Interest Group of Music systems) of IPSJ. Currently Rencon is funded by Kayamori Foundation and JST (Japan Science and Technology Corporatoin). References http://shouchan.ei.tuat.ac.jp/-rencon/ Katayose,H.,Goto,M.,Horiuchi,Y.,Matsushima,T.,Murao,T.,Shimura,S.,Rai,T.,and Hirata,K.:Report on the panel discussion "computer music research as computerscience",96-MUS-15,pp.91 - 98, 1996. Hiraga,R.,Katayose,H.,Koike,H.,Suzuki,T.,Noike,K.,and Hoshishiba,T.:Performance Rendering 2000 - Demonstration and Panel Discussion,2000-MUS35,pp.67 -70, 2000. Hiraga,R.,Horiuchi,Y.,Murao,T., Takeuchi,Y.,and Rencon workshop members:RENCON:a workshop for planning a piano contest by performance rendering systems - workshop report and panel discussion,2001 -MUS41,pp.37 - 42, 2001. Katayose,H.,Hiraga,R.,Hirata,K.,and Noike,K.:On Performance Rendering Concours for Piano -System WG Activity-,2002-MUS-44,pp.19 - 24,2002. Hiraga,R.,Hirata,K.,and Katayose,H.: RENCON:the ambitious pianist!,IPSJ,pp. 136 - 41,2002. 360