A Computer Accompaniment System with IndependenceSkip other details (including permanent urls, DOI, citation information)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. Please contact firstname.lastname@example.org to use this work in a way not covered by the license. :
For more information, read Michigan Publishing's access and usage policy.
Page 418 ï~~A Computer Accompaniment System With Independence Yasuo Horiuchi lory~cs.titech.ac.jp Hozumi Tanaka tanaka(cs.titech.ac.jp Department of Computer Science, Tokyo Institute of Technology 2-12-1Oookayama Meguro-Ku Tokyo 152 Japan Abstract In this paper, we introduce a new accompaniment system which takes independence from the soloist into consideration. In order to make a natural-sounding ensemble, we allow our system to change its independence dynamically from the soloist according to the musical situation. We introduce three new techniques into our accompaniment system. "The performance plan" allows the system to perform with expression as a human player would. "The independence rate" corresponds to how independently the system should perform from the soloist. "The time of the ensemble" provides information to enable the system to correct its tempo according to the independence rate. In order to evaluate the system, we carried out a listening experiment where the subjects listened to three versions of the same musical piece. The first was accompanied by a real human being, while the other two were computer-accompanied, where the first system simulated past researches which lacked the techniques mentioned above, while the second incorporated them. The results of the experiment show that our system which incorporated the three techniques sounds better than the system based upon past researches, and that it even sounds as good as a human accompanist. 1 Introduction An accompaniment system is a system which accompanies the human soloist. In this paper, we assume an accompaniment system is given a score containing parts for the soloist and for the accompanist. Past researches of accompaniment systems ([Dannenberg. 19841 [Vercoe, 19841 among others) aimed at the accompaniment keeping pace closely with the soloist's tempo without taking the effects of its own accompaniment on the soloist into consideration. But in order to make a naturalsounding ensemble, a two-way interaction between the soloist and the system needs to be established. For example, when there is little or no information from the soloist (such as when the soloist plays a long note or is pausing), past systems perform unnaturally, as these systems can not perform independently of the soloist. In such a situation, the accompaniment system needs to take the initiative in the ensemble and allow the human soloist to adjust his own tempo to that of the system's. It is not enough to merely have the system adjust itself to the soloist as in past researches, because an accomplished ensemble requires that both players adjust themselves cooperatively to each other's performances. In order to make a natural-sounding ensemble, we allow our system to change its independence dynamically from the soloist according to the musical situation. For example, under a situation when the soloist leads the accompanist, the system adjusts itself to the soloist in the same way as past researches. However, when the accompanist leads, the system performs expressively and independently of the human soloist; the human soloist should then adjust himself to the system's tempo. (We believe it is such a case in the ensemble consisting only of human players.) In the remainder of this paper, we will first give an overview of our system. Then, we will explain the aforementioned three techniques, and the method of using them to correct the tempo of the accompaniment. Finally, we will describe the listening experiment and present its result. 2 Overview of the Accompaniment System Our system consists of the LISTENER and the PERFORMER. The LISTENER (Horiuchi et al., 1992] tracks the performance of the soloist by matching the soloist's score (which is input into the system previous to the performance) and the soloist's performance using a DP matching algorithm. The LIsTENER then sends the information of the soloist's tempo to the PERFORMER. Using this information, the PERFORMER plays the accompaniment part. In the next section, we describe how the PERFORMER works in detail. 3 The Performer In this section, we describe three new techniques: "the performance plan," "the independence rate," and "the time of the ensemble". At the end of this section, we explain their use to correct the tempo of the accompaniment. 3.1 The Performance Plan We are in accordance with the theory that a human performer plays according to his own performance plan. In an ensemble with other performers, a human corrects his plan dynamically in order to adjust his (future) tempo to that which he predicts from the (present) tempo of the other player. In order to make a natural-sounding ensemble, we allow the system to have its own performance plan. This plan consists of the accompanist's score and additional expressive data which are not written in the score (for example, tempo perturbation, variance of loudness, pedal information etc.) and is given to the system before the performance. The plan, therefore, enables the system to play more expressively and independently from the soloist. 3.2 The Independence Rate "The independence rate" is defined as a weight corresponding to how independently the system should perform from the soloist. It is a function defined over metrical time (the virtual time represented within the musical score, measured in "beats"), its value ranging from 0 to 100, inclusive. When the independence rate is high, the system should perform independently of the soloist. On the other hand, when the independence rate is low, the system should adjust its tempo to the soloist. 7P.05 418 ICMC Proceedings 1993
Page 419 ï~~The independence rate is decided by the following rules and is given to the system before the performance: " It is low when the soloist performs the melody. " It is high when the accompaniment system performs the melody. " It is 100 when the soloist is in a long pause (more than 1 measure in our system). " It is 0 when the accompaniment system should match its timing to that of the soloist. " It becomes slightly higher when the accompaniment system plays shorter notes than the soloist (e.g. when the soloist plays quarter notes while the accompaniment system plays continuing sixteenth notes). This is because if the accompaniment system changes its tempo to that of the soloist, the ensemble sounds unnatural and distorted at times. 3.3 The Time of the Ensemble The "Time of the Ensemble" is the average of the times of performance of the soloist and the system, weighted by the independence rate. The time of the soloist's and the system's performance is defined in real time represented by the clock within the computer (measured in "seconds"). At metrical time s, the time of the soloist's performance at s is taken as tolo; the time of the system's performance at s is taken as tacco; and the independence rate at s is taken as Indep. We define the time of the ensemble at s as: tenembe =I ndept + 100 - Indeptoo (1) t.,c=100 tco+ 100 Therefore, when the independence rate is high, the time of the ensemble approaches the time of the system's performance at s; conversely, when the independence rate is low, the time approaches the time of the soloist's performance at s. 3.4 Correcting the Tempo of the System Using these three techniques (the performance plan, the independence rate, and the time of the ensemble), the system is able to correct its tempo to follow the soloist using the following method. metrical time [bat] 100-Indep:lndep S +1[.... Z-,-, wen-- - -- - "acco(Stheccext1)beat (S+1 ]system's plan of;;x" syste will be here. ithe performance Now, at etrical ime $ t e xcompnmetoys tcotn thacmaie the system h S +.1) repr nin predictedtempo Th iefteytmofseomn at h +1, t(s+y),eisgaue d lowinge theimo fe stem' performanhe ataimt~, and syte to of the performance plan at 5, Vpian(s). 1 tacco(s + 1) = t.cco( ) + 1 (2) Yp1 1 9}((2 The time of the soloist's performance at s + 1, t8o10(s + 1), is predicted from the history of the performance of the soloist in the past few beats (1 measure in our system). Using tacco(S + 1) and t,010(s + 1), the time of the ensemble at s + 1, tensemble (S + 1) is calculated below. tcnsamle(s + 1) = In: aeo(slne() + 1) + IO071ndep(a)O0 taolo(S + 1) (3) 100 aco 1 100 The tempo of the system is recalculated as Vacco(S), shown below. =1 + ) o(-) -=(1 tensemble (s + 1) - tacco(S) (4) If it happens in (4) that tensemble(s+ 1) < tacco(s) and the tempo calculated the above expression results in a negative value, the system waits until the soloist catches up to the system's performance. If the independence rate is 100, (3) results in the system performing independently of the soloist, according to its own performance plan. As an exception to equation (3), if the independence rate is 0 at metrical time s (when the accompaniment system needs to match its timing to that of the soloist), the system will perform the note at s directly after the performance of the soloist's note at s (with perhaps a few milliseconds in between). When the system's performance lags behind the soloist's, the system's metrical time jumps ahead to s; when it is ahead, it holds off playing the note at s until the soloist catches up. 4 Evaluation Because there is almost never a universal and objective method for evaluating the artistic merit of music by computational means alone, we have opted for a listening experiment involving human subjects to evaluate the output performance of our system. As most researchers in the past evaluated their own system by themselves without conducting listening experiments, their results may not have included the opinions and assessments by others, not to mention their evaluation may have been influenced by some subjectivity on their part. Our listening experiment enabled us to compare our system to both a human accompanist and a system based upon past researches. 4.1 Listening Experiment 4.1.1 Method Materials A tape was prepared consisting of three recordings of an ensemble of a human soloist accompanied by the following three types of accompaniment. The soloist in all three recordings was a professional pianist. Acco.1 A human accompanist: A student of music. Acco.2 An accompaniment system based upon past researches: A system which always adjusts its tempo quickly to the soloist's tempo. Acco.3 Our new accompaniment system: A system which changes the system's independence dynamically from the soloist according to the musical situation. ICMC Proceedings 1993 419 7P.05
Page 420 ï~~The only difference between Acco.2 and Acco.3 was the method of correcting its tempo to follow the soloist; the variance of loudness through the score was kept the same for both systems. The musical piece used in this experiment was the beginning 40 measures (about 2 minutes) of the first movement of "Sonata for Violin and Piano" by Cesar Franck. Procedure The subjects were given the following list of evaluation items before listening to any of the recordings. They were then informed that they could listen to the recordings repeatedly as many times as they needed to make the evaluation, however they had to listen to all three recordings again and could not listen to any one specific performance. Each performance was then presented at a comfortable level to the subjects by means of a loudspeaker. The subjects were asked to rank order the three performances in the following evaluation items. Q1. How smoothly does the accompaniment follow the soloist? Q2. How closely matched is the accompaniment to the soloist's timing? Q3. How expressively does the accompaniment perform? Q4. How accomplished on the whole is the ensemble performance? Subjects 36 undergraduate and graduated students belonging to the orchestra of Tokyo Institute of Technology participated in the experiment. They were divided into 6 groups, each group assigned one combination out of all 6 possible combinations of the three types of performances. After the experiment, each subject completed a short questionnaire asking him/her to describe his/her musical experience and ensemble experience. The level of musical experience of the subjects ranged from 1 to 20 years, and their ensemble experience ranged from 1 to 14 years. 14 out of the 36 subjects (39%) turned out to have heard this piece before. 4.1.2 Results and Discussion Table.1 shows a part of the overall results of the ratings, showing the number of subjects who ranked each of the performances as the best for each evaluation item (Q1 through Q4). The Friedman test was used to reveal any significant differences for all three performances, and the binomial test was used to reveal any significant differences between any two performances. Table.2 presents the results of those tests. For Q2 and Q4, our system was evaluated better than thle system simulating past researdhes (z = 2.5, p <.01), and for Qi and Q3 our system showed a tendency to be evaluated better (z = 1.5, p <.1). Our system was evaluated for QI and Q4 as perfornming as well as the human accompanist. To our surprise, for Q3, our system showed a tendency to be evaluated slightly better than the human accompanist (z = 1.5, p <.1). We believe that from the evaluations presented above, we can conclude that the performance plan made the system's performance more expressive, and our method of correcting the tempo of the accompaniment system using the independence rate and the time of the ensemble made the system's performance very similar to that of a human accompanist. Table 1: The results of the ratings Acco 1 2 31 Acco!1 2 3 EQ1 15 8 13 Q3 9 11 161 Q2 20 6 10 Q4 15 5 16 Table 2: The results of the tests 1. How smoothly does the accompaniment follow the soloist? S = 4.17 Acco.1 > Acco.2 z = 1.5 p <.1 df = 2 Acco.1 = Acco.3 z = -0.17 p <.2 Acco.2 < Acco.3 z =1.5 p<.1 2. How closely matched is the accompaniment to the soloist's timin? = - 12.06 l Acco.1: Acco.2 z = 2.5 p <.01 df =2 Acco.1 > Acco.3 z =1.5 p <.1 p <.01 j Acco.2 4C Acco.3 z = 2.5 p <.01 Q3. How expressively does the accompaniment perform? = 4.22 Acco.1 > Acco.2 z = 0.17 df =2 Acco.1 < Acco.3 z =1.5 p <.1.2 Acco.2 < Acco.3 z =1.5 p <.1 Q4. How accomplished on the whole is the ensemble performance? S - 9.39 Acco.1 ), Acco.2 z = 2.2 p <.05 df = 2 Acco.1 = Acco.3 z = -0.17 p <.01 Acco.2 4r Acco.3 z = 2.5 p <.01 Note - The left side of this table shows the result according to Friedman test. The right side of this table shows the result according to the binomial test. The value at the right of 'p <' shows the level of significance. 5 Conclusion In order to make a natural-sounding ensemble, we allow our system to change its independence dynamically from the soloist according to the musical situation. We introduce three new techniques into our accompaniment system: "the performance plan," "the independence rate," "the time of the ensemble." In order to evaluate the system, we carried out a listening experiment of the ensemble of the human soloist and-three accompaniment systems (a human accompanist, a system based upon past researches, and our new system). The results show that our system sounds better than past researches, and that it even sounds as good as a human accompanist. We feel we improved the performance of the accompaniment system introducing the independence of the accompanist. The results of the listening experiment gave suggestions that our accompaniment system was able to perform at a level very close to that of a human accompanist. References [Dannenberg, 1984j Roger B. Dannenberg: An On-Line Algorithm for Real-Time Accompaniment, in Proc. of ICMC, pp.193-198, 1984. [Horiuchi et aL., 1992J Yasno Horiuchi, Atsushi Fujii, Hozumi Tanaka: A Computer Accompaniment System Considering Independence of Accompanist, in Proc. of Japan Music And Computer Science Society, Summer Symposium, pp.73-78, 1992, (in Japanese). [Vercoe, 19841 Barry Vercoe: The Synthetic Performer in the Context of Live Performance, in Proc. of ICMC, pp.199-200, 1984. 7P.05 420 ICMC Proceedings 1993