Page  405 ï~~The Recognition and Editing System for Shakuhachi Score Toshiaki Matsushima Toho University Department of Information Sciences 2-2-1, Miyama, Funabashi, Chiba 274, JAPAN Abstract In order to facilitate and broaden musical information processing, it is necessary to input the musical information into the computer. Unique tablatures have been used for Japanese traditional musical scores and the computerization for such tablatures is far behind. The necessary musical information for such processing is input manually using special codes. In order to improve these conditions, the author is developing the recognition and editing system for Shakuhachi score. At the last ICMC, the author reported on the system, focusing on Shakuhachi score recognition. Another major part of the system, an interactive editor for Shakuhachi score, is mainly reported in this paper. 1. Introduction The author is developing the Computerized Japanese Traditional Music Processing System. The system aims to provide easy operation environments for Japanese traditional music composers and players. The Tozanryu Shakuhachi (Japanese traditional wind instruments made of bamboo) score [Uen86] is chosen as the start up target. The system is intended to be used in the following sequence: 1) The score image is scanned into the personal computer and the image data is transferred to the workstation. 2) The image data is recognized and the results (logical data) is transferred to the personal computer. 3) The recognition results are reconstructed and represented on the LCD/tablet device, and the user performs edit operations interactively. A unique feature of the system is that a combined LCD/tablet equipment provides an interactive score editing environment for the novice users. Because Shakuhachi scores are written by hand, this user interface is effective for the people in the music field who are not accustomed to computers. The necessary functions for Shakuhachi score editing are implemented and most of the symbols used in Shakuhachi scores can be input by easy pen operations. At the last ICMC, the author already reported on the system, focusing on Shakuhachi score recognition [Mat92]. The interactive editor for Shakuhachi scores is mainly reported in this paper. 2. System Configuration The system consists of two sub-systems: the optical recognition system and the interactive editing system for Shakuhachi scores. The former is implemented on a high-performance workstation because there is a large amount of image data to be processed. The latter is implemented on a personal computer which is more convenient for the on-line real-time recognition necessary for the interactive editing. These computers are connected via Ethernet and the processed data can be transferred mutually. 3. Recognition System Fuji (note characters which represent finger formations and which define pitch and timbre) and Hyoshi-sen (duration symbols) are identified and recognized from the scanned Shakuhachi score image. Shakuhachi scores are written in handwritten Japanese characters and geometrical symbols. Generally speaking, handwritten Japanese character recognition is not so easy because there exists a large number of characters and many similar-shaped characters. As only relatively simple characters are used for Shakuhachi scores, it is not so difficult to recognize them if these symbols are completely separated from each other. To separate individual symbols, vertical and horizontal projection profiles are used. The pattern matching method is used for note recognition. The templates are normalized images which are made from the average pattern of real images. Previously, 23 templates were prepared for the recognition, but some of the symbols were confused with similar-shaped symbols or rarely-occurring symbols. The survey of 126 Shakuhachi" score examples showed that the frequency of appearance of the individual characters in Fuji clearly varies, as is shown in Table 1 [Shi93]. The templates for Fujis were then restricted to the characters RO, TU, RE, TI, HA, HI, P1, TA and RU, and other Fuji characters were input directly by using the interactive editor. As for the Hyoshi-sen symbols, symbols denoting half note to 16th note can be recognized in this system. Fig.2 is the recognition result of Shakuhachi score Fig.1. There still exists some recognition errors. These are mainly caused by separation errors between Fuji and Hyoshi -sen, and confusion of similar Fuji characters. ICMC Proceedings 1993 405 4P.12

Page  406 ï~~4. Interactive Editor One of the major problems for automated recognition is the difficulty, both of the correction of recognition errors and of the addition of unrecognizable symbols. In this kind of system, it is very important to provide an easy operable editor for the novice computer users. This system thus adopts an integrated LCD/tablet device and the pen operation method for the editor manipulation. This interface is also suitable for inputting Shakuhachi scores, as they are written by hand. As shown in Fig.2, the system adopts the Fuji box interface because in the Tozan-ryu Shakuhachi score it is possible to draw boxes for each Fuji characters. This solves the problem of determining which stroke belongs to which character [Tap88, 90]. After the compression of stroke data, strokes are categorized into following four types. 1) Target indication strokes: Strokes pointed in the Fuji box or "<" shape strokes across the right boarder of a Fuji box are considered to specify a single Fuji. "<" shape strokes across the right boarder of two Fuji boxes are considered to specify the interval between the two boxes. 2) Editor command strokes: Strokes drawn in the menu area located on the right hand side of the screen signify commands. Assigned editor commands are evoked (Fig.3). 3) Fuji data strokes: Strokes drawn in an empty Fuji box are considered as a part of Fuji data. When a stroke is written outside of the box, the recognition process is evoked. 4) Hyoshi-sen data strokes: Strokes drawn in the right area of the Fuji box are considered as a part of Hyoshi-sen data. Like the Fuji data, the recognition process is evoked when the next stroke is written in an other area. All Fuji characters can be recognized, including the ones eliminated in the score recognition system. Other special symbols, such as the rests and the breath, are also supported. As for Hyoshi-sen, half to 32nd duration symbols can be recognized. Japanese Katakana characters and their modified forms are used for Fujis. Vertical lines and short line segments are used for Hyoshi-sen. Cursive lines rarely appear in the strokes composing Fuji and Hyoshi-sen, so strokes can be classified into 10 elementary strokes (Fig.4). Fuji characters and Hyoshi-sen symbols are recognized by the number of strokes and their shapes. Hasei notes (Fuji characters which are derived from fundamental ones -- there are four types of Hasei notes: Han, Mer, Omeri and Kari notes) are input easily, accurately and rapidly by pointing at the menu. The input of octave signs (Kan and Otsu) adopts the same method. Fig.5 is an example of editing operations and Fig.6 is the editing result. Fig.5 shows that the editing and input of Shakuhachi scores can be done by easy operations. 5. Conclusion The basic structure of the recognition and editing system for Shakuhachi score has been completed. As for the recognition system, there still exist some recognition errors even for the fundamental Fujis. The accuracy of separation of Fuji and Hyoshi-sen and the precise classification of similar patterns are still necessary. The editing system supports the necessary functions for Shakuhachi score editing, and most of the symbols used in Shakuhachi scores can be input by an easy pen operation method. Further research plans are as follows: 1) Modification of operability and editing functions by reflecting user comments. 2) Utilizing other system functions such as the printing out of the standard western music notation form or the score conversion for Shakuhachi of different lengths. 3) Minimizing unsupported symbols such as tempo, time signatures, and unique symbols used in Shakuhachi scores. Acknowledgements The author expresses his gratitude to Professor Satoshi Shimura, Osaka University of Arts, and Professor Kuniharu Tsuboi, Hamamatsu Polytechnic College, for the fruitful discussions and advice concerning this research. The author is also indebted to Professor Shuji Hashimoto, Waseda University, for his useful advice. Finally, the author wish to thank the students of his laboratory for their assistance. References [Uen86J K.Ueno, "Shakuhachi Ongaku no tameno Gakuten,"N Shimada Music Publishing, Tokyo, 1986 (in Japanese) [Tap88] C.C.Tappert, C.Y.Suen and T.Wakahara, "On-line handwriting recognition - A Survey," IEEE ICPR, pp.1123 -1132, 1988 [Tap90] C.C.Tappert, C.Y.Suen and T. Wakahara, "The State of the Art in On-Line Handwriting Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, 12(8), pp.787-808, 1990 [Mat92] T.Matsushima, "Computerized Japanese traditional music processing system," ICMC'92, pp. 121-124, 1992 [Shi93] S.Shimura, K.Tsuboi and T.Matsushima, "Information Processing of Traditional Japanese Music -- A Case of Shakuhachi --," IPS Japan, SIGMUS 2-3, pp.17-24, 1993 (in Japanese) 4P.12 406 ICMC Proceedings 1993

Page  407 ï~~Table 1 Count of Fuji characters Fuji 1 Count Ao HA 1." RE %~iTU ~RO STI ' HI ~TA )L-p RU:'KARA 9JRI FURA Others 1419 1244 1077 1009 938 140 64 14 10 3 3 2 0. A4A4.. Fig.1 Shakuhachi Score...... tl.. CUT C LFk rJE / I Fig.3 Command Menu r I 1I. 0' Fig.4 Elementary Strokes Fig.2 Recognition Result (including errors) Fig.6 Compelete Editing Result 1. Specify Fuji character by pointing. ='IL) I 11A l. 2. Point to 'CUT' to delete FuiP' L(RE). 3. Draw Hyoshi-sen on the right side of Fuji. ('UT I.I1! 6. Write FuA "'J(TU)' in the empty Fuji box. CIR INS roii Q -r t 4.Specify Fuji by stroke and the Hyoshi-sen drawn in 3 is automatically recognized. [F.L 4I [.I1 i\r........ f......... INS 7. Point to '*(Han) to add Hasei information, and written Fuji is recognized as Han-note of 0'Y. 5. Point to 'CLR, and the specified Fuji and Hyoshi-sen are cleared. Fig.5 Example of Editing Operations ICMC Proceedings 1993404P2 407 4P.12