Page  121 ï~~Computerized Japanese traditional music processing system Toshiaki Matsushima Toho University, Department of Information Sciences 2-2-1, Miyama, Funabashi, Chiba 274, JAPAN ABSTRACT Computerization of Japanese traditional music is far behind that of Western music. The goal of the system discussed in this paper is to provide Japanese traditional music composers and players with easy operation environments for their music. The Shakuhachi score is chosen as the initial target. This score is a kind of tablature which mainly consists of handwritten Japanese characters and geometrical symbols. The score is scanned into the computer by the high resolution image scanner. Then the symbols necessary to performing Shakuhachiis recognized using a pattern matching technique. Unique feature of the system is that a combined Liquid Crystal Display (LCD)/Tablet equipment provides an interactive score editing environment for novice users. This is a report on the current state of the system. 1. INTRODUCTION In order to process the music information broadly and easily, it is necessary to input the music information into the computer. For the western standard music notation, what is called 'score', many researches and commercial products are already known and convenient computerized environments are available for composers and plays. Unique tablatures have been used for the Japanese traditional musical scores and the computerization of such tablatures is far behind. In order to improve such circumstances, this research aims to develop the Japanese music information input environment. The author engaged in the development of integrated music information system (Matsushima 1988) and found that the serious problem of automated recognition system is the difficulty of recognition error correction. This system is designed to input music information by automated pattern recognition and user friendly interactive editor. Shakuhachi is chosen as the initial target. Shakuhachi is a Japanese traditional wind instruments made of,.. bamboo. Among the several types of Shakuhachi tablature, the Tozan-ryu type tablature is selected here as the first example. 2. SHAKUHACHI SCORE NOTATION The right of Fig.3 is an example of Shakuhachi score. Shakuhachi score is a kind of tablature (Ueno 1986) consisting mainly of handwritten Japanese characters and geometrical symbols. The Shakuhachi tablature notation system is described briefly from the point of pattern recognition view. The score is a sequence of measures and each measure is enclosed in a rectangular frame in which notes, duration and additional symbols are written. The score starts from the top-right rectangle and is read downwards. Shakuhachi notes represent the finger formation, and pitch information can be obtained from them. Handwritten Japanese Katakana characters and some modified Katakana characters are used for note symbols. Generally speaking, handwritten Japanese character recognition is not so easy because there exists a large number of characters and many characters of similar shape. As the note characters used for the Shakuhachi score description is restricted, recognition of these characters are expected to be done easier by familiar pattern matching technique. The durations of notes, which are located at the right hand of the notes, are represented by straight lines and short line segments drawn across them. Recognition of these line segments are not expected to be very difficult, either. As you can see from the example, some symbols are connected with each other and are slightly deformed when these notes have shorter duration. The major problem for recognition is in the separation of each symbols

Page  122 ï~~and in dealing with deformation of the symbols. As for the on-line recognition of input strokes from the tablet, almost the same discussion as that above mentioned is applied except that there is a need to distinguish the score data from the editor commands. This can be solved by devising the display layout and assigning the special strokes to commands. 3. SYSTEM CONFIGURATION The system configuration is shown in Fig.1. A personal computer (NEC PC9801 RA, CPU 80386, MS-DOS Ver.3.30B) is used as a host machine for a score image input and an interactive score editor. As the image input device, a high resolution image scanner (EPSON GT6000, Maximum resolution 600 DPI) is connected to the personal computer. An integrated LCD/Tablet (WACOM HD-640A, LCD 640 x 400 dots, Tablet resolution 0.05mm) is also attached to it and used for the interactive editor equipment. Shakuhachi score image recognition is performed on a workstation (Asahi Stellar GS1000, 40 MFLOPS, a UNIX based OS:Stelix and X window system). The workstation and the personal computer are connected to each other with the Ethernet line and the data can be transferred immediately via the network. The system is intended to be used in the following sequence: 1) The score image is scanned into the personal computer and the image data is transferred to the workstation. 2) The image data is recognized and the results (logical data) are transferred to the personal computer. 3) The recognition results are reconstructed and represented on the LCD/Tablet device, and the user performs edit operations interactively. 4. SCORE RECOGNITION As most of the symbols used in the Shakuhachi score are not so complex, it is not very difficult to recognize them if these symbols are completely separated from each other. In order to separate individual symbols, the recognition process is performed in the following order (Fig.2): 1) Dividing a score into measures By the extraction of horizontal and vertical long line segments, border of measures can be detected. 2) Dividing a measure into segments of symbols From the distribution of horizontal projection profile, a measure is divided into some segments in which symbols share the same vertical position. In these segments, notes and duration symbols may be included. 3) Separating duration symbols from note symbols From the distribution of vertical projection profile, a segment obtained by the above process is divided into note symbols and duration symbols. Their border can be found as an explicit gap or narrow connection. When there are no explicit line segments to the right of a segment, it is necessary to recognize this absence and omit this separation process. 4) Recognition of symbols 4-1) Recognition of notes The pattern matching method is used for note recognition. Templates are normalized images which are made as the average pattern of real image. The surrounding rectangle of a target image is measured and a target image is normalized in order to fit the template size. If the height of the surrounding rectangle is too large or the height/width ratio is unbalanced compared to other surrounding rectangles, further division of the segment is executed. 4-2) Recognition of duration Types of duration symbol can be detected by scanning across the duration symbol area vertically. 5) Reconstruction of notes and durations Recognition results of notes and durations are combined and the score information is reconstructed. The relation of the notes and durations can be determined from their positions. The left of Fig.3 is the recognition result of the measure 1st to 4th. 5. INTERACTIVE SCORE EDITOR One of the major problems for automated recognition is the difficulty of the correction of recognition errors and the addition of unrecognizable symbols. It is very important to provide an easy operable editor for novice 122

Page  123 ï~~computer users for this kind of systems. From this point, this system adopt an integrated LCD/Tablet device and pen operation for the editor manipulation. Any editor commands and score data are input to a computer only by stylus pen operation. Several special strokes are interpreted as editor commands. There are three kinds of command strokes: target indication strokes, position indication strokes, and operation strokes. Target indication strokes are ones that are written in the existing symbol areas. Position indication strokes are ones that are written across the measure borders and are the shape of an angle bracket ">". Operation strokes are either ones that point to the menu area or two crossing strokes "X" on the existing symbol area, which are interpreted as a deletion of the note symbol/measures. Any strokes written in the empty note symbol box is assumed as score data. Target symbols for recognition can be restricted in Katakana characters and some simple geometrical symbols. Moreover, there seems to be no problem about the restriction that note characters are to be written inside the prepared character box. Then the on-line character recognition for Shakuhachi score is not expected to be very difficult to realize (Tappert 1988). As of now, the user interface design and its prototype are completed and implementation work is now in progress (Fig.4). 6. CONCLUSION As for the example score, image recognition results are almost satisfactory. The prepared templates are the major and frequently used ones. Thus the expansion of templates for all Shakuhachi symbols is required. As for the interactive editor, complete implementation is now underway. From the prototype, I got the feeling that the integrated LCD/Tablet device is suitable for Japanese traditional musical score editing, especially the Shakuhachi score. Field testing by the Shakuhachi players and composers is planned after complete implementation. The project was started this winter. An improved system will be demonstrated next year. ACKNOWLEDGEMENTS The author expresses his thanks to Professor Satoshi Shimura, Osaka University of Arts, and Professor Kuniharu Tsuboi, Hamamatsu Polytechnic College, for giving me the hint of this research. The author also expresses his thanks to Professor Shuji Hashimoto, Waseda University, for his useful advises. Finally, the author would like to thank to the students of his laboratory for their assistance. REFERENCES T.Matsushima, S.Ohteru, S.Hashimoto, "An Integrated Information Processing System: PSB-er," ICMC'88, 1988, (Included in ICMC'89 proceedings, pp.191-198) Katami Ueno, Shakuhachi Ongaku no tameno Gakuten, Shimada Music Publishing, Tokyo, 1986 (In Japanese) C.C.Tappert, C.Y.Suen and T.Wakahara, "On-line handwriting recognition - A Survey,' IEEE ICPR, pp.1123-1132, 1988 Ethernet Personal Workstation Computer Image Integrated Fg1Sse ofgrto qr, Qnncir I,, ' ~t-r"I Fig..SytemConfgurtio 123

Page  124 ï~~measures )' 2 --* RO S4tHA -* 4th duration symbols Fig.2 Recognition Sequence A, I r r r - " < -;............; " Â~.1 T....................A l Â~ A 0.......,..........." ", ",1 " "I "......... " L EiJ| CUT PASTE MOVE COPY UNDO MENU......... o Â~c e " " __I Fig.4 Interactive Editor Prototype Fig.3 Sakuhachi Score Example (right) and its Recognition Results (left) 124