Page  110 ï~~Computerized Recognition of Hand-Written Musical Notes Alex Bulis, Roy Almog, Moti Gerner & Uri Shimony Laboratory of Computer Music Engineering Department of Electrical Engineering Technion, Israel Institute of Technology, Haifa, Israel, Introduction This work describes a software unit called CNR (Computerized Note Recognition) whose function is to interpret and recognize hardwritten musical notes. The handwritten notes are read from paper by alj optical scanner and the image of the page is stored in memory as a. blit-rma.p file. This file serves as an input to the CNR software. Note recognition is done by the histogram. method: each symlbol is specified in a library of symbols by prerecorded honizonta.l and vertical histograls. IIistograns of an unknown symbol is compared witih the library histograms and the symbol with the best fit to the unknown symbol is chosen. A histogram describes the sum of on-pixels for each column of the symbol. Similarly, a vertical histogram describes the sum of on-pixels for each row of the symbol, as seen in the Figure. Figure. lHorizontal and vert ica~l histograms of a hial f-note symbol Th upu fCR.i usclalh-uerccd, ecibn h"pthsofnts simple rerec re.y 0 h-oitora a nd eatily onst ructed Ii brlarf-er odds m10 htog-amsThhioga1 method of identifyingon re"symle bolfi son imes faIs, seciial lvrwens 110

Page  111 ï~~the symbols are too colmplica.ted and the histograms cannot characterize them well. However, in the case of musical notes thme nu ml)er of symbols is not too great, the symbols are simple enough and they do not, change ruch in size, so that the histogram method works very well. As in other pattern recognition cases, one cannot expect always perfect identification and some human help is need to correct machine errors and put the fina.l touch. The Recognition Process The recognition process consists of two main stages. The first, one is the staff search stage in which all the staffs in a page are identified. The second one is the symbol identification stag: in which every musical symbol is identified and recognized one after the other, in the order they on the staff. There is an option of manual pixel or line editing of the image. This option is to be used in situations where noise an(I snriidges appear in the image. Early h l-nian intervention can ai(l the ide ltificatiol wlic lfollows. The staff search stage A preliminary operation of 'ocation is often required to align the lines of the page image parallel to the linies of the dis)lay, because usually the page in the scanner is not aligned perfectly. A special check decides if rotation is required and performs the rota.tion.Care imiust l)e taken that, in the rotation extreme parts of the image are not lost. To search for the staffs, the program scans the image of the page horizontally from top down. A line is found if the lllnmber of on-pixels is greater than a, certain threshold va.lue, usually taken a.s 60%. This threshold c.aiilbe changed by the operator of the software, from 0% to 1.00%, as required. When the image of the page is "noisy" or "dirty" this threshold is lowered, otherwise the lines cannot be detected well. A staff line should be only one pixel wide, so that heavier horizontal lines are not interpreted as staff lines. Sometimes an unwanted bottom horizontal line appears in the image due to the edge of the scanned pa.l)er. Therefore, a single bottom line is disca.rded by default, unless the operator chooses to look a.t it first. To define a stafl, five horizontal lines having a. specified spacing between them should be found. The gaps found betwcell Iicstaffs should be within prelefined boti u(aries which may also be changed by the opeera.tor. Once the whole staff setup of the page is found, corrections are done to straighten slaff lines. This is done by moving up or clown by few pixels parts of the staff with the symbols written on them. The whole l)rocess of staff search can be automatic or it can be executed step by3. step. In the latter case the operator can follow the progress of the l)1ocess and interfere if something goes wrong. The symbol identification stage There are two p)OssileI mlode(s of" symblol identification, autom;atlic and manuval. In the automatic mode thle progr'am ch~ooses thle b~est, fit b)etweenl library hlistogramns anld th]e h istogram s of an unknown symlo to l)e identified. In the mnanua.l mode thle operator is presented on 111

Page  112 ï~~the display with the best ten Ilistograni its, in order of fit quality, and he has to make the choice. The symbol to be i(lent ifie( mulst conform to size tolcrances, otherwise it, is not considered as a, candidate for identification. The default tolerances are usually between 50% and 150%. Before a. symbol is sulbjected to the identification process it is cleaned from the staff lines that cross it. Its histograms are then formed and compared with the histogram collection in the library. The vertical position on the staff is of course retained before the the cleaning of the symbol so that the pitch can be determined later. The program provides a special menu option for creating and expanding the library of symbol histograms. Every time that the user discovers that a certain symbol is consistently not identified well, he can redefine the histograms in the library. He can also add new symbols to the library. For best, resilts the library of histograms should be constructed specially for each hanlxdwri ting. Lir)i'ary iistograms are also defiuied from symbols whiich were cleaned from horizon tal liles cross i g it. The idel.tiJicatiol dccision is based oi the calculation of a difference function which sums the squares of' (ifferences between the library histograms and the unknown symbol iistograi, s. Usually eqtua.l weights are assigned to tIie horizontal and vertica. lIiistogranis. Sometimes however it is.etter to give the vertical difference a. higher weight than the horizon ta.l differeiice, because( deta-ils of the mlsical symbols are usually more pronouiicecl in the vertical directioii. weigllts of between 50%/50% and 70%/30% were found to work well. I)uring the identification )rocess the ol)erator may choose to look at the difference function and judge himself the qlua.lity of identification. The present version of CNiR works on any compatible PC equipped with a scanner. Beca use of memory shortage, the image is loaded from the hard disk pa.rt by part,, a's requiired. The whole operation is automatic and transparent to the user, using a virtual menory mode built into CNR.. The scope of the present version of CNR was limited to symbols which are not connected graphically to each other, in order to simplify things. A later version will overcome this limita.tion enable to ideitify also chord notes and notes connected by ('ightll bar, sixteenth bar, etc. Conclusion The CNR project shows that, computerized recognition of musical notes can be accomplished by a. relatively simple mletliod of comparing horizontal and vertical histograms of symbols. AltLough the ilentifica,,iorn of not-es is not always perfect, it is highly efficient and ca n save mnuch time inl the l)r)cess of ii1l~),tting mnusical notes, even printed notes, iiito computer. 112