Page  329 ï~~Studio Report: A New Icon Musical Notation System - TAL* Uri Shtimony, Moti Gerner, Sllonio Markel Laboratory of Compu ter M usic Engineering Del)artment of Electrical Engineering Lechnion, Israel Institute of Technology, Haifa, Israel, and Josef Tal Music Department, The Hebrew University Jerusalem, Israel. Introduction This rsearchh pe'sclits a new conceptual approapp 't I o writ, illg lnotes of cornputIcr mulsic. It gives freedom to use notes from a, continuum of frequencies rather than from a. set of discrete frequencies. It also incorporates timbre parameters into the icon notes, thus enabling easy and continuous movement through different timbres, instead of Usiig fixe( ti ml)res. In this new musical notation icon.symbols replace and expand tle meaning of the "circles" of notes in the Conventional Musica.l Notation (CMN). The icons include in their format hints of the i)hysical parameters of tle timl)res of the tones. Using this notation the composer can shift icons continuously in the )lalne of Pitch (or log of fundamenta.l frequency) vs. Time, and can use freely and continuously microtone sounds, as seen fit. He can also change freely the timbre of tones by changing the icon parameters, such as WI4ave Form. (WF, i.e. the spectrum of sound), amplitude envelope (AM, the AttackDecay-Sustain-Release characteristics) and frequency modulation dependence (FM, effects like vibrato, pitch slides, portarnento, glissando etc.). A psychological advantage of the icon symbol is that the human eye is sensitive to small changes in closed figures, and forms of icons canl be easily recognized. A musician can therefore easily identify the changing icons ill a. musical score and follow the timbra.l changes. This unique concept of icon notation is realized in the computerized TAL system, built in our Lab, which is an integrated system of hardware and software. The hardware includes a.n advanced personal computer (or workstation, in a future configuration), peripheral units, a multi-oscillator real-time sound generator, audio sound equipment and optical scanner. The software l)ackage deals with the input, graphics, editing, printing and driving the sound generation. A composer working with this system can either write his music on a computer screen by3 using th~e computer keyboard and a. mouse, or alternatively can write it on paper. In th~e latter case thle page with the icon notation is scanned optically and converted into bit-map) im~age of th~e page. Tis is interpretedl and edited. The music canl then be heard through the audio system, as the b~uilt-in sound generator follows th~e notation instructions. "This project was sutpported by tlhe Volkswagen Foundaations, Hanover, Germany, Grant numb~er 1 / 64 292. 329

Page  330 ï~~Principles of writing notes in TAL 1) The written icon music can be heard at any time, short or long sections. Thus interactive music writing is possible, as well as learning the association between llthe written icol notes and their sounds. 2) Time dependence is 11ea.sure(l along the horizontal coordinate of the score, in the PitchTime plane, from left to right, as in CMN. The horizontal position of tle point of origin of the note (lefines the exact timing of the beginning of a note. The beginning of a. note call be (Iefilie(l conit1,inulously in tine, and is lot r(st'icte( lto (lis('ret('e time points. 3) The pi/ch of a note is defined by the height of its origin along the vertical axis, as in CMN. elre a.gain it is not restricted to a set of discrete pitch points (such as the 12 pitches per octave), and car be moved conti n notlsly by using rn icrotonie ma rking. A Amplitude Kt A(t) ~st Time,1) he tAM envelope for each note is represented inl polar coordinates, forming a. closed iconic form, as seen in tihe Figure. The magnitude of amplitude is represented by the distance to the note origin (i.e. the polar radius vector). Time chan.ge of the amplitude is represented by the polar angle (also called the azimuth an gle). 'Iypically, the icon note has a "kidnc" shape, since the amplitule starts with zero value at polar angle 0, and (e11(1s with zero value a~t polar angle:360. In order that the same icon represent the same timbre irrespective of its duration the following conventions are adopted: - The AM envelope of the note is divided into three A D-S-R segmen.ts, Attack+Decay, Sustain andl IRelease. - The wh~ole Jpolar angle of 360 legrees is (Ii vi(Ied into th~ree parts, th~e first sector of 180 degrees is assigned to thle Attack-+Decay segment, the(. next sector of 90 degrees is assioned t.o tlie Sut~sain segmnt.l andl t~tme last sector of 90 legree(s is assigned to the 330

Page  331 ï~~- Within each sector the polar angle increases proportionally with time. However, time scales for each sector is different, depending on the actual total time duration assigned to each segment. An additional statistical element may be introduced into the playing of icon notes, if the musician prefers it. Two AM envelopes are drawn for each note, e.g. by two different colors. These two co-origin icons mark the minimum and maximum magnitude between which the actual amplitude can play. It is controlled within these boundaries by statistical functions, fractals, wind, etc. 5) The actual duration time of each icon is represented by a horizontal line section drawn from the note origin to the right direction. The actual duration of each of the three AD-S-R. segments is marked on this line. 6) Volume is defined separately for each icon note. For graphic reasons, all the notes in the score of Pitch-Time plane appear the samne size, with their envelopes normalized according to Jeak valies. IIowever, when the score is played, each icon is playedl according to its assigne( volume. 7) The F/t function is dirawn horizontally as a ca.rtezial curve. The duration time line (described above, 5) drawn to the right of the icon, serves as a reference axis to this curve which represents the pitch (or frequency) deviation relative to the note's center pitch. 8) Between one and four waveforms (WF) are associated with each AD-S-H, segment of an icon note. Time evolving mixes of two WFs out of four is possil)le, by changing the weights of any two of the WFs. WFs may be defined by either one of three ways: A) The WFs are taken from a library of pre-defined or recorded WFs. Tie interpreted icons extracted from scanned icon score written on paper, are also included in this group. B) A WF is formed by superposing harmonic Fourier components. C) A VF can be drawn graphically on the screen using a mouse. Smoothing of WF is provided to avoid high frequency components which might cause aliasing and distortion. 9) Coincidence of two note icons with the same pitch and different timbres may lead to ambiguity. Therefore, when the situation gets complicated, with many coincident notes of the same pitchl and different timbres, two or more parallel and synchronous scores are written, as is common practice in CMN. Tne parallel scores are played together. 10) Every, icon note can have an Echo option activated. It is then followed by a series of delayed identical icon notes with diminishing amplitudes. These amplitudes can decrease linearly in time, or as a decaying exponential, or as any other function programmed by the user. In the score the Echo function is marked by "E" to the right of the icon. 331

Page  332 ï~~Technical Details The latest software version is called TAL3. It was implemented on a. 386-PC with Super VGA graphics using Object-Oriented lurbo Pascal 6.0. A new version, TAL4, uses Windows and is being written in Turl)o Pascal Windows. The sound genera/o-r" hardware is a PC pluggable boa.rd called Music Magic System. It is ca)able of simultaneous real-time generation of up to eight oscillators, using the additive synthesis method. lip to three such cards can be plugged in one PC. A new lranspul:r sound generation hardware unit is being developed [2]. It is a rultiprocessin g unit of hardware and software in which several transputers operate in parallel. A tree configuration which can be expanded by adding more transputer cards, can provide hundreds and thousands of rea.l-time simultaneous oscillators. IconEye is the name of the associated software package that interprets and identifies the icon pa.Iameters fr'oil the in age of the sca.il 1cIel pa.per-w'ittell icon score [3]. TLP (TAI, Learning Program) is a. Sottware package [41], whose a,im is to teach musicians low to use the 'Tl l, systei i. It la.kes the user' on a lea.iiing tour along lhe various feat it'es of the ssteml-, asks lii1 1to do some exercises and to jpcr'orm- certain tasks, making the userlearn the various possibilities by doing them himself. Conclusion 'T"]e lll control of all sound components brings about a wealth of information which (lema.nds very exacting perception in clua.ntlity and quality. Witlout a notation in combination with the comp ter i ust.irn ment,, the composer cannot fully (exploit the new possi)ililies in musical thinking. The icoloogra. )l ic notation allows full command of all p)arameters. The computer, abl)le to learn the reading of the iconogra.plic score, realizes all intentions of the composer. In this way the history of music opens a new area of compositorica.l thinking, a. perfect example of seamless am algaima.tion of science into arts. References [1] 'Icon Notation for Electroacoustic and Comlputer Musi', Uri Shimony, Shlomo Markel and Josef Ta.l, Proceedings of the Interna.tiona.l CoipultNer M usic Conference, 1CMC-88, Cologne, Germany, 1-.-24 September 1988, pp. 4:0-435. [2] 'An Expandable IHea.l-Tiime Transput.er Sound Generator", Avi Pa.rash and Uri Shimony. Proceedings of the International Computer Nu sic Conference, 1CM C-91, Montreal, Canada., 16-20 Oct~ober" 1991, pp. 2"26-'228. [.3] 'IconEye version:3.0", User's an d Programmaer's Manuals. Internal publlication, Laboratory of Coin puter NI usic Engineering, Dept.. of Electricl Engi neeri ng,Tech n ion. [41] 'TL1I - Tlal Leri~ I rogra.in, UJser's and Programmer's Manurals. Internal lpl~llication, Labora tory of (Coin pu tir Ni usic E'-ngi neeri ng, Dept. of" Electrical Lngi ne'eri ng ji ccliiiion. 332