Page  00000282 VScore: A Real-time Visual Application for Scoring Music David Topper and Juraj Kojs Virginia Center for Computer Music University of Virginia topper, kojs@virginia.edu Abstract In this paper, we propose VScore: A Real-time Visual Application for Scoring Music, which enables the composer to create musical score from digital image and movie files which can be played back seamlessly together in synchrony. Further, the application allows the performer to insert text performance remarks directly into the score. We explain how the visual score format eliminates performance distractions caused by page turning and stopwatch following. VScore is particularly suitable for scoring compositions that involve live performance and electronics, by allowing for any type of image to represent a musical idea. The application is available under the GNU Public License. 1 Introduction In the recent years, a number of composers have structured their compositions in absolute time. As opposed to the musical time, which is represented by beats and measures, absolute time presents seconds and minutes as temporal units. Scoring music in absolute time usually results in a design of graphical score. Graphic scores have been developed concurrently with the absolute time composition. Composers such as Earle Brown, John Cage, Cornelius Cardew, George Crumb, Luciano Berio, Karlheinz Stockhausen, Christian Wolff, and others took advantage of absolute time organization of their music (Cage 1973). Scoring music in absolute time presents a number of organizational, notational and performance issues. Absolute time based scores are usually organized into pages with either fixed or variable durations. In other words, each page can either reveal the same duration (e.g. 30 seconds per page), or duration varies from page to page. An example of a score with fixed duration of 1 minute per page is displayed in Figure 1. To follow a composition based on absolute time, the performer is often provided with a stopwatch. When multiple performers and/or electronics are involved, usage of synchronized stopwatches for performance coordination becomes rapidly cumbersome. In addition to performing music and following the time on a stopwatch, the performers also turn their pages. Needless to say, these issues increase performer's distress and often unwillingness to perform such music. In this paper, we propose VScore: A Real-time Visual Application for Scoring Music, an application, which enables complete digitization of the score and eliminates the problems such as cumbersome time following and page turning. Simply, the digital score movie presents continuously repeated score images over-layered by a moving timeline. To enhance an interpersonal relationship between the digital score and the performer, VScore enables the performer to create performance notes in the score. Thus, the performer needs only a laptop with VScore application to perform a piece of music organized in absolute time. For performances that involve spatially separated performers who read from a single score, we suggest using a projector to enhance the score visibility. Figure 1. Juraj Kojs: Revelations. An example of graphical score with fixed page duration (30 seconds). 2 Background Research Music scored in absolute time can be designed in notation programs such as Finale (Finale 2006), Sibelius (Sibelius 2006), MagicScore (MagicScore 2006), Nightingale 282

Page  00000283 (Nightingale X 2006), ABC (ABC 2006), LilyPond (LilyPond 2006), and others. In fixed page duration scores, a single performer or smaller ensemble can even read the digital score off the computer screen following the time pointer in the playback player in such programs. However, designing graphical scores in these applications is awkward. Graphically oriented scores may be created in Adobe applications such as Adobe Photoshop (Adobe Photoshop 2006) and Adobe Illustrator (Adobe Illustrator 2006) or any combination between notational and image editing applications. Moving presentations of such scores can be assembled in Microsoft PowerPoint (Microsoft PowerPoint 2003) and Macromedia Flash (Macromedia Flash 2006). Other resources for digital music scoring are reviewed in (Hewlett 2001). In general, the avaliable applications are costly. Whether designing traditional or graphical score, a composer is now invited to assemble his/her score a free application VScore. 3 Simple Score Construction Before assembling the score movie in VScore, the composer creates image or movie files of his score in another application. The composer can also opt to scan existing paper scores and prepare digital images for VScore. Whether digitally generated or scanned from paper, these files should be saved at the highest resolution possible. The high resolution will ensure the display quality of the assembled movie. Particularly when projected on a screen, the score clarity is crucial. In the future version of VScore, we would like to enable the composer to create simple graphical scores directly in the application. In this initial version, the composer defines the timeline according to the fixed, variable, or combined duration of the score pages once the files are imported to VScore. In all cases, the duration of each page should be divisible by a single chosen unit (e.g. 1 second). Thus, a composer can present pages of different durations (e.g. 30", 45", etc.), however these should be dividable by the frame speed at which the pointer will move (e.g. 1 frame per second). In order to create score, one starts with inserting layers (eg., images or movie files) into the timeline editor window. Once compiled, the user can then select what interval the score cursor should move across the "page." Options are either continuous, with the cursor moving constantly in realtime, or discrete with the cursor moving every X seconds (eg., 1, 2, 10, etc...). Both the performer and composer can then insert "notes" into the timeline by scrolling playback to a specific moment, clicking on the screen, and entering the note text. An additional layer can be added as needed by both the performer and composer. Text based notes are added by scrolling playback to the desired temporal location, then clicking on the main score playback window. Text boxes may be placed directly on top of the score, with specific popup time, duration and location. The layer can be saved with the project or deleted afterwards. 4 VScore VScore is the standalone application currently under development. Using the GTK/GNOME library to manage the general interface (eg., pulldown menus, widget placement) and QuickTime for image and movie files. 4.1 Editor The basic working space or editor consists of the timeline and layers into which the composer imports images and movies. By clicking on the timeline in the editor, the composer is able to select both still frames and/or movie events to be displayed at specific times. The editor is displayed in Figure 2. Left column presents the layers and right column displays the content inside the created layer. Selected frame becomes highlighted. A frame in which a new score image appears is marked in color. The content of the layers can be moved, copied, and deleted. Figure 2. VScore editor displaying layers and frames. The layers eventually create the main display for the composition's video score. Once constructed, the score movie can be viewed and controlled via the playback controls. 4.2 Timeline and Player The playback representation of the timeline involves a ruler, which scrolls across the page in real-time. The timeline is also an internal structure which checks at predefined update intervals (eg. 1 second or 0.5 seconds) for events. An example of a graphical score in VScore player is displayed in Figure 3. The top box displays the time line. The smaller numbers positioned on the top of the timeline portray fixed 10-second segments as set by the composer. The larger numbers inside the timeline (5 and 8) indicate the current time (as a one digit second) of the cursor in the score in real-time. Further, the dotted line cursor line conveniently displays the exact place in time where the performers should be. The large horizontal arrow suggests the motion of the cursor from point 5 to point 8. 283

Page  00000284 ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` ` `::TO:^^^^^^^mmmmmmmmmmm ':': |:: *:S S:::::::;:: ::;:; i:5:......I.... Figure 3. VScore playback window of J. Kojs's En Una Noche Oscura scored for four performers and computer. 4.3 Control Panel The control panel displays the current time and allows the performers to play, pause, forward, and reverse the score. The panel is conveniently designed to resemble the standard player panels from the audio and video applications. It can be moved around the screen and closed if needed. Hitting the space bar will start the movie, while hitting it twice will stop it. Figure 3 shows the control panel. ^| i~ ^^^^^^ fr^ I ^ ^:1 ~ - |,~*s ^ p ~^__^^^^. ^^S.^ i t*:\:%**.~?"',^:::,| ^ _____,.^;,<;*:.*,..,.1 ' ---- ~ Figure 4. Performer's annotations in the highlighted text boxes exemplified in the VScore of J. Kojs' Garden of the Dragon. 5 Conclusion VScore presents a novel approach to scoring music. The application facilitates performance of compositions based on the absolute time. It's real strength lies in its simplicity: an elegant means of displaying images and movies in a coordinated timeline. In the near future, we would like to embed a graphical editor that would allow the composers to generate complete graphical scores in VScore. Although given the wide variety of free image processing software, we don't plan on writing an application to compete with Adobe Photoshop or the GIMP. Further research will look into networked based versions of VScore. Using multiple laptops and/or displays, a single composition may be distributed and synchronized between multiple performers, each with specific sections, playback displays, and notes. Figure 3. VScore control panel with additional time display and playback control functions. 4.4 Performer's Notes By clicking on the actual image or movie in the editor, the composer or performer is able to add performance comments, which will pop up in at the user-defined time during playback. Personalizing the score is an important element in the interaction between the performer and score. The layer with the performance notes may be edited, deleted, or saved for later usage. Figure 4 shows an instant from the score of J. Kojs' Garden of the Dragon for three performers and computer (the tops staff). Two text annotations made by the Performer 1 are placed inside of a text box and marked in color. The entire composition is saved as a project, consisting of images, movies, and VScore specific data stored in an XML file. References [1] ABC. Retrieved from [2] Adobe Illustrator CS2. www.adobe.com. 2006. [3] Adobe Photoshop CS2. www.adobe.com. 2006. Retrieved from Retrieved from [4] Cage, J. and Knowles, A. Notations. Reprint Services Corp. 1973. [5] Finale. Retrieved from i. 2006. 284

Page  00000285 [6] Hewlett, Walter B. and Selfridge-Field, E.. eds. The Virtual Score: Representation, Retrieval, Restoration. MIT Press, Boston. 2001. [7] LilyPond v. 2.9.11. Retrieved from www.lilypond.org/web. 2006. [8] MagicScore. Retrieved from www.musicaleditor.com. 2006. [9] Nightingale X, v. 5.2. Retrieved from www.ngale.com. 2006. [10] Macromedia Flash v. 8. Retrieved from www.macromedia.com. 2006. [11] Microsoft PowerPoint. Retrieved from office.microsoft.com. 2003. [12] Sibelius. Retrieved from www.sibelius.com. 2006. 285