Page  00000001 PAGE TURNING AND IMAGE SIZE IN A DIGITAL MUSIC STAND Tim Bell, Annabel Church Department of Computer Science and Software Engineering, University of Canterbury, New Zealand ABSTRACT This paper investigates attributes of the electronic display of sheet music necessary for the development of a digital music stand. We explore the two conflicting goals of minimising page turning effort and maximising the readability of images by conducting two user experiments. In our first experiment participants trialed various page turning methods through a sight-reading exercise, and filled out a questionnaire upon completion. In the second experiment participants viewed music at different sizes while listening to an audio rendition of the piece, and were asked to note any mistakes that occured in the played audio. Results from our experimentations showed that scrolling techniques did not work as well as page replacement methods, and that some reduction in the size of the music displayed is possible without significantly degrading reader accuracy. 1. INTRODUCTION The availability of inexpensive portable computers with mass storage, flat screens, and pen interfaces has made the digital music stand feasible, and several offerings have appeared on the market recently, including Freehand, 1 Espresso, 2 and eStand. 3 Because the display is controlled by a computer, options for displaying music and page-turning are available that could not be contemplated with conventional sheetmusic and books. This includes versatility in the enlargement of the image displayed, providing more flexible methods for turning the page, and animating the display to assist the performer. In this paper we report on experiments with musicians to evaluate the effectiveness of various computer-based page-turning methods, and we investigate issues surrounding the size of the music displayed. Digital music stands offer a number of benefits. As well as providing smooth page turns, they can act as a 1 www.freehandsystems.com 2 www.digitalmusicstand.com 3 www.estand.com John McPherson, David Bainbridge Department of Computer Science, University of Waikato, New Zealand portable digital library, so a musician can carry the equivalent of a shelf-full of music is a small case and quickly search for music without being concerned about wear and tear on the pages (including annotations). Being able to adjust the magnification on the screen has benefits that are explored later in this paper; there are also benefits of minimising the noise made by the page turning (particularly for recording), and in an orchestral setting it can avoid an audible dip in the volume level as some of the performers stop playing to turn a page. Of course, there are also disadvantages. The cost of a system can be prohibitive, and the effort involved in capturing documents is not insignificant. There are copyright issues (although as the technology matures there are likely to be DRM systems to deal effectively with these). The possibility of failure of the system during a performance (software, hardware, or power) may be unacceptable for some. Even if a digital system is more reliable than paper, a problem such as lost music, damage to the pages, or pages falling off the stand might be preferred to a more technical problem. The device itself may generate an unacceptable level of noise from disks or cooling fans. Finally, aesthetics are an important aspect of most performances, and a digital system may detract (or add) to this. Regardless of the genre of music, there are generally three distinct stages of interaction with sheet music that have different needs: individual practice, group rehearsal and performance. Note that a variety of contexts could occur, including a beginner preparing a solo to play in an exam, a small group preparing for a recording, or a large orchestra giving a major performance. A number of methods have been proposed for triggering page turns on an electronic music stand; a foot pedal is a simple solution that gives the performer full control over when pages change, but it is also possible to automate the turning, for example, by timing it from a backing track, or having the computer attempt to follow the sound during live performance.

Page  00000002 2. PAGE TURNING METHODS An effective page turning system should allow the user to read the music continuously without having even a short period where the "next" few notes are not visible. Furneaux and Land [4] found that performers read about one second ahead, while Sloboda [9] suggests that a musician needs to be able to read at least seven notes ahead of the current position. One possible method to avoid a gap at the end of a page is to stream the music as one long horizontal line. While this provides a natural and continuous display that relates directly to the temporal nature of the music, it does not usually make good use of the space available, and limits the total amount of music visible, which reduces contextual information. Scrolling the music vertically makes better use of the screen space, but raises the problem of how the musician keeps track of the current position in the score if the music is constantly moving. Another possible method for updating the displayed music is to not use scrolling, but rather overwrite past music with future music. This means that the musician must keep track of where on the screen both the current page and any preview is, although visual cues can help the musician quickly locate old, current and upcoming music. There are two approaches that can be taken here. If a single page is displayed then the top of the page can be replaced with the top of the next page while the performer is still playing in the lower part of the page. The second approach is when there are two facing pages displayed, in which case a whole "stale" page can be replaced while the other one is being used. That is, any time while the user is reading from page 2, the computer can replace page 1 with page 3. Once page 3 is being read, page 2 can be replaced with page 4, and so on. Variations of the single page method have been proposed in the past. McPherson [7] and Bellini et. al. [3, 2] suggest gradually overlaying the "stale" music, while Kosakaya [6] has two steps to replace the whole page, with user defined parameters to determine what percentage is replaced in the first step, and how long the delay is until the rest of the page is automatically overwritten. The two page system that replaces "stale" pages is suggested in the Muse system [5]. We performed user trials to evaluate six methods for page turning that are representative of the approaches described above. The methods were as follows: The single page method displayed one page of music, and lines of music that had already been played were eventually replaced with new music, so that when the musician got to the bottom of the screen, the music could be read from the top again. This method had the option of greying out the old lines of music as they were completed. This is the method shown in Figure 1. In the figure, the musician has already played the first three lines and is currently playing the fourth. The third line was greyed out when it was completed, and at the same time the second line was updated to show new music. Similarly, when the musi-::: '':::II""'" I`"` 'I~l II I...............111 1................................. T1 a...........-... -.. i i i, 1 i...i....-.......... Figure 1. The "single page" method, showing the top part of the page being replaced with the beginning of the next page. cian had just finished playing the second line, it became grey and the first line was updated to show the next line after the bottom of the page is reached. This continues in a circular fashion until the end of the piece is reached. In the double page method the musician saw the music in the form of two pages, side by side. This method updated the left-hand-side pages of music when the musician was reading from the right-hand-side, and vice versa. This method also had the ability to grey out the old staves. The horizontal scrolling method involves one continuous line of staves, taking up the width of the screen and scrolling from the right. This method has the ability to modify the scrolling speed. The music moved to the left so that the bar under the cursor (a red rectangle near the middle) was always the current bar (apart from the initial few bars before that point). The horizontal jump method is the same as horizontal scrolling, but the scrolling is done in large jumps rather than slowly moving the music. The vertical scrolling method makes the traditional sheets of paper appear as a scroll, with pages joined top to bottom. This method also allows the scrolling speed to be changed. The vertical jump method is as for vertical scrolling, but with fast jumps. For all of these methods we provided a cursor (a small red rectangle) to indicate where the current point in the music is. The goal was to get feedback about the general methods so that we can focus on the most promising ones in the future. Our experimental system provided limited animation in the form of the red cursor and gray areas to denote "stale" music. The reader is referred to Picking [8] for a more detailed study on the usefulness of animations for helping a reader to follow displayed music.

Page  00000003 Method Single Double Vertical scrolling Vertical jump Horizontal scroll Horizontal jump Average Rating 2.5 2.4 3.3 2.2 2.8 1.7 Range 1-5 1-4 1-5 1-4 2-4 1-3 Table 1. Ratings of methods of page turning on a 5-point Likert scale. The Wizard of Oz method, where the experimenter mimics the expected behaviour of the fully-functioning system, was used to keep the displayed music synchronised with the performer. This mimics the best score-following functionality that could be expected in a digital page turning system, and eliminates any factors from imperfections in that process. The study involved the musicians sightreading from the monitor, and filling out a questionnaire comparing their perceived ease of reading for each method with the ease of reading from paper. The responses were on a five-point Likert scale [1]. The evaluation study described in the previous section was carried out with six musicians from the Christchurch Polytechnic School of Jazz, New Zealand. Three were piano players, two were bass players (both used electric bass for the study) and the other was a trumpet player. The numeric results given in the Table 1 below measure the users' response on a Likert scale comparing reading from the display with reading from paper, where 1 represents 'reading from the display is much harder', and 5 represents 'reading from the display is much easier'. They have been averaged across all responses. In addition, participants were asked to rate the overall clarity of the screen display compared with paper. The average response for the clarity was 2.8, that is, on average they found it marginally harder to read than paper. None of the participants recorded an extreme response ("much harder" or "much easier"). The large range of responses and small number of participants make it difficult to draw significant conclusions about preferences between the methods, except to note that for all of the page turning methods there are both people who preferred it over paper and vice versa. Thus any digital music stand system would do well to offer a variety of methods to suit different user preferences and different types of music. Some additional observations can be made: There was a general trend of the three pianists agreeing with each other for each question. The pianists also consistently rated each method lower than the non-pianists, with the exception of horizontal jumping, which none of the musicians preferred. Also, both bass players preferred vertical movement over horizontal movement for both jumping and scrolling, especially when the music was made full-sized. The three non-pianists rated vertical scrolling either 'easier' or 'much easier' to use than paper. The main reason given was the ability to zoom the music to full screen, making the score larger than printed sheet music. Every participant considered Jumping either as hard as or harder than smooth scrolling for both Vertical movement and Horizontal movement. The pianists commented that the double pages would have been rated higher if the music had been larger. One of the bass players also said that the music was difficult to read due to its size for this method. This suggests that the 17 inch monitor was too small for evaluating this method. One interesting point is that the musicians did not find the animation (a small red marker) useful - they found that they did not really notice it. This seems to reinforce Picking's findings [8]. Three of the musicians also said that any movement at all became distracting. 3. SIZE OF MUSIC In the previous experiments the size of the music was chosen largely to match the conventional size on paper. However, a computer display offers the option of scaling the music to suit the user, including magnification to assist legibility, and reduction to enable the user to get more of the page on the screen. The preferences between these two might be different for the same musician depending on whether they are practising, rehearsing or performing. We conducted an experiment to find out whether users can read small music notation accurately on a computer screen, since the smaller the notation is, the more flexible the page turning mechanism can be, and the more context the user can see on the screen at once. We performed a within-subjects experiment testing two score sizes, large and small. The large score size had a stave height of 9mm, and the small small score height was 5mm. These sizes were larger and smaller than regular scores respectively, so they represent extremes compared with what musicians would be used to. The experiment measured the accuracy in identifying pitch and rhythm errors in music presented to the participants. Music was played when the participant was shown the score. Each piece played contained either two pitch errors or two rhythm errors compared with the displayed score. There were three dependent variables in this experiment: the number of errors correctly identified, the number of false positives (errors incorrectly identified) and false negatives (errors that were not identified at all). Both pitch and rhythm errors were used in this experiment to see if there was a significant increase in the error between the identification of these two elements as the score size decreased, i.e. was one element harder to spot at the smaller size? A study by Picking [8], does a similar evaluation, but chooses not to use rhythmic errors because it might cause confusion for the participants because of a possible flowon effect. For our evaluation, the pieces are only eight bars long, so there would be a negligible flow-on effect in this respect. There were sixteen participants, six boys and ten girls,

Page  00000004 all 15 to 16 years old, from an advanced music class. Students that had a reasonable level of sight reading ablity were invited to join the evaluation by their teacher. The experiment consisted of thirty short eight-bar exercises, of which fifteen where of the large size and fifteen of the small size. Each size category had a set of exercises that ranged in difficulty and contained either pitch or rhythmic errors. At the end of the set of exercises the participants filled out a questionare indicating how hard they found the two differing sizes were to read from, and which one they would prefer to learn or perform from. They were also given the opportunity to comment on either size, the experiment and computers in music in general. For each task, the participant can either make a correct error identification, miss observing an error (false negative), or incorrectly find an error (false positive). Overall we found that the number of correct identifications of pitch errors (average 11.8 out of 15) was statistically significantly worse (p < 0.002) than those for rhythm errors (average 14.6 out of 15). However, there was no significant difference between the number of correct error identifications between the large and small score sizes - although participants performed marginally worse on the smaller scores on average. For false positives (detecting a non-existent error) there was also no significant difference, although the average for the smaller scores was marginally higher. A similar effect was observed for false negatives for rhythm. From this we can conclude that relatively small scores (about 75% of the size of normal printed scores) can be read reasonably accurately on a computer screen. However, the results from the questionnaire show a marked preference for the larger sized score as it was the easist to read. The preference of size, in the area of difficulty in readablity, showed a significantly higher result for the larger images, giving a reliable result (Wilcoxon Signed Ranks Test, Z[15] = 3.38, p < 0.000363). Thus, while the participants can cope with small music surprisingly well, it can require more concentration and all else being equal, they would prefer a "normal" size. For a digital music stand this bodes well, as smaller images can be used, for example, when the performer is familiar with the music and only needs to refer to it occasionally. 4. CONCLUSION Our experiments have explored the two conflicting goals of minimising page turning effort and maximising the legibility and readability of images on a digital music stand. The page turning evaluation study carried out gave some insight into determining musicians' preferences for various methods of page turning. An interesting result of this study showed that scrolling is the hardest page turning method to use on a digital music stand, which highlights how unusual this application is since scrolling is very common on GUI editing systems, including notation editors. There seems to be a preference for the musician to be in control of when changes occur, and so a simple foot pedal or button is likely to be suitable for most needs. The experiment to determine how well students could read large and small scores on a computer screen found only insignificant differences between presenting music in two different sizes, which were (respectively) smaller and larger than conventional paper music. However, they did express a significant preference for the larger music. In practice there is a useful tradeoff available between the size of the image that is displayed and the frequency with which page turns must be made. Even fairly small displays can be acceptable for some situations, and it is likely that in general musicians will use enlarged displays with frequent page turns for practising, and as they become more familiar with the music, move to a smaller image with fewer page turns which is more suitable for a performance. 5. REFERENCES [1] V. Barnett. Sample Survey principles and methods. Hodder, 1991. [2] P. Bellini, P. Nesi, and M.B. Spinuc. Cooperative visual manipulation. of music notation. ACM Transactions on Computer-Human Interaction (TOCHI), 9(3):194-237, September 2002. [3] Pierfrancesco Bellini, Fabrizio Fioravanti, and Paolo Nesi. Managing music in orchestras. IEEE Computer, 32(9):26 - 34, September 1999. [4] S. Furneaux and M. F. Land. The effects of skill on the eye-hand span during musical sight-reading. Proceeding of the Royal Society of London Series B, 266(1436):2435-2440, 1999. [5] C. Graefe, D. Wahila, J. Maguire, and 0. Dasna. Designing the Muse: A digital music stand for the symphony musician. In Conference proceedings on Human factors in computing systems, pages 436-441, April 1996. [6] Juichi Kosakay, Miyuki Miyazawa, and Masumi Kizaki. Research and result of a performerfriendly electronic music stand. Technical Report 106, IPSJ SIGNotes Human Interface, 2003. http://www.ipsj.or.jp/members/SIGNotes/Eng/1 1/2003/106/index.html. [7] John McPherson. Page Turning -Score Automation for Musicians. Honours report, University of Canterbury, 1999. http://www.cosc.canterbury.ac.nz/research/reportslHonsReps/1999/honsD905.pdf. [8] Richard Picking. Reading music from screens vs paper. Behaviour and Information Technology, 16(2):72-78, 1997. [9] John A. Sloboda. The Musical Mind. Oxford University Press, 1985.