Page  00000001 ILEARN: AN ITUNES AUDIO INDEXING PLUG-IN FOR EDUCATION G. McAllister, P.S. Taylor and M. Rodgers Sonic Arts Research Centre Queen's University Belfast ABSTRACT This paper details iLearn, a plug-in which allows recorded audio to be tagged and subsequently retrieved using streaming and the familiar environment of Apple's iTunes. In particular, the plug-in allows multiple sections within a recording to be indexed and associated with a unique timestamp. Therefore, students can search for a particular keyword within a one hour lecture (or other lengthy audio files) and have the ability to jump straight to that particular section of the lecture. Initially, iLearn was developed for the indexing of lectures, however features were added to allow for other possibilities. The paper will detail the development of the plug-in and the user interface design. The paper will conclude with other potential applications for the plug-in and future work in the area. 1. INTRODUCTION The role of technology in eduction is typically concerned with using software and digital storage media to represent or support the usual methods of teaching (mainly textual documents, or encyclopedias on CD-ROM). This is usually performed on a local computer, not connected to the Internet. Distance learning, or eLearning, is an approach which is designed to use electronic communication methods to achieve effective teaching. Such methods would typically use technologies such as websites, PDF documents, webcasts or MP3 audio files. In October 2002, MIT released OpenCourseWare [1]which currently offers the materials to over 900 undergraduate and graduate courses free of charge online. For any given course, lectures in PDF format and audio transcripts in Real Audio format are available for download. It is the issues of finding and navigating within audio lectures that are the focus of this paper. There are two main issues regarding lectures in audio format: 1. which lecture should I download (selection) 2. where in the file is the section of audio that I need to listen to (navigation). In the MIT model, which is similar to others including the Open University [2], a lecture item (notes or auditory transcript) is located by searching for a keyword. While this is moderately successful for documents (PDFs may not always be searchable for text), finding an audio lecture which contains a key phrase is not possible. Also, if the user is streaming an audio lecture but are only interested in a specific section, then they may have to listen for the entire length of the lecture to locate the part they are interested in. In order to address these problems, a system is required which enables fast and flexible searching of audio files, and also allows the user to easily navigate within the file. Another requirement is that the system should allow for the streaming of audio, the educational establishment may not wish for the audio files to be downloaded to hard disc permanently. After evaluation of these requirements Apple's iTunes [3] audio manager was considered an ideal software base to extend from. In particular, iTunes offers the following: 1. it allowed for rapid searching using the built in search engine 2. it is scriptable, meaning its functionality can be controlled and extended using a high level programming language (Applescript [4]) 3. it has built-in support for streaming of audio. In addition there are other advantages in using the iTunes software as a target platform. Firstly, it has a familiar user interface to millions of users already, particularly students. Secondly, iTunes has seamless integration with Apple's iPod, so lectures could be listened to on the move. It is intended that by using iTunes and extending its current behaviour, searching of audio files will become as easy and as powerful as text-based document searches. The following section will detail how the plug-in was developed. 2. ILEARN PLUG-IN The description of the plug-in development shall be segmented into the GUI design and functional development. 2.1. User Interface Design The look and feel, and general usability of the plug-in was designed to resemble the main iTunes interface as much as possible. The reason for this consistency was to ensure that users would not have to invest unnecessary time learning a new interface style. The UI was designed using

Page  00000002 Figure 1. iLeam User Interface Design be able to search for a keyword and skip directly to section of the lecture where it is discussed. The user would also be able to see the title of the lecture associated with the keyword to confirm its relevance. In order to obtain the current time-stamp from iTunes, the following Applescript code segment was used, the result being passed back to the iLearn UI: on getpos() tell application "iTunes" if selection exists then copy selection to sel set currentpos to player position end if end tell return currentpos end getpos() The code fragment above shows the English-like nature of Applescript. Tag data is stored in the Comments field in the form of a tuple <keyword, timestamp>, with multiple entries being allowed (typically around 20 entires, or until the 256 byte limit was reached for the Comments field). 3. FEATURES The iLeam GUI can be initiated by selecting it from the Scripts menu at the top of the iTunes menu bar. If an audio file is currently being played in iTunes, iLearn will be populated by the information from the current track (title, artist etc). When the user changes tracks in iTunes, all display data is automatically refreshed in the iLearn GUI. Some of the features of the iLearn GUI will now be discussed. 3.1. Load Track The Load Track feature (see Figure 2) allows the user to import an external file into iTunes for tagging, any current ID3 tag data will be displayed. Interface Builder [5], part of the Applescript Studio development environment on Mac OS X. Figure 1 shows the developed UI. This interface is designed to be used by two main types of user; the lecturer who is responsible for inserting the keywords into the audio file, and the student who will navigate through the audio file by clicking on specific keywords in the list. 2.2. Functional Design It is important to remember the two main criteria that were to be addressed by the plug-in; (1) searching of files, and (2) automatic navigation within a file. The first requirement was achieved by storing keywords within the ID3 tag [6] of the audio file. In iTunes, the ID3 tag has a visual editor built-in which contains fields such as Artist, Album, Year, Track, Composer and Comments. Most of the fields are self explanatory however Comments is a free-form field where any user-defined information can be entered. By entering keywords pertaining to the associated lecture, the audio file would now be displayed in the search results if any of the key words are entered into the iTunes search box. The ID3 tag user-interface is consistent regardless of the format of the audio file being tagged (AAC, MP3, AIFF). By storing the tags within each audio file, it ensures that the tagging information travels with the audio file. Keying the keyword list directly into the comments field directly however, is not user-friendly, nor does it associate the keyword with a specific timestamp. In order to solve this problem and achieve the second requirement (automatic navigation with a file), the behaviour of iTunes needs to be extended. The main aim was to allow the user to associate a keyword with a particular timestamp within the file. Using this approach the user would Figure 2. Loading an external audio file 3.2. Delete Track The user can delete a track using the iLearn GUI, this is necessary as the user may wish to import another version of the audio file which contains additional comments.

Page  00000003 3.3. Edit-On / Off In order for a lecturer or other privileged user to enter tag data, they must enable the Edit-On button, and then enter a valid username and password. The user is now free to enter the tag information; keywords and time-stamps. When the user has finished tagging, they end the session by clicking on the Edit-Off button. Figure 3. Validating a user to allow tagging 3.4. Navigating Once tagged, the user may now 'skip' around the audio file in a non-linear method by clicking on the associated keyword (see Figure 4). 1. The lecturer would record their lecture and convert the file to AAC, MP3 or similar (if it wasn't in that format already). Conversion of file formats is a builtin feature of iTunes. 2. They would then import the audio file into iTunes directly, or using the import feature in the iLearn GUI. 3. Once imported, they will enter edit mode, and tag the audio file as necessary. 4. The lecturer will then place the tagged audio file onto the audio server, which is possibly their own computer, and ensure that the sharing option is enabled in iTunes preferences. 5. At this point students can now search for the lecture using the tags or the title of the lecture by using their own local version of iTunes on the network (their client) It is worth noting that it is possible for many students to stream the same audio file simultaneously. In addition to offering a searchable online audio lecture system for the students on a particular module, this system would also allow for an organized body of knowledge to be constructed across schools or faculties within the university. This type of facility is becoming increasingly important as university pathways attempt to offer module exchange schemes, e.g. a Computer Science school might accept students from Electronic Engineering pathways on certain modules. Implementing online resources is one approach to simplifying module exchange schemes. Further built-in features are also available to help with the assessment and feedback of the module, these include: Ratings - the students will be able to rate a lecture according to their own criteria. This criteria could be agreed between the lecturer and student, and is rated out of 5. Playcount - lecturers can easily determine which of their lectures is the most popular by examining the playcount for each lecture. Smart Playlists - both lecturers and students will be able to organize their audio lecturers in any format they wish using the built-in smart playlist feature of iTunes. For example, it would be easily possible to view all lectures in the schools of both Computer Science and Music which contain the keyword "DSP" from the year 2000 onwards. In addition to the student being able to re-listen to lectures via streaming, they are also able to record their own audio files, or commentaries on a lecture. Using this approach, students are able to build up their own thoughts and reflections on a lecture, which can subsequently be shared out to other students (or even the lecturer). Figure 4. Tags inserted and time-stamped 3.5. Loop Feature When the user clicks the Loop-On button they will be asked to click on the tag from the list where they want the loop to start. Once that's confirmed, they will be asked to click on the tag where the loop should stop. The audio file will now play from the start tag to the stop tag and loop around until the user selects the Loop-Off button. The iLearn GUI also replicates the Play, Stop and Pause transport functionality of iTunes. 4. DISCUSSION As this system may be used by a lecturer from any school within a university (not just Computer Science or Engineering), the GUI and the entire operational process must be intuitive. It is intended that the system would be used in the following approach.

Page  00000004 5. CONCLUSIONS AND FUTURE WORK This paper has presented a system which extends the behaviour of iTunes by implementing a plug-in to allow for users to skip directly to key areas of interest within audio files. It has catered for two types of user, a lecturer, who is responsible for inserting and controlling the tag data and associated temporal placement, and the student who can stream the audio file sequentially, or skip to key sections as required. It has explained its rationale for using iTunes as the basis for the software application and shown how the default functionality can be extended using Applescript. The paper has outlined how the system is used, and how it can be extended to create a online body of knowledge to allow universities to offer their lectures to others. However, the system is still in development and several features are still to be implemented, most notably, the ability to synchronize slides with an audio lecture. This feature will be implemented using the Album Art field within the ID3 V2 tag structure (see Figure 5) and Applescript to facilitate the syncing. The slides would appear in iTunes in the album cover art area (user scalable). This feature would allow the audio file to encapsulate the complete data for a lecture, audio, slides and notes in PDF format. of this program was to, facilitate the use of information technology in innovative ways within the classroom and across campus. It is intended that the system detailed in this paper is one such use of innovative technology which has a close integration with the iPod. This paper has focussed on the use of facilitating and sharing lectures online, however it is envisaged that the same software could easily be used for particular niche areas such as music education, where particular areas of classical works could be tagged according to selection criteria, and used for analysis by students. Using this approach, the tags are not only used to skip to a particular section of a musical composition, but may also be used to define the form of the piece. One extension which is currently being used is to include the score for classical works in the iTunes library in PDF format. This enables the user to search for a composition and the associated score file, if available, will also be viewable. The next extension to this particular use of the software is to control the turning of the pages of the PDF score in sync with the audio playback. This paper has presented only some of the potential possibilities that the plug-in has to offer. As the tag data is essentially meta-data which is defined by the lecturer or student, it is hoped that unexpected uses of this system will be discovered and made available. 6. REFERENCES [1] MIT OpenCourseWare Online, [2] Open University, [3] Apple iTunes, [4] Applescript Documentation, [5] Apple Interface Builder, [6] The Official ID3 Specifications Website, [7] Duke University iPod Programme, Figure 5. ID3 V2 Tag Frame Format In August 2004, Duke University [7] gave 20GB iPods to each first-year student starting its university. The goal