Page  328 ï~~A Theory of Poetry as Music and Its Exploration through a Computer Aid to Composition Fred Lerdahl, USA and Xavier Chabot, IRCAM 31 Rue Saint Merri, 75004 Paris, France Abstract Lerdahl and Halle (in press) draws an extended mapping between between the sound structure of poetry and the hierarchical representations in Lerdahl and Jackendoffs (1983) theory of music. These parallels suggest transferences between the two arts by means of digital technology. Our first objective is to effect a step-by-step transformation of a poetic text into musical sounds while preserving perceptually the structures represented by the theory. This research can then serve as a platform for musical composition. To accomplish these goals we have designed and implemented a computer aid to composition within the framework of the PatchWork environment on the Macintosh. This paper presents the current status of the project through a live demonstration. The theory is outlined, the environment is presented, a sampled line from a Shakespeare sonnet is manipulated from PatchWork, and Chant synthesis is performed under the control of the same environment. The Music-Poetry Theory Lerdahl and Jackendoff (1983) models musical listening by means of four hierarchical components: grouping structure, metrical structure, time-span reduction, and prolongational reduction. Aspects of linguistic structure have a comparable organization, as the following summary outlines. (1) The prosodic hierarchy (which is distinct from the syntactic segmentation) is a hierarchical grouping of spoken segments, comprising the units of the syllable, the word, the clitic phrase, the phonological phrase, the intonational phrase, and the utterance (Hayes 1989). (2) Within each of the units from word to utterance, certain syllables receive more stress than others. In most poetry the stresses occur at regular enough intervals to induce a sense of meter. Though rarely isochronous, a poetic meter has the same formal structure as its musical counterpart, a leveled grid expressing periodic patterns of strong and weak beats. (3) Recent work in linguistic theory (Halle & Vergnaud 1987) notates phonological stress by a combined grid and grouping notation that can be represented by the musical time-span component, which assigns pitch stability within rhythmic structure. Similarly, the linguistic time-span analysis represents syllabic prominence within the prosodic hierarchy and in relation to the poetic meter. (4) An important feature of poetry is the pattern of full or partial recurrences of syllables, whether by rhyme, alliteration, or assonance. These patterns resemble the departure and return of pitch events described by prolongational reduction, but in the timbral domain. The resulting analysis is like the timbral prolongational structures explored in Lerdahl (1987). (5) A more pitch-like analogy to prolongational reduction concerns the melodic contour of an utterance. Speech contour in turn depends on the meaning conveyed (Bollinger 1986); in particular, contour peaks are governed by content words as projected by the prosodic hierarchy. Contour analysis thus becomes a kind of semantic prolongational structure. The following summarizes these componential mappings between music and poetry. Music Poetry grouping prosodic hierarchy meter meter time-span reduction phonological stress hierarchy timbral prolongation phonological prolongational structure pitch prolongation semantic prolongational structure ICMC 328

Page  329 ï~~We pass over specific derivations and consider only their outputs in the form of notations as in the figure below. The text is the first line of Shakespeare's Sonnet 29. The tree above the line models the semantic prolongational structure and the slurs beneath represent its phonological prolongational structure. The metrical structure is given by the dot grid. The prosodic hierarchy appears at the bottom as the timespan segmentation, with the time-span reduction notated by syllables at different levels. Semantic (contour) prolongational structure When in dis--grace with for-tunea men's eyes Phonological ___ (timbral) prolongational structure...... 0 Metrical grid -grace r-mnsesTime-span segmentation -grace for I eyes and - aed t for-grace I time-span reduction Implementation with PatchWork A computer-aided composition environment is a set of tools for the composer to deal with musical material at a precompositional stage where imagination and rigor compete for expression. Such an environment must provide an extensible set of tools not only to create material but to represent it, play it back, and modify it. The environment must be able to communicate with other programs for editing, performance, notation, and sound synthesis. It must allow the composer to reuse earlier stages of work for further development and to embed material within a structure at a larger hierarchical level. This point is essential if there is to be a relationship between material and form. PatchWork is an interactive environment for composition and synthesis control. The present version has been written in Common Lisp by Mikael Laurson, Camilo Rueda and Jacques Duthen (Laurson & Duthen, 1989, 1990). PatchWork allows the composer to create and manipulate musical material (scores, rhythms, envelopes, formal structures), to exchange this material with other environments such as MAX or a music notation program, to implement algorithms for manipulating musical objects and formal structures, and to control various synthesis models (MIDI synthesizers, csound, CHANT). PachWork's basic paradigm is that of functional programming through graphic patches. Patches are convenient for implementing functional trees that represent the structures assigned by the music-poetry theory, to perform actions at specific nodes in the structure, and to control sound playback and synthesis. The figure below shows the various parts of the PatchWork implementation of the music-poetry theory. All components are from the regular PatchWork environment except the tree node module. The functional tree represents a particular hierachical structure. The tree-collector module first evaluates the functional tree and applies a list of actions specified at each node to the sound synthesis model; then it schedules and generates instances of the sound model in an event sequence stored in and displayed by the music notation editor. The playback and synthesis commands are interpreted depending on the type of note objects collected in the score. ICMC 329

Page  330 ï~~The tree below once evaluated produces a functional expression such as: (amp 60 (pitch -110 (leaf 0 leaf 1)) (delay 10 (leaf2 leaf 3))) Choosing one action (pitch transposition, delay to change rhythm, amplitude) from the popup menu at a particular point in the tree displays the parameter values for this action at every node in the tree, and allows editing of this action parameter. In the example above, parameters for change in amplitude are displayed, but other actions (transposition and delay) appear in the expression because they have been set before. Actions from node to node are additive. head elab head ebab head Ielab head elab arg IeaOf arg 1leati arg Ileaf2 arg Ieaf3 Leaves tree tree.tree 0 tree elab head Ielab head tree ptree r* -- Action = Amplitude labl Iead ajustement du 60 **** parametre local tree PpMe t... T,-,.-P Popup M enu T Choix de lfaction Void Rese t The Functional Tree in Action For the demonstration, the functional tree is applied to two different types of sound objects: (1) sample objects that control the playback of samples in a sampling machine; (2) CHANT instrument objects that control CHANT synthesis (Barriere, et al., 1991). Concerning (1), the samples are from a reading of Sonnet 29 cut into syllables. The syllables are flattened out in timing, pitch, and amplitude to establish a baseline from which subsequent modifications can be measured. Acting on the sound objects through a functional tree ensures that the processing follows the rules assigned by the theory. A functional tree built from the semantic prolongational tree automatically assigns pitch contour, and a tree built from the metrical and time-span structures automatically assigns a durational sequence. When played with the sampled voice-objects using both trees at the same time, the result is a kind ICMC 330

Page  331 ï~~of test of the theory. To the extent that the theory is correct, the phrase should sound natural. MNc3 1_ when in dis- -grace with for- -tune and men-s eyes x-off y-off x-val y-val xzoom trans I o o 238 5900 1000 Foutrd EZZ F -]H W Finally, an example with CHANT synthesis control is presented in the demonstration. The CHANT sound objects replace the sampled voice-objects as the sound input to the tree collector. The CHANT objects follow the sequence of vowels in the Shakespeare text, creating a melodic simulation of the sampled version. The project was originally stimulated by the hybrid music-poetic compositions of Berio, Dodge, Lansky, and Ligeti. Its approach shares features with Pope (1990). The computer aid revives ideas in Lerdahl and Potard (1986).In future developments, we envision controlling a Phase Vocoder (DePalle & Poirot 1991) processing voice sounds, so that the time and frequency domains can be treated separately. The CHANT objects need to be enriched with appropriate interpolated consonants. At some point research will give way to composition. References Barriere, J.-B., X. Chabot, F. lovino, & M. Laurson. 1991. A New Chant Synthesizer in C and Its Control Environment with Patchwork. Proceedings of the International Computer Music Conference. Bollinger, D. 1986. Melody in Spoken English. Stanford: Stanford University Press, 1986. DePalle, P. & G. Poirot. 1991. SVP: A Modular System for Analysis, Processing, and Synthesis of Sound Signals. Proceedings of the International Computer Music Conference. Duthen, J., & M. Laurson. 1990. A compositional environment based on preFORM II, PatchWork, and Esquisse. Proceedings of the International Computer Music Conference. Halle, M., & J.-R. Vergnaud. An Essay on Stress. Cambridge, Mass.: MIT Press, 1987. Hayes, B. 1989. "The prosodic hierarchy in meter", in Kiparsky & Youmans, eds., Phonetics and Phonology: Rhythm and Meter. New York: Academic, 1989. Laurson, M., & J. Duthen. 1989. Patchwork: A Graphical Programming Environment in preFORM. Proceedings of the International Computer Music Conference. Lerdahl, F. 1987. Timbral Hierarchies. Contemporary Music Review, 2, 135-160. Lerdahl, F., & J. Halle. In press. Some Lines of Poetry Viewed as Music. In J. Sundberg, ed., Music and Language. New York: Academic. Lerdahl, F., & R. Jackendoff. 1983. A Generative Theory of Tonal Music. Cambridge, Mass.: MIT Press. Lerdahl, F., & Y. Potard. 1986. La composition assistee par ordinateur. Rapports de Recherche, 41. Paris: IRCAM. Pope, S. 1990. Producing Kombination XI. International Computer Music Conference. ICMC 331