Page  00000001 The Transformation Engine "Always the same, but never the same way." Heinrich Schenker B. Graziano Degazio ijfemusic@sym a Abstract The Transformation Engine is a software music composition system for the Macintosh computer, based on a hierarchical model of musical structure derived from the theories of Heinrich Schenker. It implements processes of musical transformation in real time, i.e. Schenkerian prolongations ("composing-out") can be executed while the user listens to a high performance MIDI rendering of the music. The Transformation Engine also employs technical devices derived from the musical theories of Joseph Schillinger. The software has been used for algorithmic composition using planetary position data and chaotic processes as drivers for musical transformations. It also has applications to traditional forms of composition, sonification and soundtrack composition for visual media. 1 Background and Foreground The Transformation Engine takes a hierarchical view of musical structure and implements both background and foreground levels of the Schenkerian model (Schenker 1969). 1.1 Background Structure Any correctly structured MIDI phrase can be used as a background structure, either created directly within the software or imported as a Standard MIDI File from other applications. Example 1 shows a typical background structure phrase. Note that for convenience each bar specifies a complete chord consisting of up to eight notes; the actual time scaling is determined during performance, as described below. The Transformation Engine carefully preserves not only the pitch content of each chord but also the voice leading from chord to chord. These chords, modified by the controls described below, provide the background structure for the resulting musical composition. 1 2 3 4 5 6 7 8 i l: Wi P. IhI 1.2 Foreground Structure The background structure becomes "activated" by means of foreground elaboration into an actual musical surface. Like the background structure, the details of this foreground elaboration are specified with a MIDI phrase, which can be imported from other MIDI applications or recorded within the Transformation Engine software itself. These phrases are complete MIDI performances recorded with rhythmic detail to 1/240th of a quarter note. They can also employ continuous controllers and pitchbend data. Up to ten of these phrases may be dynamically combined in realtime into a single resultant phrase. The foreground elaboration can attain a high degree of complexity by this means. 2 Resultants The use of resultants as a principal technique is derived from the musical theories of Joseph Schillinger (Schillinger 1941, 1948), where they have been elaborated in sometimes excruciating detail. 2.1 Rhythmic Resultants Rhythmic resultants are the patterns produced by the 4ý 3 Firnd lthe ata ~- fl LiL IT!1J P f ff f P ff $ A.o Example 2 - Rhythmic resultant of 4 - 3 (from Schillinger's System of Musical Composition) superimposition of simple repeating patterns, and are Example 1 - Typical Background Structure phrase Proceedings ICMC 2004

Page  00000002 exactly analogous to the interference patterns produced by sound waves of different frequencies as they combine with one another. For example, a pattern of four units duration combined with a pattern of three units duration produces the rhythmic resultant of example 2, well known to all musicians. 2.2 Melodic Resultants Melodic resultants arise from the combination of two or more melodic contours. For example, if a short ornamental figure (such as the 16th note turn, Theme A in example 3 below) is combined with a rising pitch pattern in quarter notes (Theme B), the melodic resultant is a series of matching ornamental figures rising along the steps of the pattern: Theme A Theme B Melodic Resultant, _ Example 3 - Simple Melodic Resultant This technique is at the heart of the compositional process in the Transformation Engine. It corresponds to Schenker's notion of "prolongation" or "composing-out" - the compositional process whereby the composer takes the relatively static background structure and "activates" it into a musical foreground of rhythmic vitality and interest. 3 BASIC TRANSFORMATIONAL PROCESSES This section consists of a brief description of the principal operational features of the Transformation Engine, for the purpose of explaining its potential for musical transformation. 3.1 Background Structure Transformations The background structure can be altered in both pitch and time by means of the controls in the "Background Structure" area of the screen (figure 1). The key can be changed with the Transposition control. The Time Scaling sliders adjust the duration of the Background Structure phrase relative to the Foreground Structure. The two sliders are read as a fraction so that musically useful time scalings are easily arranged. For example, if the Background Structure is to proceed at twice the original speed, the upper integer is set to "2" and the lower integer to "1". To proceed at two thirds of the original speed the upper integer is set to "2" and the lower integer to "3". E Background Structure Setting either numerator or denominator to "0" ProkofiefGavott freezes the background structure at its current Time Sainn.: position. Lastly, the "Loop Start" and "Loop 2:. o8 Length" parameters are used to select the active portion of the MIDI.Loop Start: W 000:000 phrase. Loop Length: 10:00:000 Effective Length: 40:00:000 | 3.2 Foreground Transposition: 0 semitones Structure Transformations TransformationS Fig. 1 - Background Structure Up to ten Controls foreground structure phrases may be used simultaneously, grouped together with some additional information as a Transform Set (figure 2). The ten phrases can be used either individually or summed together according to the rules of musical resultants described above. The power weighting for each of the ten phrases is specified with the short horizontal slider to its right. These are controlled en masse by means of the large vertical slider labeled "Dyn." (Master Dynamics). The "Tessitura" slider controls the overall pitch register of the resultant. The "Width" slider controls the amount of pitch used in the melodic resultant calculation and also allows for melodic inversion when set to negative values. "Articulation" controls the length of the notes; less than 1.0 means that notes will be played shorter than their recorded values, and greater than 1.0 means that notes will be played longer than recorded. The "Dynamic Note Reduction" slider controls the density of notes in time. Set to either extreme, all recorded notes are used in the resultant, while intermediate settings allow notes to be ignored based on their rhythmic placement. The Time Scaling, Loop Start and Loop Length parameters all function as they do with the Background Structure. All of the above controls may be adjusted in real-time in up to three ways: by user interaction with the on-screen faders; by means of pre-recorded data formatted as MIDI continuous controls; and by direct computation through an algorithm programmed by the user (in the host language, Forth). The Transformation Engine uses not only pitch and timing information, but also continuous control information such as mod wheel, breath pressure and pitch bend. A Proceedings ICMC 2004

Page  00000003 Transform Set phrase is therefore a complete MIDI performance recording. the author's Amerika First, based on a march by John Philip Sousa. 4.2 Algorithmic Composition Applications of the Transformation Engine to algorithmic composition include the use of fractals or chaotic functions (Degazio 1986, 1988, 1993) as drivers to the engine parameters. This comprises such well known chaotic processes as the Lorenz attractor (used in weather simulations), strange attractors such as the Logistic Equation, and fractal structures well known from their application to graphic data, such as the Mandelbrot Set. This quickly branches off into related areas of mathematical research such as simulations of natural processes (e.g. wind, tides, biological reproduction). Audio recordings of natural phenomena such as ocean surf, and research data such as shockwave pressure measurements derived from nuclear weapons tests can also drive the Transformation Engine parameters. A particular application along these lines, already implemented by the author, is the use of planetary positional data. In this case the result is a true "Music of the Spheres" rendered with a precision and musical elaboration unimaginable by Pythagoras or Kepler. The music resulting from such applications corresponds dynamically to the simulated or recorded process that drives it. In other words it is possible to produce music with a dynamic contour (and pitch contour, note density contour, etc) directly related, to the particular process; in fact, the Transformation Engine can be used to produce a musical simulation of any natural phenomenon that can be recorded or modeled. This leads directly to its promise as a "scientific visualization" engine for large quantities of laboratory data, a process known as sonification. 4.3 Sonification The rendering of large quantities of numerical data in such a way as to make patterns within the data comprehensible is known in general as scientific visualization, a name that implies that its most common method is graphic or visual. But some types of data, typically time-varying, are more suitably rendered for the ear than for eye. This rendering, known as sonification, is commonly carried out by a simple mapping of data dynamics to an easily audible sonic parameter such as pitch or loudness. Fig.2 - Foreground Structure Controls 4 Compositional Applications The software is truly an "engine" which can be applied to many musical tasks. The following briefly describes four such tasks: Traditional Composition, Algorithmic Composition, Sonification, and Soundtrack Composition. 4.1 Traditional Composition The Transformation Engine lends itself to a traditional model of musical composition whereby a limited number of "themes" or "motifs" are employed repeatedly, but are continually varied - as elegantly stated in Schenker's epigram taken as the motto for this paper. This mode of composition has been thoroughly documented by Schenker and others from the works of composers of the Classic and Romantic periods such as Haydn, Beethoven and Wagner and further on to modem composers such as Stravinsky (Schenker 1969). A "post-modern" extension of this mode of composition has been used to create musical works referring to pre-existent works or styles of music, such as Proceedings ICMC 2004

Page  00000004 While the Transformation Engine can carry out b this sort of mapping R Ba ctVground Stri easily, it goes far ProkofiefGavo beyond them by allowing mappings to Time Scaling: intrinsically musical 2:8 parameters such as note density and Loop Start: -0 durations as well. In Loop Length: H1 addition the software Effectie Length: 41 allows highly multidimensional data Transposition: mappings through the DNR Rh thm Map fifteen parameters Ave Regina Co( controlling each resultant (the ten F;1yth CI'H2 foreground phrases, LipLSt H Master Dynamics, Tessitura, Pitch Current DNR: 127 Width, Duration, and Dynamic Note Reduction). GStmicture urtomat ON Trak 08Output 4.4 Soundtrack Composition for Visual Media Fig. 3 - Transformation Engine, Main Control Screen A further application of the Transformation Engine is to film or theatrical scoring and other forms of music combined with image. The precise control of high-level compositional parameters simultaneously coordinated with moving images is a film composer's dream. Because the Transformation Engine performs MIDI music in real time, a composer experimenting with musical ideas can hear immediately the results of their experiments and see them along with the corresponding images. 5 Acknowledgments The Transformation Engine was originally conceived and developed as a non-realtime application under Canada Council Media Arts Research Grant, 1999, Harmonices Vitae: The Music of Life. The realtime version discussed here was subsequently developed by the author. Many thanks are due to other artists and musicians whose ideas and enthusiasm for the project have helped it along, including Henry Jesionka, Gustav Ciamaga, Charles Bagnall and Alyssa Ryvers. References DEGAZIO, B (1986), Musical Aspects of Fractal Geometry PROCEEDINGS OF THE INTERNATIONAL COMPUTER MUSIC CONFERENCE DEGAZIO, B (1988), Fractal Geometry and the Schillinger System Proceedings of Diffusion!, Jean-Francois Denis, ed. DEGAZIO, B (1993) New Software Composition Tools Proceedings of the Fourth Biennial Arts and Technology Symposium, The Center for Arts and Technology, Connecticut College, New London, Connecticut, USA SCHENKER, HEINRICH, (1969) Five Graphic Music Analyses Dover Publications, New York SCHILLINGER, JOSEPH, (1941) The Schillinger System of Musical Composition Da Capo Press, New York SCHILLINGER, JOSEPH, (1948) The Mathematical Basis of the Arts Da Capo Press, New York Proceedings ICMC 2004