Page  27 ï~~A HIGH-LEVEL SYSTEM FOR MUSIC COMPOSITION Antonio Camurri (*), Corrado Canepa (*), Marcello Frixione (*) (), Carlo Innocenti (*), Claudio Massucco (*), and Renato Zaccaria (*) (*) DIST - Department of Communication, Computer and System Sciences University of Genoa, Via Opera Pia 11A, 1-16145 Genoa, Italy E-mail:music@distdist.unige.it Tel: +39-10-3532983 Fax +39-10-3532948 (") Department of Philosophyr University of Genoa, Italy. ABSTRACT This paper describes the framework and the prototype high-level system for intelligent composer's assistant called HARP (Hybrid Action Representation and Planning), that we have recently developed. HARP is a system able to store and process music and sounds, and to carry out plans for manipulating this material according to the composer's goals. The system is able to generate new music pieces as well as manipulate existent ones, on the basis of given material: to this end, the system provides also formal analysis capabilities on both music and sounds, for extracting information useful in subsequent synthesis processes. The system is hybrid, since it is based on the integration of two different formalisms. The sound itself (samples, codes, algorithms), particular analysis processes, are managed in HARP with procedures, included in an object-oriented concurrent environment (HARP analogical subsystem). Higher level scores, composition rules, definitions in general, descriptions of pieces of music, are stored in a declarative symbolic environment (HARP symbolic subsystem), based on a multiple-inheritance semantic network formalism derived from KL-ONE. The paper briefly describes both the methodological framework, and the major features of the system. In particular, the concept of abstract potentials (or force fields), a class of sound and music manipulators allowing the composer to think and operate in term of intuitive natural metaphors, is described. 1. Introduction This paper is a contribution to the problem of intelligent composer's assistance [1]. It describes the framework and the prototype high-level system HARP (Hybrid Action Representation and Planning) [2], that we have recently developed. HARP is a system able to store and process music and sounds, and to carry out plans for manipulating this material according to the composer's goals. Therefore, the system is able to generate new music pieces as well as manipulate existent ones, on the basis of given material: to this end, the system provides also formal analysis capabilities on both music and sounds, for extracting information useful in subsequent synthesis processes. In other words, HARP is a system for the representation and manipulation of music knowledge as regards composition: this is a vast field of knowledge which can be suitably managed by a twofold formalism. The sound ICMC 27

Page  28 ï~~itself (samples, codes, algorithms), real-time performance, particular analysis processes, are easily managed by procedures, included in an object-oriented concurrent environment (HARP analogical subsystem). Higher level scores, composition rules, definitions in general, descriptions of pieces of music, are stored in a declarative symbolic environment (HARP symbolic subsystem), based on a multiple-inheritance semantic network formalism derived from KL-ONE [3]. A formal link between the two subsystems exists, so that, for example, asking the semantic network to generate a particular instance of a music object, automatically "fires" the appropriate procedures at the analogical level. The model we developed does not imply any constraint on the musical style (western tonal music, contemporary music): we think, however, that this approach could be particularly useful in contemporary (computer) music composition. In the paper, we develop an example regarding the modeling of a contemporary music piece (Anceps Imago, composed by Corrado Canepa in 1989). The current version of the system has been implemented in Arity Prolog and Microsoft C on PC-IBM under the Microsoft Windows environment. 2. System structure In this section we introduce the overall architecture of the HARP knowledge representation system. Here and in the remainder of the paper we use the term music action, as a general term for referring to groups, phrases, "chunks of music"; in literature, the term music object is often used with a similar meaning, but we prefer "action" to underline their dynamic nature. 2.1 HARP knowledge representation scheme We propose a hybrid scheme, symbolic and analogical, in which the inferences, regarding the discovery of properties and the manipulation of music actions for compositional goals, are partially devolved to measurements on analogical models connected to the symbolic representation. Such measurements constitute a source of information that can be transferred to the symbolic level. The interaction between the symbolic and the analogical level is due to the fact that procedures are connected to some symbols, allowing generating, updating and verifying low-level music and sounds. The HARP overall hybrid scheme is shown in figure 1. This hybrid model utilizes, for the symbolic component, a representation system based on the distinction between terminological and assertional knowledge [5]. The terminological component adopts a KL-ONE-like SI-Net formalism [3] extended by temporal primitives. This kind of formalism gives the composer a structured representation of music actions in a visual and executable formalism. The assertional component is a subset of first order predicate calculus. The symbolic component is equivalent to a subset of firstorder logic. Queries are answered by means of its formal deductive apparatus. A classifier [3] has been implemented for organizing concepts in the taxonomy. The symbolic formalism we choose has a relatively limited expressive power, because most of the inferences are ICMC 28

Page  29 ï~~devolved to the analogical component. smbolic tcompostoa! musical k.b. environment. symbolic (music actions) (planning) component 1 analogical ' analogical experts: component sonologic procedures atomic actions k.b. t(simulative engine)J Qo Q Figure 1: HARP overall architecture The analogical representation is generated by a set of concurrent procedures, organized in a simulative engine (a set of procedures which acts as a counterpart of the symbolic deductive apparatus) and in an analogical knowledge base, regarding mainly the sonologic level of representation (in part based on the cmusic language [6]). An important part of the analogical subsystem regards the mechanism of abstract potentials, an intuitive natural way of operating on music and sounds by navigating in attractor fields. The simulative engine, enriched with this metaphor, is able to derive analogical inferences, integrating the capabilities of the symbolic deductive apparatus. 2.2 The knowledge base In figure 2, a part of the HARP symbolic knowledge base is shown: ellipses correspond to concepts in the taxonomy; double arrows are IS-A links; arcs with little boxes correspond to roles (slots, relations between pairs of concepts). As most object-oriented languages in artificial intelligence, the inheritance mechanism allows to "specialize" (rather then "aggregate") objects in the taxonomy. For example, a music action IS-A music fact, and one of its roles which "specialize" it with respect to music fact is intensity, whose value is amplitude (that is, the intensity of a music action is an amplitude, see figure 2, box "a"). The language used here is derived from KL-ONE, a well-known language for knowledge representation, which recently inspired a number of prototype implemented systems [4]. Two basic music entities are defined in the system: the music action and the compositional action (see figure 2). The former can represent music material at different levels of abstraction: from the sonologic level (low-level descriptions of sounds), to the whole musical form of a piece (for example, in western tonal music, fugue or sonata). compositional actions are a sort of meta-action, i.e. they are the "manipulators" which operate on music actions (both classes and instances) to produce new music actions, or to ICMC 29

Page  30 ï~~perform analysis tasks useful for subsequent manipulation. compositional actions allow a composer to manipulate music actions according to his current objectives: for example, he can operate on a music action at its sonologic level for the "tuning" of particular sound features; while planning the overall structure of a part, the same music action can be seen as a more abstract, high-level symbolic entity, working on its possible relations with other music actions. The composer can introduce into the system both sub-classes of music actions and instances. The former are reusable objects, skeletons of music scores or sound definitions, the latter correspond to complete, individual objects which can be heard reproduced by the HARP sound output channels. HARP is a framework which does not refer to a particular music style or context. Every music context requires the creation of suitable definitions in the symbolic part, and the related analogical procedures. As an example, in a western tonal music context, the composer can define the sub-class canon, with its structure (i.e. its component music actions antecedent and consequent, as in box "b" of figure 2, with their features). Successively, the composer can introduce, or ask the system to generate, a particular canon, characterized by a given antecedent and a suitable consequent. Basic features of music actions include: (i) a temporal connotation (the begin and end roles), allowing the composer to relate music actions to each other in the time domain; (ii) a set of relations, such as dynamic evolution, timbral and density content, pitch, metrics and rhythmic properties. Let us assume that a composer wants to specify in a compositional action that a music action (a part of a piece) N is generated related to, say, the evolution, simultaneous or not, of another music action M, expressed simply as a score fragment. Given an instance of M, its analogical component is formed by a score fragment (hooked to the concept M) and by a set of functions, or methods, hooked to its relations, which extract the related feature from the score: a query can be made on the intensity of M, which corresponds to a call to the intensity method of M, whose result is a presentation of the behaviour of the dynamics of M, extracted from its score fragment. In this way, a reasoning process is carried out, partially in procedural form. Note that no a priori assumption has been superimposed on the leading features of music actions: in classical western music, the melodic aspect could be preponderant in the manipulation of music actions, but the system allows the composer to modify, or re-define their relations and "manipulators" according to his needs. A compositional action is characterized by a temporal connotation (with a time axis different from music actions, defined by the Mbegin and Mend temporal roles), and a procedural description of its behaviour. As for any other object defined in the system, the composer can define both sub-classes and instances of compositional actions. Compound compositional actions can be defined in the system, as sets of given compositional actions, whose execution is ruled by proper constraints on their temporal relations. 2.3 Human-computer interaction HARP allows the composer to use different visual interfaces for communicating with the system: in particular, a visual presentation of the semantic network formalism is available. The composer interacts either visually or textually with this part of the system by introducing/editing elements in the net, and specifying queries, whose results can be ICMC 30

Page  31 ï~~graphically presented, to the system. Queries can be made regarding both the definitional part of the network (i.e. on classes and subclasses), and on instances, i.e. individual music entities. Subsets of the whole music material can be asked to be shown to the user, allowing different views of the same material. For example, in figure 2 the query "show all objects from which canon inherits properties", has the result of giving prominence to a path in the taxonomy by coloring in grey some elements of the network. Another simple query could be "show all about the object music action except temporal relations": the result of this is to show the concept music action with all its relations except the begin and end inherited from music fact (figure 2, box "a"). Music actions have "hooks" to score fragments, and compositional actions have "hooks" to code "chunks". This underlying level of procedural definitions constitutes the analogical level of HARP, which is structured into a sonologic and a simulative component. Both components are driven by the symbolic level, since proper activations of a node in the net correspond to calls to its hooked code. Mechanisms for a visual interaction with this level of representation of music information are also available in the system, based on both music notation, sound graphic descriptions and on visual metaphors. Here, queries correspond to the activation of code modules, with the passing of proper parameter values. An example is the above mentioned query on the intensity of the music action M. Another example is the following: in figure 2 (box "b") we see that a canon is generated by an imitation process. A particular canon subclass, say per diminutionem canon, can be defined, as generated by a subclass of imitation, say per diminutionem imitation. More detailed procedures can be hooked to this last concept, and one of them can implement a proper diminution algorithm. An instance of per diminutionem canon can be generated, starting from the above concepts and an instance of antecedent, by querying a proper consequent for this canon. The system answers by activating the proper procedure hooked to per diminutionem imitation, passing as parameters the antecedent instance. In conclusion, let us analyse now in more details the canon musical example, which will be used in the next section. Figure 2 (box "b") shows the concept polyphony as a subconcept of music action; canon is subsumed by polyphony (obviously, in a richer taxonomy there can be other intermediate concepts). Canon is composed by a set of antecedents and consequents, whose number is at least 1, as shown by the 1/NIL number restriction of its roles. Two Role Value Maps (roughly, constraints on roles), not shown in the figure, are needed to express that in a canon the antecedents correspond to the fillers of the input role of compositional action, and the consequents correspond to the fillers of the output role. Different types of canon can be defined, differing in the imitation involved (see figure 4): for example, simple and complex canons, retrograde and contrario motu canons, canon per augmentationem and per diminutionem. Each of them is characterized by (a set of) different analogical procedures, able to perform suitable transformations. 2.4 Force fields An important set of analogical descriptions are those based on the metaphor of the force fields: this allows the composer to think and perform a set of compositional actions in terms of the intuitive natural dynamics of navigation in attractor fields, instead of using a rule-base. ICMC 31

Page  32 ï~~In brief, the metaphor of force fields appears to be useful for (i) describing continuous changes in music actions: the resultant of several force fields easily models complex behaviours (such as continuous timbral, melodic, or rythmic evolutions), otherwise very difficult to model as rules; (ii) giving a different point of "view" on music entities (including visual metaphors), as well as different powerful, easy-to-use manipulation primitives. 3. A musical example In this section we describe a fragment of the model of a particular music piece, called Anceps Imago, composed by Corrado Canepa in 1989, for two harpsichords and synthesized sounds. The piece is structured as a three-part concert (Andante, Adagio, and Allegro), whose structure is (partially) shown in the net of figure 3. compound begin d end F r A rp rt Anceps Imago can be asserted as a particular instance of a three-part concert: three-par-concert(%Anceps-Imago) part-of-l(Anceps-Imago, Andante) part-of -2 (Anceps-Ima go, Adagio) part-of -3 (Anceps-Imago, Allegro). part-of(Andante, Evolution) part-of(Andante, Coda) The first part (Andante) is structured in a double canon musical form, for two harpsichords. In particular, the excerpt that we consider in the paper, which is a fragment of Evolution (see figure 4), is structured in four voices forming a double (i.e. with a couple of antecedents), direct (i.e. direct imitation), rigid (i.e. strict imitation) canon at the unison. Therefore, Evolution inherits the structure of these particular canons, partially described in figure 4, and asserted with the following formulas: ICMC 32

Page  33 ï~~g en eratedby IV 0 0 unison Kcompress canon end Figure 4: HARP Knowledge Base (excerpt of Anceps Imago) ICMC 33

Page  34 ï~~Z e 4 "" t- O6 " M 9 a 23 N v col 0 T "ri U 4' 4- F.I *J 4- 4 I ~ ~ - t dJc 1pC'1H '- d u2 op.t~jo3! Saqu cs lp c!UOns a t l qua3t ne l3anp JBII Sod S 0u d

Page  35 ï~~rigid-canon(Evolution) direct-canon(Evolution) double-canon(Evolution) canon-at-unison(Evolution) In figure 4 these formulas are substituted by the (grey) individuation links. The two voices forming each couple are organized in a free (not imitative) counterpoint. The traditional score-like description level is shown in figure 5. The score of Evolution, as well as that of Coda, which can be similarly represented (as discussed later on in the paper), can be generated starting from the set of concurrent procedures (the analogic experts) hooked to the concepts at the symbolic level, each of them performing the particular canon transformation. It is worth noting that the conceptual model partially described in figures 2, 3 and 4 is re-utilized also in the two other parts of Anceps Imago, by extending it with other particular concepts. 3.1 An example of force fields The force fields play a significant role in the generation of some parts of Anceps Imago. For example, at the ending of the first part, there is a music action which is generated starting from a phrase (the incipit in figure 6), a definition of the sound parameters, and two concurrent force fields operating as follows. The initial incipit is imitated with continuous timbral variations. At the same time, the aspect of pitch intervals progressively degrades giving place to the timbral aspect. In HARP terms, there is a force field operating on the timbral space, which performs a continuous timbral disassembling according to its field function. Concurrently, another force field, operating on the pitch axis, gradually superimposes a variation of the initial incipit: the result is that the application of this force field (which has a function with local minima on particular pitches) to the incipit, causes a gradual pitch-stretching, which conduces to a final different melodic contour, taken as a theme in a fugato which ends the first part of the piece. Such kinds of complex analogical queries can be formulated, involving complex activations of interacting procedures, with the aim of building high-level, complex music actions. 4. Final remarks Visual, ipermedia interaction mechanisms can be defined as force fields to control the computer performance or particular score parameters, toward the definition of "animated scores", or music choreographies. This is a stimulating route which we are currently experimenting. Enhancements on the representation model as well as the design and implementation of new other features in the system are in course of development, basing on the feedbacks from HARP users. These and other topics will be covered in a further paper. ICMC 35

Page  36 ï~~Acknowledgements This work was partially supported by the "Progetto Finalizzato Sistemi Informatici e Calcolo Parallelo" of C.N.R. under grant no. 90.00716.PF69, pos. 115.14231, and by MURST Special Projects (Italian Ministry of University and Research). References [1] A. Camurri, "On the role of artificial intelligence in music research", Interface, 19(2-3), Special issue on Artificial Intelligence and Music, A. Camurri (Ed.), Swets & Zeitlinger, Lisse, The Netherlands, 1990, pp.219-248. [2] A. Camurri, C. Canepa, M. Frixione, and R. Zaccaria, "HARP: A system for intelligent composer's assistance", IEEE Computer, Special issue on Computer Generated Music, D.Baggi (Ed.), Vol. 24, No. 7, July 1991, pp.64-67, IEEE Computer Society Press. [3] RJ. Brachman, and J.G. Schmolze, "An overview of the KL-ONE knowledge representation system", Cognitive Science, Vol. 9, 1985, pp.171-216. [4] C. Rich (Ed.), SIGART Bulletin, Special Issue on Implemented Knowledge Representation and Reasoning Systems, Vol. 2, No. 3, June 1991, ACM Press. [5] RJ. Brachman, R.E. Fikes, and HJ. Levesque, "Krypton: a functional approach to knowledge representation", in RJ. Brachman and H.L. Levesque (Eds.), Readings in knowledge representation, Morgan Kaufman, Los Altos, CA, 1985, pp.411-429. [6] F.R. Moore, "Elements of computer music", Prentice Hall, Englewood Cliffs, N.J., 1990. Figure 6: Anceps Imago: score #2 ICMC 36