Page  583 ï~~A Semiotic Approach to Music Interaction Fernando lazzetta Center for New Music and Audio Technologies University of California, Berkeley email: Abstract: This paper aims at analyzing human/computer interaction and the way it works in the field of music activity. For this purpose, we introduce the view of interaction as a kind of communication process and try to understand music interaction through a semiotic perspective. The paper has its foundations in the semiotic theory established by the American philosopher C. S. Peirce. Peirce's triadic categories are presented and used to classify interactive music systems. INTRODUCTION This paper is part of a much more extensive study of interaction in computer music systems through a semiotic approach. Musical activity involves, in its essence, interactive processes in all of its levels: composition, performance and listening. It shows different characteristics if the interactive agents include only musicians (as in an orchestral performance), only machines (as in a music box), or both (as in interactive computer music systems). Our interest is focused on musician/machine interaction, and specially in the case of those systems that make use of computer technology. We will introduce the view of music interaction as a communicative process whose elements can be analyzed under the "doctrine of signs" or semiotics developed by the American philosopher C. S. Peirce (Peirce 1931-58). Peirce's theory is based on triadic relations that can be related to any and all phenomena. Despite the complexity of relations that this triadic thought involves, for now we can summarize those logical categories as follows: firstness: comprises the qualities of the phenomena, like redness or goodness. They are not the phenomena itself, but once there is a phenomenon there is such a quality. They are units, mere possibility of being; secondness: is related to the actual facts and comprises our experience and all things that happen in time and space; thirdness: the category of generality, processes, habits, and thought. It is important to notice that firstness, secondness and thirdness are not discrete categories since they happen as a continuos flow, the qualities generating facts, and facts generating habits and laws. INTERACTION AND SEMIOTICS Interaction is a reflexive process. The actions performed by each interactive agent do not only determine the system's responses but are also influenced by those responses. It works as a functional loop where each action acquires its meaning in function of other co-related actions. It implies a constant exercise of adaptation to contextual situations. Interaction occurs in a communication context when multiple agents are able to perform or take part in one action over one object. For instance, during a dialog two individuals (agents) express themselves by formulating their opinion (action) in the form of speech. It is important to notice that not only the agents are affected during this process but also the subject of discussion (object) undergoes transformations as the agents reflect upon it. Interactive systems can be thought of as semiotic machines. They have a large capacity for sign generation, and signs tend to generate semiosis, which in Peirce's theory means the process of action of signs. Music interaction operates in a continuum that "could be evaluated by three variables: frequency (how often one could interact); range (how many choices are available); significance (how much the choices really affected matters)" (Laurel, 1993: 20). In fact there are many ways to arrange these variables to reach a considerable level of interaction. It is not necessary to have a system where frequency, range and significance show high values to get highly interactive responses. The level of interaction based on these variables does not necessarily show a linear behavior where the increase of the values, would proportionally reflect as an increase in the system's level of interaction. Interaction is highly context-dependent and if the system cannot respond to significant aspects of each particular music state, it will be hard to feel that an interactive process is going on. Interactive computer music systems involve three different semiotic levels: structure, organization, and language. These three levels would correspond to the general logic categories, firstness, secondness, and thirdness, established by C. S. Peirce. The first level of an interactive system, the structure, refers to the IC M C PROCEEDINGS 199558 583

Page  584 ï~~universe of signs that compose the system as well as the system interfaces. Structure is related to elements that compose the system, but not to the relations that exist among those elements. Therefore, it does not give any information about how it works. It only reflects the system's possibilities. The second level, or organization, "signifies those relations that must be present in order for something to exist" (Maturana & Varela, 1987: 42). It emerges from the relation among structural elements and encompasses the interaction between parts of the system and its interfaces. It reflects the system's actuality. Language, as the third level, is taken here in a broad sense of general laws that guide the creation, use, and understanding of signs. It is related to the way agents manipulate the signs that take part in the interaction and involves the rules that operate on the system in different hierarchical levels. Operating over the elements of the systems and its relations, this level implies the notion of grammar and language and, thus, reflects the system's potentialities. Interactive music systems can operate in a very wide semiotic spectrum by allowing the communication among different levels of music creation. In this case, interactive creation differs from the composition in early stages of computer and electronic music because instead of generating particular music structures, "as does a composer of traditional music, the creator of an interactive composing system composes a mode of functioning for computer system and performer that, in operation, generates a new particular structure in every performance" (Chadabe, 1983: 147). Each interactive music system will tend to concentrate their procedures in operations that can be focused in the generation, transformation, or interpretation of music data according to which kind of action they perform. These procedures can be defined by three functions: 1) the detection of context and of the material the system receives; 2) the processing applied to this material, and; 3) the elaboration of responses. These three functions are very well depicted by Robert Rowe as part of three interactive stages: sensing stage, processing stage, and response stage (Rowe, 1993: 9). These three levels are interrelated in the very same way the Peircean triadic categories are. Each level presupposes the existence of its predecessor in any stage of the musical production. Thus, generative systems are related to the development of creative processes and operate over structural material, detecting situations that are nothing more than possibilities of development. The input of interactive music systems which emphasize generative capacities consists of 'guidelines' instead of strict scores. In this situation the system is presented to some kind of structure (a formula, a diagram), but not the data to feed this structure. In another way, a transformative system is directed, not to creation of new material, but to the re-elaboration of previously conceived music or music fragments. In this case, the system operates over organized material. It receives not only formulas but also the data that feeds these formulas. In other words, it manipulates existent music data which can be pre-established in such different ways as traditional scores, midi files or recorded sound data. Finally, in interpretative systems, the main goal neither is to originate new data nor to modify pre-existent music fragments. This kind of system aims to establish general relations among music structures by identifying or organizing musical patterns. These systems frequently are based on artificial intelligence techniques and their behavior exhibit adaptive or evolutionary characteristics. CONCLUSION Interactive music systems must operate in a full semiotic range in order to create conditions of development of musical creativeness and stimulate sign growth. The composer of interactive music may pay attention not only to traditional subjects of music composition, but also explore the expansive semiotic capabilities that the interactive processes embody. The dynamics brought to music by interaction is not a consequence of changes occurred in the context of performance. Indeed, this dynamic is much more related to the formulation of actions (composition and system design) than to the actions (performance) themselves. Interactive music systems might be conceived to freely permit the arousal of music sings, their creative connections and limitless expansion. REFERENCES Chadabe, J. (1983). Interactive Composing: An Overview. In C. Roads (Ed.), The Music Machine (pp. 143-148). Massachusetts: The MIT Press. Laurel, B. (1993). Computers as Theatre. Reading, Mas.: Addison-Wesley Publishing Company. Peirce, C. S. (1931-1958). The Collected Papers of Charles Sanders Peirce. C. Hartshome & P. Weiss Ed. (Vols. IVI); A. W. Burks Ed. (Vols. VII-VIII). Cambridge: Harvard University Press. Rowe, R. (1993). Interactive Music Systems. Cambridge, MA: The MIT Press. 584 4IC MC PROC EE D I N G S 1995