Page  1 ï~~DIGITAL TIME: COMPLEXITY AND PERCEPTION Dante Tanzi Laboratorio di informatica Musicale Dipartimento di Informatica e Comunicazione University degli Studi di Milano Via Comelico 39/41 20 135 Milano ABSTRACT This paper deals with the experience of time discontinuity that derives from digital practices and discusses observations by composers and inter-medial artists. Within this frame, the paper hypothesises that the digital processing of both musical time and inter-medial events lead to a subsequent loss in emotional sensitivity. Some communication theorists have explained how digital technology affects the sense of time. While digital culture reshaped the experience of time distancing it from the heritage of linear narratives, time itself has become digital. Digital time is an objective entity, a material with modifiable features, used for building and shaping musical structures. Through digital time one can modify the re-presentation of musical and inter-medial events. Since overlapping, re-scaling, splicing and reversing time affect musical consistency, they influence the elaboration of the symbolic functions that music psychologists consider important. Within the intermedial environment, time processing may cause perceptual imbalance: and when perception requires to be frequently re-modulated the detachment provoked by time discontinuity affects the emotional sphere. 1. INTRODUCTION Musical sounds create spaces structured by time-driven relationships which lead to a re-definition of the "inner" and "outer" space, and extend the concept of space itself. Free from the material nature of bodies, sounds greet us directly without any barrier between subject and object. According to the music anthropologist Victor Zuckerkandl, the relationship between sounds and subjective experiences is expressed by emotion [1]. The music psychologist Michel Imberty goes further. He said that the conditions of integration or disintegration of interior life and of the Ego represent the basis of musical time and give a deep sense to the musical plot independently of the cultural filters that may condition the listener [2]. But because it allows the production of different temporal textures, digital technology makes every configuration of change accessible. And it often happens that the sensitivity to musical time can be questioned when discontinuity due to overlapping times is experienced. In effect, Imberty stated that every symbolic function derives its representative elements from sensory-motorial activity, and that the connection implied in musical meaning between the body and emotion influences the relationship between interior life and musical time. But the detachment provoked by time discontinuity affects the emotional sphere of subjects and cause perceptual imbalance. Many artistic experiences require perception to be frequently re-modulated, not only through the dynamics of inter-medial environments, but also through time re-configurations. Within localized and networked installations artists like to use perceptual re-modulation to allow new meanings to emerge. Valentina de Angelis, aesthetics theorist, stated that the difficulty involved in dealing with new models of knowledge, even in their undetermined aspects, may affect the categories of reflection and perception. According to Paul Virilio, instead, the real time technologies and their accompanying dimension of "liveness" seems to have prompted the disappearance of physical space. While ethno-musicological research suggested diverse forms of digital communication may produce different mechanisms of musical involvement. These are the reasons why inter-medial artists and composers have to analyze the differences between digital time-paces, while bridging the disparate worlds of the local and remote. 1.1. Time into material According to Maurizio Lazzarato, digital processes seem to favour the development of a sensitivity based on interruption of the coexistence between time, images and sensory-motorial mechanisms. He maintains that digital processes produce crystallised time, a time transformed into material, "a time matrix, simulating constantly renewable moments that can come into being" [3]. As a digital artist, Randall Packer showed how such interruption could be artistically treated, in order to reflect and subvert the difference between past and future. Presenting his project "Pavilion", Packer explained why and how he adopted a method of blurring times by spherically mirroring user's images: "this notion has inspired the use of live-motion capture to deconstruct the viewer's perception of time, the past superimposed with the present, spread and blurred as a collage of time. The viewer's image is captured, transformed [...] and displayed on projection screens that are reciprocally reflected into the mirrors, creating visual feedback not unlike a hall of mirrors" [4]. Digital processing of time appears to be capable of questioning the permanence of musical meanings, and time, as a product, can be artistically represented and experienced. 2. AESTHETHICS OF DIGITAL TIME One of the tasks of digital aesthetics has been to reveal and represent the contradiction between the power of transferring digital objects to diverse digital domains,

Page  2 ï~~and the difficulty involved in following and describing such mutations [5]. This contradiction has manifested itself in various ways but, according to Lazzarato, mutations referring to the networked environments seem to be ontological: the net highlights the emergence of a machinic temporality, revealed through the polarization of "minimum and maximum" velocity, "fixity and simultaneity". This polarization "appears to be the very ontological consistency of digital technologies". Jordan Crandall [6] and Peter Lunenfeld [7] have dealt with the problem of the real status of digital objects, saying that their intrinsic instability requires them to be continuously re-tracked. Lev Manovich expressed a similar approach: after stating that it is possible to think of all representational art as a kind of mapping, he described the conditions in which it is appropriate (and more interesting) to use the term mapping for describing what new media does to old media. According to him, "Software allows us to re-map old media objects into new structures - thus turning media into what I call meta-media" [8]. 2.1. Digital time as a tool and a product According to Manovich, the objectivity of digital contents can be considered as a product of an intellectual operation, whose final result depends on cultural background and processing methods. In effect Manovich's approach could be adopted to describe what the new digital time does to old forms of time. Once it has become digital, time is considered to be a specific communication tool; there is no reason why we should not think of it as a specific medium. In the past the linear flow of time was considered to be a unique model; today artists and composers distinguish between natural and artificial forms of time: they can now think of digital time as a product of digital technology and as a specific form of communication. Whether it is a communicative form or a product, it may differ from "external" time (chronological); from "internal" time (implied in almost all narrative plots); and finally, from "subjective" time. Far from being a mere unit of measure, digital time has connotations resembling those of a multidimensional space, and hence capable of undergoing hybridisation and metamorphosis. This may raise further ontological issues, for the former mixes erstwhile distinct connotations of temporal objects, while the latter seems to extend their existing role. 2.2. Re-mapping times This is why the composers can indifferently think of placing musical events within a variable configuration of time or, vice-versa, of ascribing temporal features to music materials. They can do this by replacing, striating, diluting, accumulating or inverting temporal textures, both before and after they have been built. In other words, their use of digital time as a material reveals that it has also become a communication tool. Applying Lev Manovich's approach to time, one can say that composers actually search for mappings that allow them to access and transform "old temporal objects". Remapping time may represent one of the most fruitful directions for research in compositional strategy, since it attributes pre-eminence to communication of timechange rather than to musical construction. According to Mario Costa, "The aesthetics of communication emerges as a phenomenology of technological events and relies on forms of digital time" [9]:in fact, not only does the aesthetics of communication set the concept of an artistic event within multiple technological domains, but it also develops and incorporates the remaining aesthetics forms. Accordingly, one could say that sampling compels musical contents to be completely subject to the needs of communication. While recording techniques introduced the possibility of dividing, reversing and pasting musical matter, sampling facilitated interconnection among the same operations, and paved the way to using non-linear temporal facets and nonsynchronised loops. As Simon Waters observed the ability of composing within a digital time framework developed thanks to the culture of sampling [10]: while permitting "step-by-step" control, sampling offered opportunities for a cross between a number of different time-grids. 2.3. Re-visitation of time Re-visitation in itself stimulates curiosity, allowing the activation of cognitive paths and the emergence of temporal structures that may differ from musical ones: in fact re-visitation relates to cyclical organisation of both subjective and musical time. Every experience of revisitation can create issues of expectation and control. Furthermore, the effort of putting together and periodically re-visiting structures based on different time granularities may inhibit or enrich the way the emotions relate to music. Besides those provoked by the recombination of digital sounds, other aspects of detachment are revealed through interactivity and through using the net. In his book "Composing Interactive Music" Todd Winkler explained how lack of emotion due to digital processing may resound along the chains of musical performance. Regarding on-line communication, many musicians began to consider the net as a multi-temporal environment, for differences between the velocity of transmission gave the idea of an n-dimensional access to time. W. Duckworth chronicled how, in 1990, Phil Burk and John Maxwell Hobbs tried to overcome the latency problem, through letting "the interaction between musicians take place 'out of time' where the arrival time of message is not critical". The solution chosen was based on the use of the looping pattern, which "avoided the problem of network latency, by giving each home user access to a central loop and the illusion of working in real time, no matter where on the loop their non-synchronised computer happened to locate itself during log-on" [11]. In effect, while the declared aim was to avoid the effects of latency, any concern

Page  3 ï~~about it was removed, by transforming it into an opportunity to re-think, reorganise, and simplify musical thought. Unfortunately Duckworth does not reveal whether, in terms of emotional expression, such simplification implied loss or gain. 3. CHRONOLOGICAL WISDOM There are good reasons for avoiding the effects of temporal latency: while producing detachment, the perception of being outside chronological time modifies the capacity to understand musical events. It also modifies chronological wisdom, that is, the ability to activate predictive processes, to jump to certain conclusions while reserving the right to modify them when new evidence comes to light. While referring to repetition, deferment, dilution, redundancy and omission of messages, chronological wisdom relies on the familiarity with the changeable nature of time. While depending on shared time scales and contexts, this familiarity relates to subjective attitudes, differentiates the styles of communication and gives rise to a range of individual modulations. In the context of net communication [12], the idea of conversing while managing interruption and sharing more than one time scale has been proposed and practiced as an artistic choice: both in inter-medial and musical contexts, it reshuffles the dimension of continuity. Condensing, splicing, dislocating, overlapping or mosaicing times compel the audience to modify their hypotheses about narrative and conversational settings. Besides staging the representation of time, these mechanisms change the audience's attitude towards the acceptation of unpredictable events, create detachment, and thus influence the emotional impact. 3.1. Describing detachment This detachment can regard both emotional and linguistic dimension. In the article "Musical Implications of Media and Network Infrastructures: Perturbations of traditional artistic roles", Atau Tanaka gives a clear assessment of temporal shifts in his article "Musical Implications of Media and Network Infrastructures: Perturbations of traditional artistic roles." He verified this assessment through using 'MP3q', "a musical piece viewed on standard web browser, which streams and mixes multiple MP3 channels audio coming from different servers on the net". The multiple streams are mixed according to displacement and movement of an abstract cube-like structure comprised of text (URL's); besides, visitors can participate by adding links which relate to their MP3 sonorous objects. What interests here, is that Tanaka's description looks at different status, levels of abstraction and interpretations expressed by the same musical process. In fact, Tanaka first noted that client-server architecture provokes temporal delay, a "before and after stage", and he clearly showed the difference between musical data and digital data, which refers to musical objects, as related to their different accessibility over time. According to him "at the beginning, the piece exists only as code that generates a display of hypertext links, depending on interaction with a server". Tanaka then moved along different linguistic nuances, using border-paradoxical observations like: "Musically, these pieces are empty shells, contentless compositions", or, also: "Music does not reside in the piece". He went on to conclude that "'MP3q' exercises a form of displaced content storage", since the actual sound data remains at the point of origin, and is delivered only at the time of listening [13]. It is worth noting how Tanaka emphasises the separation between the musical project and the actual staging: through using an emphatic language, aimed to reveal (and perhaps suitable to counterbalance) the recognition of an emotional detachment. 3.2. Detachment as artistic practice Within interactive, inter-medial environments, a sequence of gestures may also contribute to structure artistic and musical forms. At the same time, the gestures can be mirrored and represented through different media. Often the interpretation of musical meaning has to deal with the presence of erratic events, partial unpredictability of results, and also with temporal shift (even blurring) between visual representation, symbolic description of sonorous events, and their aural presence. This also causes detachment. To the extent that it is ignored or emphasized, detachment influences the relationships between inter-medial events and emotional indexes of sonorous space. But it can be artistically treated when a networked environment is thought of as a "chamber of resonance". It is the case of project "Pavilion" mentioned above, as explained by Randall Packer: "As online viewers enter into and out of network, as the remote audience expands and contracts, the spatial dimensions of virtual network fluctuate, network activity ebbs and flows, and the sonic textures of the work evolve with the continuous change. A unique dialogue ensues between the local and remote viewer: virtual participants plant the IP address to shape a virtual architecture while on-site viewers asset their physical presence--an interplay between the real and the virtual that create an hybrid space (third space)-- dissolving the physical being of the viewer into the ghosted resonances of network presence". Being based on trying to blur differences between chronological time and virtual times, the experience of "Pavilion" may provoke an interruption of the coexistence between time, images, and sensory-motorial mechanisms, thus recalling Lazzarato' s concerns. 4. EMOTIONS RE-GAINED In effect, composers have already started to reflect on the different degrees of detachment: those deriving from online, networked communication, and those deriving from the practices of re-combining digital sounds. Agostino Di Scipio stated that, while increasing the depth of meaning

Page  4 ï~~without ascribing any real specification to it, "those practices (of recombination) seem to be unrelated to the materials used, and imply detachment from the recombined objects" [14]. In effect, both the removal of dialogue with the audience and the difficulty in finding a balance between the rational content and the emotional aspects of music have become critical factors that, together, have substantially weakened the relationships between the acts of production and listening [15]. But it has also been shown that the bodily and tactile interaction with sounds may give emotional involvement, or, at least, compensate for its lack: this is the case of those interfaces that contemporaneously allow auditory, tactile and visual control of musical processes. 4.1 Re-presenting time In many cases body movements can add further meanings to sound, and sometimes an audience prefers a combined audio-visual representation of the creative process, rather than to just listen. This may be because tracking gestures over time serves to reconstruct unity between chronological time, subjective time and the time of visual narration. Thus, paraphrasing the concepts introduced by M. Costa, one could say that, under particular conditions, musical production ends up by being part of the more general paradigm of representation, through two different levels. On one level the hybridisation and temporal re-location of sonorous events dictate the conditions for the emergence of musical meanings. On the other level musical performances planned within networked spaces imply a continuous re-articulation of time frames and affect sonorous gestures. This means that performing music through online technologies requires attention both to local and global emerging processes. Interesting both time and gestures, they may introduce different layers of musical intelligibility, making it more demanding. But one should not take for granted that this awareness could suffice to convince listeners to re-set their internal clocks, modify their physical involvement and adapt listening behaviour [16]. 5. CONCLUSION The sharing time knowledge between transmitters and receivers depends on both the actual feedback and on tolerable mutations in time scales and paces. Digital processing of time and interactivity has provoked instability of communication codes, for they may be jeopardised by non-homogeneous time frames. Even in instantaneous interaction, differences between digital time paces and chronological time have to be considered in order to interpret aims and rules of conversation. Besides, the re-scaling and re-directing time and reacting on its different granularities are all functions which should be negotiated to avoid incoherence and detachment. The interaction between gestures and inter medial environments produces events which require new "chronological wisdom" and a capacity to take part in conversational settings. This can be done through codifying new rules of artistic and musical communication, without forgetting the old ones. Depending on this balance between the 'old' and 'new', the concept of 'digital time' can be seen. as a cultural product, as a filter for human sensitivity, as a concrete musical entity and as a tool aimed at re-organising musical experience. 6. REFERENCES [1] Zuckerkandl, V. Sound and Symbol, Bollingen Series, Princeton University Press, Princeton, 1956. [2] Imberty M. Suoni Emozioni Significati, per una semantica psicologica della musica, a cura di L. Callegari e J. Tafuri. Editrice CLUEB, Bologna 1986. [3] Lazzarato, M. Videofilosofia - La percezione del tempo nel postfordismo, Roma, Manifestolibri, 1996. [4] Packer, R. Composing with Media: zero in time and space. Contemporary Music Review, vol. 24 Part 6, 2005, 516 -522. Editor Peter Nelson. Issue Editor Andrew Hugill. [5] Tanzi, D. Musical objects and digital domains. Proceedings of EMS-05 Conference. Montreal, Quebec, October 19-22, [6] Crandall J. Precision + Guided + Seeing, Ctheory, Arthur and Marilouise Kroker Editors. [7] Lunenfeld, P. Snap to grid. A user's guide to digital arts, media, and cultures, The MIT Press, Cambridge MA 2001. 124-126. [8] Manovich, L. The Anti-Sublime Ideal in Data Art. [9] Costa, M. Estetica dei Media: Avanguardie e Tecnologia. Roma, Castelvecchi, 1999. [10] Waters, S. Beyond the acousmatic, in: Music, Electronic Media and Culture, edited by Simon Emmerson, Ashgate, Aldershot U.K. 2000. [ 11] Duckworth, W. Virtual music. How the web got wired for sound. Routledge, 2005, 132-135. [12] Tanzi, D. Time, proximity and meaning on the Net, CTheory, 2000, Arthur and Marilouise Kroker Editors. http://www.ctheory.netiarticles.aspx?i d= 124 [13] Tanaka, A. Musical Implications of Media and Network Infrastructures: Perturbations of traditional artistic roles. [14] Di Scipio, A. Uno scenario post-digitale? Riflessioni ambivalenti sul suono in Internet, 2001. [15] Tanzi, D. The Cultural Role and Communicative Properties of Scientifically Derived Compositional Theories. Leonardo Music Journal: Vol. 9, Issue 1. Cambridge, MA: MIT Press, 1999: 103-106. [16] Tanzi, D. Musical Experience and On-line Communication, Crossings: eJournal of Art and Technology, University of Dublin, Trinity College, December 2003. M. Haar and E. Drew, editors.