Page  00000445 jMax recent developments FranCois Dechelle Maurizio De Cecco Enzo Maggi dechelleQircam.fr dececcocircam.fr maggi@ircam.fr Norbert Schnell schnellcircam. fr IRCAM - 1, place Stravinsky - 75004 Paris Abstract In this paper, we present recent developments of jMax. Currently supported platforms are listed. The evolution of the jMax computation engine toward a multi-threaded architecture and the new scripting tools of jMax's graphical user interface are described. Musical applications are presented, together with considerations on Max's programming paradigm. 1 Introduction jMax is a new implementation of the Max language. It is a programming environment for real-time performance interaction and audio processing. It features a high level of portability, and is based on a Java graphical user interface. 2 Platforms jMax is currently supported on the following platforms: * SGI stations, both mono and multi-processors, using the SGI standard audio interfaces * Linux on Intel compatible processors, using the Open Sound System audio and MIDI drivers The Linux version features a very good performance and has shown that a machine comparable, in term of computing power, to the ISPW [LIN 91] can now be found in entry-level PCs. For detailed benchmarks of supported platforms, the reader is invited to visit jMax home page at www.ircam.fr/jmax for the latest informations. 3 The FTS kernel The framework of the jMax environment, based on a client/server architecture, has been described in [DEC 95, DEC 98a, MAG 96]. The two components of the system are the C written real-time computation engine, already known as FTS, and the JAVA graphical user interface. The architecture of the FTS server is described in [DEC 95, DEC 98a, DEC 99]. It consists mainly of a kernel containing the message system interpreter, the DSP computation engine [DEC 95, DEC 96], the scheduler and the device based input/output system [DEC 95]. The FTS kernel has been rewritten to become completely multi-threaded in order to make full use of modern symmetric multi-processors machines, which are now most of the time based on cachecoherent memory architecture and support thread level programming interfaces. The currently used thread API is the Posix thread API, but the multithreading capabilities are not restricted to Posix and will use the native threading API on different platforms. The first use of the multi-threading capabilities has been the re-implementation of the direct-to-disk services. Now based on a device, the sound files accesses are executed in separate threads, thus being able on multi-processors machines to run on a different processor from the processor running the DSP code interpretation. This feature allows the user to build applications using many channels of direct-to-disk real-time access, and still have one processor completely dedicated to signal processing and control execution. Another use of the multi-threaded architecture, still in development, is the capability to separate control computation into "synchronous control", running in the same thread as the DSP computation, and "asynchronous control", running in a low priority thread. The advantage given by this possibility is that the "asynchronous control" computation load peaks will not interrupt the execution of signal processing and synchronous control, eliminating the need to increase the audio buffer size and therefore the latency, and keeping tight synchronization between signal processing and synchronous control computation. 4 Scripting The jMax user interface integrates a scripting language [DEC 98a, DEC 99], used at several levels: configuration (choice and parameterization of devices or choice of packages), definition of packages ICMC Proceedings 1999 - 445 -

Page  00000446 (i.e. dynamically loaded libraries of objects and Java editors), extension language (for instance definition of new edition commands). The first implementation of scriptability in jMax used a JAVA implementation of Tcl, which has shown poor JAVA integration and lack of support. It has therefore been decided to switch to another scripting language, and the choice has been to move to Java implementations of the Scheme Lisp dialect. Scheme features numerous Java implementations that all exhibit good runtime performance and high Java integration. 5 Applications Recent application development include real-time physical modeling based on the Modalys package [ECK 95, IOV 99], additive synthesis using inverse FFT [ROD 92] and score recognition for realtime soloist synchronization. The implementation of these applications was the occasion to exercise the variables mechanism introduced in jMax [DEC 98a] and to experiment different computation paradigms. 5.1 Experimenting with and reference types variables The new implementation of modal synthesis [ECK 95, IOV 99] allows to construct modal synthesis algorithms using named variables to define bidirectional interactions between two objects and the association of an object to a computation context. Since the jMax message system does not provide an adequate mechanism (including garbage collection) to handle copying reference types at runtime, the use of references were first restricted to objects instantiation. Recent developments explore as well the use of reference types in messages. For messages containing arguments of simple types, Max gives a notion of a data flow language (it must be noted that sending a message always invoke a function call with arguments passing by value). Without the introduction of a complex handling of reference types including garbage collection, this notion can not be kept for reference types. Even if the introduction of computation by side effect finally breaks the data flow illusion of the Max language, it represents an efficient and powerful way of computation which is worth exploring. The example of figure 1 shows a simple jMax patch of two graphs doing the same operations in two different ways. A table of amplitudes is filled with data from a file before it is multiplied, element by element, by a vector of weights and finally multiplied by 2. By convention the operator (like load and * in the example) output the received reference after applying their operation to it. Since the t object in the second graph outputs the incoming variable in right to left order from each output, both graphs produce identical outputs on the display. Figure 2 shows an example of additive synthesis using computation by side effect in a different way. The object FMO outputs a reference to an optimized representation of an empty FFT spectrum for each synthesis frame (every 128 samples) in a message. Each of the FMOharmonics objects accept this message, add the harmonics received by their parameter inlets to the spectrum and forward it to the outlet. These calculations are done at the same logical time during jMax's message cycle driven by a periodic alarm ran by the FMO object. When the FMO object is scheduled by the DSP subsystem it computes an inverse FFT (256 samples) on the spectrum created from the three sets of harmonics and overlaps it with the previous synthesis frame before the sound output. This way efficient additive synthesis algorithms can be created in a modular way, while all modules share the same computation model and the same inverse FFT. Other classes are implemented, which add arbitrary par tials and noise bands to an FMO spectrum using the same paradigm as FMOharmonics. Traditionally, the Max programming paradigm allows two ways of defining an interaction between objects: 1. "graphical connections from an object's outlet to another object's inlet represent either the flow of messages or audio signals 2. an object references another object by name in order to share a part of its state The data transported by messages was so far restricted to array of atoms (i.e. type tagged values), each atom representing an integer, a floating-point number or a symbol. Recent developments implement atoms of reference types containing complex data like vectors of integer or floating-point numbers, floating-point matrices or spectral representations References by global names (like between the table objects) where replaced by the concept of named variables [DEC 98a]. This concept allows an object to define a variable by exporting an atom of an arbitrary type which will be bound to a local name when the object is instantiated (e.g. mytab: table 100). Another object in the same scope can use a reference to this variable as an instantiation argument, using a $-operator with the variable name like in table $mytab. -446 - ICMC Proceedings 1999

Page  00000447 5.2 The score recognition project A set of experimentation have been carried out with jMax in relation to the score recognition problem, in order to offer a wider set of tools and strategies for this somehow classical area of research. Given a computer-readable "solo" partition and a flow of data coming from a external performer, the task is to be able to recognize from the input stream specific events coded in the solo partition, and react accordingly following a second, "accompanist" partition. The researches started from the analysis of the available tools, almost all based on variations of the Dannenberg strategy as presented in [DAN 84]. Some limitations of this model have arised, regarding the input domain, the matching criteria, and the error handling. A new model that isolate the matcher from the control (what to do if the input does not match) have been implemented, together with a language to express the matcher partition as a set of logical rules of arbitrary complexity. Furthermore, different strategies have been experimented, that use classical approximate pattern matching techniques already used in character pattern matching. The work in this direction has shown promising results. Last, a research have been carried out on a specific subproblem: the recognition of the pitch of a singer's voice. The available tools in this field are somehow inaccurate and unstable, and this degradates all the score recognition process when it is used in live situations. 6 Conclusion As performance is the main issue, jMax evolutions naturally targets improvements in this area, featuring extensions of the multi-threading kernel toward multi-processors execution of DSP as well as control. Other extensions focus on language extensions, in particular more explorations of the different computing paradigms already introduced. It must be noted that at the time of the writing of this article, jMax is distributed under GNU's General Public License, thus making its development opened to the computer music community. References [DAN 84] R. DANNENBERG. An On-Line Algorithm for Real-Time Accompaniment In Proceedings of the 1984 International Computer Music Conference, San Francisco, 1984. International Computer Music Association. [DEC 95] F. DECHELLE AND M. DE CECCO. The IRCAM real-time platform and applications. In Proceedings of the 1995 International Computer Music Conference, San Francisco, 1995. International Computer Music Association. [DEC 96] F. DECHELLE, M. DE CECCO, E. MAGGI AND N. SCHNELL. New DSP applications on FTS. In Proceedings of the 1996 International Computer Music Conference, San Francisco, 1996. International Computer Music Association. [DEC 98a] F. DECHELLE, R. BORGHESI, M. DE CECCO, E. MAGGI, B. ROVAN AND N. SCHNELL. Latest evolutions of the FTS real-time engine: typing, scoping, threading, compiling. In Proceedings of the 1998 International Computer Music Conference, San Francisco, 1998. International Computer Music Association. [DEC 98a] F. DECHELLE, R. BORGHESI, M. DE CECCO, E. MAGGI, B. ROVAN AND N. SCHNELL. jMax: a new JAVA-based editing and control system for real-time musical applications. In Proceedings of the 1998 International Computer Music Conference, San Francisco, 1998. International Computer Music Association. [DEC 99] F. DECHELLE, M. DE CECCO, E. MAGGI AND N. SCHNELL. jMax: An Environment for Real Time Musical Applications. to be published in Computer Music Journal, 1999. [ECK 95] G. ECKEL, F. IOVINO AND R. CAUSSE. Sound synthesis by physical modeling with Modalys. In Proceedings of the International Symposium of Musical Acoustics, Le Normant, Dourdan. 1995. [IOV 99] F. IOVINO AND N. SCHNELL. Modalys in jMax: real-time modal synthesis. In Proceedings of the 1999 International Computer Music Conference, San Francisco, 1999. International Computer Music Association. [LIN 91] E. LINDEMANN, F. DECHELLE, M. STARKIER AND B. SMITH. The Architecture of the IRCAM Musical Workstation. Computer Music Journal, 15(3):41-50, 1991. [MAG 96] E. MAGGI AND F. DECHELLE. The evolutions of the graphic editing environment for the IRCAM musical workstation. In Proceedings of the 1996 International Computer Music Conference, San Francisco, 1996. International Computer Music Association. [ROD 92] X. RODET AND P. DEPALLE. Spectral Envelopes and Inverse FFT Synthesis. In Pro ceedings of 93rd AES Convention, New York, 1992. AES. ICMC Proceedings 1999 - 447 -

Page  00000448 File Edit Text Tools WIndows Help computation by side effect L& (- tab 166 defwie variable Wtp as table of ampitudes vweqht(tab 166 dethne variabe lwe tnts as table of weights Sai sampo outputs reference to table of ampttudes when triggered 10&d US.fblat& 4 f aM des ftom fie o tatble t tt 3 slput first to right than to lft outlet tweights 0 *2. S itweights h0ad asci anms.float ads low tud from fie into table 12. Asplay Lamp S weeihts splay Samp Stweqhts 2 dIsay tamp tweghts 2 Figure 1: A jMax patch demonstrating computation by side effect FIe Edit Text Tools WIndows Help additive synthesis by inverse FFT *FFT minus one' Control oprC... example of choir sound using preloaded data files sod transose from harmonic analysis of single voices FMO- 128 anwrsa oand outpat ate(tO referwens of overaoad fratwa to wraty 0ectrlu" 0op Loco rmorve sorre canad calt fwerOAy v eto r of trtansto m " U ar-Vtte, 6110 Mid swrr normaons stwioav twiaw to.CctuM FOhartonice 100 100 ogutput FMO referlree to tpectrum *1 0t 'opiwr a rmWns I wLoro armo II w4upo-e k'o@ tr friatrey vetor of trantto contrdt tor add aNr hw %ar% to sacw FMoha rmonics 100 100 ~n~h~nrsn~s oo n Ien output FMO referemq 6-d sesra ot -9 ý % cts W MIA atari (P44) trlgvwietta tO ~ ~ ~ mOY ttatrettt @ Figure 2: A jMaix patch implementing additive synthesis - 448 - ICMC Proceedings 1999