Page  28 ï~~Graphical NeXTSTEP Objects as FTS Clients to Control Instruments in the new FTS Client/Server Architecture Todor Todoroff and Caroline Traube Facult6 Polytechnique de Mons todor@musique.fpms.ac.be Abstract We are designing graphical objects within NeXTSTEP which connect as Clients to the new FTS Server. Those objects provide better ways of controlling and visualising the FTS processes than those presently available within MAX/FTS, offering new multidimensional graphical controllers and viewers as well as better table editing tools. They are also intended to help mapping and visualising incoming data from a PowerGlove, from MIDI faders and from various other sensor devices. At the conference, we will demonstrate those tools in interaction with several instruments, performing granular morphing, spatialisation of sounds and transformation through analysis-resynthesis. 1 Introduction The MAX environment is a very powerful tool for developing and prototyping sound processing algorithms. But, though it might seem easy for the experimented MAX user to modify a patch in such a way that it responds as he wishes to external MIDI interfaces, composers who usually lack that knowledge and experience are often helpless when it comes to modify rather simple things. Specially as a MAX/FTS window is usually invaded by patcher boxes and connections one cannot hide. MAX also lacks of any kind of multidimensional controller: there are buttons, number boxes, sliders and tables, but one cannot call the table object a real multidimensional controller to play with. And composers who have worked with programs like Hyperprism or GRM Tools know the difference a simple x-y controller can make when it comes to controlling sound processing. Going further, a graphical interpolation tool like the one offered on the SYTER at the GRM can transform a well designed sound processing tool into a very expressive instrument. Such a graphical tool was programmed by Adrien Lefevre for Opcode MAX, but was not yet available for NeXT+ISPW users. We designed one that increases further the expressiveness by making it controllable by any MIDI controller. All this has been made possible by the new FTS Client/Server Architecture [D6chelle and De Cecco, 1995] which allows bi-directional communications with several clients at a time in a well documented manner. We will first describe with an example how the communication system between NeXTSTEP and FTS works. Then we will propose several graphical objects. 2 NeXTSTEP objects as FTS Clients Several clients may connect themselves dynamically to the FTS server. Each client may either be on the same machine or on a different one connected through Ethernet. To allow communication from a graphical NeXTSTEP object to the FTS server, the client has only to initialise the connection and send messages to the desired "receive" object in a MAX Patch. To receive messages from MAX, it is necessary for the client to create its own "receive" objet within FTS (with the same name as the corresponding MAX "send" object) and a special object of the type "gobj" that will transfer the information to the client. The client then establishes a connection between those two objets and has to poll regularly to check whether some new messages have arrived. Those modules created by the client are not visible in the MAX patch, as they are directly created within FTS5. We show on figure 1 how we implemented a simple 2-D controller as well as a 2-D visualisation tool. They might of course be combined, but we will use this as an example. The simple MAX Patch depicted allows to receive lists of x-y co-ordinates through a "receive" object named "from_client" and to send lists of x-y co-ordinates through a "send" object named "to_client". The graphical client has two windows: one allows to move a circle with the mouse or with sliders, and the other receives the x-y co-ordinates lists and displays a circle at the corresponding position on a 2-D map. Special care has to be taken in dealing with the amount of messages sent to the graphical client and with the frequency of the polling performed by the client. In order to keep the patch simple, we didn't show those modules limiting the sending of new values to one every 100 ins. Without this, the graphics may appear very slow, displaying every intermediate value and needing time to catch up before the last one is displayed. More efficient procedures, like allowing the client to get the entire queue of messages in one call, might be implemented in the future, as FTS is still in Beta version. This mechanism of sending and receiving values allows one client to Todoroff & Traube 28 ICMC Proceedings 1996

Page  29 ï~~access several "send" and "receive" boxes, as well as several clients to access the same ones. It is therefore possible to open many graphical clients at the same time. Name of the destinationvhere the FTS Client sends Its messages to r from client x v 1 - 7 Is to Clent Name of receive box crealed by the FTS Client 0 0 are 1: 2-D control and visualisation 3 The interpolation window Inspired by the "Interpol" window which was one of the strong points of the SYTER at the GRM [Terrugi, 1991], our object has some added features, like allowing to choose between several interpolation laws and allowing the current point to be moved either with the mouse or through messages sent by FTS that may come from MIDI controllers, like the PowerGlove that we use with the STEIM SensorLab. Each circle may be activated or deactivated and the configurations may be stored and recalled. The compare button is enlightened when the selected preset has been modified. The lists of weights are sent to FTS to control the sound processing. The right part of the window may be hidden at any time to free space on the screen. It is not usually needed as moving and resizing circles is done with the right mouse button. In order to make full use of the PowerGlove as a 3-D controller, we plan to implement a 3-D version of this interpolation concept. ICMC Proceedings 1996 29 Todoroff & Traube

Page  30 ï~~4 Control over the spatialisation of sounds A simple movement generator, with several types of movement will be further developed to include a larger library of shapes and to be able to interact smoothly with MIDI controllers. Though the generator is presently coded within NeXTSTEP, we will move it to an external object within FTS, controlled by the graphical object. This will lower the flow of messages between the client and the server, as well as authorise higher speed and better precision. While the sphere is moving within the 3-D space, it draws behind itself the last ten points of its trajectory, giving both an indication about the shape of the movement and about its speed. This tool may also be used to control the current position in the interpolation window, creating in a timber space a trajectory of which speed and shape may be controlled by a MIDI controller. C) r,. } 1 l M.' t' U. l34 S...........,.,..,..,. -..o...,iV1 n-. e..,. 1; S 'r, r,r: rf; is, r'r ' 1,., %:%; e 3: The movement generator w a 3-D viewer As we want to use the fingers bending to change some of the parameters, we also designed a little viewer to visualise them. We plan to place next to each finger a little button that will open a table to adjust the bending sensitivity according to the user's morphology and to the sound process he wants to control. Todoroff & Traube 30 ICMC Proceedings 1996

Page  31 ï~~5 A table editing tool We wanted a easier way to create and scale table values. With this tool, x and y may be floating points values, they don't have to start at zero, may be rescaled simply by entering new limits in the number fields, and the number of points may be changed with ease. Segments of lines may also be easily drawn and the exact values of the points may be displayed in the right side of the window. That part of the window may also be hidden when it isn't needed anymore.::::::.:::....:,,::: {; i, \::..: i::::: ':'::':":::......';"10 0 0:::': ": /....:...UIF:t r Y8.248 38 332::::::; 2 62 327.309 k"1123 79 0 8.67 ''" '"04 1 24 43 4.1.:'':::/ ',.;..; - ', I,:; l ' '.:::":.;:. fl': 000-:".'5 '13"'5"38.:3:113 ', i;f::": 6 18 86 941.748 _"ti i18 8 44.941 t; 9 29589 257 20 620 284.796:l. 1 372 3 690.04 23 73 91715 butili the iv an ii i idaowhte wan to ofe a. more intuiive! GIfo the copoer o eleci.iroacousticr. muic!!!i~~~~~~~~~~~ii!~~~~i i!! i!: i ":::!!ili:::.::. 14 i~l I434 815.894e "::::! i! r'' i " "! i "" i { {!:i~ii!!.',~: "" 1 49 94.7S:. thnst ie irr fmliisional-controllers and viewers W a wurie:Te aX user and esiora modulation without having to add or remove MAX modules and connections. This will help giving a more transparent access to composers who are not necessarily familiar with the MAX language. For each signal processing instrument (granular synthesis, spatialisation of sound, transformations by analysisresynthesis,....), several sets of, efficient mappings will be proposed and may be later modified and saved by the composer for future use. This will give a fast idea of the importance a different mapping may have on the use of an instrument and on the sound it produces. Using those presets, the composer will be able to design his own mappings much faster than starting from scratches. We have discussed in [Todoroff, 1995] how we think that this approach might also help to generate more convincing sound morphologies by creating more "specialised" instruments to choose from on top of more universal and less directive sound processing techniques. Due to the fact that those tools are build within NeXTSTEP, they offer a high level of modularity and allow the user to easily customise his/her working environment: using tables or scaling whenever needed, opening several 2-D or 3-D interpolation spaces at the same time, visualising different time or frequency-domain signals, editing filter transfer functions, etc. References [D~chelle and De Cecco, 1995] Franvois D~chelle and Maurizio De Cecco, The Ircam Real-Time Platform and Applicaticons, Proceedings ICMC, Banff, 1995, pp. 77-83. [Terrugi, 1991 ] Daniel Terrugi, manuel du stage SYTER, INA-GRM, september 1991. [Todoroff, 1995] Todor Todoroff, Instrument de synthdse granulaire dans Max/FTS, Proceedings ISEA, Montreal, 1995. [Traube and Todoroff, 1996] daroline Traube and Todor Todoroff, Interfaces graphiques NeXTXTEP pour ia commande d'instruments virtuels d'aide d la composition et d l'interprtation de musique electroacoustique' Troisimes joum6es d'Informatique Musicale JIM'96, 1996, pp. 98-102. This research has been funded by the Region Wallonne in Belgium. ICMC Proceedings 1996 31 Todoroff & Traube