jMax Configuration
Despite its architecture that offers interesting features for
a use in a distributed application framework, the jMax
environment is used most of the time as a "standalone"
application. An application as a synthesis engine
integrated in a larger distributed application have been
approached in a previous version of jMax, the MAX/FTS
environment, with its multi-client capabilities (Dechelle
1995, Dechelle 1996). However, the use of jMax as a
synthesis engine driven by data coming from another
environment has not been tested prior to the
implementation of the Coney Island installation.
The chosen implementation has been to add to jMax
network communication objects, using UDP as transport
protocols. The udp object receives on a UDP socket a
stream of datas of simple types (numbers and strings),
encoded with the same protocol as used in the
communication between jMax's JAVA user interface and
real-time server (Dechelle 1999a, Dechelle 1999b). The
udp object outputs messages that can be processed by a
usual control patch. The udp objects were used to receive
from VSS data coming from the virtual reality processing.
The formatting and scaling was then realized by patches
using the standard jMax objects set, and the results were
used to drive both granular synthesis and Spat. The choice
of this architecture offers several advantages: portability,
inter-operability, flexibility and good latency.
VSS Configuration
VSS communicates with several other real-time audio
protocols, such as Midi, OpenSound Control (Wright
1997), real-time audio streaming, RAT Mbone
teleconferencing (Varakliotis 1998), and Jmax. This
flexibility has allowed VSS to be used for diverse
integrated applications: Midi controllers driving software
sound synthesis; data from scientific applications driving
Midi and OpenSound Control sound generators; streaming
sound and control data from a musical instrument to a
Max patch running on a computer across campus, and
then streaming the sound back to the instrumentalist; and
in the present case, sending control signals from a virtual
environment (VE) to a real-time 8-channel sound
spatializer (while another VSS, also controlled by the VE,
computes the sound being spatialized). All this can be
done with VSS running under operating systems including
several versions of Irix, Linux, and Windows.
Such interoperability is made possible by VSS's flexible
internal message-passing architecture, and by its lowoverhead C++ class hierarchy. C has become a de facto
machine-independent assembly language; most packages
have a low-level interface written in C or C++, and it has
proven fairly straightforward to embed such interfaces in
wrappers in the form of shared libraries (DSO's or DLL's),
which is how all but the very core of VSS is constructed.
The connection between VSS and Jmax is built on an
internet-domain socket. As far as Jmax is concerned, VSS
looks like just another Jmax patch running on a remote
machine. First, this "patch" in VSS is initialized to
establish a communications socket with a hostname and
port number, where it expects Jmax to be listening.
Then it can receive commands from other parts of
VSS (other "actors") which cause it to send data
messages through the socket. Upon termination, it
closes the socket cleanly. Part of this installation used
both Jmax and VSS on the same machine. In this case
VSS acted solely as controller, not synthesizer, so it
did not need to access the sound hardware; Jmax
therefore had its usual exclusive access to the sound
hardware.
The format of the data messages sent from VSS to
Jmax is an internal Jmax format. The C source code
which encodes and decodes these messages into C
structures was provided by IRCAM. The VSS shared
library correspondingly encodes and decodes these C
structures into the message format internally
understood by VSS. For this project a simple interface
sufficed. From the point of view of other actors in
VSS, the Jmax actor was something to which they
could send a command in the form of a small number
of strings, integers, and floating-point numbers. This
involved a certain amount of copy/pasting in the C++
code, but kept the code simple and reliable; this was a
primary concern given the short amount of testing and
stabilizing time we had. A more general interface
would allow arbitrarily long argument lists, but would
also demand a formal description of such lists instead
of encoding instances on a case by case basis. We
considered streaming the individual audio channels
from VSS to Jmax over Ethernet, but the possibility of
dropouts was a concern (eight uncompressed 44.1
KHz audio signals, a sustained rate of over 5 megabits
per second, is impractical over 10base-T ethernet).
Since the computers running VSS and Jmax were
physically close to each other, an ADAT optical audio
cable sufficed for our application.
5. Discussion
From Coney Island we are able to assess four areas for
further development:
1. diagnostic tools for aiding the cross-platform
coupling of software synthesis modules,
2. middleware for coordinating virtual environments
and audio architectures,
3. multiple-user solutions for interactive
environments, and
4. sound feedback for enhancing group interaction
(1) The installation demonstrates that it is feasible
with current operating systems to realize a distributed
real-time application combining virtual reality and
complex sound processing. But once the
communication architecture is established, a deep
lack of analysis and debugging tools becomes
obvious: the global latency, from user action to sound,
cannot be measured, and no guarantee can be given on
its upper bound. The use of MIDI to transmit users'
actions was the source of greatest latency: an
intelligent MIDI filter was required in ScoreGraph to
discard redundant values when user activity increased.
Without the filter when latency increased users tended
