2. The Coney Island Scenario
Coney Island is a VR installation designed to explore and
demonstrate advanced auditory display of spatial and
temporal models'. The installation provides an interactive
tour of an archipelago of mechanized islands that comprise
a fantastic carnival playground. The islands are driven by
simulated mechanics and particle system dynamics, as well
as advanced geometric, lighting and camera models for
computer graphics. For each island MIDI-enabled drum
pads allow up to ten observers to play simultaneously.
Simulations provide a time-critical environment where
players can impart forces and see and hear the resulting
mechanical actions and particle system collisions. The tour
continues underwater where the players can impart force to
currents that activate sound-producing clusters of floating
objects. In each case equations of motion convert the forces
into motions of graphical objects and in parallel into
sounds.
Coney Island was designed as a case study for close
coupling of audio signal processing to spatial and temporal
VR paradigms. Multiple independent sound-producing
events are determined by sensor data combined with
simulated mechanics of rigid polygonal bodies and particle
systems. Sensor, graphic and sound events must be
scheduled to provide satisfactory temporal feedback. At the
same time overlapping audio events must be rendered in a
spatial model that allows each player the proper orientation
with respect both to a view of the virtual world and a
position in the real world adjacent to other players.
I Figure 1: Coney Island setup at IRCAM
The sound system consisted of three multi-channel
computer sound sources, a mixer and a 4-channel
diffusion system with monitors positioned in the
corners of the room. Audio software performed in realtime on linux, NT and Irix platforms. Data was
transmitted from ScoreGraph to VSS and from VSS to
jMax using udp. Audio sources included 2-channel VSS
on Linux and NT PCs, 2-channel jMax on Linux PC
and 4-channel jMax on an SGI Octane.
Figure 2: IRCAM visitors interact with Coney
Island simulations using MIDI drum pads.
Figure 3: Players' view of Coney Island.
Hardware Configuration
IRCAM Studio 5 was arranged with a large-format video
projection on the wall opposite its entrance. Graphics and
simulations were rendered in ScoreGraph on an SGI Onyx
and the image transmitted to the projector. Figure 1 shows
ten MIDI-enabled drum pads positioned in the center of the
installation facing the projection screen, with a solo
joystick on the left and sound computation hardware on the
right. Signals from players' actions were input to
ScoreGraph simulations, and the resulting movement
events passed to graphics and sounds. Figure 2 shows a
group of players at the IRCAM installation; Figure 3 shows
the players' view of a Coney Island scene.
Coney Island was presented at IRCAM during the June
1999 Portes Ouvertes.
3. Coney Island: VR architecture and graphics
Coney Island uses a software framework named
ScoreGraph to organize its numerical simulations and
interactive graphics, to manage input from a user
interface, and to send audio control signals to VSS.
ScoreGraph is a system for authoring and managing the
presentation of interactive, real-time graphics and
sound applications. ScoreGraph provides a scheduler
and libraries for data computation and multi-threaded
communication. A ScoreGraph application consists of
reusable software modules written in C++ and a script
that specifies the configuration and behavior of those
modules at runtime. Application components are
roughly divided into input devices, computational
models, and graphics and sound displays. Individual
components, called nodes, are organized into a directed
graph, the edges of which represent control signal flow.
When the application is run it is organized into parallel
threads that manage the execution of its nodes. The
service rates of the threads are independent of each
