/ Model based interactive sound for an immersive virtual environment
ï~~Model-based interactive sound for an immersive virtual environment. Robin Bargar*, Insook Choi*, Sumit Dast, Camille Goudeseune* *Audio Development Group, National Center for Supercomputing Applications, University o f Illinois at Urbana-Champaign tElectronic Visualization Lab, EECS, University of Illinois at Chicago rbargar@ncsa.uiuc.edu Abstract We discuss an audio rendering pipeline that provides real-time interactive sound synthesis for virtual environments. Sounds are controlled by computational models including experimental scientific systems. We discuss composition protocols and software architecture for hierarchical control and for synchronization with graphics. Rendering algorithms are presented for producing sound from a physically-based simulation of a chaotic circuit and from higher-dimensional topological structures. 1. Introduction In this paper we discuss the implementation of a rendering pipeline designed to bring sound synthesis and composition as research components into virtual environments (VE). We find that VE research provides a platform for projects closely related to computer music composition. We also find the VE research community is interested in the potential relevance of composition for their work, and the relevance of their work for composers. We have been developing a software-based sound synthesis and composition protocol to enhance the possibilities of collaboration. This protocol defines a pipeline from computational models to sounds. Along this pipeline we identify endeavors related to computer music including real-time sound synthesis, gesture-based interaction, composition algorithms, physically-based sound production models, and techniques for synchronizing sound with graphical events. A rendering pipeline encourages composers to consider the entire synthesis process as a composition. Generating and controlling complexity is among the most difficult tasks of computer music, and the pipeline model is valuable for connecting complex systems with rendering engines. Rapid low-level communication from complex model to sound synthesis engine permit the composer to use interactive control of the complex model to control the sound. 2. VE Systems Virtual environments are multiple-engine computation systems converging toward solutions for immersive human interface. Immersion has been associated with two classes of experience: fictional constructs and feedback constructs1. Music listening may be identified primarily with the first class and music performance primarily with the second. In order that sound provide aspects of both classes of experience we embark toward a familiar goal: techniques for generating dynamical audio spectra informed by real-world analysis, using efficient numerical encoding that allows synthesis with interactive variation in real-time. Existing computer music systems provide some of these capabilities in specialized hardware. Rather than adopt existing music systems we have focused on the importance of demonstrating that sound synthesis is relevant for general-purpose computing. We want researchers to have immediate access to sound computation in the same language, operating system and control flow that supports standard computing and graphics rendering engines. Therefor our pipeline is written in unix/c/c++ to maintain potential portability and scalability and stay close to graphics architectures and their user communities. 2.1 3D Primitives The reality image in computer graphics is encoded in an objective 3D embedding space. This objective encoding describes the range of potential subjective views that may be obtained from the image. Actual 3D rendering depends on hardware that is downstream from the objective 3D encoding. We adopt similar subsystems for audio. A primary software language encodes oscillations, envelopes, and sound propagation information, and manages rendering of dry (non-localized) sources. Secondary, scalable display subsystems are used to generate localization and depth cueing. Regardless of rendering hardware, 3D features of the propagation environment are encoded as attributes of the primitive description of a sound. 2.2 The CAVE virtual environment The primary testbed for our sound system is the CAVE, a surround-sound, surround-screen, projection-based VR Figure 1. CAVE Automatic Virtual Environment University of Illinois 1. Fictional constructs involve an observer's "willing suspension of disbelief' that supports the diegesis (the narrative world created in literature, theater and cinema). Feedback constructs are "everyday experiences" that an observer constructs by taking actions and observing their consequences through multiple sensory modalities. ICMC Proceedings 1994 471 Acoustics
Top of page Top of page