COMBINING ANALYSIS AND SYNTHESIS IN THE
CHUCK PROGRAMMING LANGUAGE
Ge Wang
Princeton University
Dept. of Computer Science
Princeton, NJ USA
[email protected]
Rebecca Fiebrink
Princeton University
Dept. of Computer Science
Princeton, NJ USA
[email protected]
Perry R. Cook
Princeton University
Dept. of Computer Science
(also Music)
Princeton, NJ USA
[email protected]
Synthesis
(physical modeling, live
performance, composition,
on-the-fly programming,
etc.)
Analysis
(spectral processing,
feature extraction, rapid
prototyping, etc.)
Programming
Model
precise timing
real-time control
(HID, OSC, MIDI)
concurrency
(parallel analysis
synthesis tasks)
dynamic
dataflow
rapid
experimentation
Figure 0. A ChucK-based programming model for building audio analysis and synthesis programs.
ABSTRACT
In this paper, we present a new programming model for
performing audio analysis, spectral processing, and
feature extraction in the ChucK programming language.
The solution unifies analysis and synthesis in the same
high-level, strongly-timed, and concurrent environment,
extending and fully integrating with the existing
language framework. In particular, we introduce the
notion of a Unit Analyzer (UAna) and new constructs
for dataflow, data types and semantics for operations in
analysis domains, and mechanisms for seamlessly
combining analysis and synthesis tasks in a precise,
sample-synchronous manner. We present the motivation
of our system, and describe new language-level
syntaxes, semantics, and the underlying implementation.
We provide code examples and discuss potential uses
and benefits of the system for audio researchers,
performers, and teachers.
1. MOTIVATION
Combining analysis and synthesis in the same
framework can lead to interesting applications, as
exemplified by the works of Roger Dannenberg and
Chris Raphael [7], Nick Collins [4], and many others.
Existing systems are mostly implemented in
combinations of low-level C modules, open source
libraries and frameworks, high-level languages, and
proprietary software. We'd like to enable more people
to experiment with, prototype, and create new tools and
systems like these, starting from a single unified, highlevel platform, without minimizing the need to develop
plug-ins in other languages (e.g., C) or to write custom
low-level modules from scratch.
Such a unified programming platform, we believe,
should successfully address the following issues. First,
in many musical applications, analysis and synthesis
inform one another. Therefore, the environment should
facilitate and encourage this symbiotic relationship,
placing equal emphasis on the two, and providing
flexibility and ease of programming for both. Next, the
system should present a precise and flexible
programming model with which programmers can
rapidly prototype and implement analysis and synthesis
tasks - and perhaps even do on-the-fly. Additionally,
the high-level abstractions in the system should expose
essential low-level parameters while doing away with
syntactic overhead, thereby providing a highly flexible
and open framework that can be easily used for a variety
of tasks. Finally, it's extremely important that the
written code represent the underlying algorithms and
dataflow precisely and clearly. Overall, we envision a
language that meets these criteria and that can be
equally suitable for audio research (e.g., synthesis,
spectral processing, feature extraction), pedagogy,
composition, and musical performance.
There exist frameworks and languages that
effectively address some components of our goals,
including synthesis systems that accommodate analysis
tasks and vice versa, and standalone systems that
perform a specialized analysis/synthesis task. What we
hope to achieve, in this work, is to produce a single
programming platform that meets the need of a broad
audience by offering solutions from programming
language perspective. In doing so, we hope to encourage
new and different ways to think about audio
programming for audio and synthesis.
In this paper, we present our programming model for
specifying precise audio analysis and synthesis tasks in
the ChucK programming language - specifically
designed to address the goals we outlined above. We
35
