REAL-TIME MUSIC SYNTHESIS IN JAVA
USING THE METRONOME GARBAGE COLLECTOR
Joshua Auerbach
IBM Research
David F Bacon
IBM Research
Florian B<rimers
Bome Software
Perry Cheng
IBM Research
ABSTRACT
Automatic memory management via garbage collection is the key to the safety, portability, and high productivity of modern programming languages like Java. However, until now no truly real-time garbage collector has
existed for Java. As a result, the extreme real-time requirements of interactive music synthesis and processing
have made it impossible to build such systems in Java.
We have developed a hard real-time garbage collector,
called Metronome, around which IBM has built a production real-time Java virtual machine running on a realtime variant of Redhat Enterprise Linux. In this paper we
demonstrate the real-time capabilities of our virtual machine with a MIDI music synthesizer that we built entirely
in standard Java using the full object-oriented feaures of
the language.
We show that even with the addition of another thread
allocating 8 MB/second of data, our garbage collector is
able to sustain error-free 44 KHz music synthesis down to
buffer sizes of 1ms, achieving keyboard-to-speaker latencies of 5.0~0.75ms, comparable to a Kurzweil K2000R
synthesizer (3.9+~Inms) and suitable for high-fidelity interactive performance.
1. INTRODUCTION
Interactive music processing is one of the most demanding
of all real-time applications, demanding worst-case latencies of 5-10ms, with only 1-4ms of jitter [5, 18]. In comparison, other paradigmatic hard real-time systems have
much less stringent requirements: 20-40ms for helicopter
flight control, 10-50ms for financial arbitrage, and 20 -40ms for telecommunications switches.
As a result, the programming of such systems has required sacrificing many of the software engineering advantages enjoyed by programmers of high-level languages
like Java: security, portability, ease of debugging, freedom
from memory errors, and a large body of portable standard
libraries.
The primary reason for this has been Java's use
of garbage collection, which has introduced nondeterministic interruptions of tens or hundreds of milliseconds.
But garbage collection is the key to many of Java's
most important advantages: freedom from memory errors
reduces the number of run-time exceptions and makes the
remaining ones much easier to find. It also prevents many
kinds of security attacks. Furthermore, automatic memory
reclamation makes programs simpler because the application does not need to be concerned about which functions
have responsibility for allocating and de-allocating memory.
Over the past several years at IBM Research, we
have developed a hard real-time garbage collector called
Metronome, first as a research prototype [2] and then as
the central component of a new real-time Java virtual machine product [16]. The technology has been adopted for
time- and safety-critical systems by companies in a number of industries, including its use by Raytheon for the
development of the software for the next-generation Navy
destroyer.
The real-time Java virtual machine runs on top of a
customized Linux kernel, co-developed by IBM and the
open-source community. These modifications to Linux
are making their way into the mainstream, with most of
them being incorporated into the real-time edition of Redhat Enterprise Linux 5 (RHEL5 RT).
The combination of these technologies raises the possibility of programming hard real-time systems on commodity hardware using a widely-distributed open-source
operating system and a highly portable, high-level objectoriented language.
In the work described in this paper, we set out to systematically investigate the capabilities and limits of our
technology in the domain of interactive music synthesis.
To this end we built a music synthesizer from scratch, entirely in Java. We made extensive use of Java's objectoriented features, including frequent dynamic allocation
of objects. This allowed us to significantly simplify and
yet also generalize the system, relative to our previous experience implementing synthesizers in C and C++.
We make no claims for the innovativeness of our synthesizer from a musical perspective; it is a wavetable synthesizer based on the SoundFont 2 standard. However, it
is the manner of its construction and the achievable performance results that are radical.
After describing the real-time virtual machine and
garbage collector, and the design of the synthesizer itself,
we evaluate the system from a number of angles. First
we run it with successively smaller buffer sizes and investigate the limits of the system in terms of the number
of samples it must buffer in order to maintain error-free
performance. We also compare the real-time virtual ma
103
0
