ï~~Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada
August 16-21, 2009
3. MUBU'S FRIENDS
Figure 2 shows the overall architecture of the components
assembled around the MuBu data structure that are briefly
described in the following sub-sections.
Figure 2. Overview of the assembled tools
3.1. Audio Analysis Tools
In the prototype applications that have guided our developments, the IrcamDescriptors and IrcamBeat command-line
tools are used to perform off-line analysis of audio files written into SDIF files and loaded into MuBu. In addition, we
have used the AudioSculpt application[1] to perform segmentation based on SuperVP also exported to SDIF files.
IrcamDescriptors has been partly developed within the
SampleOrchestrator project based on the previously developed software and Matlab libraries. It basically implements
the audio descriptors described in [5].
The IrcamBeat [6] command-line tool as well as AudioSculpt have been used to generate automatic segmentation
for concatenative synthesis. While IrcamBeat performs an
overall tempo analysis of rhythmic musical material to generate markers fitting a metric grid, the SuperVP engine of
AudioSculpt generates markers at transitions based on spectral transitions without any further musical model. The two
techniques are complementary and useful for different audio
materials and concrete applications.
For the efficient implementation of a variety of interesting interactive real-time synthesis paradigms explored in
the framework of the SampleOrchestrator project using similarity and distance measures, our collection of tools integrates the KdTree library [8]. The K-Nearest Neighbors
(KNN) algorithm implemented by the library optimises the
selection of segments or frames in a pre-analysed source
audio stream based on the distance between numeric descriptor values. The KdTree library has been developed for
the project and integrated with the other tools into a set of
Max/MSP modules.
3.2. SuperVP Real-Time Modules
The SuperVP extended phase vocoder [7] has existed for some time as a command-line tool and as the audio processing
engine behind AudioSculpt before being released as realtime modules that also have been integrated into Max/MSP
as audio processing modules3. In the framework of the SampleOrchestrator project, these modules have been extended
and currently integrate the following set of features:
* Time-stretching/compression and scrubbing with optional transient and waveform preservation
* Pitch transposition with optional transient, waveform
and spectral envelope preservation
* Spectral envelope transformation (frequency and amplitude scaling)
* Remixing of spectral transient, sinusoidal and noise
components
* Generalised cross-synthesis
* Source-filter cross-synthesis
In addition to the excellent sound transformation quality of the phase vocoder practically free of artefacts, the
connection of SuperVP with MuBu provides the possibility of applying sound transformations as a function of previously extracted audio descriptors and segmentation. This
permits to impose within certain limits an arbitrary evolution of temporality, monophonic pitch, loudness and spectral
envelope to a given sound and create sounds by hybridisation of feature extracted from two or multiple sounds. In our
prototype applications, we have used SuperVP to transform
and hybridise strictly monophonic sounds recorded from solo instruments and voices.
3.3. ZsaZsa
The ZsaZsa library implementing a granular synthesis and
concatenative synthesis engine has been developed as a second synthesis engine connected to MuBu complementary
to SuperVP4. The design of ZsaZsa is based on the experiences with Gabor [4] in FTM & Co5, a set of audio analysis and synthesis modules for Max/MSP allowing for highly
modular programming of overlap-add audio processing. The
ZsaZsa library even if integrated into Max/MSP as monolithic signal processing module, internally maintains a modular design separating the copying of the grains from the
source sound, windowing, resampling and reinsertion of the
3The SuperVP Max/MSP modules are released in the IRCAM Forum.
http:/ /forumnet, ircam, fr/708, html?L~1
4We like to present ZsaZsa as SuperVP's little sister.
5http: //ftm. ircam. fr/
425
