ï~~Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada
August 16-21, 2009
2.1. Implementation Details
Each track of MuBu is implemented as a contiguous memory array containing the data over time and allowing for vectorised access. Similar to the SDIF standard, numerical data
is represented as a sequence of two-dimensional matrices of
floating-point values (often reduced to vectors and scalars).
The time axis of a MuBu track can be represented either by
a time-tag for each element or by a constant sampling rate
defined for the whole track. Thus, uniformly sampled data
streams of multiple rates and non-uniformly sampled data
streams can be represented aligned to each other. In addition to a reference time-tag, a track can optionally define a
start and end time for each of its elements giving the possibility to associate data not just to an instant but also to a
period of time.
A track can define a name tag for each column of the
contained data matrices or vectors that may refer to the type
or semantic of the data in a particular column. The rows
of the matrix are simply indexed and typically represent
one or multiple instances of the structure defined by the
columns. As an example, vectors of different descriptors
stored in a track would be represented as elements of one
row with column names referring to the different descriptors such as loundness, pitch and brilliance. Similarly, a MIDI note would be represented as a single row with columns
corresponding to pitch, velocity and channel with the option of representing chords as multiple lines of notes within
in the same element of the track. Following the same logic, spectral coefficients would be represented as a single
column (e.g. named MFCC or MEL).
Since the matrix data is stored in a contiguous memory
space, the capacity of a track in terms of the maximum number of elements as well as the matrix size for each track element has to be defined in advance whenever possible. This
choice has been taken with respect to previous experiences
with similar approaches showing that the vast majority of
the envisaged applications can easily cope with this restriction while the implementation of the data structure and the
access to the data can be enormously simplified and easily
optimised in this way.
If in a particular data stream the elements of a track can
have a variable number of rows (e.g. chords of notes or harmonics), the maximum number of rows must be defined and
an additional vector is created within the track giving the
actual number of rows for each element of the stream. For
non-numerical data or for handling predefined data structures that can not be expressed with two-dimensional matrices, a track can define an additional optional vector of user
data handles for each element requiring customised access
methods.
A MuBu track can have a type tag such as AudioSamples, Harmonics or MidiNotes. This type tag can be used by
an application or a particular set of software modules to co
herently identify and process the stored data. We have started to provide a few predefined type tags and specified an
API for user defined types. The type tag of the track also
specifies the data type for the user data handles if used.
For the association of meta-data to a track, each track
includes a name/value table. This table also allows to completely represent the most important file formats with one
or multiple MuBu tracks and to enable the regeneration of
identical or modified files from the stored data.
2.2. Display and Graphical Interaction
In parallel to the SampleOrchestrator project we have worked on a portable library for the display and editing of data
streams such as those represented by MuBu. The library has
been implemented based on the Juce framework2 and provides a set of graphical displays and editors that remain independent from any data structure by providing interfaces for
the access to the data. The first integration of these editors
has been implemented for the FTM library in Max/MSP.
Figure 1. The IMTR Editor graphical component displaying
multiple superposed data streams
In summary, the current version of the IMTR editor library provides the following features already fitting the needs
of a graphical interface for MuBu:
* Editable graphical interfaces for time-tagged scalars
(e.g. bpfs), arrays of sampled values (e.g. waveforms)
* Non-editable displays for sonogram and trace (e.g.
harmonics with pitch and amplitude developing over
time) representations
* Various display styles for each interface (e.g. bpf, steps
or peaks for time-tagged values)
* Superposition or juxtapositon of horizontally aligned
displays with controllable opacity
* Scrolling, zooming, selection and cursor tools
2http: / /www. rawmaterialsoftware. com/juce/
424
