MEDIATE:
Key Sonic Developments in an Interactive Installation for
Children with Autism
Hans Timmermans*, Gerard van Wolferen*, Paul Newlandt, Simon Kunatht
*Utrecht School of Music and Technology, Utrecht School of the Arts
hans.timmermans@kmL hk, i. U niv, ersi ardNtsanremouthl nCkmthkwn
tUniversity of Portsmouth
simnon.kuimnath:@port.ac.u k, paul.newlx'and&porLt.acx.uk
ht p:// `Nebbport,.a.c.ukimediatei
Abstract
Our paper initially gives a brief overview of MEDIATE, a
multi-sensory, interactive environment created with the aim
to encourage expressive engagement from children with
autism. We then detail the particular innovations required
to support such a context when incorporating sonic and
tactile interfaces which through a 'signature analyzer' gain
the ability to both detect and respond to user behaviours
within this environment.
1 Introduction
MEDIATE is a 5th Framework EU project funded by the
IST (Information Society Technologies) strand undertaken
from October 2001 until April 2004. Developed by three
European design teams from Universities in Portsmouth
(UK), Barcelona (Spain) and Utrecht (Netherlands),
constructed by Show Connections and evaluated by
Goldsmiths and Kings Colleges, London, the acronym
MEDIATE stands for (Multi-sensory Environment Design
for an Interface between Autistic and Typical
Expressiveness). The project has culminated in the
production of a multi-sensory, responsive environment
designed to stimulate interaction and expression in children
with autism through visual, sonic and vibrotactile means.
1.1 Touch
MEDIATE wished to allow touch and vibration to grow
as creative expression - countering the historic trend for
poor exploration of this modality. Although a fundamental
means of communication, touch and vibratory feedback are
usually under played: through work at the School of Art,
Design and Media, University of Portsmouth, this modality
is encouraged by means of a responsive multi-textured
'Tune Fork' and vibrating 'Impression Wall'.
1.2 Aural
The modality of sound production and creative feedback
is discovered by the user through MEDIATE's capacity to
transpose movement via a sensory floor into audio input and
adapt a user's vocalisations. Utrecht School of Music and
Technology, Utrecht School of the Arts developing
approaches which allowed the user to interactively engage
with the generated output - filling the environment with
multi-channel aural compositions under the user's control.
1.3 Visual
Interactive Audiovisual Group, Universitat Pompeu
Fabra enhanced an infrared detection system (EyesWebTM)
and worked with particle based light compositions projected
on two independent environment walls. Drawing the
individual into participation these visually exciting surface
patterns are animated and influenced by the user's position,
wall contact and full body gesture. Through this control,
poetic and flexible interaction scenarios can evolve as the
user moves within MEDIATE.
1.4 Signature
Integral to the functioning of MEDIATE is an intelligent
inbuilt pattern detection system, 'signature', that recognises
repetitive behaviour in subjects and reacts by continually
adjusting and enhancing its response. Together this multisensory interface is designed to facilitate expression through
generative, full body interaction and allow children with
autism to experience a sense of agency over their
surroundings without need for the usual socio-linguistic
skills.
Proceedings ICMC 2004
