Page  00000281 Sonification of Social Data Alberto de Campo, Graz/Austria Marianne Egger de Campo, GrazlAustria Abstract This paper explores the usefulness of sonification for displaying and analyzing social data. Prototypes for an interactive sonification framework were implemented in the synthesis language SuperCollider2. The authors have experimented with auditory display of aggregate data (executions in the USA, and election results in Austria) and have explored a data matrix of survey respondents. Sonification can increase the awareness for problems in statistical analysis methods, it complements statistics particularly well with lower sample sizes, and it shows promise for multidimensional data analysis. 1. Introduction Social data are generally concerned with multivariate relations. Thus appropriate analysis methods should take into account a multitude of dimensions. Multidimensional data, however, are difficult to display in a meaningful and rich way. Most social processes require complex models that are beyond the dimensional limits of human visual perception. Thus sociologists try to extract meaning hidden in their data sets by subjecting them to statistical transformations. The result is often highly abstract and the relation between the resulting coefficients and the original data is hard to trace. Sonification offers an alternative approach to exploratory analysis of sociological data: Auditory perception is perfectly able to track multiple dimensions of the sound properties in data-generated streams of events. Sonification of social data replaces statistical analysis methods with perceptual abilities. 2. Analysis of Social Data In sonification, data are translated to sound by mapping data parameters onto properties of the generated sound events, or streams of sound events. Auditory display is the simple translation of aggregate data, e.g. poll results [1] Here, values such as percentage can be mapped to parameters of a sound stream, e.g. repeat rate. For many types of aggregate data, auditory display is straightforward and immediately clear to non-expert listeners. Exploratory data analysis is the other end of the continuum of possible approaches. When using multidimensional data, the richness in meaning lies in the distinct constellation of parameter values for each case; this richness can be kept intact by mapping the data for each case onto the sound properties of one event. Social data often contain variables that cannot be quantified (i.e. nominal level variables like gender or race) which suggests grouping into subsets (e.g. by gender, race), which can be sonified as multiple simultaneous streams. To make these identifiable, marker parameters are needed. The most natural marker parameter is spatial position, simply because it is the most constant feature of a sound source in the real world. The choice of sounds is crucial. In general, we have found the neutrality of abstract sounds to be less manipulative and distracting than e.g. imitated musical instrument sounds. Musical structures such as traditional scales and consonant intervals can be misleading by making streams merge for musical, not perceptual reasons. Since the ear can not resolve all sound properties equally well, mapping social variables to sound parameters has to consider that the values of a variable are acoustically distinguishable. The highest resolution is achieved in pitch perception, so continuous data (like age or income) are usually best mapped to pitch. 3. Examples We have experimented with the following data sets: 1. The number of executions in the USA since 1977 (the reintroduction of the death penalty). Each year is assigned a specific time period during which every single execution is represented by a sound event. Space is also mapped onto space, here in a symbolic way: The streams in each channel play the number of executions in each of four regions of the US: Northeast, South, Midwest, and West. 2. A time series of all results of parliamentary elections in Austria since World War II. Here the political spectrum was mapped to the ICMC Proceedings 1999 - 281 -

Page  00000282 spatial distribution (i.e. socialist parties are toward the left, conservatives to the right). For clarity, the streams were also differentiated by pitch. The number of votes for each party were mapped to the repeat rate of parallel streams of repeating pitches. Since non-voters are fundamentally different from voters, they were represented with a different sound stream: continuous noise. 3. Interview data of some 300 school children (age 8 to 14) in Graz, Austria: the self-reported self confidence of each student as measured on a 5 point scale is mapped on pitch. Each student of the sample is made audible as a sound event. In addition the sample was divided into two data streams representing male and female students on different spatial positions and with different timbre. The data were sorted hierarchically by age and by pitch (i.e. self confidence) within each age group. 4. Advantages of Sonification of Social Data Sonification provides free benefits, e.g. the "common sense" mapping of time onto the temporal dimension of an auditory display. A visual display of time series does not offer this option. Geographical distributions can be mapped onto spatial sound properties; a visual display of the added spatial dimension requires condensing or reducing the other represented information. In contrast to statistical data analysis, sonification need not subject the data to various transformations to extract relevant information; in fact, the more pristine the data are the better - the ear can evaluate the data with its ability, to perceive complex patterns. The process of experimenting with and gradually developing a more and more complex stream of sonified data is in itself highly instructive. In contrast to visual displays or tables the immediacy of an auditory display adds a "human dimension" to the analysis of social data. Representing 100 persons with 100 events is much more intuitive than the abstract number 100. Sonification therefore can help "to bring men back into" social science [2]. Sonification allows to represent quite complex data; whereas visual displays are limited to three dimensions, the ear is capable of perceiving more dimensions simultaneously. With sonification one often wishes for richer data, e.g. in our example of executions in the USA using the exact day of each execution for the time axis, instead of a number of equally distributed events. This would also allow for a cross-comparison with possibly relevant factors going on at the time, e.g. changes in political power. Sonification is able to make both relative and absolute frequency (percentage and number of cases) perceivable at the same time. The oneevent-per-case stream concept keeps the number of cases inherently present. Low total case numbers make statistical results less reliable; and there is no way to miss that fact in sonification. Our sonification of the Austrian election results illustrates a fact that is generally neglected by ordinary data display (in tables or graphs): the increasing noise for low turnout simultaneously represents both the validity and the (proportional) frequency of votes. 5. Problems and Solutions Complexity The complexity and richness of the displayed data can turn into a problem when researchers become overwhelmed by the complexity of the generated sound streams and consequently lose confidence in their untrained hearing abilities. Some guidelines for using a sonification system should help avoid this: Start with a blank stream, then add parameters one at a time. Make the scaling and centering of each parameter accessible in a GUI. Unless a specific mapping is very compelling, (such as time to time) it helps to keep mappings changeable, to have on/off switches for temporarily disabling distracting parameters, and to include options for inverse scaling of data parameters (to neutralize biases in favor of e.g. brighter sounds). Since most people hear only relative pitch, (and timbre, volume, attack time and so on!), maximum, minimum and center value examples (beacons [3]) for the current center and scaling settings of each parameter should always be available for reference. Access to data subsets, e.g. looping specific groups or regions and jumping between looped subsets should be easily available. Artificial time dimension Social data like opinion polls or surveys are not time-based; i.e. sonifying them adds an artificial time dimension. This time axis can become a powerful guide through a data set: E.g. sorting all cases by the respondents' age will make the age distribution of other mapped parameters audible. Hierarchically sorting the cases within each age group by a second parameter - e.g. selfconfidence helps to reduce complexity further. -282 - ICMC Proceedings 1999

Page  00000283 Finally group borders can be made audible by introducing rests. Splitting data sets into simultaneous streams Discrete parameters imply data subsets, e.g. gender can be made audible as spatial position in a single stream. This generates a distracting jumping effect; mapping the subsets to two simultaneous streams is a better solution. If the groups represented by the data streams differ in size, we have to decide whether it makes more sense to keep the time frame for each age group constant or rather to scale time to the total number of each age group, so that the overall speed is constant In any case the procedure will make evident that proportions based on small total numbers have to be treated with more caution - a fact often neglected in statistics. Volume Mapping a parameter to volume will lead to soft events becoming less and less noticeable. Care must be taken to do this only when the parameter is meant to portray insignificance. Timbre Timbre and pitch interact. While it is easy to hear whether two abstract sounds with different timbre have equal pitch, there is no unambiguous way for two abstract sounds with different pitch to have equal timbre. (Identification of musical instruments depends on very specific constellations of attack transient parameters, and only partially on spectral content of the steady state sound. Thus, mapping data parameters onto these sensitive characteristics can easily destroy instrument identification.) This cannot be resolved, only alleviated; e.g. by reducing the pitch range and increasing the timbre range. (One can limit fundamental pitch to 100-500 Hz and filter resonance frequency to 1000 -6000 Hz.) Missing values Very often a data set consists of incomplete cases, i.e. a respondent has forgotten or refused to answer some of the questions. Statistical procedures allow to include incomplete interviews by setting the missing value to zero. Sonification, however, combines multiple answers onto one sound event (which is characterized by the various sound parameters) and therefore cannot ignore a missing value. While the exclusion of incomplete cases may lead to an unwanted reduction of the data set available for analysis, marking missing values with a very different sound may increase complexity and therefore impede fruitful exploration of the data. Size of data sets In general, large data sets as used in survey research will have too many data points for an exploratory data analysis with sound. They can, however, be broken down to aggregate data which again can be represented in a meaningful auditory display. Exploratory data analysis by sonification works well with smaller data sets and can therefore complement ordinary statistical models that in general require higher case numbers. 6. Conclusion Sonification turns out to be a helpful tool for sociological data analysis. It invites an informal, exploratory approach of "getting to know" a data set by giving a more tactile presence to data dimensions that are otherwise easy to lose track of. The flexibility and interactive possibilities of the framework suggested here make it well suited as a complement to standard statistical methods. More research is needed on laypersons' reactions to aesthetic design choices such as the palette of sound synthesis algorithms. Note: Prototypes of the framework suggested in this paper have been implemented in Supercollider 2 and are available from the authors. Write to: References [1] Kramer, Gregory, 1994. "An Introduction to Auditory Display." In: Auditory Display. Sonification, Audification, and Auditory Interfaces. (ed. Gregory Kramer) Santa Fe Institute Studies in the Sciences of Complexity pp. 1-77. [2] Homans, George C., 1964. "Bringing Men Back In". Presidential Address delivered at the Annual Meeting of the American Sociological Association. American Sociological Review Vol 29, 6, pp. 809-818. [3] Kramer, Gregory, 1994. "Some Organizing Principles for Representing Data with Sound." In: Auditory Display. Sonification, Audification, and Auditory Interfaces. (ed. Gregory Kramer) Santa Fe Institute Studies in the Sciences of Complexity pp. 185-221. ICMC Proceedings 1999 - 283 -