Page  00000270 WATER MUSIC: SONIFICATION OF OCEAN BUOY SPECTRAL DATA Bob L. Sturm Scripps Institution of Oceanography Coastal Data Information Program mail code 0214, La Jolla, California, 92103 USA bobs@splash.ucsd.edu ABSTRACT The Coastal Data Information Program has been collecting data on ocean wave conditions since late 1975, first using arrays of pressure sensors, and more recently using directional buoys. Taking a Fourier transform of the data reveals the spectral and directional content of the wave-driven motions at the location of the buoy. Scaling these frequencies within the range of hearing, taking an Inverse Fourier transform, and spatializing the spectral components with their respective directions from which they originate, synthesizes an aurally interesting and comprehensively illuminating sonification of ocean dynamics. Sonification can squeeze these large datasets temporally to reveal long-term trends that are hard to see when using other means of visualization. Further collaboration will be done with oceanographic scientists to determine how this auditory display can be useful and strengthened. The work done so far has only been guided by artistic wonderings. Several examples of ocean data sonification will be presented at the conference, each illustrating important aspects of ocean dynamics. The most obvious sonic event is the onset of storm energy, which sounds like gradual upward frequency sweeps. An entire year of data has been sonified over a ten minute duration for buoys in different regions. These demonstrate dramatic seasonal and regional differences. 1. INTRODUCTION Since 1975 the Coastal Data Information Program (CDIP)1, within the Scripps Institution of Oceanography (SIO)2, at the University of California, San Diego (UCSD)3, has measured, disseminated and archived coastal environment data for use by coastal engineers, planners, and managers as well as scientists, mariners, and surfers. CDIP owns and operates approximately twenty off-shore and near-shore buoys which record ocean conditions such as wave height and period, sea temperature, wind velocity and direction, and barometric pressure. Such multidimensionality invites aural display experimentation. The sonification mapping is quite direct. The frequencies in one domain are the scaled frequencies in the other. Amplitudes are determined by energies present in particular frequencies. The spatialization of the frequency is determined by the direction from which that component is originating. The sonification of several 1 http://cdip.ucsd.edu 2http://www.sio.ucsd.edu/ 3http://www.ucsd.edu spectral measurements is created by combining these signals windowed to unity. Loud low frequencies are long period swells, which create good surfing conditions. Loud high frequencies are short period waves, which create choppy water. During some intervals there are recognizable frequency sweeps from low to high. This signifies the effects of storm energy reaching the buoy. Since long period energy travels faster in water than short [1], the storm's effects should sound like a long frequency sweep from low to high. The decay of the storm's effects are just the opposite, a frequency sweep from high to low, but happen over a shorter timespan. 2. THE BUOYS AND DATA The buoy in figure 1 is a 0.9 meter diameter spherical steel hull that weighs approximately 180 kg, and is tethered to the ocean bottom with a mooring line and anchored with 450 kg of ballast chain.4 This buoy can measure accelerations in the three dimensions and temperature. Wave height is determined by measuring the vertical acceleration of the buoy. Wave direction is determined by the accelerations in the xy-plane. These buoys are typically in water approximately 500 m deep, but are used closer to shore in 20 meter depths. The directional buoys make Figure 1: A Directional Buoy 2,048 measurements over twentysix minutes and then take four minutes to transmit via radio the data to an onshore field station consisting of a computer and receiver. This field station is queried every thirty minutes by CDIP, which retrieves the data, processes it5, and then disseminates it to the National Weather Service and displays it on the CDIP homepage. Latencies are usually 30 minutes to an hour after data retrieval. The total size of the current database is over 50 gigabytes (GB) and spans twenty-six years. It increases each day by approximately 270

Page  00000271 045 OCEANSIDE BUOY NOVEMBER 2001 -0.12Hz 0.10Hz 0.08Hz 0.06Hz 0.04Hz 18s. 14 s 12s. 10s. 270 n? Figure 2: Spectral Energies for the Oceanside Buoy, November 2001 15-20 megabytes (Mb), or about 1 Mb of data for each station. 3. THE SONIFICATION Figure 2 shows spectral data for a single month of the Oceanside Buoy (OsB). It is a waterfall plot of spectral results from every sixth hour. For the month of November, 2001, the OsB made 1,416 measurements, about two per hour. For the year 2000, OsB made 17,540 measurements. This spectral plot makes it easy to see how this data lends itself to sonification. Figure 3 is a polar plot of the directional wave spectrum, for one spectral file. This plot depicts the distribution of spectral energy as a function of wave period and direction measured at the buoy. It shows two sources of energy: a significant one coming from the NW and a very small one coming from the SW. The energy is peaked near the frequency of 0.08 Hz coming from the NW direction of 310 degrees. The stereo field is centered at 270 degrees, facing due West. North is panned to the right and South to the left. Energy further from the center is higher pitched, and the darker the component the louder it is. A spectral file is produced from each 2,048 sample measurement. This file contains the mean energies and directions for each frequency bin, usually 64 or 128 bins, 0.025 - 0.58 Hz. Of these the most significant for CDIP's purposes are periods above 4 seconds and below 306, but all bins can be included in the sonification. The sonification of this wave spectral data is straight-forward. Each spectral file is used to synthesize a short signal by taking an Inverse Fourier Transform of the scaled data. This in a sense reconstructs the original wave-height time-series but without the phases. These windowed signals are then ordered to form a longer representation of the wave spectral data. The general formula for 4Buoys made by Datawell, http://www.datawell.nl 5The data processing and reporting standards, e.g. Fast Fourier Transform of the directional data, are outlined in [2]. 6There is not much energy of interest outside this region. Figure 3: Directional Wave Spectrum Centered on the Buoy sonifying a spectral file is: N S(n) = w(n) RA( k, Q)A(Ek, Dk)eiR(wk Q2,)kn/FS k=l (1) where n is the sample number, w(n) is a window function, N is the total number of frequency bins, RA (wk, Qi) is an amplitude scaling function-Qj being any set of extra parametersA(Ek, Dk) is the amplitude and spatialization function based on a bin's mean energy, Ek, and mean direction Dk; R, (wk, Q2) is a frequency scaling function, wk is the kth angular frequency, and Fs is the sampling frequency of the sonification. The process is basically granular synthesis with parameters derived from ocean spectral data [3, 4]. Each grain is synthesized with the spectral information of the buoy wave-driven motions, and these grains are then combined in the same order as the measurements to form a mosaic of localized ocean dynamics. Figure 4: The sin2 Windows Combined The first window function used was a rectangular one, which created discontinuities in the sonification at the edges. A smoother sin2 window or a rounded rectangular one gives less irritating results. Equation 2 is an example of a window function used to make the sound more fluid. 271

Page  00000272 wm, (n) = sin2 (Sf (m - 1)-) 2 (2) where m is the frame number, (m - 1) - < n < (m + 1) - and sy is the number of samples per frame. When these frames are combined they add to unity. An example with five windows is shown in figure 4, where each window number corresponds to a spectral observation of the buoy. The scaling functions, RA (Wk, Q1) and R, (wk, Q2), are arbitrarily assigned according mapping needs. Qj can be any extra set of parameters, such as time, temperature, barometric pressure, etc. The author has made RA a function derived from Fletcher-Munson equal loudness scaling so that no particular frequency is perceived louder than it should [5]. The frequency scaling function, R,, is the means to make the sub-sonic wavelengths of the ocean audible. Experimentation has been done with linear, quadratic, and higher order functions. The amplitude function, A(Ek, Dk), determines the volume of the spectral component based upon the amount of energy present within that bin, Ek. In creating a two-channel sonification, the direction of the frequency component, Dk, is used to determine the amplitude of the spectral component in each stereo field: 2700 is pan center, 360~ is pan right, and 180~ is pan left. The sonification mapping is simple and direct. Spectral frequency of the sound is determined by the scaled spectral frequency of the wave-driven motions observed by the directional buoy. The energy within that spectral component is mapped to the amplitude of the spectral component in the sound. The spatialization of that spectral component is determined by the direction from which the spectral component is observed originating. Finally, for each measurement made by the buoy, one sonic grain is synthesized with a smooth window function and placed into the continuum of other sonified measurements. 4. RESULTS The sonifications using these methods are immediately interesting and captivating. The sound is much like a large bell having its harmonics excited individually. It is as if a set of wind chimes are placed in the ocean and resonate when fundamental frequencies are present. Without knowing the science, the sounds are still significant. One co-worker suggested that if he knew the data sounded like that his job would be more interesting. Other auditors have remarked how calming it is, and wonder at the relationship between ocean rhythms and circadian rhythms.7 Figure 5 shows the amplitude silhouette of a 10-minute sonification of the entire year 2000, for Point Reyes Buoy (PRB). The buoy, off the coast of Northern San Francisco, CA, USA, made 17,028 measurements that year, usually about 2 measurements per hour. With a sonification sample rate of 44.1 kHz, each of the grains has 1,554 samples and lasts 40 ms. This creates a nice macroscopic perspective of ocean activity. What is most apparent in figure 5 are the differences between the summer and winter. There is much more activity, or energetic phenomena, during the winter than in any time of the year. This is clear while listening to the sound as well. The most prominent features of this sonification are frequency sweeps. An upward sweep signifies the presence of storm energy passing the buoy. Basic oceanography proves that in water long 7This music could become popular if other sounds were subtly added, such as fog horns, seagulls, and whale sounds. January February March Apnl May June July August September October November December o I 0. L. 0 time 600 Figure 5: Amplitude vs. Time for the Sonification of Point Reyes Buoy, Jan - Dec 2000, San Francisco, CA, USA period energy travels faster in water than short period [1]. In addition there are descending sweeps, which is the decay, or the trough, of the storm's effects. Comparing these two phenomena one hears the up-sweeps happening over a longer time span than the downward ones, which is confirmed by basic ocean science. 5. CONCLUSIONS The spectral ocean wave data from CDIP lends itself well to auditory display by its being a measurement of dynamic waves. Furthermore since the database is very large it provides a nice collection of data for sonification experimenting. Though visual means of inspecting data over short duration is efficient, looking at large durations becomes a mess whereby details become blurred. Sonification of these larger datasets reveal quite well the activity of the ocean and any trends present. Storms are readily recognized and seasonal activity is quite apparent in these long duration sonifications. Further work will be completed by the time of the conference so that a better understanding of the usefulness of these routines to oceanographers can be had. 6. ACKNOWLEDGMENTS Much appreciation for the help, input, and direction of Dr. Richard J. Seymour, founder and director of the Coastal Data Information Program at Scripps Institution of Oceanography. 7. REFERENCES [1] W. Bascom, Waves and Beaches: The Dynamics of the Ocean Surface, second edition, Anchor Doubleday, New York, 1980. [2] M. Earle, D. McGehee, M. Tubman, Field Wave Gaging Program, Wave Data Analysis Standard, U.S. Army Corps of Engineers Instruction Report, CERC-95-1, Mar. 1995. [3] D. Gabor, "Acoustical Quanta and the Theory of Hearing," in Nature, vol. 159, no. 4044, pp. 591-594, 1947. [4] I. Xenakis, Formalized Music, Thought and Mathematics in Music, revised edition, Pendragon Press, New York, 1992. [5] B. Moore, An Introduction to the Psychology of Hearing, third edition, Academic Press, San Diego, 2001. 272