TANGIBLE COMPUTING FOR INTERACTIVE SONIFICATION OF MULTIVARIATE DATA

We present a novel tangible computing system for interactively controlling real-time and offline data sonifications. Tangible objects serve as physical representation for data series such as EEG channel time series, and their arrangement on our Tangible Desk (tDesk) surface is used to interactively explore features of interest in the real-time rendered sonifications. A listener object and its distances to the data objects are used to select how salient the channels’ sonic representation is in the overall sonification. Selector objects allow intuitive specification of data sets. This interface enables the user (a) to identify groups of correlated rhythmical behavior, (b) to control multiple channels simultaneously, and (c) to explore the data collaboratively in a team. We give a full account on hard-/software of the system and we demonstrate the system at hand of offline- and real-time sonification of EEG and stock market data.