Interactive sonification of complex data

In this paper we present two experiments on implementing interaction in sonification displays: the first focuses on recorded data (interactive navigation) and the second on data gathered in real time (auditory feedback). Complex synthesised data are explored in the first experiment to evaluate how well the known characteristics present in the data are distinguished using different interaction methods, while real medical data (from physiotherapy) are used for the second. The addition of interaction to the exploration of sonified recorded data improves the system usability (efficiency, effectiveness and user satisfaction), and the real-time sonification of complex physiotherapy data can produce sounds with timbral characteristics that audibly change when important characteristics present in the data vary.

[1]  Jonathan Berger,et al.  Sonimime: Sonification of fine motor skills , 2005 .

[2]  Sigurd Saue,et al.  A model for interaction in exploratory sonification displays , 2000 .

[3]  Fredrik Winberg,et al.  QUALITATIVE ASPECTS OF AUDITORY DIRECT MANIPULATION A CASE STUDY OF THE TOWERS OF HANOI , 2001 .

[4]  Hugo Fastl,et al.  Psychoacoustics: Facts and Models , 1990 .

[5]  Thomas Hermann,et al.  A MALLEABLE DEVICE WITH APPLICATIONS TO SONIFICATION-BASED DATA EXPLORATION , 2006 .

[6]  Thilo Hinterberger,et al.  Parametric orchestral sonification of EEG in real time , 2005, IEEE MultiMedia.

[7]  Stephen Barrass,et al.  Responsive Sonification of Well-logs , 2000 .

[8]  Helge Ritter,et al.  Supplementary Material for "Gesture Desk -- An Integrated Multi-Modal Interface for Interactive Sonification" , 2003 .

[9]  Alan B. Craig,et al.  Using sound to extract meaning from complex data , 1991, Electronic Imaging.

[10]  Karon E. MacLean,et al.  D'GROOVE: A HAPTIC TURNTABLE FOR DIGITAL AUDIO CONTROL , 2003 .

[11]  Gregory Kramer,et al.  An introduction to auditory display , 1994 .

[12]  Perry R. Cook,et al.  An auditory display system for aiding interjoint coordination , 2000 .

[13]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[14]  Emil Jovanov,et al.  Acoustic rendering as support for sustained attention during biomedical procedures , 1998 .

[15]  Andy Hunt,et al.  A Toolkit for Interactive Sonification , 2004, ICAD.

[16]  Mikael Fernström,et al.  After direct manipulation---direct sonification , 1998, TAP.

[17]  Thomas Hermann,et al.  Sonification for Exploratory Data Analysis , 2002 .

[18]  Alfred Effenberg,et al.  Movement sonification: Effects on perception and action , 2005, IEEE MultiMedia.

[19]  Helge Ritter,et al.  TANGIBLE DATA SCANNING SONIFICATION MODEL , 2006 .

[20]  William M. Hartmann,et al.  Psychoacoustics: Facts and Models , 2001 .

[21]  J Dixon.  An electromyographic analysis of quadriceps femoris in patients with osteoarthritis of the knee. , 2004 .

[22]  Bruce N. Walker,et al.  Ecological Psychoacoustics and Auditory Displays: Hearing, Grouping, and Meaning Making , 2004 .

[23]  Andy Hunt,et al.  Guest Editors' Introduction: An Introduction to Interactive Sonification , 2005, IEEE Multim..

[24]  Thomas Hermann,et al.  An introduction to interactive sonification , 2005 .

[25]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .