Three Orthogonal Dimensions for Psychoacoustic Sonification

Objective: Three perceptually orthogonal auditory dimensions for multidimensional and multivariate data sonification are identified and experimentally validated. Background: Psychoacoustic investigations have shown that orthogonal acoustical parameters may interfere perceptually. The literature hardly offers any solutions to this problem, and previous auditory display approaches have failed to implement auditory dimensions that are perceived orthogonally by a user. In this study we demonstrate how a location in three-dimensional space can be sonified unambiguously by the implementation of perceptually orthogonal psychoacoustic attributes in monophonic playback. Method: Perceptually orthogonal auditory attributes are identified from literature research and experience in music and psychoacoustic research. We carried out an experiment with 21 participants who identified sonified locations in two-dimensional space. Results: With just 5 minutes of explanation and exploration, naive users can interpret our multidimensional sonification with high accuracy. Conclusion: We identified a set of perceptually orthogonal auditory dimensions suitable for three-dimensional data sonification. Application: Three-dimensional data sonification promises blind navigation, e.g. for unmanned vehicles, and reliable real-time monitoring of multivariate data, e.g., in the patient care sector.

[1]  Jürgen Meyer,et al.  ACOUSTICS AND THE PERFORMANCE OF MUSIC , 2024, Spring Conference 1981.

[2]  Tim Ziemer.  Source Width in Music Production. Methods in Stereo, Ambisonics, and Wave Field Synthesis , 2017 .

[3]  Tim Ziemer,et al.  A Psychoacoustic Sound Design for Pulse Oximetry , 2019, Proceedings of the 25th International Conference on Auditory Display (ICAD 2019).

[4]  David Black,et al.  Psychoacoustic sonification for tracked medical instrument guidance , 2017 .

[5]  Gerd Schmitz,et al.  Dual Mode Gait Sonification for Rehabilitation After Unilateral Hip Arthroplasty , 2019, Brain sciences.

[6]  Penelope M. Sanderson,et al.  Designing Sonification for Effective Attentional Control in Complex Work Domains , 2004 .

[7]  H. J. Noordmans,et al.  The impact of auditory feedback on neuronavigation , 2005, Acta Neurochirurgica.

[8]  Albrecht Schneider,et al.  Perception of Timbre and Sound Color , 2018 .

[9]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..

[10]  Thomas Hermann,et al.  Laboratory Methods for Experimental Sonification , 2011 .

[11]  Ron Kikinis,et al.  Auditory feedback to support image-guided medical needle placement , 2017, International Journal of Computer Assisted Radiology and Surgery.

[12]  Norberto Degara,et al.  SONEX: An Evaluation Exchange Framework for Reproducible Sonification , 2013 .

[13]  Francesca Pedrielli,et al.  Just noticeable differences of loudness and sharpness for earth moving machines , 2008 .

[14]  Penelope Sanderson,et al.  Sonification design for complex work domains: dimensions and distractors. , 2009, Journal of experimental psychology. Applied.

[15]  Stephen Barrass,et al.  Auditory information design , 1998 .

[16]  Sara Ann Bly.  Sound and computer information presentation , 1982 .

[17]  Bruce N. Walker,et al.  Effect of Beacon Sounds on Navigation Performance in a Virtual Reality Environment , 2003 .

[18]  Holger Schultheis,et al.  Psychoacoustical Interactive Sonification for Short Range Navigation , 2018, Acta Acustica united with Acustica.

[19]  Densil Cabrera,et al.  Using psychoacoustical models for information sonification , 2006 .

[20]  F. Müller,et al.  Movement Sonification in Stroke Rehabilitation , 2018, Front. Neurol..

[21]  Zoran Vukic,et al.  Teleoperated path following and trajectory tracking of unmanned vehicles using spatial auditory guidance system , 2018 .

[22]  Monique Noirhomme-Fraiture,et al.  Complementing Visual Data Mining with the Sound Dimension: Sonification of Time Dependent Data , 2008, Visual Data Mining.

[23]  David Black,et al.  Psychoacoustic sonification design for navigation in surgical interventions , 2017 .

[24]  Rob Gray,et al.  Looming Auditory Collision Warnings for Driving , 2011, Hum. Factors.

[25]  Florian Grond,et al.  Listening-Mode-Centered Sonification Design for Data Exploration , 2013 .

[26]  Stephen A. Brewster,et al.  Evaluation of psychoacoustic sound parameters for sonification , 2017, ICMI.

[27]  Richard Kronland-Martinet,et al.  Comparison and Evaluation of Sonification Strategies for Guidance Tasks , 2016, IEEE Transactions on Multimedia.

[28]  Stephen Barrass,et al.  A perceptual framework for the auditory display of scientific data , 2005, TAP.

[29]  R. Shepard.  Circularity in Judgments of Relative Pitch , 1964 .

[30]  Jiping He,et al.  Journal of Neuroengineering and Rehabilitation Open Access Recent Developments in Biofeedback for Neuromotor Rehabilitation Review of Early Biofeedback Therapy Current Developments in Biofeedback in Neurorehabilitation Table 1: Function of Basic Modules in Multisensing Biofeedback Systems for Task T , 2022 .

[31]  B. Cardozo,et al.  ESTIMATES OF ANNOYANCE OF SOUNDS OF DIFFERENT CHARACTER , 1981 .

[32]  Holger Schultheis,et al.  Psychoacoustic auditory display for navigation: an auditory assistance system for spatial orientation tasks , 2018, Journal on Multimodal User Interfaces.

[33]  Thomas Hermann,et al.  Sonification for Exploratory Data Analysis , 2002 .

[34]  Stephen Barrass,et al.  Using sonification , 1999, Multimedia Systems.

[35]  Ernst Terhardt.  Aspekte und Möglichkeiten der gehörbezogenen Schallanalyse und -bewertung , 1981 .

[36]  Holger Schultheis,et al.  A Psychoacoustic Auditory Display for Navigation , 2018, Proceedings of the 24th International Conference on Auditory Display - ICAD 2018.

[37]  Rebecca Stewart,et al.  Spatial auditory display for acoustics and music collections , 2010 .

[38]  Gregory Kramer,et al.  Pitch and loudness interact in auditory displays: can the data get lost in the map? , 2002, Journal of experimental psychology. Applied.

[39]  Edward S. Yeung,et al.  Pattern recognition by audio representation of multivariate analytical data , 1980 .

[40]  Liming Wu,et al.  Sonification as a possible stroke rehabilitation strategy , 2014, Front. Neurosci..

[41]  J Edworthy,et al.  Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency , 1991, Human factors.

[42]  J Edworthy,et al.  Improving Auditory Warning Design: Quantifying and Predicting the Effects of Different Warning Parameters on Perceived Urgency , 1993, Human factors.

[43]  Stephan Riek,et al.  Concurrent 3-D Sonifications Enable the Head-Up Monitoring of Two Interrelated Aircraft Navigation Instruments , 2014, Hum. Factors.

[44]  Marcus Watson,et al.  Sonification Supports Eyes-Free Respiratory Monitoring and Task Time-Sharing , 2004, Hum. Factors.

[45]  Norberto Degara,et al.  Fast and Accurate Guidance - Response Times to Navigational Sounds , 2014 .

[46]  Penelope M Sanderson,et al.  Advanced patient monitoring displays: tools for continuous informing. , 2005, Anesthesia and analgesia.

[47]  Nikola Miskovic,et al.  Teleoperated trajectory tracking of remotely operated vehicles using spatial auditory interface , 2016 .

[48]  Yi Yu,et al.  Using Psychoacoustic Models for Sound Analysis in Music , 2016, FIRE.

[49]  Jiajun Yang,et al.  Reproducible sonification for virtual navigation , 2014, 2014 IEEE VR Workshop: Sonic Interaction in Virtual Environments (SIVE).

[50]  W. Aures,et al.  Ein Berechnungsverfahren der Rauhigkeit , 1985 .

[51]  Wolfgang Ellermeier,et al.  Predicting annoyance judgments from psychoacoustic metrics: Identificable versus neutralized sounds , 2004 .

[52]  Etienne Parizet,et al.  Investigation on localisation accuracy for first and higher order ambisonics reproduced sound sources , 2013 .

[53]  Perry R. Cook,et al.  An auditory display system for aiding interjoint coordination , 2000 .

[54]  Penelope M. Sanderson,et al.  Intelligibility of Sonifications for Respiratory Monitoring in Anesthesia , 2001 .

[55]  James P. Bliss,et al.  SONIFICATION AND RELIABILITY - IMPLICATIONS FOR SIGNAL DESIGN , 2007 .

[56]  W. H. Lichte.  Attributes of complex tones. , 1941 .

[57]  W. R. Garner.  The Processing of Information and Structure , 1974 .

[58]  Stephen Barrass,et al.  Stream-Based Sonification Diagrams , 2008 .

[59]  Francis Rumsey,et al.  Spatial Audio: Binaural Challenges , 2014 .

[60]  Tapio Lokki,et al.  Navigation with auditory cues in a virtual environment , 2005, IEEE MultiMedia.

[61]  Luis De Florez,et al.  True Blind Flight , 1936 .

[62]  Joe Wolfe,et al.  Does timbral brightness scale with frequency and spectral centroid , 2006 .

[63]  Bruce N. Walker,et al.  Navigation Performance With a Virtual Auditory Display: Effects of Beacon Sound, Capture Radius, and Practice , 2006, Hum. Factors.

[64]  J W Grau,et al.  The distinction between integral and separable dimensions: evidence for the integrality of pitch and loudness. , 1988, Journal of experimental psychology. General.