Immersive sonification of protein surface

To extend pre-existing protein visualisation methods, we present a new approach of immersive sonification to represent protein surfaces through sound. The protein surface is first discretized so each point of the surface is attached to a sound source spatialized in such a way the user is immersed in the center of the protein. We add a spherical filtering system, that the user can control, to select the surface points that would be rendered in order to reinforce the auditory interpretation of the 3D shape. Several questions, which can benefit the HCI community, are discussed both on audio and audiographical filtering consistency, and on multimodal integration of data coming from different point of view and point of listening.

[1]  Gokhan Ince,et al.  Sonification of 3D Object Shape for Sensory Substitution: An Empirical Exploration , 2017, ACHI 2017.

[2]  Florian Grond,et al.  SONIFICATION OF 3 D POINT CLOUDS FOR SUBSTITUTION OF VISION BY AUDITION FOR BLIND USERS , 2018 .

[3]  Robert Höldrich,et al.  3D binaural sound reproduction using a virtual ambisonic approach , 2003, IEEE International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2003. VECIMS '03. 2003.

[4]  Davide Rocchesso,et al.  The Sonification Handbook , 2011 .

[5]  D. Goodsell,et al.  Visualization of macromolecular structures , 2010, Nature Methods.

[6]  Thomas Ertl,et al.  Enhancing visualization of molecular simulations using sonification , 2015, 2015 IEEE 1st International Workshop on Virtual and Augmented Reality for Molecular Science (VARMS@IEEEVR).

[7]  Jörn Loviscach,et al.  SoundTorch: Quick Browsing in Large Audio Collections , 2008 .

[8]  Matthieu Chavent,et al.  Advances in Human-Protein Interaction - Interactive and Immersive Molecular Simulations , 2012 .

[9]  Pierre Guillot,et al.  The HOA library, review and prospects , 2014, ICMC.

[10]  Paul Vickers,et al.  Interactive Sonification for Structural Biology and Structure-based Drug Design , 2016 .

[11]  Robert P. Bywater,et al.  Melody discrimination and protein fold classification , 2016, Heliyon.

[12]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[13]  Florian Grand,et al.  Sumo. A Sonification Utility for Molecules , 2008 .

[14]  Daphne Maurer,et al.  Do small white balls squeak? Pitch-object correspondences in young children , 2004, Cognitive, affective & behavioral neuroscience.

[15]  Huseyin Seker,et al.  The sound of proteins , 2012, Proceedings of 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics.

[16]  Grigori Evreinov,et al.  An Exploration of Volumetric Data in Auditory Space , 2014 .

[17]  Michael Pettitt,et al.  Curve shape and curvature perception through interactive sonification , 2012, TAP.