Haptic and sound correlations: pitch, loudness and texture

People process information through multiple modalities and many studies have been done that show interaction among the auditory, haptic and visual modes. This paper reports on a correlation between the haptic property of roughness and each of the audio properties of pitch and loudness. There were four experiments: given a roughness, match a loudness level to it and vice versa and given a roughness, match a pitch to it and vice versa. Subjects showed a strong correlation in the first two experiments, associating louder sounds with rougher textures and softer sounds with smoother textures. They also showed, although not always in the same direction, a strong correlation between pitch (high/low) and roughness. Applications of this study are to multimodal user interfaces and to music cognition.

[1]  Chris Chafe,et al.  Haptic interactions for audio navigation , 2004 .

[2]  Susan J. Lederman,et al.  Relative performance using haptic and/or touch-produced auditory cues in a remote absolute texture identification task , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[3]  Özgür Izmirli,et al.  Modeling the Tempo Coupling between an Ensemble and the Conductor , 2001, ICMC.

[4]  Susan J. Lederman,et al.  Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[5]  M. Cutkosky,et al.  Roughness Perception of Haptically Displayed Fractal Surfaces , 2000, Dynamic Systems and Control: Volume 2.

[6]  Dinesh K. Pai,et al.  Contact Interaction with Integrated Audio and Haptics , 2000 .

[7]  Stephen A. Brewster,et al.  The Effective Combination of Haptic and Auditory Textural Information , 2000, Haptic Human-Computer Interaction.

[8]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[9]  Janet M. Weisenberger,et al.  Multisensory roughness perception of virtual surfaces: effects of correlated cues , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[10]  Stephen Brewster,et al.  Feeling Rough: Multimodal Perception of Virtual Roughness , 2001 .

[11]  Thomas Kerwin,et al.  The role of multisensory feedback in haptic surface perception , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[12]  Stephen A. Brewster,et al.  Evaluation of multimodal graphs for blind people , 2003, Universal Access in the Information Society.

[13]  S. Lederman.  Tactile roughness of grooved surfaces: The touching process and effects of macro- and microsurface structure , 1974 .

[14]  Stephen A. Brewster,et al.  Haptic Graphs for Blind Computer Users , 2000, Haptic Human-Computer Interaction.

[15]  Christine L. MacKenzie,et al.  Auditory, graphical and haptic contact cues for a reach, grasp, and place task in an augmented environment , 2003, ICMI '03.

[16]  Kay M. Stanney,et al.  Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations , 2004, IEEE Computer Graphics and Applications.

[17]  Durand R. Begault,et al.  Sensitivity to haptic-audio asynchrony , 2003, ICMI '03.