Effect of Auditory Feedback on Tactile Intensity Perception in a Touchscreen Application

This article presents the effect of auditory feedback on tactile intensity perception, which may be of interest to haptic or audiotactile interaction engineers. An experimental setup consisted of a touchscreen, an electrodynamic shaker, and a closed-back headphone for a subject to interact with the touchscreen and to feel audiotactile feedback. In the experiment, participants were asked to judge perceived tactile intensity, using the magnitude estimation method, in the absence and presence of simultaneous auditory feedback. All data collected from the subjects were analyzed statistically, and then the effect of auditory feedback was investigated focusing on the following aspects: whether the presence of auditory feedback changes perceived tactile intensity, whether the frequency component of auditory feedback affects tactile intensity perception, and whether the coincidence of tactile and auditory frequencies influences on tactile intensity perception. Besides, changes in Stevens's exponent were analyzed to discuss how tactile intensity perception varies due to the auditory feedback. Finally, an equal intensity contour, in the domain of sensation level and frequency of tactile stimulation, was drawn. It can be applied to adjust the level of tactile stimuli for haptic feedback designers to provide a constant perceived tactile intensity considering the presence of auditory feedback.

[1]  Susan J. Lederman,et al.  Multisensory Texture Perception , 2010 .

[2]  Susan J. Lederman,et al.  Relative performance using haptic and/or touch-produced auditory cues in a remote absolute texture identification task , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[3]  Mark Mulder,et al.  The Effect of Haptic Support Systems on Driver Performance: A Literature Survey , 2015, IEEE Transactions on Haptics.

[4]  Stephen Brewster,et al.  Feeling Rough: Multimodal Perception of Virtual Roughness , 2001 .

[5]  Michael S. Beauchamp,et al.  Sound enhances touch perception , 2009, Experimental Brain Research.

[6]  Julia Seebode.  Emotional Feedback for Mobile Devices , 2015 .

[7]  J. Zwislocki,et al.  Absolute scaling of sensory magnitudes: A validation , 1980, Perception & psychophysics.

[8]  Hsin-Yun Yao,et al.  Perceived Vibration Strength in Mobile Devices: The Effect of Weight and Frequency , 2010, IEEE Transactions on Haptics.

[9]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[10]  S. Merchel,et al.  FEELING THE SOUND: AUDIO-TACTILE INTENSITY PERCEPTION , 2011 .

[11]  Gi-Hun Yang,et al.  Development of Vibrotactile Pedestal With Multiple Actuators and Application of Haptic Illusions for Information Delivery , 2019, IEEE Transactions on Industrial Informatics.

[12]  Sebastian Merchel,et al.  Electrotactile Feedback for Handheld Devices with Touch Screen and Simulation of Roughness , 2012, IEEE Transactions on Haptics.

[13]  Sebastian Merchel,et al.  Audiotactile Feedback Design for Touch Screens , 2009, HAID.

[14]  M. Ercan Altinsoy,et al.  The Effect of Auditory Cues on the Audiotactile Roughness Perception: Modulation Frequency and Sound Pressure Level , 2008, HAID.

[15]  J. F. Dammann,et al.  Temporal Frequency Channels Are Linked across Audition and Touch , 2009, Current Biology.

[16]  Topi Kaaresoja,et al.  Perception of short tactile pulses generated by a vibration motor in a mobile phone , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[17]  Seungmoon Choi,et al.  Vibrotactile Perceived Intensity for Mobile Devices as a Function of Direction, Amplitude, and Frequency , 2013, IEEE Transactions on Haptics.

[18]  M. Ercan Altinsoy,et al.  Auditory tactile interaction in virtual environments , 2006 .

[19]  J. Edward Colgate,et al.  The application of tactile, audible, and ultrasonic forces to human fingertips using broadband electroadhesion , 2017, 2017 IEEE World Haptics Conference (WHC).

[20]  M. Kuefer.  Psychophysics Introduction To Its Perceptual Neural And Social Prospects , 2016 .

[21]  M. Hollins,et al.  Perceived intensity of vibrotactile stimuli: the role of mechanoreceptive channels. , 1996, Somatosensory & motor research.

[22]  Bruce Banter,et al.  Touch Screens and Touch Surfaces are Enriched by Haptic Force‐Feedback , 2010 .

[23]  Thorsten Alexander Kern,et al.  Novel Thin Electromagnetic System for Creating Pushbutton Feedback in Automotive Applications , 2012, EuroHaptics.

[24]  Roland Werthschützky,et al.  Vibrotactile Force Perception – Absolute and Differential Thresholds and External Influences , 2016, IEEE Transactions on Haptics.

[25]  Sebastian Merchel,et al.  Tactile intensity perception compared to auditory loudness perception , 2015, 2015 IEEE World Haptics Conference (WHC).

[26]  Cristy Ho,et al.  Tactile and Multisensory Spatial Warning Signals for Drivers , 2008, IEEE Transactions on Haptics.

[27]  Cagatay Basdogan,et al.  Effect of Waveform on Tactile Perception by Electrovibration Displayed on Touch Screens , 2017, IEEE Transactions on Haptics.

[28]  Vincent Hayward,et al.  Vibrotactile Stimulation Can Affect Auditory Loudness: A Pilot Study , 2012, EuroHaptics.

[29]  Sebastian Merchel,et al.  Touch the Sound: Audio-Driven Tactile Feedback for Audio Mixing Applications , 2012 .

[30]  Dejan Todorović,et al.  Context effects in visual perception and their explanations , 2010 .

[31]  Louis D Braida,et al.  Integration of auditory and vibrotactile stimuli: effects of phase and stimulus-onset asynchrony. , 2009, The Journal of the Acoustical Society of America.

[32]  Seungmoon Choi,et al.  Vibrotactile Feedback for Information Delivery in the Vehicle , 2010, IEEE Transactions on Haptics.

[33]  Mandayam A. Srinivasan,et al.  The Effect of Auditory Cues on the Haptic Perception of Stiffness in Virtual Environments , 1997, Dynamic Systems and Control.

[34]  R. T. Verrillo,et al.  Vibration Sensation in Humans , 1992 .

[35]  Seungmoon Choi,et al.  Vibrotactile Display: Perception, Technology, and Applications , 2013, Proceedings of the IEEE.

[36]  Louis D Braida,et al.  Integration of auditory and vibrotactile stimuli: effects of frequency. , 2010, The Journal of the Acoustical Society of America.

[37]  J. Edward Colgate,et al.  The Application of Tactile, Audible, and Ultrasonic Forces to Human Fingertips Using Broadband Electroadhesion , 2018, IEEE Transactions on Haptics.

[38]  Shuichi Sakamoto,et al.  Selective effects of auditory stimuli on tactile roughness perception , 2008, Brain Research.

[39]  Allison M. Okamura,et al.  Reality-based models for vibration feedback in virtual environments , 2001 .

[40]  S. S. Stevens.  Tactile vibration: Change of exponent with frequency , 1968 .

[41]  Riitta Hari,et al.  Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. , 2004, The Journal of the Acoustical Society of America.

[42]  Gary Weismer,et al.  Direct magnitude estimates of speech intelligibility in dysarthria: effects of a chosen standard. , 2002, Journal of speech, language, and hearing research : JSLHR.

[43]  Ercan Tunca,et al.  Advantages of Active Haptics on Touch Surfaces , 2016, AutomotiveUI.

[44]  Uwe Koch,et al.  Info- and Entertainment of the BMW 7er Series , 2008 .

[45]  Jon Sprouse.  Magnitude Estimation and theNon-Linearity of Acceptability Judgments , 2008 .

[46]  Hakan Tüzün,et al.  Usability testing of a 3D touch screen kiosk system for way-finding , 2016, Comput. Hum. Behav..