Influence of auditory and visual feedback for perceiving walking over bumps and holes in desktop VR

In this paper, we present an experiment whose goal is to investigate the role of sound and vision in the recognition of different surface profiles in a walking scenario. Fifteen subjects participated to two within-subjects experiments where they were asked to interact with a desktop system simulating bumps, holes and flat surfaces by means of audio, visual and audio-visual cues. Results of the first experiment show that participants are able to successfully identify the surface profiles provided through the proposed audio-visual techniques. Results of a second experiment in which conflictual audiovisual stimuli were presented, reveal that for some of the proposed visual effects the visual feedback is dominant on the auditory one, while for the others the role of dominance is inverted.

[1]  Luca Turchet,et al.  Physically Based Sound Synthesis and Control of Footsteps Sounds , 2010 .

[2]  Andy Farnell.  Marching onwards Procedural synthetic footsteps for video games and animation , 2007 .

[3]  Hiroo Iwata,et al.  Gait Master: a versatile locomotion interface for uneven virtual terrain , 2001, Proceedings IEEE Virtual Reality 2001.

[4]  Laurent Étienne,et al.  Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures , 2004, CHI.

[5]  B. Stein,et al.  The Merging of the Senses , 1993 .

[6]  Luca Turchet,et al.  Do You Hear a Bump or a Hole?: An Experiment on Temporal Aspects in Footsteps Recognition , 2010 .

[7]  Haruo Noma,et al.  Simulating side slopes on locomotion interfaces using torso forces , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[8]  Perry R. Cook,et al.  Modeling Bill's Gait: Analysis and Parametric Synthesis of Walking Sounds , 2002 .

[9]  Thomas P. Caudell,et al.  Using wavelets to synthesize stochastic-based sounds for immersive virtual environments , 2005, TAP.

[10]  Luca Turchet,et al.  Sound synthesis and evaluation of interactive footsteps for virtual reality applications , 2010, 2010 IEEE Virtual Reality Conference (VR).

[11]  Federico Fontana,et al.  Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds , 2003 .

[12]  Maud Marchal,et al.  Walking up and down in immersive virtual worlds: Novel interactive techniques based on visual feedback , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).