Sound synthesis and evaluation of interactive footsteps for virtual reality applications

A system to synthesize in real-time the sound of footsteps on different materials is presented. The system is based on microphones which allow the user to interact with his own footwear. This solution distinguishes our system from previous efforts that require specific shoes enhanced with sensors. The microphones detect real footsteps sounds from users, from which the ground reaction force (GRF) is estimated. Such GRF is used to control a sound synthesis engine based on physical models. Evaluations of the system in terms of sound validity and fidelity of interaction are described.

[1]  R. Pastore,et al.  Auditory event perception: The source—perception loop for posture in human gait , 2008, Perception & psychophysics.

[2]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[3]  Richard D. Sanders,et al.  The effect of sound delivery methods on a user's sense of presence in a virtual environment , 2002 .

[4]  A.W. Law,et al.  A multi-modal floor-space for experiencing material deformation underfoot in virtual reality , 2008, 2008 IEEE International Workshop on Haptic Audio visual Environments and Games.

[5]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[6]  Perry R. Cook,et al.  Real Sound Synthesis for Interactive Applications , 2002 .

[7]  Bill Kapralos,et al.  Auditory Cues in the Perception of Self Motion , 2004 .

[8]  Federico Fontana,et al.  Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds , 2003 .

[9]  Michael Zyda,et al.  Interactions in Perceived Quality of Auditory-Visual Displays , 2000, Presence: Teleoperators & Virtual Environments.

[10]  Joseph A. Paradiso,et al.  Interactive Music for Instrumented Dancing Shoes , 1999, ICMC.

[11]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[12]  Joseph A. Paradiso,et al.  A wireless modular sensor architecture and its application in on-shoe gait analysis , 2003, Proceedings of IEEE Sensors 2003 (IEEE Cat. No.03CH37498).

[13]  Philip H. Marsden,et al.  Designing Auditory Spaces to Support Sense of Place: The Role of Expectation , 2002 .

[14]  Daniel Västfjäll,et al.  Travelling without moving: Auditory scene cues for translational self-motion , 2005 .

[15]  Perry R. Cook,et al.  Physically Informed Sonic Modeling (PhISM): Synthesis of percussive sounds , 1997 .

[16]  Andy Farnell.  Marching onwards Procedural synthetic footsteps for video games and animation , 2007 .

[17]  Vincent Hayward,et al.  Single state elastoplastic friction models , 2002, IEEE Trans. Autom. Control..

[18]  Ming C. Lin,et al.  Interactive sound synthesis for large scale environments , 2006, I3D '06.

[19]  Perry R. Cook,et al.  Modeling Bill's Gait: Analysis and Parametric Synthesis of Walking Sounds , 2002 .

[20]  Rolf Nordahl.  Increasing the Motion of Users in Photo-realistic Virtual Environments by Utilising Auditory Rendering of the Environment and Ego-motion , 2006 .

[21]  Daniel Västfjäll,et al.  The Subjective Sense of Presence, Emotion Recognition, and Experienced Emotions in Auditory Virtual Environments , 2003, Cyberpsychology Behav. Soc. Netw..

[22]  Jean-Marie Adrien,et al.  The missing link: modal synthesis , 1991 .

[23]  Bruno L. Giordano,et al.  Material identification of real impact sounds: effects of size variation in steel, glass, wood, and plexiglass plates. , 2006, The Journal of the Acoustical Society of America.

[24]  Bruno L. Giordano,et al.  Non‐visual identification of walking grounds , 2008 .

[25]  Robert J. Logan,et al.  Perception of acoustic source characteristics: walking sounds. , 1991, The Journal of the Acoustical Society of America.

[26]  Daniel Västfjäll,et al.  Perception of Self-motion and Presence in Auditory Virtual Environments , 2004 .

[27]  Davide Rocchesso,et al.  MODELING COLLISION SOUNDS: NON-LINEAR CONTACT FORCE , 2001 .