Audio Texture Synthesis for Complex Contact Interactions

This paper presents a new synthesis approach for generating contact sounds for interactive simulations. To address complex contact sounds, surface texturing is introduced. Visual textures of objects in the environment are reused as a discontinuity map to create audible position-dependent variations during continuous contacts. The resulting synthetic profiles are then used in real time to provide an excitation force to a modal resonance model of the sounding objects. Compared to previous sound synthesis for virtual environments, our approach has three major advantages: (1) complex contact interactions are addressed and a large variety of sounding events can be rendered, (2) it is fast due to the compact form of the solution which allows for synthesizing at interactive rates, (3) it provides several levels of detail which can be used depending on the desired precision.

[1]  Perry R. Cook,et al.  Real Sound Synthesis for Interactive Applications , 2002 .

[2]  Dinesh K. Pai,et al.  Modal Synthesis for Vibrating Objects , 2007 .

[3]  Y. Chien,et al.  Pattern classification and scene analysis , 1974 .

[4]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Ken Perlin,et al.  An image synthesizer , 1988 .

[6]  Chen Shen,et al.  Synthesizing sounds from rigid-body simulations , 2002, SCA '02.

[7]  R A Young,et al.  The Gaussian derivative model for spatial vision: I. Retinal mechanisms. , 1988, Spatial vision.

[8]  Dinesh K. Pai,et al.  Scanning physical interaction behavior of 3D objects , 2001, SIGGRAPH.

[9]  Davide Rocchesso,et al.  MODELING INTERACTIONS BETWEEN RUBBED DRY SURFACES USING AN ELASTO-PLASTIC FRICTION MODEL , 2002 .

[10]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[11]  Ming C. Lin,et al.  Haptic display of interaction between textured models , 2004, IEEE Visualization 2004.

[12]  Tapio Takala,et al.  Sound rendering , 1992, SIGGRAPH.

[13]  Dinesh K. Pai,et al.  Synthesis of shape dependent sounds with physical modeling , 1996 .

[14]  Dinesh K. Pai,et al.  Interactive Simulation of Complex Audiovisual Scenes , 2004, Presence: Teleoperators & Virtual Environments.

[15]  Davide Rocchesso,et al.  Physically-based audio rendering of contact , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[16]  Ming C. Lin,et al.  Interactive sound synthesis for large scale environments , 2006, I3D '06.

[17]  George Drettakis,et al.  Fast modal sounds with scalable frequency-domain synthesis , 2008, ACM Trans. Graph..

[18]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).