Audio-Material Reconstruction for Virtualized Reality Using a Probabilistic Damping Model

Modal sound synthesis has been used to create realistic sounds from rigid-body objects, but requires accurate real-world material parameters. These material parameters can be estimated from recorded sounds of an impacted object, but external factors can interfere with accurate parameter estimation. We present a novel technique for estimating the damping parameters of materials from recorded impact sounds that probabilistically models these external factors. We represent the combined effects of material damping, support damping, and sampling inaccuracies with a probabilistic generative model, then use maximum likelihood estimation to fit a damping model to recorded data. This technique greatly reduces the human effort needed and does not require the precise object geometry or the exact hit location. We validate the effectiveness of this technique with a comprehensive analysis of a synthetic dataset and a perceptual study on object identification. We also present a study establishing human performance on the same parameter estimation task for comparison.

[1]  T. Shimogo Vibration Damping , 1994, Active and Passive Vibration Damping.

[2]  M. Grassi Do we hear size or sound? Balls dropped on plates , 2005, Perception & psychophysics.

[3]  Dinesh K. Pai,et al.  Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources , 2006, SIGGRAPH 2006.

[4]  Jim Woodhouse,et al.  LINEAR DAMPING MODELS FOR STRUCTURAL VIBRATION , 1998 .

[5]  Xavier Serra,et al.  A sound analysis/synthesis system based on a deterministic plus stochastic decomposition , 1990 .

[6]  T. Caughey,et al.  Classical Normal Modes in Damped Linear Dynamic Systems , 1960 .

[7]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ming C. Lin,et al.  Example-guided physically based modal sound synthesis , 2013, ACM Trans. Graph..

[9]  E. Grushka Characterization of exponentially modified Gaussian peaks in chromatography. , 1972, Analytical chemistry.

[10]  Dinesh K. Pai,et al.  The Sounds of Physical Shapes , 1998, Presence.

[11]  Wojciech Matusik,et al.  Computational design of metallophone contact sounds , 2015, ACM Trans. Graph..

[12]  Sondipon Adhikari,et al.  Damping modelling using generalized proportional damping , 2006 .

[13]  Doug L. James,et al.  Toward wave-based sound synthesis for computer animation , 2018, ACM Trans. Graph..

[14]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[15]  Doug L. James,et al.  Eigenmode compression for modal sound models , 2014, ACM Trans. Graph..

[16]  Kevin Karplus,et al.  Digital Synthesis of Plucked-String and Drum Timbers , 1983 .

[17]  Eero P. Simoncelli,et al.  Summary statistics in auditory perception , 2013, Nature Neuroscience.

[18]  Davide Rocchesso,et al.  Integration of acoustical information in the perception of impacted sound sources: the role of information accuracy and exploitability. , 2010, Journal of experimental psychology. Human perception and performance.

[19]  D. J. Ewins,et al.  Complex Modes - Origins and Limits , 1995 .

[20]  Perry R. Cook,et al.  Rtmidi, Rtaudio, and a synthesis Toolkit (STK) Update , 2005, ICMC.

[21]  Doug L. James,et al.  Harmonic shells: a practical nonlinear sound model for near-rigid thin shells , 2009, ACM Trans. Graph..

[22]  Julius O. Smith,et al.  Physical Modeling Using Digital Waveguides , 1992 .

[23]  Perry R. Cook,et al.  The Synthesis ToolKit (STK) , 1999, ICMC.

[24]  Ming C. Lin,et al.  Auditory Perception of Geometry-Invariant Material Properties , 2013, IEEE Transactions on Visualization and Computer Graphics.

[25]  George Drettakis,et al.  Fast modal sounds with scalable frequency-domain synthesis , 2008, ACM Trans. Graph..

[26]  Stephen McAdams,et al.  The psychomechanics of simulated sound sources: material properties of impacted thin plates. , 2010, The Journal of the Acoustical Society of America.

[27]  John William Strutt Rayleigh,et al.  The theory of sound. Vol. 2 , 2015 .

[28]  S. Adhikari,et al.  Identification of damping: Part 1, viscous damping , 2001 .

[29]  S. McAdams,et al.  The psychomechanics of simulated sound sources: material properties of impacted bars. , 2004, The Journal of the Acoustical Society of America.

[30]  Maarten van Walstijn,et al.  An Explorative String-Bridge-Plate Model With Tunable Parameters , 2017 .

[31]  Bin Wang,et al.  Deformation capture and modeling of soft objects , 2015, ACM Trans. Graph..

[32]  Bruno L. Giordano,et al.  Material identification of real impact sounds: effects of size variation in steel, glass, wood, and plexiglass plates. , 2006, The Journal of the Acoustical Society of America.

[33]  Claudio Perez Tamargo Can one hear the shape of a drum , 2008 .

[34]  Doug L. James,et al.  Precomputed acceleration noise for improved rigid-body sound , 2012, ACM Trans. Graph..

[35]  Dingzeyu Li,et al.  Interactive Acoustic Transfer Approximation for Modal Sound , 2015, ACM Trans. Graph..

[36]  Ming C. Lin,et al.  Interactive sound synthesis for large scale environments , 2006, I3D '06.

[37]  Erwin Coumans,et al.  Bullet physics simulation , 2015, SIGGRAPH Courses.

[38]  Ming C. Lin,et al.  Interactive modal sound synthesis using generalized proportional damping , 2016, I3D.

[39]  Thomas L. Szabo,et al.  Time domain wave equations for lossy media obeying a frequency power law , 1994 .

[40]  Daniel J. Inman,et al.  Survey of modern methods for modeling frequency dependent damping in finite element models , 1993 .

[41]  Takeo Igarashi,et al.  Interactive physically-based sound design of 3D model using material optimization , 2016, Symposium on Computer Animation.

[42]  Doug L. James,et al.  Toward high-quality modal contact sound , 2011, SIGGRAPH 2011.

[43]  Paul Graham,et al.  Large Scale Physical Modeling Sound Synthesis , 2013 .

[44]  Naga K. Govindaraju,et al.  Sound synthesis for impact sounds in video games , 2011, SI3D.

[45]  Shin'ya Nishida,et al.  Audiovisual integration in the human perception of materials. , 2014, Journal of vision.

[46]  Chen Shen,et al.  Synthesizing sounds from rigid-body simulations , 2002, SCA '02.

[47]  Ming C. Lin,et al.  MaterialCloning: Acquiring Elasticity Parameters from Images for Medical Applications , 2016, IEEE Transactions on Visualization and Computer Graphics.

[48]  Josh H McDermott,et al.  Statistics of natural reverberation enable perceptual separation of sound and space , 2016, Proceedings of the National Academy of Sciences.

[49]  Davide Rocchesso,et al.  MODELING COLLISION SOUNDS: NON-LINEAR CONTACT FORCE , 2001 .

[50]  Julius O. Smith,et al.  Spectral modeling synthesis: A sound analysis/synthesis based on a deterministic plus stochastic decomposition , 1990 .