Example‐based synthesis for sound of ocean waves caused by bubble dynamics

We present an automatic approach for the semantic modeling of indoor scenes based on a single photograph, instead of relying on depth sensors. Without using handcrafted features, we guide indoor scene modeling with feature maps extracted by fully convolutional networks. Three parallel fully convolutional networks are adopted to generate object instance masks, a depth map, and an edge map of the room layout. Based on these high‐level features, support relationships between indoor objects can be efficiently inferred in a data‐driven manner. Constrained by the support context, a global‐to‐local model matching strategy is followed to retrieve the whole indoor scene. We demonstrate that the proposed method can efficiently retrieve indoor objects including situations where the objects are badly occluded. This approach enables efficient semantic‐based scene editing.

[1]  Diemo Schwarz,et al.  DESCRIPTOR-BASED SOUND TEXTURE SAMPLING , 2010 .

[2]  Matthias Teschner,et al.  SPH Fluids in Computer Graphics , 2014, Eurographics.

[3]  Luca Turchet,et al.  Sound Synthesis and Evaluation of Interactive Footsteps and Environmental Sounds Rendering for Virtual Reality Applications , 2011, IEEE Transactions on Visualization and Computer Graphics.

[4]  Dani Lischinski,et al.  Synthesizing Sound Textures through Wavelet Tree Learning , 2002, IEEE Computer Graphics and Applications.

[5]  Jim R. Parker,et al.  Creating audio textures by example: tiling and stitching , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  H. Pumphrey,et al.  Underwater sound produced by individual drop impacts and rainfall , 1989 .

[7]  Steven L. Means,et al.  Low-frequency sound generation from breaking surf , 2004 .

[8]  Qiang Yin,et al.  Sounding Solid Combustibles: Non-Premixed Flame Sound Synthesis for Different Solid Combustibles , 2018, IEEE Transactions on Visualization and Computer Graphics.

[9]  Zygmunt Klusek,et al.  Acoustic noise generation under plunging breaking waves , 2013 .

[10]  Preston S. Wilson,et al.  On the resonance frequency of an ideal arbitrarily shaped bubble , 2013 .

[11]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[12]  Doug L. James,et al.  Harmonic fluids , 2009, SIGGRAPH 2009.

[13]  Richard M. Heitmeyer,et al.  Low-frequency sound generation by an individual open-ocean breaking wave , 2001 .

[14]  Rüdiger Westermann,et al.  Narrow Band FLIP for Liquid Simulations , 2016, Comput. Graph. Forum.

[15]  Diemo Schwarz,et al.  Interactive Sound Texture Synthesis Through Semi-Automatic User Annotations , 2013, CMMR.

[16]  Dinesh Manocha,et al.  Sounding liquids: Automatic sound synthesis from fluid simulation , 2010, TOGS.

[17]  Diemo Schwarz,et al.  Principles and Applications of Interactive Corpus-Based Concatenative Synthesis , 2008 .

[18]  T. Leighton The Acoustic Bubble , 1994 .

[19]  Shiguang Liu,et al.  Haptic interaction based SPH fluid control , 2017 .

[20]  Steve Marschner,et al.  Motion-driven concatenative synthesis of cloth sounds , 2012, ACM Trans. Graph..

[21]  Eero P. Simoncelli,et al.  Article Sound Texture Perception via Statistics of the Auditory Periphery: Evidence from Sound Synthesis , 2022 .

[22]  Timothy G. Leighton,et al.  An acoustical hypothesis for the spiral bubble nets of humpback whales and the implications for whale feeding , 2007 .

[23]  Yongning Zhu,et al.  Animating sand as a fluid , 2005, SIGGRAPH 2005.

[24]  Doug L. James,et al.  Animating fire with sound , 2011, SIGGRAPH 2011.

[25]  Yoshinori Dobashi,et al.  Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics , 2003, ACM Trans. Graph..

[26]  Diemo Schwarz,et al.  A Modular Sound Descriptor Analysis Framework For Relaxed-Real-Time Applications , 2010, ICMC.

[27]  Kees van den Doel,et al.  Physically based models for liquid sounds , 2005, TAP.

[28]  Gerd Bruder,et al.  Exploring the effect of vibrotactile feedback through the floor on social presence in an immersive virtual environment , 2017, 2017 IEEE Virtual Reality (VR).

[29]  Fredrik Kjolstad,et al.  Why New Programming Languages for Simulation? , 2016, ACM Trans. Graph..

[30]  Doug L. James,et al.  Real-time sound synthesis for paper material based on geometric analysis , 2016, Symposium on Computer Animation.

[31]  Cumhur Erkut,et al.  Synthesis of Hand Clapping Sounds , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[32]  M. Longuet-Higgins An analytic model of sound production by raindrops , 1990, Journal of Fluid Mechanics.

[33]  Andrea Prosperetti,et al.  Bubble Dynamics in Oceanic Ambient Noise , 1988 .

[34]  Doug L. James,et al.  Toward animating water with complex acoustic bubbles , 2016, ACM Trans. Graph..

[35]  M. Strasberg The Pulsation Frequency of Nonspherical Gas Bubbles in Liquids , 1953 .

[36]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[37]  Shiguang Liu,et al.  Sounding fire for immersive virtual reality , 2015, Virtual Reality.

[38]  Neil L. Aaronson,et al.  Testing and extending the Woodworth model , 2013 .