Inverse-Foley animation

In this paper, we introduce Inverse-Foley Animation, a technique for optimizing rigid-body animations so that contact events are synchronized with input sound events. A precomputed database of randomly sampled rigid-body contact events is used to build a contact-event graph, which can be searched to determine a plausible sequence of contact events synchronized with the input sound's events. To more easily find motions with matching contact times, we allow transitions between simulated contact events using a motion blending formulation based on modified contact impulses. We fine tune synchronization by slightly retiming ballistic motions. Given a sound, our system can synthesize synchronized motions using graphs built with hundreds of thousands of precomputed motions, and millions of contact events. Our system is easy to use, and has been used to plan motions for hundreds of sounds, and dozens of rigid-body models.

[1]  Joe Marks,et al.  N-body spacetime constraints , 1995, Comput. Animat. Virtual Worlds.

[2]  Thanh Giang,et al.  Evaluating the visual fidelity of physically based animations , 2003, ACM Trans. Graph..

[3]  Doug L. James,et al.  Rigid-body fracture sound with precomputed soundbanks , 2010, ACM Trans. Graph..

[4]  Doug L. James,et al.  Precomputed acceleration noise for improved rigid-body sound , 2012, ACM Trans. Graph..

[5]  Doug L. James,et al.  Many-worlds browsing for control of multibody dynamics , 2007, SIGGRAPH 2007.

[6]  In-Kwon Lee,et al.  Automatic Synchronization of Background Music and Motion in Computer Animation , 2005, Comput. Graph. Forum.

[7]  H. Pottmann,et al.  Energy-minimizing splines in manifolds , 2004, SIGGRAPH 2004.

[8]  Okan Arikan,et al.  Interactive motion generation from examples , 2002, ACM Trans. Graph..

[9]  Michel Chion,et al.  Audio-Vision: Sound on Screen , 1994 .

[10]  Steven M. Seitz,et al.  Interactive manipulation of rigid body simulations , 2000, SIGGRAPH.

[11]  Doug L. James,et al.  Backward steps in rigid body simulation , 2008, ACM Trans. Graph..

[12]  Doug L. James,et al.  Rigid-body fracture sound with precomputed soundbanks , 2010, ACM Trans. Graph..

[13]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[14]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[15]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[16]  David A. Forsyth,et al.  Sampling plausible solutions to multi-body constraint problems , 2000, SIGGRAPH.

[17]  이현철.  Automatic synchronization of background music and motion in computer animation , 2005 .

[18]  Rustam Stolkin,et al.  Physical simulation for monocular 3D model based tracking , 2011, 2011 IEEE International Conference on Robotics and Automation.

[19]  John F. Hughes,et al.  Plausible motion simulation for computer graphics animation , 1996 .

[20]  Doug L. James,et al.  Harmonic shells: a practical nonlinear sound model for near-rigid thin shells , 2009, SIGGRAPH 2009.

[21]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[22]  Steven M. Seitz,et al.  Motion sketching for control of rigid-body simulations , 2003, TOGS.

[23]  M. Cardle,et al.  Music-driven motion editing: local motion transformations guided by music analysis , 2002, Proceedings 20th Eurographics UK Conference.

[24]  Calin Belta,et al.  An SVD-based projection method for interpolation on SE(3) , 2002, IEEE Trans. Robotics Autom..

[25]  Doug L. James,et al.  Backward steps in rigid body simulation , 2008, SIGGRAPH 2008.

[26]  James F. O'Brien,et al.  Synthesizing sounds from physically based motion , 2001, SIGGRAPH.

[27]  Tapio Takala,et al.  Sound rendering , 1992, SIGGRAPH.

[28]  Jeffrey C. Trinkle,et al.  Interactive Simulation of Rigid Body Dynamics in Computer Graphics , 2014, Eurographics.

[29]  Steven M. Seitz,et al.  Computing the Physical Parameters of Rigid-Body Motion from Video , 2002, ECCV.

[30]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH '08.

[31]  Michael F. Cohen,et al.  Interactive spacetime control for animation , 1992, SIGGRAPH.

[32]  James F. O'Brien,et al.  Synthesizing Sounds from Physically Based Motion , 2001, SIGGRAPH Video Review on Animation Theater Program.

[33]  Doug L. James,et al.  Harmonic shells: a practical nonlinear sound model for near-rigid thin shells , 2009, ACM Trans. Graph..

[34]  Sung Yong Shin,et al.  Rhythmic-motion synthesis based on motion-beat analysis , 2003, ACM Trans. Graph..

[35]  Sung Yong Shin,et al.  Computer puppetry: An importance-based approach , 2001, TOGS.

[36]  James J. Kuffner,et al.  Effective sampling and distance metrics for 3D rigid body path planning , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.