Interactive Animation of 4D Performance Capture

A 4D parametric motion graph representation is presented for interactive animation from actor performance capture in a multiple camera studio. The representation is based on a 4D model database of temporally aligned mesh sequence reconstructions for multiple motions. High-level movement controls such as speed and direction are achieved by blending multiple mesh sequences of related motions. A real-time mesh sequence blending approach is introduced, which combines the realistic deformation of previous nonlinear solutions with efficient online computation. Transitions between different parametric motion spaces are evaluated in real time based on surface shape and motion similarity. Four-dimensional parametric motion graphs allow real-time interactive character animation while preserving the natural dynamics of the captured performance.

[1]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[2]  Michael Gleicher,et al.  Motion editing with spacetime constraints , 1997, SI3D.

[3]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[4]  Martin Klaudiny,et al.  Global Non-rigid Alignment of Surface Sequences , 2013, International Journal of Computer Vision.

[5]  Sung Yong Shin,et al.  A hierarchical approach to interactive motion editing for human-like figures , 1999, SIGGRAPH.

[6]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[7]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[9]  Olga Sorkine-Hornung,et al.  Differential Representations for Mesh Processing , 2006, Comput. Graph. Forum.

[10]  Michael Garland,et al.  Free-form motion processing , 2008, TOGS.

[11]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[12]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Hans-Peter Seidel,et al.  Video-based characters: creating new human performances from a multi-view video database , 2011, ACM Trans. Graph..

[14]  Adrian Hilton,et al.  Space-time Editing of 3D Video Sequences , 2011, 2011 Conference for Visual Media Production.

[15]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[16]  Alla Safonova,et al.  Achieving good connectivity in motion graphs , 2008, SCA '08.

[17]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[18]  Tomohiko Mukai,et al.  Geostatistical motion interpolation , 2005, SIGGRAPH '05.

[19]  Adrian Hilton,et al.  Parametric Motion Blending through Wavelet Analysis , 2001, Eurographics.

[20]  Adrian Hilton,et al.  Global temporal registration of multiple non-rigid surface sequences , 2011, CVPR 2011.

[21]  Zoran Popovic,et al.  Physically based motion transformation , 1999, SIGGRAPH.

[22]  Takashi Matsuyama,et al.  Dynamic surface matching by geodesic mapping for 3D animation transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..

[24]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[25]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[26]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[27]  Adrian Hilton,et al.  Temporal Alignment of 3D Video Sequences Using Shape and Appearance , 2010, 2010 Conference on Visual Media Production.

[28]  S. Bhulai,et al.  Active Appearance Models for Face Recognition , 2007 .

[29]  Jean-Yves Guillemaut,et al.  4D parametric motion graphs for interactive animation , 2012, I3D '12.

[30]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[31]  Yen-Lin Chen,et al.  Interactive generation of human animation with deformable motion models , 2009, TOGS.

[32]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[33]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[34]  Sebastian Thrun,et al.  Video-based reconstruction of animatable human characters , 2010, ACM Trans. Graph..

[35]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[36]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, SIGGRAPH 2008.

[37]  Jovan Popović,et al.  Semantic deformation transfer , 2009, SIGGRAPH 2009.

[38]  Adrian Hilton,et al.  Human motion synthesis from 3D video , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[40]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[41]  Michael Gleicher,et al.  Automated extraction and parameterization of motions in large data sets , 2004, SIGGRAPH 2004.

[42]  Atsushi Nakazawa,et al.  Human video textures , 2009, I3D '09.

[43]  Michael Gleicher,et al.  Parametric motion graphs , 2007, SI3D.

[44]  Kun Zhou,et al.  Gradient domain editing of deforming mesh sequences , 2007, ACM Trans. Graph..