Parametric animation of performance‐captured mesh sequences

In this paper, we introduce an approach to high‐level parameterisation of captured mesh sequences of actor performance for real‐time interactive animation control. High‐level parametric control is achieved by non‐linear blending between multiple mesh sequences exhibiting variation in a particular movement. For example, walking speed is parameterised by blending fast and slow walk sequences. A hybrid non‐linear mesh sequence blending approach is introduced to approximate the natural deformation of non‐linear interpolation techniques whilst maintaining the real‐time performance of linear mesh blending. Quantitative results show that the hybrid approach gives an accurate real‐time approximation of offline non‐linear deformation. An evaluation of the approach shows good performance not only for entire meshes but also with specific mesh areas. Results are presented for single and multi‐dimensional parametric control of walking (speed/direction), jumping (height/distance) and reaching (height) from captured mesh sequences. This approach allows continuous real‐time control of high‐level parameters such as speed and direction whilst maintaining the natural surface dynamics of captured movement. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, SIGGRAPH 2008.

[2]  Jovan Popović,et al.  Semantic deformation transfer , 2009, SIGGRAPH 2009.

[3]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[4]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[5]  Adrian Hilton,et al.  Shape Similarity for 3D Video Sequences of People , 2010, International Journal of Computer Vision.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  Tomohiko Mukai,et al.  Geostatistical motion interpolation , 2005, SIGGRAPH '05.

[8]  Olga Sorkine-Hornung,et al.  Differential Representations for Mesh Processing , 2006, Comput. Graph. Forum.

[9]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[10]  Michael Garland,et al.  Free-form motion processing , 2008, TOGS.

[11]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[12]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[13]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[14]  Kun Zhou,et al.  Gradient domain editing of deforming mesh sequences , 2007, ACM Trans. Graph..

[15]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[16]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Sung Yong Shin,et al.  A hierarchical approach to interactive motion editing for human-like figures , 1999, SIGGRAPH.

[18]  Yijie Han,et al.  Concurrent threads and optimal parallel minimum spanning trees algorithm , 2001, JACM.

[19]  Michael Gleicher,et al.  Motion editing with spacetime constraints , 1997, SI3D.

[20]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[21]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[22]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[23]  Sebastian Thrun,et al.  Video-based reconstruction of animatable human characters , 2010, ACM Trans. Graph..

[24]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[25]  Adrian Hilton,et al.  Global temporal registration of multiple non-rigid surface sequences , 2011, CVPR 2011.

[26]  Zoran Popovic,et al.  Physically based motion transformation , 1999, SIGGRAPH.

[27]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[28]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[29]  Jean-Yves Guillemaut,et al.  Parametric Control of Captured Mesh Sequences for Real-Time Animation , 2011, MIG.

[30]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[31]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[32]  Hans-Peter Seidel,et al.  Video-based characters: creating new human performances from a multi-view video database , 2011, ACM Trans. Graph..

[33]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[34]  Takashi Matsuyama,et al.  Dynamic surface matching by geodesic mapping for 3D animation transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[36]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Adrian Hilton,et al.  Human motion synthesis from 3D video , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Adrian Hilton,et al.  Hierarchical Shape Matching for Temporally Consistent 3D Video , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[39]  Michael Gleicher,et al.  Automated extraction and parameterization of motions in large data sets , 2004, SIGGRAPH 2004.

[40]  Atsushi Nakazawa,et al.  Human video textures , 2009, I3D '09.

[41]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..