Dynamic Surface Animation using Generative Networks

This paper presents techniques to animate realistic human-like motion using a compressed learnt model from 4D volumetric performance capture data. Sequences of 4D dynamic geometry representing a human performing an arbitrary motion are encoded through a generative network into a compact space representation, whilst maintaining the original properties, such as, surface dynamics. An animation framework is proposed which computes an optimal motion graph using the novel capabilities of compression and generative synthesis properties of the network. This approach significantly reduces the memory space requirements, improves quality of animation, and facilitates the interpolation between motions. The framework optimises the number of transitions in the graph with respect to the shape and motion of the dynamic content. This generates a compact graph structure with low edge connectivity, and maintains realism when transitioning between motions. Finally, it demonstrates that generative networks facilitate the computation of novel poses, and provides a compact motion graph representation of captured dynamic shape enabling real-time interactive animation and interpolation of novel poses to smoothly transition between motions.

[1]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[2]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH Classes.

[3]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[4]  Martin Klaudiny,et al.  Global Non-rigid Alignment of Surface Sequences , 2013, International Journal of Computer Vision.

[5]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[6]  Adrian Hilton,et al.  Human motion synthesis from 3D video , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[8]  Jean-Yves Guillemaut,et al.  Interactive Animation of 4D Performance Capture , 2013, IEEE Transactions on Visualization and Computer Graphics.

[9]  Hans-Peter Seidel,et al.  Fast articulated motion tracking using a sums of Gaussians body model , 2011, 2011 International Conference on Computer Vision.

[10]  Alvaro Collet,et al.  High-quality streamable free-viewpoint video , 2015, ACM Trans. Graph..

[11]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[12]  Adrian Hilton,et al.  Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture , 2015, TOGS.

[13]  Adrian Hilton,et al.  Realistic synthesis of novel human movements from a database of motion capture examples , 2000, Proceedings Workshop on Human Motion.

[14]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[15]  Slobodan Ilic,et al.  Robust Human Body Shape and Pose Tracking , 2013, 2013 International Conference on 3D Vision.

[16]  Adrian Hilton,et al.  Shape Similarity for 3D Video Sequences of People , 2010, International Journal of Computer Vision.

[17]  Lin Gao,et al.  Variational Autoencoders for Deforming 3D Mesh Models , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[19]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[20]  J. Kautz 4D Video Textures for Interactive Character Appearance , 2013 .

[21]  Jean-Yves Guillemaut,et al.  4D parametric motion graphs for interactive animation , 2012, I3D '12.

[22]  J. Collomosse,et al.  Real-Time Full-Body Motion Capture from Video and IMUs , 2017, 2017 International Conference on 3D Vision (3DV).

[23]  Christian Theobalt,et al.  MonoPerfCap , 2017, ACM Trans. Graph..

[24]  Edmond Boyer,et al.  Surface Motion Capture Animation Synthesis , 2019, IEEE Transactions on Visualization and Computer Graphics.

[25]  Yaser Sheikh,et al.  Deep appearance models for face rendering , 2018, ACM Trans. Graph..

[26]  David A. Forsyth,et al.  Motion synthesis from annotations , 2003, ACM Trans. Graph..

[27]  Alvaro Collet,et al.  Motion graphs for unstructured textured meshes , 2016, ACM Trans. Graph..

[28]  Edmond Boyer,et al.  Controllable Variation Synthesis for Surface Motion Capture , 2017, 2017 International Conference on 3D Vision (3DV).

[29]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Jovan Popović,et al.  Semantic deformation transfer , 2009, SIGGRAPH 2009.

[31]  Adrian Hilton,et al.  Hybrid Skeleton Driven Surface Registration for Temporally Consistent Volumetric Video , 2018, 2018 International Conference on 3D Vision (3DV).

[32]  Edmond Boyer,et al.  Video Based Animation Synthesis with the Essential Graph , 2015, 2015 International Conference on 3D Vision.

[33]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[34]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[35]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[36]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[37]  Michael Gleicher,et al.  Parametric motion graphs , 2007, SI3D.