Optimal and interactive keyframe selection for motion capture

Motion capture is increasingly used in games and movies, but often requires editing before it can be used, for many reasons. The motion may need to be adjusted to correctly interact with virtual objects or to fix problems that result from mapping the motion to a character of a different size or, beyond such technical requirements, directors can request stylistic changes. Unfortunately, editing is laborious because of the low-level representation of the data. While existing motion editing methods accomplish modest changes, larger edits can require the artist to “re-animate” the motion by manually selecting a subset of the frames as keyframes. In this paper, we automatically find sets of frames to serve as keyframes for editing the motion. We formulate the problem of selecting an optimal set of keyframes as a shortest-path problem, and solve it efficiently using dynamic programming. We create a new simplified animation by interpolating the found keyframes using a naive curve fitting technique. Our algorithm can simplify motion capture to around 10% of the original number of frames while retaining most of its detail. By simplifying animation with our algorithm, we realize a new approach to motion editing and stylization founded on the time-tested keyframe interface. We present results that show our algorithm outperforms both research algorithms and a leading commercial tool.

[1]  Takeshi Miura,et al.  Adaptive Keypose Extraction from Motion Capture Data , 2014, J. Inf. Process..

[2]  Qiang Zhang,et al.  Key-frame Extraction of Human Motion Capture Data Based on Least-square Distance Curve , 2012 .

[3]  Masayuki Nakajima,et al.  Motion Belts: Visualization of Human Motion Data on a Timeline , 2008, IEICE Trans. Inf. Syst..

[4]  Richard Bellman,et al.  Splines via dynamic programming , 1972 .

[5]  Sung Yong Shin,et al.  Example-based motion cloning: Research Articles , 2004 .

[6]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH Classes.

[7]  Xin Wang,et al.  3D Human Motion Editing and Synthesis: A Survey , 2014, Comput. Math. Methods Medicine.

[8]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[9]  Daniel Cohen-Or,et al.  Action synopsis: pose selection and illustration , 2005, ACM Trans. Graph..

[10]  Michael Gleicher,et al.  Animation from observation: Motion capture and motion editing , 1999, COMG.

[11]  Martin Guay,et al.  The line of action , 2013, ACM Trans. Graph..

[12]  Adrian Hilton,et al.  Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture , 2015, TOGS.

[13]  Yeongho Seol,et al.  Artist friendly facial animation retargeting , 2011, ACM Trans. Graph..

[14]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[15]  Brian Eoff,et al.  ShortStraw: a simple and effective corner finder for polylines , 2008, SBM'08.

[16]  Takaaki Kaiga,et al.  A hybrid approach to keyframe extraction from motion capture data using curve simplification and principal component analysis , 2014 .

[17]  Nicolas Courty,et al.  Motion Compression using Principal Geodesics Analysis , 2009, Comput. Graph. Forum.

[18]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[19]  Sung Yong Shin,et al.  A hierarchical approach to interactive motion editing for human-like figures , 1999, SIGGRAPH.

[20]  Sung Yong Shin,et al.  Example‐based motion cloning , 2004, Comput. Animat. Virtual Worlds.

[21]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[22]  Kenny Roy How to Cheat in Maya 2014: Tools and Techniques for Character Animation , 2011 .

[23]  V. Rich Personal communication , 1989, Nature.

[24]  Tony White,et al.  Animation from Pencils to Pixels: Classical Techniques for the Digital Animator , 2006 .

[25]  Philip J. Schneider,et al.  An algorithm for automatically fitting digitized curves , 1990 .

[26]  Dan Zhao,et al.  Optimization-based key frame extraction for motion capture animation , 2012, The Visual Computer.

[27]  Qiang Zhang,et al.  Key Frames Extraction from Human Motion Capture Data Based on Hybrid Particle Swarm Optimization Algorithm , 2016 .

[28]  Eyuphan Bulut,et al.  Key Frame Extraction from Motion Capture Data by Curve Saliency , 2007 .

[29]  George Baciu,et al.  Entropy‐based motion extraction for motion capture animation , 2005, Comput. Animat. Virtual Worlds.

[30]  Lei Feng,et al.  Keyframe Extraction for Human Motion Capture Data Based on Joint Kernel Sparse Representation , 2017, IEEE Transactions on Industrial Electronics.

[31]  Qiang Zhang,et al.  Keyframe Extraction from Human Motion Capture Data Based on a Multiple Population Genetic Algorithm , 2014, Symmetry.

[32]  Sergey Levine,et al.  DeepMimic , 2018, ACM Trans. Graph..

[33]  Chun-Fa Chang,et al.  Key Probe: a technique for animation keyframe extraction , 2005, The Visual Computer.

[34]  John P. Lewis,et al.  SketchiMo: sketch-based motion editing for articulated characters , 2016, ACM Trans. Graph..

[35]  Urs Ramer,et al.  An iterative procedure for the polygonal approximation of plane curves , 1972, Comput. Graph. Image Process..

[36]  Wei Chen,et al.  Motion track: Visualizing variations of human motion data , 2010, 2010 IEEE Pacific Visualization Symposium (PacificVis).

[37]  Rama Chellappa,et al.  Key Frame-Based Activity Representation Using Antieigenvalues , 2006, ACCV.

[38]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[39]  Jean-Yves Guillemaut,et al.  4D parametric motion graphs for interactive animation , 2012, I3D '12.

[40]  Okan Arikan Compression of motion capture databases , 2006, SIGGRAPH 2006.

[41]  J. P. Lewis,et al.  Identifying salient points , 2009, SIGGRAPH ASIA '09.

[42]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[43]  Yi Zhang,et al.  3D Human Motion Key-Frames Extraction Based on Asynchronous Learning Factor PSO , 2015, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC).

[44]  J. Kaufman,et al.  The social science of cinema , 2013 .

[45]  Daniel Thalmann,et al.  Key-posture extraction out of human motion data , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[46]  Dimitri P. Bertsekas,et al.  Network optimization : continuous and discrete models , 1998 .

[47]  George Baciu,et al.  Entropy-based motion extraction for motion capture animation: Motion Capture and Retrieval , 2005 .

[48]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.

[49]  Tolga K. Çapin,et al.  Multiscale motion saliency for keyframe extraction from motion capture sequences , 2011, Comput. Animat. Virtual Worlds.