Reconstruct 3D Human Motion from Monocular Video Using Motion Library

In this paper, we present a new approach to reconstruct 3D human motion from video clips with the assistance of a precaputred motion library. Given a monocular video clip recording of one person performing some kind of locomotion and a motion library consisting of similar motions, we can infer the 3D motion from the video clip. We segment the video clip into segments with fixed length, and by using a shape matching method we can find out from the motion library several candidate motion sequences for every video segment, then from these sequences a coarse motion clip is generated by performing a continuity test on the boundaries of these candidate sequences. We propose a pose deformation algorithm to refine the coarse motion. To guarantee the naturalness of the recovered motion, we apply a motion splicing algorithm to the motion clip. We tested the approach using synthetic and real sports videos. The experimental results show the effectiveness of this approach.

[1]  Trevor Darrell,et al.  Fast contour matching using approximate earth mover's distance , 2004, CVPR 2004.

[2]  Sung Yong Shin,et al.  A Coordinate-Invariant Approach to Multiresolution Motion Analysis , 2001, Graph. Model..

[3]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[4]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[5]  Michael J. Black,et al.  Implicit Probabilistic Models of Human Motion for Synthesis and Tracking , 2002, ECCV.

[6]  Chih-Yi Chiu,et al.  Retrieval and constraint-based human posture reconstruction from a single image , 2006, J. Vis. Commun. Image Represent..

[7]  Cristian Sminchisescu,et al.  Covariance scaled sampling for monocular 3D body tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Andrew V. Goldberg,et al.  An efficient cost scaling algorithm for the assignment problem , 1995, Math. Program..

[9]  Cristian Sminchisescu,et al.  Kinematic jump processes for monocular 3D human tracking , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Michael J. Black,et al.  Learning image statistics for Bayesian tracking , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Rongrong Wang,et al.  A Video-Driven Approach to Continuous Human Motion Synthesis , 2006, Computer Graphics International.

[12]  Raghu Machiraju,et al.  Markerless monocular motion capture using image features and physical constraints , 2005, International 2005 Computer Graphics.

[13]  Ankur Agarwal,et al.  Recovering 3D human pose from monocular images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[15]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[16]  Hans-Hellmut Nagel,et al.  Tracking Persons in Monocular Image Sequences , 1999, Comput. Vis. Image Underst..

[17]  Camillo J. Taylor,et al.  Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image , 2000, Comput. Vis. Image Underst..

[18]  Hans-Peter Seidel,et al.  Advances in Computer Graphics, 24th Computer Graphics International Conference, CGI 2006, Hangzhou, China, June 26-28, 2006, Proceedings , 2006, CGI.