Motion capture and human pose reconstruction from a single-view video sequence

We propose a framework to reconstruct the 3D pose of a human for animation from a sequence of single-view video frames. The framework for pose construction starts with background estimation and the performer@?s silhouette is extracted using image subtraction for each frame. Then the body silhouettes are automatically labeled using a model-based approach. Finally, the 3D pose is constructed from the labeled human silhouette by assuming orthographic projection. The proposed approach does not require camera calibration. It assumes that the input video has a static background, it has no significant perspective effects, and the performer is in an upright position. The proposed approach requires minimal user interaction.

[1]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[2]  Ioannis A. Kakadiaris,et al.  Model-Based Estimation of 3D Human Motion , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Ronen Basri,et al.  Actions as space-time shapes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  A. Murat Tekalp,et al.  Unsupervised dance figure analysis from video for dancing Avatar animation , 2008, 2008 15th IEEE International Conference on Image Processing.

[5]  Camillo J. Taylor,et al.  Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image , 2000, Comput. Vis. Image Underst..

[6]  Jitendra Malik,et al.  Estimating Human Body Configurations Using Shape Context Matching , 2002, ECCV.

[7]  David Salesin,et al.  A sketching interface for articulated figure animation , 2006, SIGGRAPH 2006.

[8]  U. Gudukbay,et al.  Motion Control for Realistic Walking Behavior using Inverse Kinematics , 2007, 2007 3DTV Conference.

[9]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[10]  Chung-Lin Huang,et al.  A Real-Time Model-Based Human Motion Tracking and Analysis for Human-Computer Interface Systems , 2004, EURASIP J. Adv. Signal Process..

[11]  Erwan Guillou,et al.  Real-time and Markerless Full-Body Human Motion Capture , 2007 .

[12]  Daniel Thalmann.  Physical, Behavioral, and Sensor-Based Animation , 1996 .

[13]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[14]  Jens Palsberg,et al.  Compiler technology for object-oriented languages , 1996, CSUR.

[15]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[16]  Priyanka Ashok Shisode.  Computer animation , 2015 .

[17]  Trevor Darrell,et al.  Fast pose estimation with parameter-sensitive hashing , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Rómer Rosales,et al.  Estimating 3D body pose using uncalibrated cameras , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Sharat Chandran,et al.  Markerless Motion Capture from Monocular Videos , 2004, ICVGIP.

[20]  Luca Ballan,et al.  Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes , 2008 .

[21]  Jessica K. Hodgins,et al.  Video-based 3D motion capture through biped control , 2012, ACM Trans. Graph..

[22]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[23]  J. Graham-Pole,et al.  Physical , 1998, The Lancet.

[24]  Huosheng Hu,et al.  CSM-420 A Survey - Human Movement Tracking and Stroke Rehabilitation , 2004 .

[25]  Xiaolin K. Wei,et al.  VideoMocap: modeling physically realistic human motion from monocular video sequences , 2010, ACM Trans. Graph..

[26]  Ugur Güdükbay,et al.  Realistic Rendering and Animation of a Multi-Layered Human Body Model , 2006, Tenth International Conference on Information Visualisation (IV'06).

[27]  Michael Gleicher,et al.  Evaluating video-based motion capture , 2002, Proceedings of Computer Animation 2002 (CA 2002).

[28]  Carissa Callini.  Virtual Reality Hardware , 2015 .

[29]  Hans-Peter Seidel,et al.  Motion capture using joint skeleton tracking and surface estimation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Paul A. Viola,et al.  Learning silhouette features for control of human motion , 2004, SIGGRAPH '04.

[31]  Szilard Kiss,et al.  Computer animation for articulated 3D characters , 2002 .

[32]  Ruigang Yang,et al.  Accurate 3D pose estimation from a single depth image , 2011, 2011 International Conference on Computer Vision.

[33]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  David A. Forsyth,et al.  Automatic Annotation of Everyday Movements , 2003, NIPS.