Markerless human motion tracking with a flexible model and appearance learning

A new approach to the 3D human motion tracking problem is proposed, which combines several particle filters with a physical simulation of a flexible body model. The flexible body model allows the partitioning of the state space of the human model into much smaller subsets, while finding a solution considering all the partial results of the particle filters. The flexible model also creates the necessary interaction between the different particle filters and allows effective semi-hierarchical tracking of the human body. The physical simulation does not require inverse kinematics calculations and is hence fast and easy to implement. Furthermore the system also builds an appearance model on-the-fly which allows it to work without a foreground segmentation. The system is able to start tracking automatically with a convenient initialization procedure. The implementation runs with 10 Hz on a regular PC using a stereo camera and is hence suitable for Human-Robot Interaction applications.

[1]  L. Verlet.  Computer "Experiments" on Classical Fluids. I. Thermodynamical Properties of Lennard-Jones Molecules , 1967 .

[2]  Takeo Kanade,et al.  Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[3]  Soon Ki Jung,et al.  Particle filter with analytical inference for human body tracking , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[4]  Rüdiger Dillmann,et al.  Sensor fusion for 3D human body tracking with an articulated 3D body model , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[5]  Raghu Machiraju,et al.  Markerless monocular motion capture using image features and physical constraints , 2005, International 2005 Computer Graphics.

[6]  Toby Howard,et al.  Real-time markerless human body tracking using colored voxels and 3D blobs , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[7]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Thomas Jakobsen,et al.  Advanced Character Physics , 2003 .

[9]  Ales Ude,et al.  Stereo-based Markerless Human Motion Capture for Humanoid Robot Systems , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[10]  Roberto Cipolla,et al.  Real-time tracking of highly articulated structures in the presence of noisy measurements , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Frédéric Lerasle,et al.  Data fusion within a modified annealed particle filter dedicated to human motion capture , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Patrick Peursum,et al.  On the Behaviour of Body Tracking with the Annealed Particle Filter in Realistic Conditions , 2006 .

[13]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[14]  Cristian Sminchisescu,et al.  Estimating Articulated Human Motion with Covariance Scaled Sampling , 2003, Int. J. Robotics Res..

[15]  R. Chellappa,et al.  Markerless Motion Capture using Multiple Cameras , 2005, Computer Vision for Interactive and Intelligent Environment (CVIIE'05).

[16]  Ian D. Reid,et al.  Automatic partitioning of high dimensional search spaces associated with articulated body motion capture , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Pascal Fua,et al.  Markerless Full Body Shape and Motion Capture from Video Sequences , 2002 .