Real-Time 3D Body Pose Tracking from Multiple 2D Images

We present a human body motion tracking system for an interactive virtual simulation training environment. This system captures images using IR illumination and near-IR cameras to overcome limitations of a dimly lit environment. Features, such as silhouettes and medial axis of blobs are extracted from the images which lack much internal texture. We use a combination of a 2D ICP and particle filtering method to estimate the articulated body configuration of a trainee from image features. The method allows articulation of the arms at elbows and shoulders and of the body at the waist; this is a considerable improvement over previous such methods. Our system works in real-time and is robust to temporary errors in image acquisition or tracking. The system serves as part of a multi-modal user-input device for interactive simulation.

[1]  Luc Van Gool,et al.  Full body tracking from multiple views using stochastic sampling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Ramakant Nevatia,et al.  Dynamic Human Pose Estimation using Markov Chain Monte Carlo Approach , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[3]  Kaleem Siddiqi,et al.  Medial Representations: Mathematics, Algorithms and Applications , 2008 .

[4]  Ramakant Nevatia,et al.  Segmentation and 3-D recovery of curved-axis generalized cylinders from an intensity image , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[5]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Rama Chellappa,et al.  View independent human body pose estimation from a single perspective image , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Trevor Darrell,et al.  3-D articulated pose tracking for untethered diectic reference , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[8]  Olivier Bernier,et al.  Real-Time 3D Articulated Pose Tracking using Particle Filtering and Belief Propagation on Factor Graphs , 2006, BMVC.

[9]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[10]  W. Eric L. Grimson,et al.  Simultaneous Pose Estimation and Camera Calibration from Multiple Views , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  Jannik Fritsch,et al.  Kernel particle filter for real-time 3D body tracking in monocular color images , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[12]  Trevor Darrell,et al.  Avoiding the "streetlight effect": tracking by exploring likelihood modes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Ahmed M. Elgammal,et al.  Inferring 3D body pose from silhouettes using activity manifold learning , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[15]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[16]  James M. Rehg,et al.  A Modular Approach to the Analysis and Evaluation of Particle Filters for Figure Tracking , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Ramakant Nevatia,et al.  Real Time Body Pose Tracking in an Immersive Training Environment , 2007, ICCV-HCI.

[18]  Claus Gramkow,et al.  On Averaging Rotations , 2004, Journal of Mathematical Imaging and Vision.

[19]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[20]  Seiji Ishikawa,et al.  Appearance-based representation and recognition of human motions , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[21]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..