Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow

In this paper, we present a method for human full-body pose estimation from Time-of-Flight (ToF) camera images. Our approach consists of robustly detecting anatomical landmarks in the 3D data and fitting a skeleton body model using constrained inverse kinematics. Instead of relying on appearance-based features for interest point detection that can vary strongly with illumination and pose changes, we build upon a graph-based representation of the ToF depth data that allows us to measure geodesic distances between body parts. As these distances do not change with body movement, we are able to localize anatomical landmarks independent of pose. For differentiation of body parts that occlude each other, we employ motion information, obtained from the optical flow between subsequent ToF intensity images. We provide a qualitative and quantitative evaluation of our pose tracking method on ToF sequences containing movements of varying complexity.

[1]  Luc Van Gool,et al.  Learning Generative Models for Multi-Activity Body Pose Estimation , 2008, International Journal of Computer Vision.

[2]  Trevor Darrell,et al.  Sparse probabilistic regression for activity-independent human pose inference , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Reinhard Koch,et al.  Time-of-Flight Sensors in Computer Graphics , 2009, Eurographics.

[4]  Nassir Navab,et al.  Manifold Learning for ToF-based Human Body Tracking and Activity Recognition , 2010, BMVC.

[5]  Sebastian Thrun,et al.  Real time motion capture using a single time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Jessica K. Hodgins,et al.  Automatic Joint Parameter Estimation from Magnetic Motion Capture Data , 2023, Graphics Interface.

[7]  Thomas B. Moeslund,et al.  Fusion of range and intensity information for view invariant gesture recognition , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[9]  Sridha Sridharan,et al.  An adaptive optical flow technique for person tracking systems , 2007, Pattern Recognit. Lett..

[10]  Philip H. S. Torr,et al.  Regression-Based Human Motion Capture From Voxel Data , 2006, BMVC.

[11]  Kikuo Fujimura,et al.  A Bayesian Framework for Human Body Pose Tracking from Depth Image Sequences , 2010, Sensors.

[12]  Joachim Hornegger,et al.  3-D gesture-based scene navigation in medical imaging applications using Time-of-Flight cameras , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[13]  Bodo Rosenhahn,et al.  Multisensor-fusion for 3D full-body human motion capture , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Behzad Dariush,et al.  Controlled human pose estimation from depth image streams , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Giuseppe Patanè,et al.  From geometric to semantic human body models , 2006, Comput. Graph..

[16]  Luc Van Gool,et al.  Markerless tracking of complex human motions from multiple views , 2006, Comput. Vis. Image Underst..

[17]  Michael Beetz,et al.  Accurate Human Motion Capture Using an Ergonomics-Based Anthropometric Human Model , 2008, AMDO.

[18]  Yoshiaki Shirai,et al.  Tracking a person with 3-D motion by integrating optical flow and depth , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Rasmus Larsen,et al.  Analyzing Gait Using a Time-of-Flight Camera , 2009, SCIA.