Head Pose Estimation Based on 3-D Facial Landmarks Localization and Regression

In this paper we present a system that is able to estimate head pose using only depth information from consumer RGB-D cameras such as Kinect 2. In contrast to most approaches addressing this problem, we do not rely on tracking and produce pose estimation in terms of pitch, yaw and roll angles using single depth frames as input. Our system combines three different methods for pose estimation: two of them are based on state-ofthe- art landmark detection and the third one is a dictionarybased approach that is able to work in especially challenging scans where landmarks or mesh correspondences are too difficult to obtain. We evaluated our system on the SASE database, which consists of 30K frames from 50 subjects. We obtained average pose estimation errors between 5 and 8 degrees per angle, achieving the best performance in the FG2017 Head Pose Estimation Challenge. Full code of the developed system is available on-line.

[1]  Wei Liang,et al.  Head Pose Estimation with Combined 2D SIFT and 3D HOG Features , 2013, 2013 Seventh International Conference on Image and Graphics.

[2]  Lijun Yin,et al.  Automatic pose estimation of 3D facial models , 2008, 2008 19th International Conference on Pattern Recognition.

[3]  Luc Van Gool,et al.  Real Time Head Pose Estimation from Consumer Depth Cameras , 2011, DAGM-Symposium.

[4]  Sergio Escalera,et al.  Joint Challenge on Dominant and Complementary Emotion Recognition Using Micro Emotion Features and Head-Pose Estimation: Databases , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[5]  Paul F. Whelan,et al.  Asymmetry Patterns Shape Contexts to Describe the 3D Geometry of Craniofacial Landmarks , 2013, VISIGRAPP.

[6]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[7]  Michael J. Jones,et al.  Real-time 3D head pose and facial landmark estimation from depth images using triangular surface patch features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Peter Robinson,et al.  3D Constrained Local Model for rigid and non-rigid facial tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Luc Van Gool,et al.  Real-time face pose estimation from single range images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[13]  Paul F. Whelan,et al.  Rotationally Invariant 3D Shape Contexts using Asymmetry Patterns , 2016, GRAPP/IVAPP.

[14]  Antonis A. Argyros,et al.  Head pose estimation on depth data based on Particle Swarm Optimization , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Gholamreza Anbarjafari,et al.  SASE: RGB-Depth Database for Human Head Pose Estimation , 2016, ECCV Workshops.

[16]  Rainer Stiefelhagen,et al.  Real Time Head Model Creation and Head Pose Estimation on Consumer Depth Cameras , 2014, 2014 2nd International Conference on 3D Vision.

[17]  José Miguel Buenaposada,et al.  Head-Pose Estimation In-the-Wild Using a Random Forest , 2016, AMDO.

[18]  Rainer Stiefelhagen,et al.  Head pose estimation using stereo vision for human-robot interaction , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[19]  Paul F. Whelan,et al.  Comparing 3D Descriptors for Local Search of Craniofacial Landmarks , 2012, ISVC.

[20]  Paul Suetens,et al.  A comparison of methods for non-rigid 3D shape retrieval , 2013, Pattern Recognit..

[21]  Witold Pedrycz,et al.  A central profile-based 3D face pose estimation , 2014, Pattern Recognit..

[22]  Nicu Sebe,et al.  Robust Real-Time Extreme Head Pose Estimation , 2014, 2014 22nd International Conference on Pattern Recognition.

[23]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[24]  Paul F. Whelan,et al.  3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features , 2015, IEEE Transactions on Cybernetics.

[25]  Limin Wang,et al.  A Comparative Study of Encoding, Pooling and Normalization Methods for Action Recognition , 2012, ACCV.

[26]  Jan Kautz,et al.  Robust Model-Based 3D Head Pose Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).