Driver head pose tracking with thermal camera

Head pose can be seen as a coarse estimation of gaze direction. In automotive industry, knowledge about gaze direction could optimize Human-Machine Interface (HMI) and Advanced Driver Assistance Systems (ADAS). Pose estimation systems are often based on camera when applications have to be contactless. In this paper, we explore uncooled thermal imagery (8-14μm) for its intrinsic night vision capabilities and for its invariance versus lighting variations. Two methods are implemented and compared, both are aided by a 3D model of the head. The 3D model, mapped with thermal texture, allows to synthesize a base of 2D projected models, differently oriented and labeled in yaw and pitch. The first method is based on keypoints. Keypoints of models are matched with those of the query image. These sets of matchings, aided with the 3D shape of the model, allow to estimate 3D pose. The second method is a global appearance approach. Among all 2D models of the base, algorithm searches the one which is the closest to the query image thanks to a weighted least squares difference.

[1]  David Beymer,et al.  Face recognition under varying pose , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Shengcai Liao,et al.  Illumination Invariant Face Recognition Using Near-Infrared Images , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Pradeep Buddharaju,et al.  Pose-Invariant Physiological Face Recognition in the Thermal Infrared Spectrum , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[5]  Liyuan Li,et al.  Head pose estimation in thermal images for human and robot interaction , 2010, 2010 The 2nd International Conference on Industrial Mechatronics and Automation.

[6]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[7]  Christophe Ducottet,et al.  Digital holography of particles: benefits of the ‘inverse problem’ approach , 2008 .

[8]  Jian-Gang Wang,et al.  Facial Feature Extraction in an Infrared Image by Proxy With a Visible Face Image , 2007, IEEE Transactions on Instrumentation and Measurement.

[9]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Irfan Essa,et al.  Head Tracking Using a Textured Polygonal Model , 1998 .

[12]  Masatoshi Okutomi,et al.  ASPnP: An Accurate and Scalable Solution to the Perspective-n-Point Problem , 2013, IEICE Trans. Inf. Syst..

[13]  Paul A. Beardsley,et al.  A qualitative approach to classifying gaze direction , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[14]  Hongyuan Wang,et al.  Novel averaging window filter for SIFT in infrared face recognition , 2011 .

[15]  Kim L. Boyer,et al.  Head pose estimation using view based eigenspaces , 2002, Object recognition supported by user interaction for service robots.

[16]  Lijun Jiang,et al.  A robust method for detecting facial orientation in infrared images , 2006, Pattern Recognit..

[17]  Ioannis A. Kakadiaris,et al.  Benchmarking 3D Pose Estimation for Face Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[18]  M.M. Trivedi,et al.  HyHOPE: Hybrid Head Orientation and Position Estimation for vision-based driver head tracking , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[19]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[21]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[22]  T. Kato,et al.  Detection of driver's posture in the car by using far infrared camera , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[23]  Larry S. Davis,et al.  Computing 3-D head orientation from a monocular image sequence , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[24]  Mohan M. Trivedi,et al.  On the design and evaluation of robust head pose for visual user interfaces: algorithms, databases, and comparisons , 2012, AutomotiveUI.

[25]  Javier R. Movellan,et al.  Monocular head pose estimation using generalized adaptive view-based appearance model , 2010, Image Vis. Comput..

[26]  Francesc Moreno-Noguer,et al.  Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Shaogang Gong,et al.  Real-time face pose estimation , 1998, Real Time Imaging.

[28]  Vincent Lepetit,et al.  Stable real-time 3D tracking using online and offline information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Shiqi Li,et al.  A Robust O(n) Solution to the Perspective-n-Point Problem , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Klaus Bengler,et al.  Infrared-Based In-Vehicle Head-Tracking , 2015 .

[31]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[32]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[33]  Vincent Lepetit,et al.  3-D Head Tracking via Invariant Keypoint Learning , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Jacob Ström.  Model-Based Real-Time Head Tracking , 2002, EURASIP J. Adv. Signal Process..

[35]  Mohan M. Trivedi,et al.  A two-stage head pose estimation framework and evaluation , 2008, Pattern Recognit..