Inner Eye Canthus Localization for Human Body Temperature Screening

In this paper, we propose an automatic approach for localizing the inner eye canthus in thermal face images. We first coarsely detect 5 facial keypoints corresponding to the center of the eyes, the nosetip and the ears. Then we compute a sparse 2D-3D points correspondence using a 3D Morphable Face Model (3DMM). This correspondence is used to project the entire 3D face onto the image, and subsequently locate the inner eye canthus. Detecting this location allows to obtain the most precise body temperature measurement for a person using a thermal camera. We evaluated the approach on a thermal face dataset provided with manually annotated landmarks. However, such manual annotations are normally conceived to identify facial parts such as eyes, nose and mouth, and are not specifically tailored for localizing the eye canthus region. As additional contribution, we enrich the original dataset by using the annotated landmarks to deform and project the 3DMM onto the images. Then, by manually selecting a small region corresponding to the eye canthus, we enrich the dataset with additional annotations. By using the manual landmarks, we ensure the correctness of the 3DMM projection, which can be used as ground-truth for future evaluations. Moreover, we supply the dataset with the 3D head poses and per-point visibility masks for detecting self-occlusions. The data will be publicly released.

[1]  R. Basri,et al.  Direct visibility of point sets , 2007, SIGGRAPH 2007.

[2]  Xin Jin,et al.  Face alignment in-the-wild: A Survey , 2016, Comput. Vis. Image Underst..

[3]  Marcus A. Magnor,et al.  Sparse localized deformation components , 2013, ACM Trans. Graph..

[4]  Thomas S. Huang,et al.  Interactive Facial Feature Localization , 2012, ECCV.

[5]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Dorit Merhof,et al.  A Thermal Infrared Face Database With Facial Landmarks and Emotion Labels , 2019, IEEE Transactions on Instrumentation and Measurement.

[7]  Dorit Merhof,et al.  Towards Analysis of Mental Stress Using Thermal Infrared Tomography , 2018, Bildverarbeitung für die Medizin.

[8]  Alberto Del Bimbo,et al.  A Dictionary Learning-Based 3D Morphable Shape Model , 2017, IEEE Transactions on Multimedia.

[9]  J. Snell,et al.  International standards for pandemic screening using infrared thermography , 2010, Medical Imaging.

[10]  Shuowen Hu,et al.  An Examination of Deep-Learning Based Landmark Detection Methods on Thermal Face Imagery , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  G Machin,et al.  New standards for devices used for the measurement of human body temperature , 2010, Journal of medical engineering & technology.

[12]  Xavier Maldague,et al.  Université Laval Face Motion and Time-Lapse Video Database (UL-FMTV) , 2018 .

[13]  Liyuan Li,et al.  Head pose estimation in thermal images for human and robot interaction , 2010, 2010 The 2nd International Conference on Industrial Mechatronics and Automation.

[14]  Dorit Merhof,et al.  A combined modular system for face detection, head pose estimation, face tracking and emotion recognition in thermal infrared images , 2018, 2018 IEEE International Conference on Imaging Systems and Techniques (IST).

[15]  Alberto Del Bimbo,et al.  Dictionary Learning Based 3D Morphable Model Construction for Face Recognition with Varying Expression and Pose , 2015, 2015 International Conference on 3D Vision.

[16]  I. Lauder,et al.  Screening for fever by remote-sensing infrared thermographic camera. , 2006, Journal of travel medicine.

[17]  Qiang Ji,et al.  Facial Landmark Detection: A Literature Survey , 2018, International Journal of Computer Vision.

[18]  Xavier Maldague,et al.  Infrared face recognition: A comprehensive review of methodologies and databases , 2014, Pattern Recognit..

[19]  Tal Hassner,et al.  Extreme 3D Face Reconstruction: Seeing Through Occlusions , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  J. Desenclos,et al.  International travels and fever screening during epidemics: a literature review on the effectiveness and potential use of non-contact infrared thermometers. , 2009, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[21]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[22]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Wei-Ta Chu,et al.  Thermal Facial Landmark Detection by Deep Multi-Task Learning , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).

[24]  Eddie Y K Ng,et al.  Analysis of IR thermal imager for mass blind fever screening. , 2004, Microvascular research.

[25]  Dorit Merhof,et al.  A fully annotated thermal face database and its application for thermal facial expression recognition , 2018, 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[26]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[27]  James B. Mercer,et al.  Fever screening and infrared thermal imaging: concerns and guidelines , 2009 .

[28]  Yung-Yu Chuang,et al.  FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Gabriel Hermosilla Vigneau,et al.  Thermal Face Recognition Under Temporal Variation Conditions , 2017, IEEE Access.

[30]  Alberto Del Bimbo,et al.  Effective 3D based frontalization for unconstrained face recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[31]  F. Staal,et al.  Describing Crouzon and Pfeiffer syndrome based on principal component analysis. , 2015, Journal of cranio-maxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery.

[32]  Yuan-Hsiang Lin,et al.  A Thermal Camera Based Continuous Body Temperature Measurement System , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[33]  Sebastian Budzan,et al.  Face and eyes localization algorithm in thermal images for temperature measurement of the inner canthus of the eyes , 2013, Infrared Physics & Technology.

[34]  Namrata Srivastava.  Using contactless sensors to estimate learning difficulty in digital learning environments , 2019, UbiComp/ISWC Adjunct.

[35]  Jürgen Beyerer,et al.  Adaptive Contour Fitting for Pose-Invariant 3D Face Shape Reconstruction , 2015, BMVC.

[36]  Satish K. Singh,et al.  Occluded Thermal Face Recognition Using Bag of CNN ($Bo$CNN) , 2020, IEEE Signal Processing Letters.

[37]  Alberto Del Bimbo,et al.  Pose Independent Face Recognition by Localizing Local Binary Patterns via Deformation Components , 2014, 2014 22nd International Conference on Pattern Recognition.

[38]  Alberto Del Bimbo,et al.  Deep 3D morphable model refinement via progressive growing of conditional Generative Adversarial Networks , 2019, Comput. Vis. Image Underst..

[39]  Min Peng,et al.  Thermal face recognition using convolutional neural network , 2016, 2016 International Conference on Optoelectronics and Image Processing (ICOIP).

[40]  Burcin Becerik-Gerber,et al.  Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment , 2019, BuildSys@SenSys.

[41]  A. Wilder-Smith,et al.  Can we contain the COVID-19 outbreak with the same measures as for SARS? , 2020, The Lancet Infectious Diseases.

[42]  Shinfeng D. Lin,et al.  Thermal Face Recognition Based on Physiological Information , 2019, 2019 IEEE International Conference on Image Processing (ICIP).