A benchmark database of visible and thermal paired face images across multiple variations

Although visible face recognition systems have grown as a major area of research, they are still facing serious challenges when operating in uncontrolled environments. In attempt to overcome these limitations, thermal imagery has been investigated as a promising direction to extend face recognition technology. However, the reduced number of databases acquired in thermal spectrum limits its exploration. In this paper, we introduce a database of face images acquired simultaneously in visible and thermal spectra under various variations: illumination, expression, pose and occlusion. Then, we present a comparative study of face recognition performances on both modalities against each variation and the impact of bimodal fusion. We prove that thermal spectrum rivals with the visible spectrum not only in the presence of illumination changes, but also in case of expression and poses changes.

[1]  César San-Martín,et al.  Fusion of Visible and Thermal Descriptors Using Genetic Algorithms for Face Recognition Systems , 2015, Sensors.

[2]  Patrick J. Flynn,et al.  Visible-light and Infrared Face Recognition , 2003 .

[3]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[4]  Marinette Revenu,et al.  Fusion levels of visible and infrared modalities for face recognition , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[5]  Andrea Salgian,et al.  A comparative analysis of face recognition performance with visible and thermal infrared imagery , 2002, Object recognition supported by user interaction for service robots.

[6]  Aly A. Farag,et al.  Face recognition in low resolution thermal images , 2013, Comput. Vis. Image Underst..

[7]  Xuan Zou,et al.  Illumination Invariant Face Recognition: A Survey , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[8]  T. Maity,et al.  Thermal imaging system and its real time applications : a survey , 2017 .

[9]  M. Saquib Sarfraz,et al.  Deep Perceptual Mapping for Thermal to Visible Face Recogntion , 2015, BMVC.

[10]  Larry S. Davis,et al.  Thermal to visible face recognition , 2012, Defense + Commercial Sensing.

[11]  Jonghyun Choi,et al.  Thermal-to-visible face recognition using partial least squares. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Chao Zhang,et al.  Converting Thermal Infrared Face Images into Normal Gray-Level Images , 2007, ACCV.

[13]  Jong Beom Ra,et al.  Multi-sensor image registration based on intensity and edge orientation information , 2008, Pattern Recognit..

[14]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[15]  Lawrence A. Klein,et al.  Sensor and Data Fusion: A Tool for Information Assessment and Decision Making , 2004 .

[16]  Arun Ross,et al.  Matching thermal to visible face images using hidden factor analysis in a cascaded subspace learning framework , 2016, Pattern Recognit. Lett..

[17]  Chao Zhang,et al.  Hallucinating faces from thermal infrared images , 2008, 2008 15th IEEE International Conference on Image Processing.