Cascaded Generation of High-quality Color Visible Face Images from Thermal Captures

Generating visible-like face images from thermal images is essential to perform manual and automatic cross-spectrum face recognition. We successfully propose a solution based on cascaded refinement network that, unlike previous works, produces high quality generated color images without the need for face alignment, large databases, data augmentation, polarimetric sensors, computationally-intense training, or unrealistic restriction on the generated resolution. The training of our solution is based on the contextual loss, making it inherently scale (face area) and rotation invariant. We present generated image samples of unknown individuals under different poses and occlusion conditions.We also prove the high similarity in image quality between ground-truth images and generated ones by comparing seven quality metrics. We compare our results with two state-of-the-art approaches proving the superiority of our proposed approach.

[1]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[2]  Brian C. Lovell,et al.  TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition , 2017, 2018 International Conference on Biometrics (ICB).

[3]  Lihi Zelnik-Manor,et al.  The Contextual Loss for Image Transformation with Non-Aligned Data , 2018, ECCV.

[4]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Vishal M. Patel,et al.  Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces , 2017 .

[6]  Marina L. Gavrilova,et al.  A linear regression model for estimating facial image quality , 2017, 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC).

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  M. Saquib Sarfraz,et al.  Deep Perceptual Mapping for Thermal to Visible Face Recogntion , 2015, BMVC.

[10]  Stan Z. Li,et al.  Standardization of Face Image Sample Quality , 2007, ICB.

[11]  Vishal M. Patel,et al.  Generative adversarial network-based synthesis of visible faces from polarimetrie thermal faces , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[12]  Lina J. Karam,et al.  A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection (CPBD) , 2011, IEEE Transactions on Image Processing.

[13]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[14]  Jonghyun Choi,et al.  Thermal-to-visible face recognition using partial least squares. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Chao Zhang,et al.  Hallucinating faces from thermal infrared images , 2008, 2008 15th IEEE International Conference on Image Processing.

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Vladlen Koltun,et al.  Photographic Image Synthesis with Cascaded Refinement Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  László Neumann,et al.  Global Contrast Factor - a New Approach to Image Contrast , 2005, CAe.

[19]  Kiran B. Raja,et al.  Assessing face image quality for smartphone based face recognition system , 2017, 2017 5th International Workshop on Biometrics and Forensics (IWBF).

[20]  M.V. Shirvaikar.  An optimal measure for camera focus and exposure , 2004, Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the.

[21]  Jean-Luc Dugelay,et al.  A benchmark database of visible and thermal paired face images across multiple variations , 2018, 2018 International Conference of the Biometrics Special Interest Group (BIOSIG).

[22]  Benjamin S. Riggan,et al.  Thermal to Visible Synthesis of Face Images Using Multiple Regions , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).