Hybrid sensing face detection and recognition

The capability to track, detect, and identify human targets in highly cluttered scenes under extreme conditions, such as in complete darkness or in battlefield, has been one of the primary tactical advantages in military operations. In this paper, we propose a new collaborative, multi-spectrum sensing solution to achieve face detection and registration under low lighting conditions. We construct a novel type of hybrid sensors by combining a pair of near infrared (NIR) cameras and a thermal camera (a long wave infrared LWIR camera). We strategically surround each NIR sensor with a ring of LED IR flashes in order to capture the “red-eye”, or more precisely, the “bright-eye” effect of the target. The bright-eyes are used to localize the 3D position of eyes and face. The recovered 3D information can be further used to warp the thermal face imagery to frontal-parallel pose so that additional tasks such as face recognition can be reliably conducted, especially with the assistance of accurate eye locations.

[1]  Lior Wolf,et al.  Learning over Sets using Kernel Principal Angles , 2003, J. Mach. Learn. Res..

[2]  Xavier Maldague,et al.  Infrared face recognition: A comprehensive review of methodologies and databases , 2014, Pattern Recognit..

[3]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[4]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Christoph von der Malsburg,et al.  Recognizing Faces by Dynamic Link Matching , 1996, NeuroImage.

[6]  Won Mok Shim,et al.  Color updating on the apparent motion path. , 2013, Journal of vision.

[7]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[8]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Xavier Maldague,et al.  Theory and Practice of Infrared Technology for Nondestructive Testing , 2001 .

[11]  Christophe Garcia,et al.  Convolutional face finder: a neural architecture for fast and robust face detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Michael M. Bronstein.  Expression-invariant 3D face recognition , 2008 .

[13]  Masashi Nishiyama,et al.  Face recognition using the classified appearance-based quotient image , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[14]  Xiaogang Wang,et al.  Recover Canonical-View Faces in the Wild with Deep Neural Networks , 2014, ArXiv.

[15]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Sang Uk Lee,et al.  Face recognition using face-ARG matching , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  D. Norren,et al.  Directional and nondirectional spectral reflection from the human fovea. , 2008 .

[19]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[20]  Hans-Peter Seidel,et al.  Exchanging Faces in Images , 2004, Comput. Graph. Forum.

[21]  Tim Chuk,et al.  Understanding eye movements in face recognition using hidden Markov models. , 2014, Journal of vision.

[22]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yann LeCun,et al.  Synergistic Face Detection and Pose Estimation with Energy-Based Models , 2004, J. Mach. Learn. Res..