Fusing the facial temporal information in videos for face recognition

Face recognition is a challenging and innovative research topic in the present sophisticated world of visual technology. In most of the existing approaches, the face recognition from the still images is affected by intra-personal variations such as pose, illumination and expression which degrade the performance. This study proposes a novel approach for video-based face recognition due to the availability of large intra-personal variations. The feature vector based on the normalised semi-local binary patterns is obtained for the face region. Each frame is matched with the signature of the faces in the database and a rank list is formed. Each ranked list is clustered and its reliability is analysed for re-ranking. To characterise an individual in a video, multiple re-ranked lists across the multiple video frames are fused to form a video signature. This video signature embeds diverse intra-personal and temporal variations, which facilitates in matching two videos with large variations. For matching two videos, their video signatures are compared using Kendall-Tau distance. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their approach when compared with the existing techniques.

[1]  D. Sculley,et al.  Rank Aggregation for Similar Items , 2007, SDM.

[2]  Yunhong Wang,et al.  Video face recognition via combination of real-time local features and temporal-spatial cues , 2014, IET Comput. Vis..

[3]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Mahmoud Hassaballah,et al.  Face recognition: challenges, achievements and future directions , 2015, IET Comput. Vis..

[5]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[6]  Xiaojun Qi,et al.  Face recognition under varying illumination based on adaptive homomorphic eight local directional patterns , 2015, IET Comput. Vis..

[7]  Jaesik Choi,et al.  Semi-Local Structure Patterns for Robust Face Detection , 2015, IEEE Signal Processing Letters.

[8]  Patrick J. Flynn,et al.  Face Recognition from Video: a Review , 2012, Int. J. Pattern Recognit. Artif. Intell..

[9]  Shiguang Shan,et al.  Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[11]  Ajmal S. Mian,et al.  Sparse approximated nearest points for image set classification , 2011, CVPR 2011.

[12]  Xiaogang Wang,et al.  Deep Learning Multi-View Representation for Face Recognition , 2014, ArXiv.

[13]  Yongkang Wong,et al.  Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition , 2011, CVPR 2011 WORKSHOPS.

[14]  A. Suruliandi,et al.  Local binary pattern and its derivatives for face recognition , 2012 .

[15]  Wen Gao,et al.  Manifold–Manifold Distance and its Application to Face Recognition With Image Sets , 2012, IEEE Transactions on Image Processing.

[16]  Simon C. K. Shiu,et al.  Image Set-Based Collaborative Representation for Face Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[17]  Arun Ross,et al.  Score normalization in multimodal biometric systems , 2005, Pattern Recognit..

[18]  Shiguang Shan,et al.  Image sets alignment for Video-Based Face Recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Himanshu S. Bhatt,et al.  On rank aggregation for face recognition from videos , 2013, 2013 IEEE International Conference on Image Processing.

[20]  Honglak Lee,et al.  Learning hierarchical representations for face verification with convolutional deep belief networks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[22]  Vladimir Pavlovic,et al.  Face tracking and recognition with visual constraints in real-world videos , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Jian Zhang,et al.  Improving the Effectiveness of Information Retrieval with Clustering and Fusion , 2001, Int. J. Comput. Linguistics Chin. Lang. Process..

[24]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[25]  Lei Zhang,et al.  Face recognition based on regularized nearest points between image sets , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[26]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Ioannis A. Kakadiaris,et al.  Addressing the illumination challenge in two-dimensional face recognition: a survey , 2015, IET Comput. Vis..

[28]  Tai Sing Lee,et al.  Image Representation Using 2D Gabor Wavelets , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[30]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[31]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.