Robust in-plane and out-of-plane face detection algorithm using frontal face detector and symmetry extension

Abstract Face detection plays an important role in many computer vision applications. In recent years, much research has focused on extending the well-established Adaboost face detector algorithm for multi-view face detection. However, detecting in-plane and out-of-plane rotated faces simultaneously is still a challenging task today. In this paper, a very robust multi-view face detection algorithm, with its core functionality based on frontal face detection, is proposed to simultaneously detect in-plane and out-of-plane rotated faces. Moreover, only the training data in the frontal face is needed and we do not require the training data from different in-plane or out-of-plane rotation angles. In the proposed algorithm, first, techniques such as the skin filter and entropy rate superpixels (ERSs) are applied to obtain face candidates. Then, angle compensation and refinement are applied to improve the accuracy of face detection in the in-plane case. Moreover, the symmetry extension technique, i.e., extending the face candidate with its flipped version to create a face similar to the frontal one, is applied to detect out-of-plane faces without the need of training data. Simulations on the FEI, Pointing'04, BaoFace, Group, Utrecht, and WWW datasets demonstrate the proposed algorithm's effectiveness and superior performance as compared to state-of-the-art face detection methods.

[1]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[2]  Larry S. Davis,et al.  SSH: Single Stage Headless Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  I. Jolliffe.  Principal Component Analysis , 2002 .

[4]  Hakan Cevikalp,et al.  Efficient object detection using cascades of nearest convex model classifiers , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Fátima N. S. de Medeiros,et al.  Rotation-invariant image description from independent component analysis for classification purposes , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[6]  Andrew Zisserman,et al.  "Here's looking at you, kid". Detecting people looking at each other in videos , 2011, BMVC.

[7]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[8]  Shengcai Liao,et al.  A Fast and Accurate Unconstrained Face Detector , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Yu Deng,et al.  Face Image Quality Assessment Based on Learning to Rank , 2015, IEEE Signal Processing Letters.

[10]  Maja Pantic,et al.  Empirical analysis of cascade deformable models for multi-view face detection , 2013, Image Vis. Comput..

[11]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[12]  A. Manzar,et al.  Rotation-Invariant Features for Texture Image Classification , 2006, 2006 IEEE International Conference on Engineering of Intelligent Systems.

[13]  Jian-Jiun Ding,et al.  An improved approach for face detection using superpixels, moment-based matching, and isosceles triangle matching , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[14]  Li-Jia Li,et al.  Multi-view Face Detection Using Deep Convolutional Neural Networks , 2015, ICMR.

[15]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[16]  Lionel Prevost,et al.  A Cascade of Boosted Generative and Discriminative Classifiers for Vehicle Detection , 2008, EURASIP J. Adv. Signal Process..

[17]  Junmo Kim,et al.  Efficient and fast multi-view face detection based on feature transformation , 2014, 16th International Conference on Advanced Communication Technology.

[18]  Rama Chellappa,et al.  Entropy rate superpixel segmentation , 2011, CVPR 2011.

[19]  H. D. Cheng,et al.  A simple and effective histogram equalization approach to image enhancement , 2004, Digit. Signal Process..

[20]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[21]  Wei Liu,et al.  Multi-view face detection based on cascade classifier and skin color , 2012, 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems.

[22]  Bin Yang,et al.  Aggregate channel features for multi-view face detection , 2014, IEEE International Joint Conference on Biometrics.

[23]  Kar-Ann Toh,et al.  Face detection based on skin color likelihood , 2014, Pattern Recognit..

[24]  Peter H. N. de With,et al.  Cascaded Face Detection Using Neural Network Ensembles , 2008, EURASIP J. Adv. Signal Process..

[25]  Charless C. Fowlkes,et al.  Occlusion Coherence: Localizing Occluded Faces with a Hierarchical Deformable Part Model , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Deepak Ghimire,et al.  A Robust Face Detection Method Based on Skin Color and Edges , 2013, J. Inf. Process. Syst..

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Kevin Curran,et al.  A skin tone detection algorithm for an adaptive approach to steganography , 2009, Signal Process..

[31]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[32]  C. Thomaz,et al.  A new ranking method for principal components analysis and its application to face image analysis , 2010, Image Vis. Comput..

[33]  Ioannis Pitas,et al.  Demonstrating the stability of support vector machines for classification , 2006, Signal Process..

[34]  Jian-Jiun Ding,et al.  Adaptive growing and merging algorithm for image segmentation , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).