A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework

The human face provides useful information during interaction; therefore, any system integrating Vision-Based Human Computer Interaction requires fast and reliable face and facial feature detection. Different approaches have focused on this ability but only open source implementations have been extensively used by researchers. A good example is the Viola–Jones object detection framework that particularly in the context of facial processing has been frequently used. The OpenCV community shares a collection of public domain classifiers for the face detection scenario. However, these classifiers have been trained in different conditions and with different data but rarely tested on the same datasets. In this paper, we try to fill that gap by analyzing the individual performance of all those public classifiers presenting their pros and cons with the aim of defining a baseline for other approaches. Solid comparisons will also help researchers to choose a specific classifier for their particular scenario. The experimental setup also describes some heuristics to increase the facial feature detection rate while reducing the face false detection rate.

[1]  Luhong Liang,et al.  A detector tree of boosted classifiers for real-time object detection and tracking , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[2]  Fabrizio Smeraldi,et al.  Saccadic search with Gabor features applied to eye detection and real-time head tracking , 2000, Image Vis. Comput..

[3]  Adam Schmidt,et al.  The Performance of the Haar Cascade Classifiers Applied to the Face and Eyes Detection , 2008, Computer Recognition Systems 2.

[4]  Larry S. Davis,et al.  Labeling of human face components from range data , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Ara V. Nefian,et al.  Speaker independent audio-visual continuous speech recognition , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[6]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[7]  Changle Zhou,et al.  Real-Time Eye Detection in Video Streams , 2008, 2008 Fourth International Conference on Natural Computation.

[8]  Larry S. Davis,et al.  Computing 3-D head orientation from a monocular image sequence , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Liya Ding,et al.  Precise detailed detection of faces and facial features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Elsevier Sdol,et al.  Journal of Visual Communication and Image Representation , 2009 .

[11]  Rainer Lienhart,et al.  Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection , 2003, DAGM-Symposium.

[12]  Carlos Hitoshi Morimoto,et al.  Real-time multiple face detection using active illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[13]  Shaogang Gong,et al.  Security and Surveillance , 2011, Visual Analysis of Humans.

[14]  Gozde Bozdagi Akar,et al.  View point tracking for 3D display systems , 2005, 2005 13th European Signal Processing Conference.

[15]  Xiaobo Ren,et al.  Robust Nose Detection and Tracking Using GentleBoost and Improved Lucas-Kanade Optical Flow Algorithms , 2007, ICIC.

[16]  Klaus J. Kirchberg,et al.  Robust Face Detection Using the Hausdorff Distance , 2001, AVBPA.

[17]  Xiaobo Li,et al.  Towards a system for automatic facial feature detection , 1993, Pattern Recognit..

[18]  Marc Hanheide,et al.  Automatic Initialization for Facial Analysis in Interactive Robotics , 2008, ICVS.

[19]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Ioannis Pitas,et al.  A novel method for automatic face segmentation, facial feature extraction and tracking , 1998, Signal Process. Image Commun..

[21]  Rajkumar Roy,et al.  Advances in Soft Computing , 2018, Lecture Notes in Computer Science.

[22]  Hong Yan,et al.  Locating and extracting the eye in human face images , 1996, Pattern Recognit..

[23]  Dmitry O. Gorodnichy,et al.  Nouse 'use your nose as a mouse' perceptual vision technology for hands-free games and interfaces , 2004, Image Vis. Comput..

[24]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Preeti R. Bajaj,et al.  Face and facial feature detection , 2011, ICWET.

[26]  John D. Fernandez,et al.  Facial feature detection using Haar classifiers , 2006 .

[27]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Chris H. Q. Ding,et al.  Tensor reduction error analysis — Applications to video compression and classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Javier Lorenzo-Navarro,et al.  Face and Facial Feature Detection Evaluation - Performance Evaluation of Public Domain Haar Detectors for Face and Facial Feature Detection , 2008, VISAPP.

[31]  Alex Pentland,et al.  Looking at People: Sensing for Ubiquitous and Wearable Computing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  B. Schiele,et al.  Fast and Robust Face Finding via Local Context , 2003 .

[33]  Oscar Déniz-Suárez,et al.  ENCARA2: Real-time detection of multiple faces at different resolutions in video streams , 2007, J. Vis. Commun. Image Represent..

[34]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[35]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[36]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[37]  Matthew Turk,et al.  Computer vision in the interface , 2004, CACM.

[38]  Raymond N. J. Veldhuis,et al.  A landmark paper in face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[39]  Alan L. Yuille,et al.  Feature extraction from faces using deformable templates , 2004, International Journal of Computer Vision.

[40]  Ioannis Pitas,et al.  Facial Feature Extraction and Determination of Pose , 1998, NMBIA.

[41]  WilsonPhillip Ian,et al.  Facial feature detection using Haar classifiers , 2006 .

[42]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Weimin Huang,et al.  A robust approach to face and eyes detection from images with cluttered background , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[44]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[45]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[46]  Andreas Zell,et al.  Detection, tracking, and pursuit of humans with an autonomous mobile robot , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[47]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[48]  Ioannis Pitas,et al.  Facial feature extraction and pose determination , 2000, Pattern Recognit..