Model-based varying pose face detection and facial feature registration in video images

This paper presents an automatic method for simultaneous human face detection and facial feature registration from colour video images. At the first stage, we use a skin colour Gaussian model to identify possible face locations under varying pose. Secondly, we compare image patterns with a varying pose face model in terms of shape and texture differences, using a combined feature-texture similarity measure (FTSM). False detections from the first stage are eliminated by setting an appropriate FTSM threshold. Moreover, one can also register the facial features (eyes, nose and mouth) by aligning a prototype face with the unknown pose faces. Experimental results show that the proposed method can achieve reliable face detection and feature registration under various conditions, including different poses, face appearances, and lighting conditions.

[1]  Alex Waibel,et al.  A model-based gaze tracking system , 1996, Proceedings IEEE International Joint Symposia on Intelligence and Systems.

[2]  William H. Press,et al.  Numerical recipes in C , 2002 .

[3]  Trevor Darrell,et al.  A virtual mirror interface using real-time robust face tracking , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[4]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  A. Ardeshir Goshtasby,et al.  Detecting human faces in color images , 1998, Image Vis. Comput..

[6]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[7]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[8]  David J. Kriegman,et al.  Illumination-based image synthesis: creating novel images of human faces under differing pose and lighting , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[9]  Shaogang Gong,et al.  Tracking and segmenting people in varying lighting conditions using colour , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  G. C. Stockman,et al.  Computer operation via face orientation , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[11]  Consumer products user interface using face and eye orientation , 1997, ISCE '97. Proceedings of 1997 IEEE International Symposium on Consumer Electronics (Cat. No.97TH8348).

[12]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[13]  Narendra Ahuja,et al.  Face detection using mixtures of linear subspaces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Michael J. Jones,et al.  Model-Based Matching by Linear Combinations of Prototypes , 1996 .

[15]  Alex Pentland,et al.  Probabilistic visual learning for object detection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Hiroshi Murase,et al.  Dimensionality of Illumination Manifolds in Appearance Matching , 1996, Object Representation in Computer Vision.

[17]  Andreas Koschan,et al.  Colour Image Segmentation: A Survey , 1994 .

[18]  Narendra Ahuja,et al.  Detecting human faces in color images , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[19]  D. Collobert,et al.  MULTRAK: a system for automatic multiperson localization and tracking in real-time , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[20]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[24]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[25]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  James M. Rehg,et al.  Vision for a smart kiosk , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Lixin Fan,et al.  A combined feature-texture similarity measure for face alignment under varying pose , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Georgios Tziritas,et al.  Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis , 1999, IEEE Trans. Multim..

[29]  David Beymer,et al.  Vectorizing Face Images by Interleaving Shape and Texture Computations , 1995 .