A novel method for automatic face segmentation, facial feature extraction and tracking

The present paper describes a novel method for the segmentation of faces, extraction of facial features and tracking of the face contour and features over time. Robust segmentation of faces out of complex scenes is done based on color and shape information. Additionally, face candidates are verified by searching for facial features in the interior of the face. As interesting facial features we employ eyebrows, eyes, nostrils, mouth and chin. We consider incomplete feature constellations as well. If a face and its features are detected once reliably, we track the face contour and the features over time. Face contour tracking is done by using deformable models like snakes. Facial feature tracking is performed by block matching. The success of our approach was verified by evaluating 38 different color image sequences, containing features as beard, glasses and changing facial expressions.

[1]  Nauchnyĭ sovet po kompleksnoĭ probleme Kibernetika,et al.  Pattern recognition and image analysis : advances in mathematical theory and applications in the USSR. , 1991 .

[2]  Luc Vandendorpe,et al.  Multi-modal person verification tools using speech and images , 1996 .

[3]  Ioannis Pitas,et al.  Nonlinear Digital Filters - Principles and Applications , 1990, The Springer International Series in Engineering and Computer Science.

[4]  David C. Gibbon,et al.  Multi-modal system for locating heads and faces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[5]  D.,et al.  Multi-modal person veri cation tools using speech and images , 1996 .

[6]  Mubarak Shah,et al.  A Fast algorithm for active contours and curvature estimation , 1992, CVGIP Image Underst..

[7]  James L. Crowley,et al.  Coordination of perceptual processes for computer mediated communication , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  Karin Sobottka Ioannis Pitas.  Localization Of Facial Regions And Features In Color Images , 1996 .

[9]  Ying Dai,et al.  Extraction of Facial Images from the Complex Background Using Color Information and SGLD Matrices , 1996 .

[10]  Hidefumi Kobatake,et al.  Extraction of facial sketch images and expression transformation based on FACS , 1995, Proceedings., International Conference on Image Processing.

[11]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[12]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[13]  Alexandros Eleftheriadis,et al.  Automatic face location detection and tracking for model-assisted coding of video teleconferencing sequences at low bit-rates , 1995, Signal Process. Image Commun..

[14]  Ioannis Pitas,et al.  Looking for Faces and Facial Features in Color Images , 1997 .

[15]  A. Zakhor,et al.  Depth based recovery of human facial features from video sequences , 1995, Proceedings., International Conference on Image Processing.

[16]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Frederic Fol Leymarie,et al.  Tracking Deformable Objects in the Plane Using an Active Contour Model , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  P. Perona,et al.  Face Localization via Shape Statistics , 1995 .

[20]  Ioannis Pitas,et al.  Segmentation and tracking of faces in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[21]  Heinrich Niemann.  Pattern Analysis and Understanding , 1990 .