Mouth region localization based on Gabor features and active appearance models

We propose a combined knowledge, feature and appearancebased method for accurate localization of the mouth region from a face image obtained using a web camera. First, using a knowledge-based approach based on geometrical properties of a regular face, we prune the mouth search region by extracting a rough segment where the mouth should be located in the given face. Next, assuming the mouth is closed, we extract near-to-horizontal features that resemble those of the dark shadow line observed in the intersection between the upper and lower lip. We use for this a set of Gabor featurebased filters, with specifically oriented and scaled parameters that are sufficient for our purpose. We then find strong features in the resultant Gabor set of images, and we apply the Hough Transform on the fused binary image in order to obtain the near-to horizontal line approximating the intersection between the upper and lower lip. Next, we use an Active Appearance Model (AAM) to extract an accurate mouth region. For this purpose, we have built a database of mouths containing 365 images of 20 different users, and we have used this dataset for training a 15-point mouth model. The extremes of the detected line are used as two of the model initialization points and after we apply a AAM fitting process, we can finally extract a very accurate region that contains the closed mouth. Our results show that the proposed method is able to detect a line in all of the images of our database. The accuracy of the mouth corners localization is approximately 95%. We also show that the detection of these points proves to be critical for the output of the AAM fitting.