Automatic lip model extraction for constrained contour-based tracking

Contour model-based tracking is more robust if an accurate reference shape model of the underlying object is available. Since lip shapes vary, the ability to automatically extract user-dependent lip models from input images is desirable. We present an unsupervised segmentation method to hierarchically locate the user's face and then the lips. Techniques employed include modeling in the hue/saturation color space using Gaussian mixture models and the use of geometric constraints. With the region of interest automatically located, the model extraction problem is then formulated as a regularized model-fitting problem. The use of a generic shape as prior information improves the accuracy of the extracted lip model which is based an a cubic B-spline representation. We also describe a method to compute automatically an optimal linear color space transform needed to obtain raw estimates of the lip boundary locations, as required by the fitting procedure.