Visual Human Face Tracking and its Application to Lip-Reading and Emotion Recognition

We describe a visual face tracking algorithm which tracks the global as well as the nonrigid face movement in three dimensions using one camera. It is based on 3D face models. Using the face movement constraints, it is possible to track robustly and in real time. The results of the face tracking have been used in a number of applications including lip reading and emotion recogntion. In both cases, fusing the visual cues with audio speech analysis improves the performance significantly. 17 Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney