Real time 3D face tracking from appearance

We propose a real time 3D tracking algorithm dedicated to the tracking of human faces in video sequences. A face is represented by a collection of 2D images called reference views. In our approach, a pattern is a region of the image defined in an area of interest and its sampling gives a grey level vector. The tracking technique involves two stages. An off-line learning stage is devoted to the computation of an interaction matrix for every reference view. This matrix relates the grey level difference between the tracked reference pattern and the current pattern sampled inside the area of interest to its "fronto parallel" movement (which do not modify its aspect in the image). The on-line stage consists in using this matrix to track the reference pattern in the current image. During this stage, appearance changes due to movements in roll are managed by switching between the different reference patterns. The reference pattern, after motion correction, giving the smallest grey level difference is supposed to be the new tracked reference pattern. We present experimental results showing the efficiency and the robustness of our approach.

[1]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[2]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Shaogang Gong,et al.  Modelling faces dynamically across views and over time , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[5]  Michel Dhome,et al.  A simple and efficient template matching algorithm , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  K. Walker,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[7]  Michel Dhome,et al.  Tracking of 3D Objects from Appearance , 2001 .

[8]  Dimitris N. Metaxas,et al.  Deformable model-based face shape and motion estimation , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Alex Pentland,et al.  Face recognition using view-based and modular eigenspaces , 1994, Optics & Photonics.

[10]  Shaogang Gong,et al.  Support vector regression and classification based multi-view face detection and recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).