Facial tracking with head pose estimation in stereo vision

Facial tracking is a key step for face modeling, coding and animation. In this paper we present a facial tracking method in stereo vision. We use an efficient representation of 3D face motion, consisting of global head motion and local facial deformation. Stereo vision is utilized to measure 3D coordinates of the face, the global and local motion parameters of the tracked face is solved respectively. From the 3D space to the left and right image space we employ reprojection to realize 3D tracking. Experimental results of our method are given to demonstrate its performance.

[1]  M. Pauline Baker,et al.  Computer Graphics, C Version , 1996 .

[2]  John E. Howland,et al.  Computer graphics , 1990, IEEE Potentials.

[3]  Alexander Zelinsky,et al.  An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Michele Covell,et al.  Eigen-points: control-point location using principal component analyses , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[5]  Surendra Ranganath,et al.  3D deformable face model for pose determination and face synthesis , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[6]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[7]  Keith Waters,et al.  Computer Facial Animation, Second Edition , 1996 .

[8]  Jian-Huang Lai,et al.  Virtual view face image synthesis using 3D spring-based face model from a single image , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[9]  David Salesin,et al.  Resynthesizing facial animation through 3D model-based tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Radek Grzeszczuk,et al.  A data-driven model for monocular face tracking , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Peter Eisert,et al.  Model-aided coding: a new approach to incorporate facial animation into motion-compensated video coding , 2000, IEEE Trans. Circuits Syst. Video Technol..

[14]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.

[15]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[16]  Dimitris N. Metaxas,et al.  The integration of optical flow and deformable models with applications to human face shape and motion estimation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[19]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Takeo Kanade,et al.  Recognizing upper face action units for facial expression analysis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[21]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[22]  Tomaso A. Poggio,et al.  Learning-based approach to real time tracking and analysis of faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[23]  Christoph von der Malsburg,et al.  Tracking and learning graphs and pose on image sequences of faces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[24]  Keith Waters,et al.  Computer facial animation , 1996 .

[25]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[26]  Rama Chellappa,et al.  Gabor attributes tracking for face verification , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[27]  A. Pentland,et al.  Real time tracking and modeling of faces: an EKF-based analysis by synthesis approach , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.

[28]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Kiyoharu Aizawa,et al.  Model-based image coding advanced video coding techniques for very low bit-rate applications , 1995, Proc. IEEE.