3D head tracking using non-linear optimization

Accurate and reliable tracking of the 3D position of human heads is a continuing research problem in computer vision. This paper addresses the specific problem of model-based tracking with a generic deformable 3D head model. Following the work of Vetter and Blanz, a collection of head models is obtained from a 3D scanner, registered and parameterized to give a generic head model which is linearly parameterized by a small number of parameters. This is the 3D analogue of Cootes and Taylor’s active appearance models. We cast tracking as a parameter estimation problem, and note that many existing solutions to the problem—such as CONDENSATION and Kalman filtering—are analogous to nonlinear optimization strategies in numerical analysis. We show how careful analysis of the error function, parameterization of the model pose parameters, and choice of optimizer allows us to robustly track 3D head pose in digital video camera footage of quickly moving heads.

[1]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[2]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[3]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[4]  Geoffrey D. Sullivan,et al.  Pose refinement of active models using forces in 3D , 1994, ECCV.

[5]  Timothy F. Cootes,et al.  Coupled-View Active Appearance Models , 2000, BMVC.

[6]  Lisa M. Brown,et al.  3D head tracking using motion adaptive texture-mapping , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[8]  Irfan Essa,et al.  Head Tracking Using a Textured Polygonal Model , 1998 .

[9]  David C. Hogg,et al.  Reactive Memories: An Interactive Talking-Head , 2001, BMVC.

[10]  Jörgen Ahlberg,et al.  Face tracking for model‐based coding and face animation , 2003, Int. J. Imaging Syst. Technol..

[11]  Ye Zhang,et al.  Robust 3D head tracking under partial occlusion , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[12]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.

[13]  Michael J. Jones,et al.  Model-Based Matching by Linear Combinations of Prototypes , 1996 .

[14]  Shaogang Gong,et al.  A Multi-View Nonlinear Active Shape Model Using Kernel PCA , 1999, BMVC.

[15]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[16]  David C. Hogg,et al.  Towards 3D hand tracking using a deformable model , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  S. Gilles.  Description and experimentation of image matching using mutual information , 1996 .

[19]  William H. Press,et al.  Numerical recipes in C , 2002 .

[20]  Mansoor Sarhadi,et al.  Non-linear statistical models for the 3D reconstruction of human pose and motion from monocular image sequences , 2000, Image Vis. Comput..

[21]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Stephen J. Maybank,et al.  Learning Enhanced 3D Models for Vehicle Tracking , 1998, BMVC.

[23]  Nira Dyn,et al.  Image Warping by Radial Basis Functions: Application to Facial Expressions , 1994, CVGIP Graph. Model. Image Process..

[24]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[25]  G. Marchal,et al.  Multi-modal volume registration by maximization of mutual information , 1997 .

[26]  Paulo R. S. Mendonça,et al.  Model-based 3D tracking of an articulated hand , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.