3D head tracking under partial occlusion

A new algorithm for 3D head tracking under partial occlusion from 2D monocular image sequences is proposed. The extended superquadric (ESQ) is used to generate a geometric 3D face model in order to reduce the shape ambiguity during tracking. Optical flow is then regularized by this model to estimate the 3D rigid motion. To deal with occlusion, a new motion segmentation algorithm using motion residual error analysis is developed. The occluded areas are successfully detected and discarded as noise. Furthermore, accumulation error is heavily reduced by a new post-regularization process based on edge flow. This makes the algorithm more stable over long image sequences. The algorithm is applied to both synthetic occlusion sequence and real image sequences. Comparisons with the ground truth indicate that our method is effective and is not sensitive to occlusion during head tracking.

[1]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[3]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[4]  Chandra Kambhamettu,et al.  Extending Superquadrics with Exponent Functions: Modeling and Reconstruction , 2001, Graph. Model..

[5]  Thomas S. Huang,et al.  Nonrigid Motion Modeling and Analysis in Video Sequences for Realistic Facial Animation , 1999 .

[6]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Dimitris N. Metaxas,et al.  The integration of optical flow and deformable models with applications to human face shape and motion estimation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Alex Pentland,et al.  Motion regularization for model-based head tracking , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[9]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Alex Pentland,et al.  Visually Controlled Graphics , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Paul W. Fieguth,et al.  Color-based tracking of heads and other mobile objects at video frame rates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Irfan Essa,et al.  Head Tracking Using a Textured Polygonal Model , 1998 .

[13]  Alex Pentland,et al.  LAFTER: lips and face real time tracker , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  James L. Crowley,et al.  Multi-modal tracking of faces for video communications , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Ye Zhang,et al.  Robust 3D head tracking under partial occlusion , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[16]  Avideh Zakhor,et al.  Edge-based 3-D camera motion estimation with application to video coding , 1993, IEEE Trans. Image Process..

[17]  Dimitris N. Metaxas,et al.  Dynamic 3D models with local and global deformations: deformable superquadrics , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[18]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.