Hybrid On-Line 3D Face and Facial Actions Tracking in RGBD Video Sequences

In this paper, we propose a hybrid model-based tracker for simultaneous tracking of 3D head pose and facial actions in sequences of texture and depth frames. Our tracker utilizes a generic wireframe model, the Candide-3, to represent facial deformations. This wireframe model is initially fit into the first frame by an Iterative Closest Point algorithm. Given the result after the first frame, our tracking algorithm combines both Iterative Closest Point technique and Appearance Model for head pose and facial actions tracking. The tracker is capable of adapting on-line to the changes in appearance of the target and thus the prior training process is avoided. Furthermore, the tracking system works automatically without any intervention from human operators.

[1]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[2]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[3]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[4]  Martin Breidt,et al.  Robust semantic analysis by synthesis of 3D facial motion , 2011, Face and Gesture 2011.

[5]  Fadi Dornaika,et al.  Real time 3D face and facial feature tracking , 2007, Journal of Real-Time Image Processing.

[6]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Radek Grzeszczuk,et al.  A data-driven model for monocular face tracking , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[9]  Maja Pantic,et al.  Hierarchical On-line Appearance-Based Tracking for 3D head pose, eyebrows, lips, eyelids and irises , 2013, Image Vis. Comput..

[10]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[11]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[12]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[13]  Richard Bowden,et al.  Robust Facial Feature Tracking Using Shape-Constrained Multiresolution-Selected Linear Predictors , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jörgen Ahlberg.  Using the active appearance algorithm for face and facial feature tracking , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[15]  Zhengyou Zhang,et al.  3D Deformable Face Tracking with a Commodity Depth Camera , 2010, ECCV.

[16]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Fadi Dornaika,et al.  Fast and reliable active appearance model search for 3-D face tracking , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[19]  C. Kanzow.  Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties , 2004 .

[20]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.