A robust head pose estimation system for uncalibrated monocular videos

We present a robust head pose estimation system that is capable of estimating the 3D pose of a human head in video sequences captured using a single camera. The proposed system is able to accurately estimate the 3D pose parameters even without the knowledge of camera parameters. The face is modelled using a parametrized face mask in 3D. SIFT is used to match consecutive image frames. We propose a novel interpolation technique that captures the 3D movement of feature points to estimate the 2D-3D correspondences between the 3D model and the face image. The pose is established using the POSIT algorithm in a RANSAC framework that fits a 3D deformable face model onto the given face image. We evaluate the performance of the proposed scheme on standard test datasets. The mean absolute errors of estimated pitch, yaw and roll are found comparable and in some cases better than the results reported in literature.

[1]  Fadi Dornaika,et al.  Face And Facial Feature Tracking Using Deformable Models , 2004, Int. J. Image Graph..

[2]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[3]  Irfan Essa,et al.  Fast Multiple Camera Head Pose Tracking , 2003 .

[4]  Herbert Edelsbrunner,et al.  Geometry and Topology for Mesh Generation , 2001, Cambridge monographs on applied and computational mathematics.

[5]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Daijin Kim,et al.  Robust head tracking using 3D ellipsoidal head model in particle filter , 2008, Pattern Recognit..

[7]  Fadi Dornaika,et al.  Fitting 3D face models for tracking and active appearance model training , 2006, Image Vis. Comput..

[8]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[9]  Rama Chellappa,et al.  3D Facial Pose Tracking in Uncalibrated Videos , 2005, PReMI.

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Bodo Rosenhahn,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Combined Region-and Motion-based 3d Tracking of Rigid and Articulated Objects , 2022 .

[12]  Takeo Kanade,et al.  Robust 3D Head Tracking by Online Feature Registration , 2008 .

[13]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Tommaso Gritti.  Toward fully automated face pose estimation , 2009, IMCE '09.

[15]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Bingpeng Ma,et al.  Robust Head Pose Estimation Using LGBP , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[17]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[18]  Jing Xiao,et al.  Robust full-motion recovery of head by dynamic templates and re-registration techniques , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[19]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[20]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[21]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[22]  Ehud Rivlin,et al.  Robust 3D Head Tracking Using Camera Pose Estimation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[23]  Jörgen Ahlberg,et al.  CANDIDE-3 - An Updated Parameterised Face , 2001 .