Full-motion recovery from multiple video cameras applied to face tracking and recognition

Robust object tracking still remains a difficult problem in computer vision research and surveillance applications. One promising development in this area is the increased availability of surveillance cameras with overlapping views. Intuitively, these overlapping views may lead to more robust object tracking and recognition. However, combining the information from the multiple cameras in a meaningful way is challenging. Our contribution in this work is a novel approach to object tracking by robustly and accurately recovering the full motion of the object from multiple cameras. This is accomplished by explicitly fusing the information from multiple cameras into a joint 3D motion calculation. We apply this approach to the tracking of faces in multiple video cameras and utilize the 3D cylinder model to realize the motion calculation. The method is demonstrated on a sequence of real data for pose estimation of the face. Also, the 3D cylinder texture map from the tracking result is used in face recognition. The performance of full-motion recovery from multiple cameras is shown to produce a significant increase in the accuracy of face pose estimation and results in a higher face recognition rate than from a single camera. Our approach may be applied to other types of object tracking such as vehicles.

[1]  Marco La Cascia,et al.  Fast, reliable head tracking under varying illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  Takeo Kanade,et al.  Pose Robust Face Tracking by Combining Active Appearance Models and Cylinder Head Models , 2007, International Journal of Computer Vision.

[3]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[5]  Pedro Jiménez,et al.  Face pose estimation and tracking using automatic 3D model construction , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Alex Pentland,et al.  Motion regularization for model-based head tracking , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[7]  Jake K. Aggarwal,et al.  Occlusion robust multi-camera face tracking , 2011, CVPR 2011 WORKSHOPS.

[8]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Jing Xiao,et al.  Robust full‐motion recovery of head by dynamic templates and re‐registration techniques , 2003 .

[10]  Jing Xiao,et al.  Fitting a Single Active Appearance Model Simultaneously to Multiple Images , 2004, BMVC.

[11]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[12]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[13]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Z. Liu,et al.  Real time head pose tracking from multiple cameras with a generic model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[16]  Rama Chellappa,et al.  3D Facial Pose Tracking in Uncalibrated Videos , 2005, PReMI.

[17]  Zicheng Liu,et al.  Robust head motion computation by taking advantage of physical properties , 2000, Proceedings Workshop on Human Motion.

[18]  Daijin Kim,et al.  Robust head tracking using 3D ellipsoidal head model in particle filter , 2008, Pattern Recognit..

[19]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Zhi-Wei Chen,et al.  Extending 3D Lucas–Kanade tracking with adaptive templates for head pose estimation , 2009, Machine Vision and Applications.

[21]  Jake K. Aggarwal,et al.  Fusing face recognition from multiple cameras , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[22]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[23]  P PentlandAlex,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997 .