Face Tracking and Recognition in Video

In this chapter, we describe the utility of videos in enhancing performance of image-based recognition tasks. We discuss a joint tracking-recognition framework that allows for using the motion information in a video to better localize and identify the person in the video using still galleries. We discuss how to jointly capture facial appearance and dynamics to obtain a parametric representation for video-to-video recognition. We discuss recognition in multi-camera networks where the probe and gallery both consist of multi-camera videos. Concluding remarks and directions for future research are provided.

[1]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[2]  Xavier Boyen,et al.  Tractable Inference for Complex Stochastic Processes , 1998, UAI.

[3]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[4]  Rama Chellappa,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Matching Shape Sequences in Video with Applications in Human Movement Analysis. Ieee Transactions on Pattern Analysis and Machine Intelligence 2 , 2022 .

[5]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[6]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[7]  Ravi Ramamoorthi,et al.  Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[9]  Trevor Darrell,et al.  Face recognition with image sets using manifold density divergence , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Dacheng Tao,et al.  Video Search and Mining , 2010, Video Search and Mining.

[11]  Bart De Moor,et al.  Subspace algorithms for the stochastic identification problem, , 1993, Autom..

[12]  Dit-Yan Yeung,et al.  Locally Linear Models on Face Appearance Manifolds with Application to Dual-Subspace Based Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[14]  Michel Dhome,et al.  A simple and efficient template matching algorithm , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[16]  Arun Ross,et al.  Handbook of Multibiometrics , 2006, The Kluwer international series on biometrics.

[17]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Rama Chellappa,et al.  A generic approach to simultaneous tracking and verification in video , 2002, IEEE Trans. Image Process..

[19]  David J. Fleet,et al.  Robust online appearance models for visual tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  K. Etemad,et al.  Discriminant analysis for recognition of human face images , 1997 .

[21]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[23]  Tsuhan Chen,et al.  Eigenspace updating for non-stationary process and its application to face recognition , 2003, Pattern Recognit..

[24]  Rama Chellappa,et al.  Characterization of Human Faces under Illumination Variations Using Rank, Integrability, and Symmetry Constraints , 2004, ECCV.

[25]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[27]  Y. Chikuse.  Statistics on special manifolds , 2003 .

[28]  Daniel D. Lee,et al.  Grassmann discriminant analysis: a unifying view on subspace-based learning , 2008, ICML '08.

[29]  Rama Chellappa,et al.  Statistical Analysis on Manifolds and Its Applications to Video Analysis , 2010, Video Search and Mining.

[30]  Stefano Soatto,et al.  Recognition of human gaits , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[32]  Amit K. Roy-Chowdhury,et al.  Pose and Illumination Invariant Face Recognition in Video , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[34]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[35]  Bart De Moor,et al.  Subspace angles between ARMA models , 2002, Syst. Control. Lett..

[36]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[37]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[40]  Rama Chellappa,et al.  Structure from Motion Using Sequential Monte Carlo Methods , 2004, International Journal of Computer Vision.

[41]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[42]  Rama Chellappa,et al.  From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Rama Chellappa,et al.  A system identification approach for video-based face recognition , 2004, ICPR 2004.

[44]  Hervé Abdi,et al.  Learning the Moves: The Effect of Familiarity and Facial Motion on Person Recognition across Large Changes in Viewing Format , 2006, Perception.

[45]  Arun Ross,et al.  Face Recognition in Video: Adaptive Fusion of Multiple Matchers , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[47]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[48]  Lei Zhang,et al.  Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Shaogang Gong,et al.  Video-based online face recognition using identity surfaces , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[50]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[51]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[52]  Rama Chellappa,et al.  Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[54]  Tammo tom Dieck,et al.  Representations of Compact Lie Groups , 1985 .

[55]  Rama Chellappa,et al.  Face reconstruction from monocular video using uncertainty analysis and a generic model , 2003, Comput. Vis. Image Underst..

[56]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[57]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[58]  Sami Romdhani,et al.  Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions , 2002, ECCV.

[59]  Alex Pentland,et al.  Bayesian Modeling of Facial Similarity , 1998, NIPS.

[60]  Michael J. Black,et al.  A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions , 1998, ECCV.

[61]  J. Sullivan,et al.  Guiding random particles by deterministic search , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[62]  Alex Pentland,et al.  Probabilistic Visual Learning for Object Representation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Roberto Cipolla,et al.  Face Recognition from Video Using the Generic Shape-Illumination Manifold , 2006, ECCV.

[65]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  David J. Kriegman,et al.  Visual tracking and recognition using probabilistic appearance manifolds , 2005, Comput. Vis. Image Underst..