Person recognition using facial video information: A state of the art

In this article we propose a detailed state of the art on person recognition using facial video information. We classify the existing approaches present in the scientific literature between those that neglect the temporal information, and those that exploit it even partially. Concerning the first category, we detail the extensions to video data of: eigenfaces, fisherfaces, active appearance models (AAMs), radial basis function neural networks (RBFNNs), elastic graph matching (EGM), hierarchical discriminative regression trees (HDRTs) and pairwise clustering methods. After that, we focus on the strategies exploiting the temporal information, in particular those analysing: the facial motion with optical flow, the evolution of facial appearance over time with hidden Markov models (HMMs) or with various probabilistic tracking and recognition approaches, and the head motion with Gaussian mixture models.

[1]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[2]  Hiroshi Murase,et al.  Unsupervised recognition of multi-view face sequences based on pairwise clustering with attraction and repulsion , 2003, Comput. Vis. Image Underst..

[3]  Masashi Nishiyama,et al.  Face Recognition with the Multiple Constrained Mutual Subspace Method , 2003, AVBPA.

[4]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[6]  Timothy F. Cootes,et al.  Learning to identify and track faces in image sequences , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[7]  Juyang Weng,et al.  An incremental learning method for face recognition under continuous video stream , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[8]  Juyang Weng,et al.  Hierarchical Discriminant Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[10]  Hiroshi Murase,et al.  Multi-category classification by kernel based nonlinear subspace method , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[11]  Hiroshi Murase,et al.  Unsupervised face recognition by associative chaining , 2003, Pattern Recognit..

[12]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Jean-Luc Dugelay,et al.  Video Face Recognition: A Physiological and Behavioural Multimodal Approach , 2007, 2007 IEEE International Conference on Image Processing.

[14]  Shaogang Gong,et al.  Modelling faces dynamically across views and over time , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  J. D. van Ouwerkerk,et al.  Image super-resolution survey , 2006, Image Vis. Comput..

[16]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[17]  Hilary Buxton,et al.  Learning identity with radial basis function networks , 1998, Neurocomputing.

[18]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[19]  Baback Moghaddam,et al.  Principal Manifolds and Probabilistic Subspaces for Visual Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[21]  Jean-Luc Dugelay,et al.  Person Recognition Using Human Head Motion Information , 2006, AMDO.

[22]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[23]  Hartmut Neven,et al.  PersonSpotter-fast and robust system for human detection, tracking and recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[24]  Rama Chellappa,et al.  Face reconstruction from monocular video using uncertainty analysis and a generic model , 2003, Comput. Vis. Image Underst..

[25]  David J. Kriegman,et al.  Visual tracking and recognition using probabilistic appearance manifolds , 2005, Comput. Vis. Image Underst..

[26]  Hong-Yuan Mark Liao,et al.  Person identification using facial motion , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[27]  Jean-Luc Dugelay,et al.  Person Recognition based on Head and Mouth Dynamics , 2006, 2006 IEEE Workshop on Multimedia Signal Processing.

[28]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[29]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[30]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Sadaoki Furui,et al.  Digital Speech Processing, Synthesis, and Recognition , 1989 .

[32]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[33]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[34]  Shaogang Gong,et al.  Recognising trajectories of facial identities using kernel discriminant analysis , 2003, Image Vis. Comput..

[35]  Timothy F. Cootes,et al.  Improving identification performance by integrating evidence from sequences , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[36]  Luis Torres,et al.  Automatic face recognition for video indexing applications , 2002, Pattern Recognit..

[37]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[38]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[39]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[40]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[41]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[42]  A. Johnston,et al.  The Role of Movement in Face Recognition , 1997 .

[43]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[44]  Y. V. Venkatesh,et al.  An integrated automatic face detection and recognition system , 2002, Pattern Recognit..

[45]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Rama Chellappa,et al.  A generic approach to simultaneous tracking and verification in video , 2002, IEEE Trans. Image Process..

[47]  Mohan M. Trivedi,et al.  Streaming face recognition using multicamera video arrays , 2002, Object recognition supported by user interaction for service robots.

[48]  Shin'ichi Satoh,et al.  Comparative evaluation of face sequence matching for content-based video access , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).