Design and Fusion of Pose-Invariant Face-Identification Experts

We address the problem of pose-invariant face recognition based on a single model image. To cope with novel view face images, a model of the effect of pose changes on face appearance must be available. Face images at an arbitrary pose can be mapped to a reference pose by the model yielding view-invariant representation. Such a model typically relies on dense correspondences of different view face images, which are difficult to establish in practice. Errors in the correspondences seriously degrade the accuracy of any recognizer. Therefore, we assume only the minimal possible set of correspondences, given by the corresponding eye positions. We investigate a number of approaches to pose-invariant face recognition exploiting such a minimal set of facial features correspondences. Four different methods are proposed as pose-invariant face recognition "experts" and combined in a single framework of expert fusion. Each expert explicitly or implicitly realizes the three sequential functions jointly required to capture the nonlinear manifolds of face pose changes: representation, view transformation, and class discriminative feature extraction. Within this structure, the experts are designed for diversity. We compare a design in which the three stages are sequentially optimized with two methods which employ an overall single nonlinear function learnt from different view face images. We also propose an approach exploiting a three-dimensional face data. A lookup table storing facial feature correspondences between different pose images, found by 3-D face models, is constructed. The designed experts are different in their nature owing to different sources of information and architectures used. The proposed fusion architecture of the pose-invariant face experts achieves an impressive accuracy gain by virtue of the individual experts diversity. It is experimentally shown that the individual experts outperform the classical linear discriminant analysis (LDA) method on the XM2VTS face data set consisting of about 300 face classes. Further impressive performance gains are obtained by combining the outputs of the experts using different fusion strategies

[1]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[2]  Michael I. Jordan,et al.  Local linear perceptrons for classification , 1996, IEEE Trans. Neural Networks.

[3]  Luc Vandendorpe,et al.  Enhancing the performance of personal identity authentication systems by fusion of face verification experts , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[4]  Robert P. W. Duin,et al.  Classifier Conditional Posterior Probabilities , 1998, SSPR/SPR.

[5]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Matti Pietikäinen,et al.  Unsupervised learning using locally linear embedding: experiments with face pose analysis , 2002, Object recognition supported by user interaction for service robots.

[7]  Josef Kittler,et al.  Multiple Classifier Systems , 2004, Lecture Notes in Computer Science.

[8]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[9]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[10]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Luc Vandendorpe,et al.  Combining face verification experts , 2002, Object recognition supported by user interaction for service robots.

[12]  Anil K. Jain,et al.  Integrating Faces and Fingerprints for Personal Identification , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Hyeong-Seok Ko,et al.  Performance-driven muscle-based facial animation , 2001, Comput. Animat. Virtual Worlds.

[14]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[15]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Ian Craw,et al.  Synthetic Images of Faces - An Approach to Model-Based Face Recognition , 1991, BMVC.

[17]  Lixin Fan,et al.  A combined feature-texture similarity measure for face alignment under varying pose , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  E. Mayoraz,et al.  Fusion of face and speech data for person identity verification , 1999, IEEE Trans. Neural Networks.

[19]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Shaogang Gong,et al.  Constructing facial identity surfaces in a nonlinear discriminating space , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nigel M. Allinson,et al.  Automatic Face Representation and Classification , 1998, BMVC.

[24]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[25]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[26]  Wen Gao,et al.  Information fusion in face identification , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[27]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[28]  Christoph von der Malsburg,et al.  Analysis and synthesis of human faces with pose variations by a parametric piecewise linear subspace method , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  M. Lincoln,et al.  Towards pose-independent face recognition , 2000 .

[30]  Josef Kittler,et al.  Independent component analysis in a local facial residue space for face recognition , 2004, Pattern Recognit..

[31]  Ralph Gross,et al.  Eigen light-fields and face recognition across pose , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[32]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[33]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[35]  Josef Kittler,et al.  Independent component analysis in a facial local residue space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[36]  Tomaso A. Poggio,et al.  Face recognition with support vector machines: global versus component-based approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[37]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  anonymous,et al.  Evaluation Report , 1990, The Aboriginal Child at School.

[39]  J Kittler,et al.  Discriminant analysis by multiple locally linear transformations , 2003 .

[40]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[41]  Ralph Gross,et al.  Appearance-based face recognition and light-fields , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[43]  David Casasent,et al.  Pose-invariant recognition of faces at unknown aspect views , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[44]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.