Pose-Invariant Face Recognition Using Markov Random Fields

One of the key challenges for current face recognition techniques is how to handle pose variations between the probe and gallery face images. In this paper, we present a method for reconstructing the virtual frontal view from a given nonfrontal face image using Markov random fields (MRFs) and an efficient variant of the belief propagation algorithm. In the proposed approach, the input face image is divided into a grid of overlapping patches, and a globally optimal set of local warps is estimated to synthesize the patches at the frontal view. A set of possible warps for each patch is obtained by aligning it with images from a training database of frontal faces. The alignments are performed efficiently in the Fourier domain using an extension of the Lucas-Kanade algorithm that can handle illumination variations. The problem of finding the optimal warps is then formulated as a discrete labeling problem using an MRF. The reconstructed frontal face image can then be used with any face recognition technique. The two main advantages of our method are that it does not require manually selected facial landmarks or head pose estimation. In order to improve the performance of our pose normalization method in face recognition, we also present an algorithm for classifying whether a given face image is at a frontal or nonfrontal pose. Experimental results on different datasets are presented to demonstrate the effectiveness of the proposed approach.

[1]  Rama Chellappa,et al.  What Is the Range of Surface Reconstructions from a Gradient Field? , 2006, ECCV.

[2]  Rama Chellappa,et al.  Automatic head pose estimation using randomly projected dense SIFT descriptors , 2012, 2012 19th IEEE International Conference on Image Processing.

[3]  Carlos D. Castillo,et al.  Using Stereo Matching with General Epipolar Geometry for 2D Face Recognition across Pose , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[5]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[6]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[8]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Rainer Stiefelhagen,et al.  Pose Normalization for Local Appearance-Based Face Recognition , 2009, ICB.

[10]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[11]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[12]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[13]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Tsuhan Chen,et al.  Fast image alignment in the Fourier domain , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Takeo Kanade,et al.  Multi-subregion based probabilistic approach toward pose-invariant face recognition , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[17]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[18]  Ralph Gross,et al.  Appearance-based face recognition and light-fields , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[20]  Wook Sung Kim,et al.  Abscess Transformation of Intracardiac Hematoma and Ventricular Rupture after Double‐Patch Repair of Postinfarction Ventricular Septal Defect , 2010, Journal of cardiac surgery.

[21]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[22]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Josef Kittler,et al.  Pose-Invariant Face Matching Using MRF Energy Minimization Framework , 2009, EMMCVPR.

[24]  Chinmay Hegde,et al.  Efficient Machine Learning Using Random Projections , 2007 .

[25]  M. Saquib Sarfraz,et al.  Probabilistic learning for fully automatic face recognition across pose , 2010, Image Vis. Comput..

[26]  Michael J. Jones,et al.  Fully automatic pose-invariant face recognition via 3D pose normalization , 2011, 2011 International Conference on Computer Vision.

[27]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[28]  William T. Freeman,et al.  The Patch Transform , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Jun Li,et al.  Boosting dense SIFT descriptors and shape contexts of face images for gender recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[30]  Jonathan Warrell,et al.  Tied Factor Analysis for Face Recognition across Large Pose Differences , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Yongsheng Gao,et al.  Face recognition across pose: A review , 2009, Pattern Recognit..

[32]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[33]  Nikos Komodakis,et al.  Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning , 2007, IEEE Transactions on Image Processing.

[34]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[35]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[36]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Shu Liao,et al.  A novel Markov random field based deformable model for face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[41]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[42]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Wen Gao,et al.  Locally Linear Regression for Pose-Invariant Face Recognition , 2007, IEEE Transactions on Image Processing.

[44]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[45]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[46]  Rama Chellappa,et al.  Pose-robust albedo estimation from a single image , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Tsuhan Chen,et al.  Learning patch correspondences for improved viewpoint invariant face recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  S. Shan,et al.  Maximizing intra-individual correlations for face recognition across pose differences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[50]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[51]  I. Jolliffe.  Principal Component Analysis , 2002 .

[52]  T. Hughes,et al.  Signals and systems , 2006, Genome Biology.

[53]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.