Sparse frontal face image synthesis from an arbitrary profile image

Frontal face image synthesis from an arbitrary profile image plays an important role in automatic video surveillance systems, and remains a challenge in computer vision. The strategies of partition are popular and promising for synthesizing frontal face images. However, conventional rectangular partition criterions fail to align corresponding patches in profile images and frontal face images. Given an arbitrary profile image, to synthesize a corresponding frontal face image which is smooth in texture and similar in appearance, we introduce a triangulation-based partition criterion and do synthesis based on sparse representation. The triangulation-based partition ensures the corresponding triangular patches are strictly aligned. And sparse representation adaptively finds the most similar patches for synthesis while abandons unlike patches. Furthermore, a confederate learning strategy is proposed to reduce the blocking artifacts caused by triangulation-based partition. Experimental results conducted on the Oriental Face database demonstrate the effectiveness of the proposed frontal face image synthesis method and advantages over previous works.

[1]  Tamás D. Gedeon,et al.  Learning-based Face Synthesis for Pose-Robust Recognition from Single Image , 2009, BMVC.

[2]  Takeo Kanade,et al.  Real-time combined 2D+3D active appearance models , 2004, CVPR 2004.

[3]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Xuelong Li,et al.  Transductive Face Sketch-Photo Synthesis , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[5]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[6]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[7]  Chunna Tian,et al.  Multi-view face recognition by nonlinear tensor decomposition , 2008, 2008 19th International Conference on Pattern Recognition.

[8]  Shaogang Gong,et al.  Support vector machine based multi-view face detection and recognition , 2004, Image Vis. Comput..

[9]  William T. Freeman,et al.  Quilting for Texture Synthesis and Transfer , 2001, SIGGRAPH 2001.

[10]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[11]  Yun Fu,et al.  Graph embedded analysis for head pose estimation , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[12]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[13]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[14]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[15]  Xinbo Gao,et al.  Photo-sketch synthesis and recognition based on subspace learning , 2010, Neurocomputing.

[16]  Bing Chen,et al.  Adaptive Neural Tracking Control for a Class of Nonstrict-Feedback Stochastic Nonlinear Systems With Unknown Backlash-Like Hysteresis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[18]  K. Walker,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Ruigang Yang,et al.  Manifold Estimation in View-Based Feature Space for Face Synthesis across Poses , 2009, ACCV.

[20]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[21]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[23]  Mohan M. Trivedi,et al.  Head Pose Estimation for Driver Assistance Systems: A Robust Algorithm and Experimental Evaluation , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[24]  Jonathan Warrell,et al.  Tied Factor Analysis for Face Recognition across Large Pose Differences , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Xuelong Li,et al.  Heterogeneous image transformation , 2013, Pattern Recognit. Lett..

[26]  Xuelong Li,et al.  A Review of Active Appearance Models , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[27]  J. Hawkins,et al.  On Intelligence , 2004 .

[28]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Xuelong Li,et al.  Face Sketch–Photo Synthesis and Retrieval Using Sparse Representation , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Igor Skrjanc,et al.  Editorial A Successful Change From TNN to TNNLS and a Very Successful Year , 2013, IEEE Trans. Neural Networks Learn. Syst..

[31]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[32]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[33]  Sethuraman Panchanathan,et al.  Biased Manifold Embedding: A Framework for Person-Independent Head Pose Estimation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Zhi-Hua Zhou,et al.  Face recognition from a single image per person: A survey , 2006, Pattern Recognit..

[35]  Jian Sun,et al.  An associate-predict model for face recognition , 2011, CVPR 2011.

[36]  Timothy F. Cootes,et al.  Statistical models of appearance for medical image analysis and computer vision , 2001, SPIE Medical Imaging.

[37]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[38]  Rex A. Dwyer.  A faster divide-and-conquer algorithm for constructing delaunay triangulations , 1987, Algorithmica.

[39]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[40]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[41]  Wen Gao,et al.  Locally Linear Regression for Pose-Invariant Face Recognition , 2007, IEEE Transactions on Image Processing.