Image-based facial sketch-to-photo synthesis via online coupled dictionary learning

Image-based sketch-to-photo synthesis has received intense attention in recent years because of its potential applications in digital entertainment and law enforcement. In order to achieve successful hallucinating synthesis of a realistic face photo, a coupled dictionary is first learned from a set of training photo-sketch pairs for modeling the relationship between patches from sketches and photos. Then, a series of overlapped photo patches can be synthesized on the basis of the learned coupled dictionary. Afterwards, these overlapped photo patches are further selected by the proposed global energy minimization framework based on a Markov network. Finally, a multi-band blending process is carried out to synthesize a realistic face photo. The experimental results convincingly validate the effectiveness of the proposed approach in comparison with the existing techniques.

[1]  Wei Liu,et al.  Bayesian Tensor Inference for Sketch-Based Facial Photo Hallucination , 2007, IJCAI.

[2]  Mignon Park,et al.  An effective method for detecting facial features and face in human-robot interaction , 2006, Inf. Sci..

[3]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[4]  Xiaogang Wang,et al.  Face Photo-Sketch Synthesis and Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Chun-xia Yang,et al.  A method of illumination compensation for human face image based on quotient image , 2008, Inf. Sci..

[6]  Joseph F. Murray,et al.  Learning Sparse Overcomplete Codes for Images , 2006, J. VLSI Signal Process..

[7]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Prabir Bhattacharya,et al.  A driver fatigue recognition model based on information fusion and dynamic Bayesian network , 2010, Inf. Sci..

[9]  H. Deutsch.  Principle Component Analysis , 2004 .

[10]  Xiaogang Wang,et al.  Face sketch recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Xuelong Li,et al.  A new approach for face recognition by sketches in photos , 2009, Signal Process..

[12]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[13]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[14]  Nanning Zheng,et al.  Example-based facial sketch generation with non-parametric sampling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Xindong Wu,et al.  Manifold elastic net: a unified framework for sparse dimension reduction , 2010, Data Mining and Knowledge Discovery.

[16]  Wei Chu,et al.  A new evolutionary search strategy for global optimization of high-dimensional problems , 2011, Inf. Sci..

[17]  Stephen Mancusi.  The Police Composite Sketch , 2010 .

[18]  Asifullah Khan,et al.  Digital image authentication and recovery: Employing integer transform based information embedding and extraction , 2010, Inf. Sci..

[19]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[20]  Ying Yu,et al.  Hebbian-Based Neural Networks for Bottom-Up Visual Attention Systems , 2009, ICONIP.

[21]  Weilin Huang,et al.  Adaptive nonlinear manifolds and their applications to pattern recognition , 2010, Inf. Sci..

[22]  Hanqing Lu,et al.  A nonlinear approach for face sketch synthesis and recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Baining Guo,et al.  Geometry-driven photorealistic facial expression synthesis , 2003, IEEE Transactions on Visualization and Computer Graphics.

[24]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[25]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[26]  Xinbo Gao,et al.  Photo-sketch synthesis and recognition based on subspace learning , 2010, Neurocomputing.

[27]  Luc Van Gool,et al.  Face/Off: live facial puppetry , 2009, SCA '09.