Real-Time Exemplar-Based Face Sketch Synthesis

This paper proposes a simple yet effective face sketch synthesis method. Similar to existing exemplar-based methods, a training dataset containing photo-sketch pairs is required, and a K-NN photo patch search is performed between a test photo and every training exemplar for sketch patch selection. Instead of using the Markov Random Field to optimize global sketch patch selection, this paper formulates face sketch synthesis as an image denoising problem which can be solved efficiently using the proposed method. Real-time performance can be obtained on a state-of-the-art GPU. Meanwhile quantitative evaluations on face sketch recognition and user study demonstrate the effectiveness of the proposed method. In addition, the proposed method can be directly extended to the temporal domain for consistent video sketch synthesis, which is of great importance in digital entertainment.

[1]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[2]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[3]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[4]  J. Tenenbaum,et al.  An example-based approach to style translation for line drawings , 1999 .

[5]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[6]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Nanning Zheng,et al.  Example-based facial sketch generation with non-parametric sampling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Xiaogang Wang,et al.  Face sketch synthesis and recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Hanqing Lu,et al.  A nonlinear approach for face sketch synthesis and recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Mislav Grgic,et al.  Independent comparative study of PCA, ICA, and LDA on the FERET data set , 2005, Int. J. Imaging Syst. Technol..

[12]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[13]  Hong Chen,et al.  A Hierarchical Compositional Model for Face Representation and Sketching , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Zihan Zhou,et al.  Demo: Robust face recognition via sparse representation , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[15]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[16]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[17]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xiaogang Wang,et al.  Lighting and Pose Robust Face Sketch Synthesis , 2010, ECCV.

[19]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[20]  Xiaogang Wang,et al.  Coupled information-theoretic encoding for face photo-sketch recognition , 2011, CVPR 2011.

[21]  Manuel Menezes de Oliveira Neto,et al.  Domain transform for edge-aware image and video processing , 2011, ACM Trans. Graph..

[22]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Hao Zhou,et al.  Markov Weight Fields for face sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[25]  Cewu Lu,et al.  Combining sketch and tone for pencil drawing production , 2012, NPAR '12.

[26]  Ariel Shamir,et al.  Style and abstraction in portrait sketching , 2013, ACM Trans. Graph..