Robust Face Sketch Synthesis via Generative Adversarial Fusion of Priors and Parametric Sigmoid

Despite the extensive progress in face sketch synthesis, existing methods are mostly workable under constrained conditions, such as fixed illumination, pose, background and ethnic origin that are hardly to control in real-world scenarios. The key issue lies in the difficulty to use data under fixed conditions to train a model against imaging variations. In this paper, we propose a novel generative adversarial network termed pGAN, which can generate face sketches efficiently using training data under fixed conditions and handle the aforementioned uncontrolled conditions. In pGAN, we embed key photo priors into the process of synthesis and design a parametric sigmoid activation function for compensating illumination variations. Compared to the existing methods, we quantitatively demonstrate that the proposed method can work well on face photos in the wild.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[3]  Xiaogang Wang,et al.  Lighting and Pose Robust Face Sketch Synthesis , 2010, ECCV.

[4]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[5]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Ming-Hsuan Yang,et al.  Real-Time Exemplar-Based Face Sketch Synthesis , 2014, ECCV.

[8]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[9]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[10]  Giulio Sandini,et al.  2nd European conference on computer vision , 1992, Image Vis. Comput..

[11]  Roberto Scopigno,et al.  Computer Graphics forum , 2003, Computer Graphics Forum.

[12]  Bin Song,et al.  Data-driven vs. model-driven: Fast face sketch synthesis , 2017, Neurocomputing.

[13]  Xinbo Gao,et al.  Face Sketch Synthesis via Sparse Representation-Based Greedy Search , 2015, IEEE Transactions on Image Processing.

[14]  Lei Zhang,et al.  End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning , 2015, ICMR.

[15]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[16]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[17]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jiawei Zhang,et al.  Fast Preprocessing for Robust Face Sketch Synthesis , 2017, IJCAI.

[20]  Liang Lin,et al.  Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning , 2017, IEEE Transactions on Image Processing.

[21]  Xuelong Li,et al.  Transductive Face Sketch-Photo Synthesis , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[23]  Hanqing Lu,et al.  A nonlinear approach for face sketch synthesis and recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[25]  Xinbo Gao,et al.  Deep Graphical Feature Learning for Face Sketch Synthesis , 2017, IJCAI.

[26]  Hao Zhou,et al.  Markov Weight Fields for face sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Proceedings of the 5th ACM on International Conference on Multimedia Retrieval , 2015, ICMR.

[28]  Ming-Hsuan Yang,et al.  Multi-objective convolutional learning for face labeling , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Sylvain Paris,et al.  Automatic Portrait Segmentation for Image Stylization , 2016, Comput. Graph. Forum.

[30]  Xuelong Li,et al.  Multiple Representations-Based Face Sketch–Photo Synthesis , 2016, IEEE Transactions on Neural Networks and Learning Systems.