Text2Sketch: Learning Face Sketch from Facial Attribute Text

Face sketch is the main approach to find suspect in law enforcement, especially in many cases when facial attribute descriptions of suspects by witnesses are available. Face sketch synthesized from facial attribute text can also be used in sketch based face recognition. While most previous work focus on face photo to sketch synthesis, the problem of sketch synthesis with facial attribute text has not been explored yet. The problem is challenging due to two facts: firstly, no database of face attribute text to sketch is available; secondly, it is hard to synthesize high-quality face sketches due to the ambiguity and complexity of text description. In this paper, we propose a face sketch synthesis approach with text using Stagewise-GAN. Our contributions lie in two aspects: 1) we construct the first text to face sketch database. The database, namely Text2Sketch dataset, is annotated with CUFSF dataset of 1194 sketches. For each sketch, an attribute description is labelled; 2) we synthesize vivid face sketches using Stagewise-GAN. We use user study, face retrieval performance with synthesized sketch, and quantitative results for evaluation. Experimental results show the effectiveness of our approach.

[1]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[2]  Xiaogang Wang,et al.  Face sketch recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[4]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[5]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Honglak Lee,et al.  Attribute2Image: Conditional Image Generation from Visual Attributes , 2015, ECCV.

[7]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Zhenan Sun,et al.  Recent Progress of Face Image Synthesis , 2017, 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR).

[9]  Jie Li,et al.  Bayesian Face Sketch Synthesis , 2017, IEEE Transactions on Image Processing.

[10]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[11]  Liang Chang,et al.  Face Sketch Synthesis via Sparse Representation , 2010, 2010 20th International Conference on Pattern Recognition.

[12]  Bin Song,et al.  Training-Free Synthesized Face Sketch Recognition Using Image Quality Assessment Metrics , 2016, ArXiv.

[13]  Yoshua Bengio,et al.  Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).