Generative Adversarial Multi-Task Learning for Face Sketch Synthesis and Recognition

Face sketch synthesis and recognition have wide range of applications in law enforcement. Despite the impressive progresses have been made in faces sketch and recognition, most existing researches regard them as two separate tasks. In this paper, we propose a generative adversarial multitask learning method in order to deal with face sketch synthesis and recognition simultaneously. Our framework is based on generative adversarial networks (GAN), in which an improved deep network named residual dense U-Net is used as generator to synthesize face sketch image and a multi-task discriminator is designed to not only guide the generator to produce more realistic sketch image, but also extract discriminative face feature. In addition, except the common adversarial loss, the perceptual loss and triplet loss are adopted for the learning of generator and discriminator, respectively. Compared with the state-of-the-art methods, the proposed method obtains better results in terms of face sketch synthesis and recognition.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Björn Ommer,et al.  A Variational U-Net for Conditional Appearance and Shape Generation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Hamed Kiani Galoogahi,et al.  Inter-modality Face Sketch Recognition , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[5]  Xinbo Gao,et al.  Random sampling for fast face sketch synthesis , 2017, Pattern Recognit..

[6]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[7]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Xinbo Gao,et al.  Anchored Neighborhood Index for Face Sketch Synthesis , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[11]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Xinbo Gao,et al.  Face Recognition from Multiple Stylistic Sketches: Scenarios, Datasets, and Evaluation , 2016, ECCV Workshops.

[13]  Anil K. Jain,et al.  Matching Composite Sketches to Face Photos: A Component-Based Approach , 2013, IEEE Transactions on Information Forensics and Security.

[14]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[16]  Lei Zhang,et al.  End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning , 2015, ICMR.

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[20]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[21]  Ladislav Lenc,et al.  Automatic face recognition system based on the SIFT features , 2015, Comput. Electr. Eng..

[22]  Hao Zhou,et al.  Markov Weight Fields for face sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Xiaogang Wang,et al.  Coupled information-theoretic encoding for face photo-sketch recognition , 2011, CVPR 2011.

[24]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Xinbo Gao,et al.  A Deep Collaborative Framework for Face Photo–Sketch Synthesis , 2019, IEEE Transactions on Neural Networks and Learning Systems.