WarpGAN: Automatic Caricature Generation

We propose, WarpGAN, a fully automatic network that can generate caricatures given an input face photo. Besides transferring rich texture styles, WarpGAN learns to automatically predict a set of control points that can warp the photo into a caricature, while preserving identity. We introduce an identity-preserving adversarial loss that aids the discriminator to distinguish between different subjects. Moreover, WarpGAN allows customization of the generated caricatures by controlling the exaggeration extent and the visual styles. Experimental results on a public domain dataset, WebCaricature, show that WarpGAN is capable of generating caricatures that not only preserve the identities but also outputs a diverse set of caricatures for each input photo. Five caricature experts suggest that caricatures generated by WarpGAN are visually similar to hand-drawn ones and only prominent facial features are exaggerated.

[1]  William T. Freeman,et al.  Synthesizing Normalized Faces from Facial Identity Features , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Andrea Vedaldi,et al.  Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Kun Zhou,et al.  CaricatureShop: Personalized and Photorealistic Caricature Sketching , 2018, IEEE Transactions on Visualization and Computer Graphics.

[5]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[6]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[7]  Lior Wolf,et al.  Unsupervised Cross-Domain Image Generation , 2016, ICLR.

[8]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[9]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jing Liao,et al.  CariGANs , 2018, ACM Trans. Graph..

[12]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[13]  K. Mardia,et al.  A review of image-warping methods , 1998 .

[14]  Hanjiang Lai,et al.  Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis , 2018, NeurIPS.

[15]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[16]  Nicu Sebe,et al.  Deformable GANs for Pose-Based Human Image Generation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[18]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Leon A. Gatys,et al.  Controlling Perceptual Factors in Neural Style Transfer , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Luiz Velho,et al.  Interactive 3D caricature from harmonic exaggeration , 2011, Comput. Graph..

[21]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Minh N. Do,et al.  Semantic Image Inpainting with Perceptual and Contextual Losses , 2016, ArXiv.

[23]  Mubarak Shah,et al.  How to Take a Good Selfie? , 2015, ACM Multimedia.

[24]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[25]  John P. Lewis,et al.  Improved automatic caricature by feature normalization and exaggeration , 2004, SIGGRAPH '04.

[26]  Anil K. Jain,et al.  Towards automated caricature recognition , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[27]  Susan E. Brennan,et al.  From the Leonardo Archive , 2007, Leonardo.

[28]  Ziqiang Zheng,et al.  Unpaired photo-to-caricature translation on faces in the wild , 2017, Neurocomputing.

[29]  Wei Xiong,et al.  CariGAN: Caricature Generation through Weakly Paired Adversarial Learning , 2018, Neural Networks.

[30]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Victor S. Lempitsky,et al.  DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation , 2016, ECCV.

[32]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[33]  Ersin Yumer,et al.  ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[35]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yinghuan Shi,et al.  WebCaricature: a benchmark for caricature recognition , 2017, BMVC.

[37]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[38]  G. Rhodes,et al.  Identification and ratings of caricatures: Implications for mental representations of faces , 1987, Cognitive Psychology.

[39]  Joan Bruna,et al.  Super-Resolution with Deep Convolutional Sufficient Statistics , 2015, ICLR.

[40]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[41]  Harry Shum,et al.  Example-based caricature generation with exaggeration , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[42]  Yinghuan Shi,et al.  WebCaricature: a benchmark for caricature face recognition , 2017, ArXiv.