CariGANs

Facial caricature is an art form of drawing faces in an exaggerated way to convey humor or sarcasm. In this paper, we propose the first Generative Adversarial Network (GAN) for unpaired photo-to-caricature translation, which we call "CariGANs". It explicitly models geometric exaggeration and appearance stylization using two components: CariGeoGAN, which only models the geometry-to-geometry transformation from face photos to caricatures, and CariStyGAN, which transfers the style appearance from caricatures to face photos without any geometry deformation. In this way, a difficult cross-domain translation problem is decoupled into two easier tasks. The perceptual study shows that caricatures generated by our CariGANs are closer to the hand-drawn ones, and at the same time better persevere the identity, compared to state-of-the-art methods. Moreover, our CariGANs allow users to control the shape exaggeration degree and change the color/texture style by tuning the parameters or giving an example caricature.

[1]  Ergun Akleman,et al.  Making caricatures with morphing , 1997, SIGGRAPH '97.

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Harry Shum,et al.  PicToon: a personalized image-based cartoon system , 2002, MULTIMEDIA '02.

[4]  Susan E. Brennan,et al.  From the Leonardo Archive , 2007, Leonardo.

[5]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Golam Ashraf,et al.  Shape Stylized Face Caricatures , 2011, MMM.

[7]  Tsai-Yen Li,et al.  Automatic Caricature Generation by Analyzing Facial Features , 2004 .

[8]  Wen Gao,et al.  Mapping learning in eigenspace for harmonious caricature generation , 2006, MM '06.

[9]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[10]  Alexei A. Efros,et al.  Toward Multimodal Image-to-Image Translation , 2017, NIPS.

[11]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Takayuki Fujiwara,et al.  On KANSEI facial image processing for computerized facial caricaturing system PICASSO , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[13]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Harry Shum,et al.  Example-based caricature generation with exaggeration , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[15]  Josiane Zerubia,et al.  Markov random field image segmentation using cellular neural network , 1997 .

[16]  Eli Shechtman,et al.  Example-based synthesis of stylized facial animations , 2017, ACM Trans. Graph..

[17]  Nenghai Yu,et al.  StyleBank: An Explicit Representation for Neural Image Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ergun Akleman,et al.  Making Extreme Caricatures with a New Interactive 2D Deformation Technique with Simplicial Complexes , 2006 .

[19]  Linda Doyle,et al.  Painting style transfer for head portraits using convolutional neural networks , 2016, ACM Trans. Graph..

[20]  William T. Freeman,et al.  Synthesizing Normalized Faces from Facial Identity Features , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[22]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[23]  Cheng Li,et al.  Unconstrained Face Alignment via Cascaded Compositional Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Gang Hua,et al.  Visual attribute transfer through deep image analogy , 2017, ACM Trans. Graph..

[25]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[26]  Erik Reinhard,et al.  Human facial illustrations: Creation and psychophysical evaluation , 2004, TOGS.

[27]  Lenn Redman,et al.  How To Draw Caricatures , 1984 .

[28]  Nenghai Yu,et al.  Coherent Online Video Style Transfer , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Lior Wolf,et al.  Unsupervised Cross-Domain Image Generation , 2016, ICLR.

[31]  Jenn-Jier James Lien,et al.  Synthesis of Exaggerative Caricature with Inter and Intra Correlations , 2007, ACCV.

[32]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[33]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[36]  Fei Yang,et al.  Facial expression editing in video using a temporally-smooth factorization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  John P. Lewis,et al.  Improved automatic caricature by feature normalization and exaggeration , 2004, SIGGRAPH '04.

[38]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[39]  Paul W. H. Chung,et al.  Use of Neural Networks in Automatic Caricature Generation: An Approach Based on Drawing Style Capture , 2005, IbPRIA.

[40]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.