Automatic makeup based on generative adversarial nets

Automatic makeup aims to compose and synthesize makeup on human face through computer algorithm, which belongs to the field of face image analysis. It plays an important role in interactive entertainment applications, image and video editing and face recognition assistance. However, as a face editing task, automatic makeup has several challenges as following: 1) It is usually difficult to meet the editing requirements while ensuring that makeup results are naturally looking without obvious artifacts. 2) Designed network should preserve the content (human identity) of input images but only edits the makeup-related parts of input images such as lip gloss and eye shadow. 3) Number of face makeup dataset is limited. To address these difficulties, we propose an automatic makeup generative adversarial network, a novel approach that can perform makeup attribute editing for multiple makeup styles. The network is an end-to-end model with several specially designed skip layers in generator network and allows simultaneous training of multiple datasets in addition to makeup datasets. Finally, the performance of proposed method was compared with state-of-art method qualitatively and quantitatively.

[1]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Alexei A. Efros,et al.  Generative Visual Manipulation on the Natural Image Manifold , 2016, ECCV.

[3]  Xiaochun Cao,et al.  Makeup Like a Superstar: Deep Localized Makeup Transfer Network , 2016, IJCAI.

[4]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yao Sun,et al.  Cross-domain Human Parsing via Adversarial Feature and Label Adaptation , 2018, AAAI.

[6]  John E. Hopcroft,et al.  Stacked Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Kun Zhou,et al.  Simulating makeup through physics-based manipulation of intrinsic image layers , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Dong Guo,et al.  Digital face makeup by example , 2009, CVPR.

[9]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[10]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Yao Sun,et al.  Face Aging with Contextual Generative Adversarial Nets , 2017, ACM Multimedia.

[12]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[13]  Hans-Peter Seidel,et al.  Computer‐Suggested Facial Makeup , 2011, Comput. Graph. Forum.