Photorealistic Facial Wrinkles Removal

Editing and retouching facial attributes is a complex task that usually requires human artists to obtain photo-realistic results. Its applications are numerous and can be found in several contexts such as cosmetics or digital media retouching, to name a few. Recently, advancements in conditional generative modeling have shown astonishing results at modifying facial attributes in a realistic manner. However, current methods are still prone to artifacts, and focus on modifying global attributes like age and gender, or local mid-sized attributes like glasses or moustaches. In this work, we revisit a two-stage approach for retouching facial wrinkles and obtain results with unprecedented realism. First, a state of the art wrinkle segmentation network is used to detect the wrinkles within the facial region. Then, an inpainting module is used to remove the detected wrinkles, filling them in with a texture that is statistically consistent with the surrounding skin. To achieve this, we introduce a novel loss term that reuses the wrinkle segmentation network to penalize those regions that still contain wrinkles after the inpainting. We evaluate our method qualitatively and quantitatively, showing state of the art results for the task of wrinkle removal. Moreover, we introduce the first high-resolution dataset, named FFHQ-Wrinkles, to evaluate wrinkle detection methods.

[1]  Xianming Liu,et al.  GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Victor Lempitsky,et al.  Resolution-robust Large Mask Inpainting with Fourier Convolutions , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[3]  Mike Rogers,et al.  A Survey on Facial Wrinkles Detection and Inpainting: Datasets, Methods, and Challenges , 2021, IEEE Transactions on Emerging Topics in Computational Intelligence.

[4]  Alireza Shafaei,et al.  AutoRetouch: Automatic Professional Face Retouching , 2021, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Chen Change Loy,et al.  Focal Frequency Loss for Image Reconstruction and Synthesis , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Jing Xiao,et al.  Image Inpainting Guided by Coherence Priors of Semantics and Textures , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Alexei A. Efros,et al.  Contrastive Learning for Unpaired Image-to-Image Translation , 2020, ECCV.

[8]  Roy Or-El,et al.  Lifespan Age Transformation Synthesis , 2020, ECCV.

[9]  Chia-Wen Lin,et al.  Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes , 2020, ECCV.

[10]  Wangmeng Zuo,et al.  Face attribute editing based on generative adversarial networks , 2020, Signal Image Video Process..

[11]  Yong Shi,et al.  Learning to Incorporate Structure Knowledge for Image Inpainting , 2020, AAAI.

[12]  Jianming Liang,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[13]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[14]  Peter Wonka,et al.  Image2StyleGAN++: How to Edit the Embedded Images? , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Faisal Z. Qureshi,et al.  EdgeConnect: Structure Guided Image Inpainting using Edge Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[16]  Edward Y. Chang,et al.  RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Xiao Liu,et al.  STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alexandr A. Kalinin,et al.  Albumentations: fast and flexible image augmentations , 2018, Inf..

[20]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Patrick Pérez,et al.  Non-Local Patch-Based Image Inpainting , 2017, Image Process. Line.

[22]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Shiguang Shan,et al.  AttGAN: Facial Attribute Editing by Only Changing What You Want , 2017, IEEE Transactions on Image Processing.

[24]  Andrew Slavin Ross,et al.  Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients , 2017, AAAI.

[25]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[26]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[27]  Guillaume Lample,et al.  Fader Networks: Manipulating Images by Sliding Attributes , 2017, NIPS.

[28]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[31]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[34]  Rama Chellappa,et al.  Detection and Inpainting of Facial Wrinkles Using Texture Orientation Fields and Markov Random Field Modeling , 2014, IEEE Transactions on Image Processing.

[35]  Masayuki Nakajima,et al.  Toward anthropometrics simulation of face rejuvenation and skin cosmetic , 2004, Comput. Animat. Virtual Worlds.

[36]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[37]  Daniel Cohen-Or,et al.  Only a matter of style: age transformation using a style-based regression model , 2021, ACM Trans. Graph..

[38]  Yadong Mu,et al.  Fast Fourier Convolution , 2020, NeurIPS.

[39]  Jiaqi Zhu,et al.  InjectionGAN: Unified Generative Adversarial Networks for Arbitrary Image Attribute Editing , 2020, IEEE Access.

[40]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[41]  R. Suthar,et al.  A Survey on Various Image Inpainting Techniques to Restore Image , 2014 .