Lifespan Age Transformation Synthesis

We address the problem of single photo age progression and regression-the prediction of how a person might look in the future, or how they looked in the past. Most existing aging methods are limited to changing the texture, overlooking transformations in head shape that occur during the human aging and growth process. This limits the applicability of previous methods to aging of adults to slightly older adults, and application of those methods to photos of children does not produce quality results. We propose a novel multi-domain image-to-image generative adversarial network architecture, whose learned latent space models a continuous bi-directional aging process. The network is trained on the FFHQ dataset, which we labeled for ages, gender, and semantic segmentation. Fixed age classes are used as anchors to approximate continuous age transformation. Our framework can predict a full head portrait for ages 0-70 from a single photo, modifying both texture and shape of the head. We demonstrate results on a wide variety of photos and datasets, and show significant improvement over the state of the art.

[1]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Anil K. Jain,et al.  Learning Face Age Progression: A Pyramid Architecture of GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Rama Chellappa,et al.  Modeling Age Progression in Young Faces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Lingyun Wu,et al.  MaskGAN: Towards Diverse and Interactive Facial Image Manipulation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  D. Perrett,et al.  Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[7]  Rama Chellappa,et al.  Computational methods for modeling facial aging: A survey , 2009, J. Vis. Lang. Comput..

[8]  Lior Wolf,et al.  Unsupervised Cross-Domain Image Generation , 2016, ICLR.

[9]  Yun Fu,et al.  Age Synthesis and Estimation via Faces: A Survey , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Qi Li,et al.  Global and Local Consistent Age Generative Adversarial Networks , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[11]  Marios Savvides,et al.  Temporal Non-volume Preserving Approach to Facial Age-Progression and Age-Invariant Face Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Daniel Thalmann,et al.  A plastic-visco-elastic model for wrinkles in facial animation and skin aging , 1994 .

[13]  Xu Tang,et al.  Face Aging with Identity-Preserved Conditional Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Xiao Liu,et al.  STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bolei Zhou,et al.  Disentangled Inference for GANs with Latently Invertible Autoencoder , 2019 .

[16]  Bo Zhang,et al.  LIA: Latently Invertible Autoencoder with Adversarial Learning , 2019, ArXiv.

[17]  Bolei Zhou,et al.  Interpreting the Latent Space of GANs for Semantic Face Editing , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Chu-Song Chen,et al.  Cross-Age Reference Coding for Age-Invariant Face Recognition and Retrieval , 2014, ECCV.

[19]  Heng Wang,et al.  Face Aging Effect Simulation Using Hidden Factor Analysis Joint Sparse Representation , 2015, IEEE Transactions on Image Processing.

[20]  Tien D. Bui,et al.  Longitudinal Face Aging in the Wild - Recent Deep Learning Approaches , 2018, ArXiv.

[21]  Shiguang Shan,et al.  A Compositional and Dynamic Model for Face Aging , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Zhenan Sun,et al.  Attribute-Aware Face Aging With Wavelet-Based Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Tien D. Bui,et al.  Longitudinal Face Modeling via Temporal Deep Restricted Boltzmann Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Hanjiang Lai,et al.  Personalized Age Progression with Aging Dictionary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Jean-Luc Dugelay,et al.  Face aging with conditional generative adversarial networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[27]  Timothy F. Cootes,et al.  Toward Automatic Simulation of Aging Effects on Face Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Wen Gao,et al.  A Concatenational Graph Evolution Aging Model , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Nicu Sebe,et al.  Recurrent Face Aging , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Bernard Tiddeman,et al.  Prototyping and Transforming Facial Textures for Perception Research , 2001, IEEE Computer Graphics and Applications.

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  Sebastian Nowozin,et al.  Which Training Methods for GANs do actually Converge? , 2018, ICML.

[33]  Yang Song,et al.  Age Progression/Regression by Conditional Adversarial Autoencoder , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Ira Kemelmacher-Shlizerman,et al.  Illumination-Aware Age Progression , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Maneesh Kumar Singh,et al.  DRIT++: Diverse Image-to-Image Translation via Disentangled Representations , 2019, International Journal of Computer Vision.

[37]  Duncan Rowland,et al.  Manipulating facial appearance through shape and color , 1995, IEEE Computer Graphics and Applications.

[38]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Nadia Magnenat-Thalmann,et al.  Simulation of Skin Aging and Wrinkles with Cosmetics Insight , 2000, Computer Animation and Simulation.

[40]  Shiguang Shan,et al.  S2GAN: Share Aging Factors Across Ages and Share Aging Trends Among Individuals , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[41]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[42]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Jaakko Lehtinen,et al.  Few-Shot Unsupervised Image-to-Image Translation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Jianping Shi,et al.  Face Parsing via Recurrent Propagation , 2017, BMVC.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[47]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[48]  Rama Chellappa,et al.  Modeling shape and textural variations in aging faces , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[49]  Jung-Woo Ha,et al.  StarGAN v2: Diverse Image Synthesis for Multiple Domains , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Takaaki Kuratate,et al.  A simple method for modeling wrinkles on human skin , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[51]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.