Transferring Rich Deep Features for Facial Beauty Prediction

Feature extraction plays a significant part in computer vision tasks. In this paper, we propose a method which transfers rich deep features from a pretrained model on face verification task and feeds the features into Bayesian ridge regression algorithm for facial beauty prediction. We leverage the deep neural networks that extracts more abstract features from stacked layers. Through simple but effective feature fusion strategy, our method achieves improved or comparable performance on SCUT-FBP dataset and ECCV HotOrNot dataset. Our experiments demonstrate the effectiveness of the proposed method and clarify the inner interpretability of facial beauty perception.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Mohamed Elhoseny,et al.  Automatic removal of complex shadows from indoor videos using transfer learning and dynamic thresholding , 2017, Comput. Electr. Eng..

[5]  Jie Xu,et al.  Facial attractiveness prediction using psychologically inspired convolutional neural network (PI-CNN) , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[7]  Honglak Lee,et al.  Learning hierarchical representations for face verification with convolutional deep belief networks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[9]  David Zhang,et al.  Computer Models for Facial Beauty Analysis , 2016, Springer International Publishing.

[10]  David Zhang,et al.  Data-Driven Facial Beauty Analysis: Prediction, Retrieval and Manipulation , 2018 .

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Yihong Gong,et al.  Predicting Facial Beauty without Landmarks , 2010, ECCV.

[13]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[14]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[15]  Daniel Cohen-Or,et al.  A Humanlike Predictor of Facial Attractiveness , 2006, NIPS.

[16]  David Zhang,et al.  Quantitative analysis of human facial beauty using geometric features , 2011, Pattern Recognit..

[17]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Eytan Ruppin,et al.  Facial Attractiveness: Beauty and the Machine , 2006, Neural Computation.

[20]  Ming Shao,et al.  Attractive or Not?: Beauty Prediction with Attractiveness-Aware Encoders and Robust Late Fusion , 2014, ACM Multimedia.

[21]  Guangming Lu,et al.  A New Face Beauty Prediction Model based on Blocked LBP , 2016, VISIGRAPP.

[22]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[23]  Yinhua Liu,et al.  Deep self-taught learning for facial beauty prediction , 2014, Neurocomputing.

[24]  Jie Xu,et al.  SCUT-FBP: A Benchmark Dataset for Facial Beauty Perception , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[25]  D. Perrett,et al.  Facial shape and judgements of female attractiveness , 1994, Nature.

[26]  Xiaohui Yuan,et al.  Multi-level structured hybrid forest for joint head detection and pose estimation , 2017, Neurocomputing.

[27]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Jie Xu,et al.  A new humanlike facial attractiveness predictor with cascaded fine-tuning deep learning model , 2015, ArXiv.

[29]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.