The beauty of capturing faces: Rating the quality of digital portraits

Digital portrait photographs are everywhere, and while the number of face pictures keeps growing, not much work has been done to on automatic portrait beauty assessment. In this paper, we design a specific framework to automatically evaluate the beauty of digital portraits. To this end, we procure a large dataset of face images annotated not only with aesthetic scores but also with information about the traits of the subject portrayed. We design a set of visual features based on portrait photography literature, and extensively analyze their relation with portrait beauty, exposing interesting findings about what makes a portrait beautiful. We find that the beauty of a portrait is linked to its artistic value, and independent from age, race and gender of the subject. We also show that a classifier trained with our features to separate beautiful portraits from non-beautiful portraits outperforms generic aesthetic classifiers.

[1]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[2]  Bernard Mérialdo,et al.  Saliency moments for image categorization , 2011, ICMR.

[3]  Tsuhan Chen,et al.  Aesthetic quality assessment of consumer photos with faces , 2010, 2010 IEEE International Conference on Image Processing.

[4]  Tomás Pevný,et al.  Steganalysis by subtractive pixel adjacency matrix , 2010, IEEE Trans. Inf. Forensics Secur..

[5]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Eric Gilbert,et al.  Faces engage us: photos with faces attract more likes and comments on Instagram , 2014, CHI.

[7]  Hai-Dong Yuan,et al.  Blind Forensics of Median Filtering in Digital Images , 2011, IEEE Transactions on Information Forensics and Security.

[8]  Jiebo Luo,et al.  Aesthetics and Emotions in Images , 2011, IEEE Signal Processing Magazine.

[9]  Jianxiong Xiao,et al.  What makes an image memorable? , 2011, CVPR 2011.

[10]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[11]  Mubarak Shah,et al.  A framework for photo-quality assessment and enhancement based on visual aesthetics , 2010, ACM Multimedia.

[12]  Mateu Sbert,et al.  Conceptualizing Birkhoff's Aesthetic Measure Using Shannon Entropy and Kolmogorov Complexity , 2007, CAe.

[13]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[14]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Luc Van Gool,et al.  The Interestingness of Images , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Marcelo H. Ang,et al.  Practical issues in pixel-based autofocusing for machine vision , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[18]  Nuria Oliver,et al.  Towards Category-Based Aesthetic Models of Photographs , 2012, MMM.

[19]  J. Weiser,et al.  Phototherapy Techniques: Exploring the Secrets of Personal Snapshots and Family Albums , 1993 .

[20]  Jan P. Allebach,et al.  Feature design for aesthetic inference on photos with faces , 2013, 2013 IEEE International Conference on Image Processing.

[21]  Jun Gao,et al.  Learning to predict the perceived visual quality of photos , 2011, 2011 International Conference on Computer Vision.

[22]  Hao Su,et al.  Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification , 2010, NIPS.

[23]  Shehroz S. Khan,et al.  Evaluating visual aesthetics in photographic portraiture , 2012, CAe '12.

[24]  Tomás Pevný,et al.  Steganalysis by Subtractive Pixel Adjacency Matrix , 2009, IEEE Transactions on Information Forensics and Security.

[25]  Albert A. Michelson,et al.  Studies in Optics , 1995 .

[26]  John Child,et al.  Studio Photography: Essential Skills , 1999 .

[27]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[28]  Rongrong Ji,et al.  Large-scale visual sentiment ontology and detectors using adjective noun pairs , 2013, ACM Multimedia.

[29]  Bernard Mérialdo,et al.  Where is the beauty?: retrieving appealing VideoScenes by learning Flickr-based graded judgments , 2012, ACM Multimedia.

[30]  Jonathon S. Hare,et al.  Privacy-aware image classification and search , 2012, SIGIR '12.

[31]  Ralph Gross,et al.  An Image Preprocessing Algorithm for Illumination Invariant Face Recognition , 2003, AVBPA.

[32]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[35]  K. Scherer,et al.  The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance , 2011, Behavior research methods.

[36]  Xiaogang Wang,et al.  Content-based photo quality assessment , 2011, 2011 International Conference on Computer Vision.

[37]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[38]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.