Bridging the Aesthetic Gap: The Wild Beauty of Web Imagery

To provide good results, image search engines need to rank not just the most relevant images, but also the highest quality images. To surface beautiful pictures, existing computational aesthetic models are trained with datasets from photo contest websites, dominated by professional photos. Such models fail completely in real web scenarios, where images are extremely diverse in terms of quality and type (e.g. drawings, clip-art, etc). This work aims at bridging and understanding this "aesthetic gap". We collect a dataset of around 100K web images with `quality' and `type' (photo vs non-photo) annotations. We design a set of visual features to describe image pictorial characteristics, and deeply analyse the peculiar beauty of web images as opposed to appealing professional images. Finally, we build a set of computational aesthetic frameworks based on deep learning and hand-crafted features that take into account the diverse quality of web images, and show that they significantly outperform traditional computational aesthetics methods on our dataset.

[1]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[2]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Zhenyang Wu,et al.  Natural color image enhancement and evaluation algorithm based on human visual system , 2006, Comput. Vis. Image Underst..

[5]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[6]  Michael Freeman,et al.  The Photographer's Eye: Composition and Design for Better Digital Photos , 2007 .

[7]  Shih-Fu Chang,et al.  Lessons Learned from Online Classification of Photo-Realistic Computer Graphics and Photographs , 2007 .

[8]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[9]  Nuria Oliver,et al.  The role of tags and image aesthetics in social image search , 2009, WSM '09.

[10]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[11]  Xian-Sheng Hua,et al.  The role of attractiveness in web image search , 2011, ACM Multimedia.

[12]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[13]  Xiaogang Wang,et al.  Content-based photo quality assessment , 2011, 2011 International Conference on Computer Vision.

[14]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[15]  Jun Gao,et al.  Learning to predict the perceived visual quality of photos , 2011, 2011 International Conference on Computer Vision.

[16]  Jianxiong Xiao,et al.  What makes an image memorable? , 2011, CVPR 2011.

[17]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[18]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Luc Van Gool,et al.  The Interestingness of Images , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Rossano Schifanella,et al.  6 Seconds of Sound and Vision: Creativity in Micro-videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Raffay Hamid,et al.  What makes an image popular? , 2014, WWW.

[22]  James Zijun Wang,et al.  RAPID: Rating Pictorial Aesthetics using Deep Learning , 2014, ACM Multimedia.

[23]  Trevor Darrell,et al.  Recognizing Image Style , 2013, BMVC.

[24]  Rossano Schifanella,et al.  An Image Is Worth More than a Thousand Favorites: Surfacing the Hidden Beauty of Flickr Pictures , 2015, ICWSM.

[25]  Miriam Redi,et al.  The beauty of capturing faces: Rating the quality of digital portraits , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[26]  Hailin Jin,et al.  Composition-Preserving Deep Photo Aesthetics Assessment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Rossano Schifanella,et al.  Leveraging User Interaction Signals for Web Image Search , 2016, SIGIR.

[28]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .