Synthesized computational aesthetic evaluation of photos

Assessing aesthetic appeal of images is a highly subjective task which has attracted a lot of interests recently. It is an interdisciplinary subject related to art, psychology, and computer vision. In this paper, we systematically study prior researches of feature extraction in this area, and category them into four groups, low level, rule based, information theory, and visual attention. In each group, the effectiveness and limitations of existing features are examined. Based on the analysis, we propose a comprehensive feature set, which include 16 novel features and 70 well proved features. With this feature set, we build the system under machine learning scheme consisting of an SVM based classifier to estimate if an image is high aesthetic or low aesthetic. The experiments are conducted on public datasets show that our comprehensive feature set outperforms conventional models that concentrate mainly on certain types of features. The combination of our features produces a promising classification accuracy of 82.4% and a good performance comparable to aesthetic rating of human. Finally, we implemented the proposed evaluation system on mobile devices. It can provide real-time feedback to help users capture appealing photos.

[1]  Zurek,et al.  Algorithmic randomness and physical entropy. , 1989, Physical review. A, General physics.

[2]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Mateu Sbert,et al.  Informational Aesthetics Measures , 2008, IEEE Computer Graphics and Applications.

[4]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[5]  Meng Wang,et al.  Visual query suggestion , 2010, ACM Trans. Multim. Comput. Commun. Appl..

[6]  W. Chu Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[7]  Chu-Song Chen,et al.  Intelligent Photographing Interface with On-Device Aesthetic Quality Assessment , 2012, ACCV Workshops.

[8]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[9]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[10]  Mu Qiao,et al.  OSCAR: On-Site Composition and Aesthetics Feedback Through Exemplars for Photographers , 2012, International Journal of Computer Vision.

[11]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[12]  Xuelong Li,et al.  Image esthetic assessment using both hand-crafting and semantic features , 2014, Neurocomputing.

[13]  Kok-Lim Low,et al.  Saliency-enhanced image aesthetics class prediction , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[14]  Richard Lind Attention and the Aesthetic Object , 1980 .

[15]  Tao Mei,et al.  Personalized Video Recommendation through Graph Propagation , 2014, TOMM.

[16]  Yi Yang,et al.  Interactive Video Indexing With Statistical Active Learning , 2012, IEEE Transactions on Multimedia.

[17]  Xuelong Li,et al.  Optimized graph-based segmentation for ultrasound images , 2014, Neurocomputing.

[18]  Xuelong Li,et al.  Automatic segmentation of breast lesions for interaction in ultrasonic computer-aided diagnosis , 2015, Inf. Sci..

[19]  Siwei Luo,et al.  Computers and Mathematics with Applications Fuzzy Aesthetic Semantics Description and Extraction for Art Image Retrieval , 2022 .

[20]  Mubarak Shah,et al.  A framework for photo-quality assessment and enhancement based on visual aesthetics , 2010, ACM Multimedia.

[21]  J. Daugman Spatial visual channels in the fourier plane , 1984, Vision Research.

[22]  Florian Hönig Defining Computational Aesthetics , 2005, CAe.

[23]  Tsuhan Chen,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[24]  James Ze Wang,et al.  ACQUINE: aesthetic quality inference engine - real-time automatic rating of photo aesthetics , 2010, MIR '10.

[25]  Wei-Ning Wang,et al.  Image emotional semantic query based on color semantic description , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[26]  Victor Ciesielski,et al.  Finding Image Features Associated with High Aesthetic Value by Machine Learning , 2013, EvoMUSART.

[27]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[28]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[29]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[30]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[31]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[32]  Jiebo Luo,et al.  Aesthetics and Emotions in Images , 2011, IEEE Signal Processing Magazine.

[33]  Wei Luo,et al.  Content-Based Photo Quality Assessment , 2013, IEEE Trans. Multim..

[34]  Penousal Machado,et al.  Aesthetic Classification and Sorting Based on Image Compression , 2011, EvoApplications.

[35]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Nuria Oliver,et al.  Supporting personal photo storytelling for social albums , 2010, ACM Multimedia.

[37]  Yili Liu,et al.  Computational modeling and experimental investigation of effects of compositional elements on interface and design aesthetics , 2006, Int. J. Hum. Comput. Stud..

[38]  Tao Mei,et al.  Probabilistic Multimodality Fusion for Event based Home Photo Clustering , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[39]  Xuelong Li,et al.  GA-SIFT: A new scale invariant feature transform for multispectral image using geometric algebra , 2014, Inf. Sci..

[40]  Xuelong Li,et al.  Exploiting Local Coherent Patterns for Unsupervised Feature Ranking , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[41]  Pere Obrador,et al.  The role of image composition in image aesthetics , 2010, 2010 IEEE International Conference on Image Processing.

[42]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[43]  Penousal Machado,et al.  Computing Aethetics , 1998, SBIA.