Deep Conditional Color Harmony Model for Image Aesthetic Assessment

As one of the important features, color provides plenty useful information to represent images. Thus, color harmony, which is defined as “two or more colors are sensed together as a single, pleasing, collective impression” [1], can also be served as a fundamental feature and plays a key role to determine the aesthetics quality of images. To reveal the inherent color harmony attribute within patch and the harmonious relations between image patches which construct the pleasing colorful images, we designed a conditional random field (CRF) based color harmony model in this paper to accomplish the image aesthetic assessment tasks. Unlike the previous learning based color harmony models, we used deep neural networks to obtain the coherence properties between original image patch pairs, and embedded these relations along with each patch's own color harmony characteristic into a CRF to measure the harmony scores of the entire image. The experimental results on a public dataset show that the proposed deep conditional color harmony model is superior to the existing color harmony models in respect of the image aesthetic assessment.

[1]  Lionel Oisel,et al.  Harmony-guided image editing , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[2]  L. Ou,et al.  Additivity of colour harmony , 2011 .

[3]  Linda Holtzschue.  Understanding Color: An Introduction for Designers , 1994 .

[4]  Chu-Song Chen,et al.  Assessment of photo aesthetics with efficiency , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[7]  Weiqi Jin,et al.  Objective color harmony assessment for visible and infrared color fusion images of typical scenes , 2012 .

[8]  Xinmei Tian,et al.  Learning Relative Aesthetic Quality with a Pairwise Approach , 2016, MMM.

[9]  Peng Lu,et al.  Towards aesthetics of image: A Bayesian framework for color harmony modeling , 2015, Signal Process. Image Commun..

[10]  Xiaoou Tang,et al.  Image Aesthetic Assessment: An experimental survey , 2016, IEEE Signal Processing Magazine.

[11]  W. Chu.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[12]  D. Spencer,et al.  Geometric Formulation of Classical Color Harmony , 1944 .

[13]  Xujun Peng,et al.  Discovering Harmony: A Hierarchical Colour Harmony Model for Aesthetics Assessment , 2014, ACCV.

[14]  Ran He,et al.  Deep Aesthetic Quality Assessment With Semantic Information , 2016, IEEE Transactions on Image Processing.

[15]  C. A. Garabedian.  Birkhoff on Aesthetic Measure , 1934 .

[16]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[17]  A. Munsell,et al.  A grammar of color : a basic treatise on the color system of Albert H. Munsell , 1969 .

[18]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Xiangmin Xu,et al.  A multi-scene deep learning model for image aesthetic evaluation , 2016, Signal Process. Image Commun..

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  James Zijun Wang,et al.  Rating Image Aesthetics Using Deep Learning , 2015, IEEE Transactions on Multimedia.

[22]  Nikos Komodakis,et al.  Learning to compare image patches via convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Xujun Peng,et al.  An EL-LDA based general color harmony model for photo aesthetics assessment , 2016, Signal Process..

[24]  Xujun Peng,et al.  Image color harmony modeling through neighbored co-occurrence colors , 2016, Neurocomputing.

[25]  Shao-Yi Chien,et al.  Scenic photo quality assessment with bag of aesthetics-preserving features , 2011, ACM Multimedia.

[26]  Hailin Jin,et al.  Composition-Preserving Deep Photo Aesthetics Assessment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Peng Lu,et al.  Learning self-adaptive color harmony model for aesthetic quality classification , 2015, International Conference on Graphic and Image Processing.

[28]  Wei Luo,et al.  Content-Based Photo Quality Assessment , 2013, IEEE Trans. Multim..

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Shuang Ma,et al.  Pose Maker: A Pose Recommendation System for Person in the Landscape Photographing , 2014, ACM Multimedia.

[31]  Krystian Mikolajczyk,et al.  Ranking Images Based on Aesthetic Qualities , 2014, 2014 22nd International Conference on Pattern Recognition.

[32]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[33]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).