Hierarchical Features Fusion for Image Aesthetics Assessment

Image aesthetics assessment is an interesting yet challenging topic which can be applied on numerous scenarios such as high quality image retrieval or recommendation systems. We propose a hierarchical features fusion aesthetic assessment (HFFAA) model for this task. HFFAA is a two-stream convolutional neural network (CNN) which is composed of two branches with heterogeneous and complementary aesthetic perceptual abilities. HFFAA learns the mapping from deep image representation into their ground-truth aesthetic labels (good or bad) in an end-to-end fashion. Extensive experiments demonstrate that the proposed model achieves superior performance on two widely evaluated public benchmark databases, i.e., CUHKPQ and AVA. We also validate the rationality of the designs of HFFAA through a series of ablation experiments.

[1]  W. Chu.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[2]  James Zijun Wang,et al.  Rating Image Aesthetics Using Deep Learning , 2015, IEEE Transactions on Multimedia.

[3]  Wei Luo,et al.  Content-Based Photo Quality Assessment , 2013, IEEE Trans. Multim..

[4]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Tao Mei,et al.  Query-Dependent Aesthetic Model With Deep Learning for Photo Quality Assessment , 2015, IEEE Transactions on Multimedia.

[6]  Le Wu,et al.  ILGNet: inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation , 2016, IET Comput. Vis..

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Hailin Jin,et al.  Composition-Preserving Deep Photo Aesthetics Assessment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Dexiang Deng,et al.  Predicting perceptual quality of images in realistic scenario using deep filter banks , 2018 .

[11]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Chong Luo,et al.  Multiple Level Feature-Based Universal Blind Image Quality Assessment Model , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[13]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Houqiang Li,et al.  Photo Quality Assessment with DCNN that Understands Image Well , 2015, MMM.

[15]  Xin Fu,et al.  Image Aesthetics Assessment Using Composite Features from off-the-Shelf Deep Models , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[16]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[18]  Xiaoou Tang,et al.  Image Aesthetic Assessment: An experimental survey , 2016, IEEE Signal Processing Magazine.

[19]  Yi Li,et al.  Finetuning Convolutional Neural Networks for visual aesthetics , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[20]  Xiangmin Xu,et al.  A multi-scene deep learning model for image aesthetic evaluation , 2016, Signal Process. Image Commun..

[21]  Mei-Chen Yeh,et al.  Relative features for photo quality assessment , 2012, 2012 19th IEEE International Conference on Image Processing.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.