Pseudo-labelling and Meta Reweighting Learning for Image Aesthetic Quality Assessment

In the tasks of image aesthetic quality evaluation, it is difficult to reach both the high score area and low score area due to the normal distribution of aesthetic datasets. To reduce the error in labeling and solve the problem of normal data distribution, we propose a new aesthetic mixed dataset with classification and regression called AMD-CR, and we train a meta reweighting network to reweight the loss of training data differently. In addition, we provide a training strategy acccording to different stages, based on pseudo labels of the binary classification task, and then we use it for aesthetic training acccording to different stages in classification and regression tasks. In the construction of the network structure, we construct an aesthetic adaptive block (AAB) structure that can adapt to any size of the input images. Besides, we also use the efficient channel attention (ECA) to strengthen the feature extracting ability of each task. The experimental result shows that our method improves 0.1112 compared with the conventional methods in SROCC. The method can also help to find best aesthetic path planning for unmanned aerial vehicles (UAV) and vehicles.

[1]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[2]  Michael S. Gazzaniga,et al.  Interhemispheric relationships: the neocortical commissures; syndromes of hemisphere disconnection , 1969 .

[3]  Xin Jin,et al.  Incremental Learning of Multi-Tasking Networks for Aesthetic Radar Map Prediction , 2019, IEEE Access.

[4]  Anselm Brachmann,et al.  Computational and Experimental Approaches to Visual Aesthetics , 2017, Front. Comput. Neurosci..

[5]  Chu-Song Chen,et al.  Aesthetic Critiques Generation for Photos , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Qilong Wang,et al.  ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jingrui He,et al.  Classification of Digital Photos Taken by Photographers or Home Users , 2004, PCM.

[8]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[9]  Xin Jin,et al.  Deep Multimodality Learning for UAV Video Aesthetic Quality Assessment , 2020, IEEE Transactions on Multimedia.

[10]  Chong Wang,et al.  Visual aesthetic quality assessment with a regression model , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[11]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[12]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[13]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[14]  Radomír Mech,et al.  Photo Aesthetics Ranking Network with Attributes and Content Adaptation , 2016, ECCV.

[15]  Kenji Doya,et al.  Meta-learning in Reinforcement Learning , 2003, Neural Networks.

[16]  Chang-Su Kim,et al.  Image Aesthetic Assessment Based on Pairwise Comparison ­ A Unified Approach to Score Regression, Binary Classification, and Personalization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[18]  Kede Ma,et al.  Perceptual Quality Assessment of Smartphone Photography , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  James Zijun Wang,et al.  RAPID: Rating Pictorial Aesthetics using Deep Learning , 2014, ACM Multimedia.

[21]  Xiaogang Wang,et al.  Content-based photo quality assessment , 2011, 2011 International Conference on Computer Vision.

[22]  Dong Liu,et al.  Composition-Aware Image Aesthetics Assessment , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Jun Gao,et al.  Learning to predict the perceived visual quality of photos , 2011, 2011 International Conference on Computer Vision.

[24]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[25]  Qi Xie,et al.  Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting , 2019, NeurIPS.

[26]  Le Wu,et al.  Predicting Aesthetic Radar Map Using a Hierarchical Multi-task Network , 2018, PRCV.

[27]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[28]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[30]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[31]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[32]  Radomír Mech,et al.  Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).