Image Style Classification Based on Learnt Deep Correlation Features

This paper presents a comprehensive study of deep correlation features on image style classification. Inspired by that, correlation between feature maps can effectively describe image texture, and we design various correlations and transform them into style vectors, and investigate classification performance brought by different variants. In addition to intralayer correlation, interlayer correlation is proposed as well, and its effectiveness is verified. After showing the effectiveness of deep correlation features, we further propose a learning framework to automatically learn correlations between feature maps. Through extensive experiments on image style classification and artist classification, we demonstrate that the proposed learnt deep correlation features outperform several variants of convolutional neural network features by a large margin, and achieve the state-of-the-art performance.

[1]  Amit Jain,et al.  A multiscale representation including opponent color features for texture recognition , 1998, IEEE Trans. Image Process..

[2]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[3]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[4]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[5]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[6]  Rongrong Ji,et al.  Large-scale visual sentiment ontology and detectors using adjective noun pairs , 2013, ACM Multimedia.

[7]  陈世峰.  Style Transfer via Image Component Analysis , 2013 .

[8]  Jorma Laaksonen,et al.  Content-Based Prediction of Movie Style, Aesthetics, and Affect: Data Set and Baseline Experiments , 2014, IEEE Transactions on Multimedia.

[9]  Trevor Darrell,et al.  Recognizing Image Style , 2013, BMVC.

[10]  Michael Felsberg,et al.  Painting-91: a large scale database for computational painting categorization , 2014, Machine Vision and Applications.

[11]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[12]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[13]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  James Zijun Wang,et al.  Rating Image Aesthetics Using Deep Learning , 2015, IEEE Transactions on Multimedia.

[15]  Wen-Huang Cheng,et al.  Gestalt Rule Feature Points , 2015, IEEE Transactions on Multimedia.

[16]  Wenwu Zhu,et al.  Learning Compact Hash Codes for Multimodal Representations Using Orthogonal Deep Structure , 2015, IEEE Transactions on Multimedia.

[17]  Tsuhan Chen,et al.  Cross-layer features in convolutional neural networks for generic classification tasks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[20]  Radomír Mech,et al.  Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Anderson Rocha,et al.  From impressionism to expressionism: Automatically identifying van Gogh's paintings , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[22]  Wei-Ta Chu,et al.  Deep Correlation Features for Image Style Classification , 2016, ACM Multimedia.

[23]  Alex J. Champandard,et al.  Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks , 2016, ArXiv.

[24]  Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation , 2014, IEEE Transactions on Multimedia.

[25]  Thomas Brox,et al.  Artistic Style Transfer for Videos , 2016, GCPR.

[26]  Keiji Yanai,et al.  CNN-based Style Vector for Style Image Retrieval , 2016, ICMR.

[27]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[29]  Yan Wang,et al.  On Branded Handbag Recognition , 2016, IEEE Transactions on Multimedia.

[30]  Yu-Chiang Frank Wang,et al.  Style retrieval from natural images , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Andrea Vedaldi,et al.  Texture Networks: Feed-forward Synthesis of Textures and Stylized Images , 2016, ICML.

[32]  Andrea Vedaldi,et al.  Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Jiaying Liu,et al.  Demystifying Neural Style Transfer , 2017, IJCAI.

[34]  Wataru Shimoda,et al.  DeepStyleCam: A Real-Time Style Transfer App on iOS , 2017, MMM.

[35]  Jonathon Shlens,et al.  A Learned Representation For Artistic Style , 2016, ICLR.