2 Related works 2 . 1 Automatic painting analysis

Selective classification (or rejection based classification) has been proved useful in many applications. In this paper we describe a selective clustering framework with reject option to carry out large-scale digital arts analysis. With the help of deep learning techniques, we extract content-style features from a pre-trained convolutional network for the paintings. By proposing a rejection mechanism under Bayesian framework, we focus on selecting style-oriented representative paintings of an artist, which is an interesting and challenging cultural heritage application. Two kinds of samples are rejected during the rejection based robust continuous clustering process. Representative paintings are selected during the selective clustering phase. Visual qualitative analysis on small painting set and large scale quantitative experiments on a subset of Wikiart show that the proposed rejection based selective clustering approach outperforms the standard clustering methods.

[1]  Lior Shamir,et al.  Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art , 2010, TAP.

[2]  Pramod K. Varshney,et al.  Multi-Object Classification via Crowdsourcing With a Reject Option , 2016, IEEE Transactions on Signal Processing.

[3]  Ling-Yu Duan,et al.  From Part to Whole: Who is Behind the Painting? , 2017, ACM Multimedia.

[4]  Pinar Duygulu Sahin,et al.  FAME: Face Association through Model Evolution , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Gang Hua,et al.  Visual attribute transfer through deep image analogy , 2017, ACM Trans. Graph..

[6]  Seung-Woo Son,et al.  Large-Scale Quantitative Analysis of Painting Arts , 2014, Scientific Reports.

[7]  J. M. Hughes,et al.  Quantification of artistic style through sparse coding analysis in the drawings of Pieter Bruegel the Elder , 2010, Proceedings of the National Academy of Sciences.

[8]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[9]  Peter L. Bartlett,et al.  Classification with a Reject Option using a Hinge Loss , 2008, J. Mach. Learn. Res..

[10]  M. Fiedler.  Algebraic connectivity of graphs , 1973 .

[11]  Eric O. Postma,et al.  Toward Discovery of the Artist's Style: Learning to recognize artists by their artworks , 2015, IEEE Signal Processing Magazine.

[12]  Kang Zhang,et al.  Computational Aesthetics: On the Complexity of Computer-Generated Paintings , 2012, Leonardo.

[13]  Pinar Duygulu Sahin,et al.  DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books , 2017, ICMR.

[14]  Emmanuel J. Candès,et al.  Robust Subspace Clustering , 2013, ArXiv.

[15]  Ravneet Singh Arora,et al.  Towards automated classification of fine-art painting style: A comparative study , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[16]  Yves Grandvalet,et al.  Support Vector Machines with a Reject Option , 2008, NIPS.

[17]  Wei-Ta Chu,et al.  Deep Correlation Features for Image Style Classification , 2016, ACM Multimedia.

[18]  Mei-Chen Yeh,et al.  Artist-based Classification via Deep Learning with Multi-scale Weighted Pooling , 2016, ACM Multimedia.

[19]  Bao-Gang Hu,et al.  Learning in the Class Imbalance Problem When Costs are Unknown for Errors and Rejects , 2012, 2012 IEEE 12th International Conference on Data Mining Workshops.

[20]  Yao Li,et al.  Mining Mid-level Visual Patterns with Deep CNN Activations , 2015, International Journal of Computer Vision.

[21]  Nicu Sebe,et al.  Who's Afraid of Itten: Using the Art Theory of Color Combination to Analyze Emotions in Abstract Paintings , 2015, ACM Multimedia.

[22]  Larry S. Davis,et al.  Representing Videos Using Mid-level Discriminative Patches , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  James Ze Wang,et al.  Digital imagery for significant cultural and historical materials , 2005, International Journal on Digital Libraries.

[24]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  James She,et al.  DeepArt: Learning Joint Representations of Visual Arts , 2017, ACM Multimedia.

[26]  Ryan P. Adams,et al.  Clustering with a Reject Option: Interactive Clustering as Bayesian Prior Elicitation , 2016, ICML 2016.

[27]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[28]  Richard P. Taylor,et al.  Fractal analysis of Pollock's drip paintings , 1999, Nature.

[29]  Jonathon Shlens,et al.  A Learned Representation For Artistic Style , 2016, ICLR.

[30]  Vladlen Koltun,et al.  Robust continuous clustering , 2017, Proceedings of the National Academy of Sciences.

[31]  J. Kittler,et al.  Multistage pattern recognition with reject option , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[32]  Nicu Sebe,et al.  Looking at Mondrian's Victory Boogie-Woogie: What Do I Feel? , 2015, IJCAI.

[33]  Radu Herbei,et al.  Classification with reject option , 2006 .

[34]  Jialie Shen,et al.  Stochastic modeling western paintings for effective classification , 2009, Pattern Recognit..

[35]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[37]  Fabio Roli,et al.  Reject option with multiple thresholds , 2000, Pattern Recognit..

[38]  Daniel Cohen-Or,et al.  Co-locating style-defining elements on 3D shapes , 2017, TOGS.

[39]  Robert P. W. Duin,et al.  Growing a multi-class classifier with a reject option , 2008, Pattern Recognit. Lett..

[40]  Nikhil R. Pal,et al.  How to make a neural network say "Don't know" , 2018, Inf. Sci..

[41]  Stefan Winkler,et al.  Inferring Painting Style with Multi-Task Dictionary Learning , 2015, IJCAI.

[42]  Vipin Kumar,et al.  Chameleon: Hierarchical Clustering Using Dynamic Modeling , 1999, Computer.

[43]  Ran El-Yaniv,et al.  Selective Classification for Deep Neural Networks , 2017, NIPS.