Chronological classification of ancient paintings using appearance and shape features

A method for classifying ancient paintings in chronology.A bag-of-visual-words approach that uses appearance features and shape features.A deep-learning network that refines the appearance features.Algorithm evaluation using a collection of 660 ancient painting images. Ancient paintings are valuable for historians and archeologists to study the humanities, customs and economy of the corresponding eras. For this purpose, it is important to first determine the era in which a painting was drawn. This problem can be very challenging when the paintings from different eras present a same topic and only show subtle difference in terms of the painting styles. In this paper, we propose a novel computational approach to address this problem by using the appearance and shape features extracted from the paintings. In this approach, we first extract the appearance and shape features using the SIFT and kAS descriptors, respectively. We then encode these features with deep learning in an unsupervised way. Finally, we combine all the features in the form of bag-of-visual-words and train a classifier in a supervised fashion. In the experiments, we collect 660 Flying-Apsaras paintings from Mogao Grottoes in Dunhuang, China and classify them into three different eras, with very promising results.

[1]  Lior Shamir,et al.  Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art , 2010, TAP.

[2]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[3]  Koen Vanhoof,et al.  Features for Art Painting Classification Based on Vector Quantization of MPEG-7 Descriptors , 2010, ICDEM.

[4]  Morten Nielsen,et al.  Stylometry of paintings using hidden Markov modelling of contourlet transforms , 2013, Signal Process..

[5]  Oguz Icoglu,et al.  Classification and indexing of paintings based on art movements , 2004, 2004 12th European Signal Processing Conference.

[6]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Li Yi,et al.  Object Detection Using Shape Codebook , 2007, BMVC.

[8]  Cordelia Schmid,et al.  Learning Color Names from Real-World Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Frédéric Jurie,et al.  Groups of Adjacent Contour Segments for Object Detection , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Bryan Pardo,et al.  Classifying paintings by artistic genre: An analysis of features & classifiers , 2009, 2009 IEEE International Workshop on Multimedia Signal Processing.

[11]  Sung-Hyuk Cha,et al.  The classification of style in fine-art painting , 2005 .

[12]  James Ze Wang,et al.  Studying digital imagery of ancient paintings by mixtures of stochastic models , 2004, IEEE Transactions on Image Processing.

[13]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[14]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[15]  Peter Stanchev,et al.  Analysis of the Distributions of Color Characteristics in Art Painting Images , 2008, Serdica Journal of Computing.

[16]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[17]  Sung-Hyuk Cha,et al.  A graphical user interface for a fine-art painting image retrieval system , 2004, MIR '04.

[18]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[20]  Osman N. Ucan,et al.  SEPARATION OF ORIGINAL PAINTINGS OF MATISSE AND HIS FAKES USING WAVELET AND ARTIFICIAL NEURAL NETWORKS , 2009 .

[21]  David G. Stork,et al.  Computer Vision and Computer Graphics Analysis of Paintings and Drawings: An Introduction to the Literature , 2009, CAIP.

[22]  Robert Sablatnig,et al.  Hierarchical classification of paintings using face- and brush stroke models , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[23]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[24]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[25]  Corneliu Florea,et al.  Perceptually-Inspired Artistic Genre Identification System in Digitized Painting Collections , 2013, SCIA.

[26]  Oguz Icoglu,et al.  Content-based access to art paintings , 2005, IEEE International Conference on Image Processing 2005.

[27]  Paul H. Lewis,et al.  An integrated content and metadata based retrieval system for art , 2004, IEEE Transactions on Image Processing.

[28]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[29]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Daniel J. Graham,et al.  Statistics, vision, and the analysis of artistic style , 2012 .

[31]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[32]  Ravneet Singh Arora,et al.  Towards automated classification of fine-art painting style: A comparative study , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[33]  Florent Perronnin,et al.  An analysis of the relationship between painters based on their work , 2008, 2008 15th IEEE International Conference on Image Processing.

[34]  Y. Liu,et al.  Bilinear deep learning for image classification , 2011, ACM Multimedia.