Texton-based analysis of paintings

The visual examination of paintings is traditionally performed by skilled art historians using their eyes. Recent advances in intelligent systems may support art historians in determining the authenticity or date of creation of paintings. In this paper, we propose a technique for the examination of brushstroke structure that views the wildly overlapping brushstrokes as texture. The analysis of the painting texture is performed with the help of a texton codebook, i.e., a codebook of small prototypical textural patches. The texton codebook can be learned from a collection of paintings. Our textural analysis technique represents paintings in terms of histograms that measure the frequency by which the textons in the codebook occur in the painting (so-called texton histograms). We present experiments that show the validity and effectiveness of our technique for textural analysis on a collection of digitized high-resolution reproductions of paintings by Van Gogh and his contemporaries. As texton histograms cannot be easily be interpreted by art experts, the paper proposes to approaches to visualize the results on the textural analysis. The first approach visualizes the similarities between the histogram representations of paintings by employing a recently proposed dimensionality reduction technique, called t-SNE. We show that t-SNE reveals a clear separation of paintings created by Van Gogh and those created by other painters. In addition, the period of creation is faithfully reflected in the t-SNE visualizations. The second approach visualizes the similarities and differences between paintings by highlighting regions in a painting in which the textural structure of the painting is unusual. We illustrate the validity of this approach by means of an experiment in which we highlight regions in a painting by Monet that are not very "Van Gogh-like". Taken together, we believe the tools developed in this study are well capable of assisting for art historians in support of their study of paintings.

[1]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[2]  D. Eckstein,et al.  DENDROCHRONOLOGICAL DATING OF OAK PANELS OF DUTCH SEVENTEENTH -CENTURY PAINTINGS , 1970 .

[3]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[4]  David G. Stork,et al.  Did early Renaissance painters trace optical projections? Evidence pro and con , 2005, IS&T/SPIE Electronic Imaging.

[5]  Thomas Hurtut,et al.  2D artistic images analysis, a content-based survey , 2010 .

[6]  Siwei Lyu,et al.  A digital technique for art authentication , 2004, Proc. Natl. Acad. Sci. USA.

[7]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[8]  Miguel Á. Carreira-Perpiñán,et al.  The Elastic Embedding Algorithm for Dimensionality Reduction , 2010, ICML.

[9]  Kenneth M. Sayre,et al.  Machine recognition of handwritten words: A project report , 1973, Pattern Recognit..

[10]  K JohnsonMicah,et al.  Computer Vision, Image Analysis, and Master Art , 2006 .

[11]  Eric O. Postma,et al.  Computer analysis of Van Gogh's complementary colours , 2007, Pattern Recognit. Lett..

[12]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[13]  MalikJitendra,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001 .

[14]  Lambert Schomaker,et al.  Using codebooks of fragmented connected-component contours in forensic and historic writer identification , 2007, Pattern Recognit. Lett..

[15]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[16]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[17]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[18]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[19]  D. Stork.  Optics and realism in Renaissance art. , 2004, Scientific American.

[20]  N. Kingsbury.  Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[21]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[22]  Jia Li,et al.  Image processing for artist identification , 2008, IEEE Signal Processing Magazine.

[23]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Laurens van der Maaten,et al.  Learning a Parametric Embedding by Preserving Local Structure , 2009, AISTATS.

[25]  David G. Stork,et al.  Estimating the location of illuminants in realist master paintings Computer image analysis addresses a debate in art history of the Baroque , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[26]  Robert Sablatnig,et al.  Hierarchical classification of paintings using face- and brush stroke models , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[27]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[28]  David G. Stork,et al.  Did Georges de la Tour use optical projections while painting Christ in the Carpenter’s Studio? , 2005, IS&T/SPIE Electronic Imaging.

[29]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[30]  C.R. Johnson,et al.  Algorithms for Old Master painting canvas thread counting from x-rays , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[31]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  R. Näätänen,et al.  Development of language-specific phoneme representations in the infant brain , 1998, Nature Neuroscience.

[33]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[34]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[35]  Geoffrey E. Hinton,et al.  Topographic Product Models Applied to Natural Scene Statistics , 2006, Neural Computation.

[36]  Antonio Criminisi,et al.  Bringing Pictorial Space to Life: computer techniques for the analysis of paintings , 2002 .

[37]  Gertjan J. Burghouts,et al.  Color Textons for Texture Recognition , 2006, BMVC.

[38]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[39]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[40]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[41]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[42]  David G. Stork,et al.  Recovering layers of brush strokes through statistical analysis of color and shape: an application to van Gogh's Self portrait with grey felt hat , 2008, Electronic Imaging.

[43]  Liliane Masschelein-Kleiner,et al.  Radiocarbon dating of canvas paintings: two case studies , 1998 .

[44]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[45]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.