Automatic analysis of artistic paintings using information-based measures

The artistic community is increasingly relying on automatic computational analysis for authentication and classification of artistic paintings. In this paper, we identify hidden patterns and relationships present in artistic paintings by analysing their complexity, a measure that quantifies the sum of characteristics of an object. Specifically, we apply Normalized Compression (NC) and the Block Decomposition Method (BDM) to a dataset of 4,266 paintings from 91 authors and examine the potential of these information-based measures as descriptors of artistic paintings. Both measures consistently described the equivalent types of paintings, authors, and artistic movements. Moreover, combining the NC with a measure of the roughness of the paintings creates an efficient stylistic descriptor. Furthermore, by quantifying the local information of each painting, we define a fingerprint that describes critical information regarding the artists’ style, their artistic influences, and shared techniques. More fundamentally, this information describes how each author typically composes and distributes the elements across the canvas and, therefore, how their work is perceived. Finally, we demonstrate that regional complexity and two-point height difference correlation function are useful auxiliary features that improve current methodologies in style and author classification of artistic paintings. The whole study is supported by an extensive website (http://panther.web.ua.pt) for fast author characterization and authentication.

[1]  Paul M. B. Vitányi,et al.  Automatic Extraction of Meaning from the Web , 2006, 2006 IEEE International Symposium on Information Theory.

[2]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[3]  Manuel Cebrián,et al.  The Normalized Compression Distance Is Resistant to Noise , 2007, IEEE Transactions on Information Theory.

[4]  David G. Stork,et al.  Image analysis of paintings by computer graphics synthesis: an investigation of the illumination in Georges de la Tour's Christ in the carpenter's studio , 2008, Electronic Imaging.

[5]  R. Niven.  Combinatorial entropies and statistics , 2009, 0902.3038.

[6]  António M. Lopes,et al.  Artistic painting: A fractional calculus perspective , 2019, Applied Mathematical Modelling.

[7]  Armando J. Pinho,et al.  On the Approximation of the Kolmogorov Complexity for DNA Sequences , 2017, IbPRIA.

[8]  Armando J. Pinho,et al.  Inverted-repeats-aware finite-context models for DNA coding , 2008, 2008 16th European Signal Processing Conference.

[9]  Patrick Le Callet,et al.  Computational modeling of artistic intention: Quantify lighting surprise for painting analysis , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[10]  Diogo Pratas,et al.  AC: A Compression Tool for Amino Acid Sequences , 2019, Interdisciplinary Sciences: Computational Life Sciences.

[11]  Antonio Restivo,et al.  A New Combinatorial Approach to Sequence Comparison , 2007, Theory of Computing Systems.

[12]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[13]  Paul M. B. Vitányi,et al.  Clustering by compression , 2003, IEEE Transactions on Information Theory.

[14]  João Bernardes,et al.  Entropy and compression: two measures of complexity. , 2013, Journal of evaluation in clinical practice.

[15]  Steven de Rooij,et al.  A Safe Approximation for Kolmogorov Complexity , 2014, ALT.

[16]  Armando J. Pinho,et al.  On the Detection of Unknown Locally Repeating Patterns in Images , 2012, ICIAR.

[17]  Jean-Paul Delahaye,et al.  Human behavioral complexity peaks at age 25 , 2017, PLoS Comput. Biol..

[18]  Hector Zenil,et al.  A Computable Measure of Algorithmic Probability by Finite Approximations , 2015, Complex..

[19]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[20]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[21]  Robert R. Sokal,et al.  A statistical method for evaluating systematic relationships , 1958 .

[22]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[23]  Wei-Ta Chu,et al.  Image Style Classification Based on Learnt Deep Correlation Features , 2018, IEEE Transactions on Multimedia.

[24]  Matthew V. Mahoney,et al.  Adaptive weighing of context models for lossless data compression , 2005 .

[25]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[26]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[27]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[28]  Seung-Woo Son,et al.  Large-Scale Quantitative Analysis of Painting Arts , 2014, Scientific Reports.

[29]  Paul M. B. Vitányi,et al.  Normalized Compression Distance of Multisets with Applications , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Siwei Lyu,et al.  A digital technique for art authentication , 2004, Proc. Natl. Acad. Sci. USA.

[31]  Richard P. Taylor,et al.  Fractal analysis of Pollock's drip paintings , 1999, Nature.

[32]  Jia Li,et al.  Image processing for artist identification , 2008, IEEE Signal Processing Magazine.

[33]  Nikolaos G. Bourbakis,et al.  Lossless compression and information hiding in images , 2004, Pattern Recognit..

[34]  J. M. Hughes,et al.  Quantification of artistic style through sparse coding analysis in the drawings of Pieter Bruegel the Elder , 2010, Proceedings of the National Academy of Sciences.

[35]  Ian H. Witten,et al.  Arithmetic coding revisited , 1998, TOIS.

[36]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[37]  Bronwen Brown.  Secret Knowledge: Rediscovering the Lost Techniques of the Old Masters , 2002 .

[38]  James She,et al.  DeepArt: Learning Joint Representations of Visual Arts , 2017, ACM Multimedia.

[39]  Jean-Paul Delahaye,et al.  Calculating Kolmogorov Complexity from the Output Frequency Distributions of Small Turing Machines , 2012, PloS one.

[40]  Aaron Hertzmann,et al.  Can Computers Create Art? , 2018, ArXiv.

[41]  Armando J. Pinho,et al.  A Method to Detect Repeated Unknown Patterns in an Image , 2014, ICIAR.

[42]  Robert Sablatnig,et al.  Estimating the original drawing trace of painted strokes , 2008, Electronic Imaging.

[43]  R. Weisberg.  Creativity: Understanding Innovation in Problem Solving, Science, Invention, and the Arts , 2006 .

[44]  Jean-Paul Delahaye,et al.  Numerical evaluation of algorithmic complexity for short strings: A glance into the innermost structure of randomness , 2011, Appl. Math. Comput..

[45]  Alexander N. Rybalov.  On the strongly generic undecidability of the Halting Problem , 2007, Theor. Comput. Sci..

[46]  Hector Zenil,et al.  A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic Complexity , 2016, Entropy.

[47]  Nando de Freitas,et al.  A Machine Learning Perspective on Predictive Coding with PAQ8 , 2011, 2012 Data Compression Conference.

[48]  David G. Stork,et al.  Recovering layers of brush strokes through statistical analysis of color and shape: an application to van Gogh's Self portrait with grey felt hat , 2008, Electronic Imaging.

[49]  Tsuhan Chen,et al.  Cross-layer features in convolutional neural networks for generic classification tasks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[50]  Michael Felsberg,et al.  Painting-91: a large scale database for computational painting categorization , 2014, Machine Vision and Applications.

[51]  Armando J. Pinho,et al.  Substitutional Tolerant Markov Models for Relative Compression of DNA Sequences , 2017, PACBB.

[52]  S. Blair Hedges.  Image analysis of Renaissance copperplate prints , 2008, Electronic Imaging.

[53]  David,et al.  Abstract Expressionism: The Politics of Apolitical Painting , 1978 .

[54]  Vladimir M. Petrov,et al.  Entropy and Stability in Painting: An Information Approach to the Mechanisms of Artistic Creativity , 2002, Leonardo.

[55]  Nikolai K. Vereshchagin,et al.  Combinatorial interpretation of Kolmogorov complexity , 2002 .

[56]  Glen G. Langdon,et al.  Arithmetic Coding , 1979 .

[57]  Xin Chen,et al.  An information-based sequence distance and its application to whole mitochondrial genome phylogeny , 2001, Bioinform..

[58]  Joost Smiers,et al.  Arts under pressure : promoting cultural diversity in the age of globalization , 2003 .

[59]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[60]  Ian H. Witten,et al.  Data Compression Using Adaptive Coding and Partial String Matching , 1984, IEEE Trans. Commun..

[61]  Armando J. Pinho,et al.  Finding unknown repeated patterns in images , 2011, 2011 19th European Signal Processing Conference.

[62]  Nikolai K. Vereshchagin,et al.  Inequalities for Shannon Entropy and Kolmogorov Complexity , 1997, J. Comput. Syst. Sci..

[63]  B. Olshausen,et al.  Applied mathematics: The statistics of style , 2010, Nature.

[64]  James Ze Wang,et al.  Studying digital imagery of ancient paintings by mixtures of stochastic models , 2004, IEEE Transactions on Image Processing.

[65]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[66]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences , 1966, JACM.

[67]  Armando J. Pinho,et al.  Efficient Compression of Genomic Sequences , 2016, 2016 Data Compression Conference (DCC).

[68]  Florent Perronnin,et al.  An analysis of the relationship between painters based on their work , 2008, 2008 15th IEEE International Conference on Image Processing.

[69]  Xavier Maldague,et al.  Non-destructive Investigation of Paintings on Canvas by Continuous Wave Terahertz Imaging and Flash Thermography , 2017, Journal of Nondestructive Evaluation.