Can we detect harmony in artistic compositions? A machine learning approach

Harmony in visual compositions is a concept that cannot be defined or easily expressed mathematically, even by humans. The goal of the research described in this paper was to find a numerical representation of artistic compositions with different levels of harmony. We ask humans to rate a collection of grayscale images based on the harmony they convey. To represent the images, a set of special features were designed and extracted. By doing so, it became possible to assign objective measures to subjectively judged compositions. Given the ratings and the extracted features, we utilized machine learning algorithms to evaluate the efficiency of such representations in a harmony classification problem. The best performing model (SVM) achieved 80% accuracy in distinguishing between harmonic and disharmonic images, which reinforces the assumption that concept of harmony can be expressed in a mathematical way that can be assessed by humans.

[1]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[2]  C. Redies,et al.  Evaluating the Rule of Thirds in Photographs and Paintings , 2014 .

[3]  H. Leder,et al.  A model of aesthetic appreciation and aesthetic judgments. , 2004, British journal of psychology.

[4]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[5]  Dong ping Tian,et al.  A Review on Image Feature Extraction and Representation Techniques , 2013 .

[6]  Erwin Bolthausen,et al.  BERNOULLI'S LAW OF LARGE NUMBERS , 2013, ASTIN Bulletin.

[7]  Yuxuan Wang,et al.  A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[8]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[9]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[10]  R. Davis.  An Evaluation and Test of Birkhoff's Aesthetic Measure Formula , 1936 .

[11]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[12]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[13]  B. Aruna Devi,et al.  A Survey on Artificial Intelligence Approaches for Medical Image Classification , 2011 .

[14]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[15]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[16]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[17]  Mark S. Nixon,et al.  Feature extraction & image processing for computer vision , 2012 .

[18]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Somnuk Phon-Amnuaisuk,et al.  Quantifying aesthetic beauty through its dimensions: a case study on trochoids , 2015, Int. J. Knowl. Eng. Soft Data Paradigms.

[20]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[21]  Anna Jordanous,et al.  A Standardised Procedure for Evaluating Creative Systems: Computational Creativity Evaluation Based on What it is to be Creative , 2012, Cognitive Computation.

[22]  G. Rizzolatti,et al.  The Golden Beauty: Brain Response to Classical and Renaissance Sculptures , 2007, PloS one.

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[24]  I. Newton,et al.  The Principia : Mathematical Principles of Natural Philosophy , 2018 .

[25]  J. Macgregor,et al.  Image texture analysis: methods and comparisons , 2004 .

[26]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[27]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.