Exploring probabilistic grammars of symbolic music using PRISM

In this paper we describe how we used the logic-based probabilistic programming language PRISM to conduct a systematic comparison of several probabilistic models of symbolic music, including 0th and 1st order Markov models over pitches and intervals, and a probabilistic grammar with two parameterisations. Using PRISM allows us to take advantage of variational Bayesian methods for assessing the goodness of fit of the models. When applied to a corpus of Bach chorales and the Essen folk song collection, we found that, depending on various parameters, the probabilistic grammars sometimes but not always out-perform the simple Markov models. Examining how the models perform on smaller subsets of pieces, we find that the simpler Markov models do out-perform the best grammar-based model at the small end of the scale.

[1]  Yee Whye Teh,et al.  A stochastic memoizer for sequence data , 2009, ICML '09.

[2]  Marcus T. Pearce,et al.  The construction and evaluation of statistical models of melodic structure in music perception and composition , 2005 .

[3]  Masataka Goto,et al.  A Vocabulary-Free Infinity-Gram Model for Nonparametric Bayesian Chord Progression Analysis , 2011, ISMIR.

[4]  Neng-Fa Zhou,et al.  Generative Modeling with Failure in PRISM , 2005, IJCAI.

[5]  Heinrich Schenker,et al.  Der freie Satz , 1935 .

[6]  Phillip B. Kirlin,et al.  Probabilistic Modeling of Hierarchical Music Analysis , 2011, ISMIR.

[7]  Isabelle Guyon,et al.  Model Selection: Beyond the Bayesian/Frequentist Divide , 2010, J. Mach. Learn. Res..

[8]  Chung-chieh Shan,et al.  Embedded Probabilistic Programming , 2009, DSL.

[9]  Jon Sneyers,et al.  Probabilistic-Logical Modeling of Music , 2006, PADL.

[10]  Harry H. Porter.  Earley Deduction , 1986 .

[11]  Cosma Rohilla Shalizi,et al.  Philosophy and the practice of Bayesian statistics. , 2010, The British journal of mathematical and statistical psychology.

[12]  B. Lindblom,et al.  Towards a generative theory of melody , 2007 .

[13]  Antti Honkela,et al.  Variational learning and bits-back coding: an information-theoretic view to Bayesian learning , 2004, IEEE Transactions on Neural Networks.

[14]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[15]  Kenichi Kurihara,et al.  Variational Bayes via propositionalized probability computation in PRISM , 2008, Annals of Mathematics and Artificial Intelligence.

[16]  Taisuke Sato,et al.  A Separate-and-Learn Approach to EM Learning of PCFGs , 2001, NLPRS.

[17]  Eric Brill,et al.  Beyond N-Grams: Can Linguistic Sophistication Improve Language Modeling? , 1998, COLING-ACL.

[18]  Phillip B. Kirlin,et al.  A Probabilistic Model of Hierarchical Music Analysis , 2014, ISMIR 2014.

[19]  Terry Winograd,et al.  Linguistics and the computer analysis of tonal harmony , 1968 .

[20]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[21]  Michael Kassler.  A trinity of essays : Toward a theory that is the twelve-note-class system, Toward development of a constructive tonality theory based on writings by Heinrich Schenker, Toward a simple programming language for musical imformation retrieval , 1967 .

[22]  Mark Granroth-Wilding,et al.  Harmonic analysis of music using combinatory categorial grammar , 2013 .

[23]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[24]  Alan Marsden,et al.  Schenkerian Analysis by Computer: A Proof of Concept , 2010 .

[25]  R. Jackendoff,et al.  A Generative Theory of Tonal Music , 1985 .

[26]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[27]  Kenichi Kurihara,et al.  Variational Bayesian Grammar Induction for Natural Language , 2006, ICGI.

[28]  Matthew Brown,et al.  PARSING CONTEXT-FREE GRAMMARS FOR MUSIC: A COMPUTATIONAL MODEL OF SCHENKERIAN ANALYSIS , 2004 .

[29]  Mark Steedman,et al.  A Generative Grammar for Jazz Chord Sequences , 1984 .

[30]  Martin Rohrmeier,et al.  Towards a generative syntax of tonal harmony , 2011 .

[31]  Rens Bod.  Probabilistic grammars for music , 2001 .

[32]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[33]  Édouard Gilbert,et al.  A Probabilistic Context-Free Grammar for Melodic Reduction ? , 2007 .

[34]  David L. Poole,et al.  Representing Bayesian Networks Within Probabilistic Horn Abduction , 1991, UAI.