Bayesian selection of grammar productions for the language of thought

Probabilistic proposals of Language of Thoughts (LoTs) can explain learning across different domains as statistical inference over a compositionally structured hypothesis space. While frameworks may differ on how a LoT may be implemented computationally, they all share the property that they are built from a set of atomic symbols and rules by which these symbols can be combined. In this work we show how the set of productions of a LoT grammar can be effectively selected from a broad repertoire of possible productions by an inferential process starting from experimental data. We then test this method in the language of geometry, a specific LoT model (Amalric et al., 2017). Finally, despite the fact of the geometrical LoT not being a universal (i.e. Turing-complete) language, we show an empirical relation between a sequence’s probability and its complexity consistent with the theoretical relationship for universal languages described by the Coding Theorem.

[1]  Pierre Pica,et al.  Geometry as a Universal mental Construction , 2011 .

[2]  Johan Wagemans,et al.  The role of vertical mirror symmetry in visual shape detection. , 2009, Journal of vision.

[3]  Murat Aydede,et al.  Language of Thought: The Connectionist Contribution , 1997, Minds and Machines.

[4]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[5]  Elizabeth M. Brannon,et al.  Space, Time and Number in the Brain , 2011 .

[6]  D. McDermott.  LANGUAGE OF THOUGHT , 2012 .

[7]  Thomas L. Griffiths,et al.  Bayesian Inference for PCFGs via Markov Chain Monte Carlo , 2007, NAACL.

[8]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[9]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[10]  D. Gentner.  Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[11]  E. Rosch,et al.  Structural bases of typicality effects. , 1976 .

[12]  Jonathan Knowles.  The language of thought and natural language understanding , 1998 .

[13]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[14]  Sang Ah Lee,et al.  Navigation as a source of geometric knowledge: Young children’s use of length, angle, distance, and direction in a reorientation task , 2012, Cognition.

[15]  G. Boole.  An Investigation of the Laws of Thought: On which are founded the mathematical theories of logic and probabilities , 2007 .

[16]  Noah D. Goodman,et al.  The logical primitives of thought: Empirical foundations for compositional cognitive models. , 2016, Psychological review.

[17]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[18]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[20]  Armando Solar-Lezama,et al.  Unsupervised Learning by Program Synthesis , 2015, NIPS.

[21]  Robert A Jacobs,et al.  Learning multisensory representations for auditory-visual transfer of sequence category knowledge: a probabilistic language of thought approach , 2014, Psychonomic bulletin & review.

[22]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[23]  H Gleitman,et al.  Spatial knowledge and geometric representation in a child blind from birth. , 1981, Science.

[24]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[25]  Noah D. Goodman,et al.  Theory learning as stochastic search in the language of thought , 2012 .

[26]  Pierre Pica,et al.  Core knowledge of geometry in an Amazonian indigene group. , 2006, Science.

[27]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[28]  Mariano Sigman,et al.  LT^2C^2: A language of thought with Turing-computable Kolmogorov complexity , 2013, ArXiv.

[29]  Jorge Luis Borges.  Ficciones (1935-1944) , 1944 .

[30]  Cristian S. Calude,et al.  Computing a Glimpse of Randomness , 2002, Exp. Math..

[31]  J. Gomez,et al.  Production and perception rules underlying visual patterns: effects of symmetry and hierarchy , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[32]  John Goldsmith,et al.  Probabilistic Models of Grammar: Phonology as Information Minimization , 2002 .

[33]  Elizabeth S Spelke,et al.  Core foundations of abstract geometry , 2013, Proceedings of the National Academy of Sciences.

[34]  Mariano Sigman,et al.  The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers , 2017, PLoS Comput. Biol..

[35]  Thomas L. Griffiths,et al.  A Rational Analysis of Rule-Based Concept Learning , 2008, Cogn. Sci..

[36]  Robert A. Jacobs,et al.  Four Problems Solved by the Probabilistic Language of Thought , 2016 .

[37]  R. Nosofsky.  Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[38]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[39]  Meaning in Mind , 2019, Mind, Meaning and World.

[40]  Barry Loewer,et al.  Meaning in mind : Fodor and his critics , 1993 .

[41]  John A. Goldsmith,et al.  Unsupervised Learning of the Morphology of a Natural Language , 2001, CL.

[42]  Noah D. Goodman,et al.  Bootstrapping in a language of thought: A formal model of numerical concept learning , 2012, Cognition.