Compositional Inductive Biases in Function Learning

How do people recognize and learn about complex functional structure? Taking inspiration from other areas of cognitive science, we propose that this is achieved by harnessing compositionality: complex structure is decomposed into simpler building blocks. We formalize this idea within the framework of Bayesian regression using a grammar over Gaussian process kernels, and compare this approach with other structure learning approaches. Participants consistently chose compositional (over non-compositional) extrapolations and interpolations of functions. Experiments designed to elicit priors over functional patterns revealed an inductive bias for compositional structure. Compositional functions were perceived as subjectively more predictable than non-compositional functions, and exhibited other signatures of predictability, such as enhanced memorability and reduced numerosity. Taken together, these results support the view that the human intuitive theory of functions is inherently compositional.

[1]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  William M. Smith,et al.  A Study of Thinking , 1956 .

[3]  William M. Smith,et al.  A Study of Thinking , 1956 .

[4]  D. Broadbent CHAPTER 5 – THE EFFECTS OF NOISE ON BEHAVIOUR , 1958 .

[5]  R. Shepard,et al.  Learning and memorization of classifications. , 1961 .

[6]  J. Carroll FUNCTIONAL LEARNING: THE LEARNING OF CONTINUOUS FUNCTIONAL MAPPINGS RELATING STIMULUS AND RESPONSE CONTINUA , 1963 .

[7]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[8]  H. Simon,et al.  Perception in chess , 1973 .

[9]  B. Brehmer The effect of cue intercorrelation on interpersonal learning of probabilistic inference tasks , 1974 .

[10]  B. Brehmer Hypotheses about relations between scaled variables in the learning of probabilistic inference tasks , 1974 .

[11]  P. Thorndyke Cognitive structures in comprehension and memory of narrative discourse , 1977, Cognitive Psychology.

[12]  G. Kanizsa,et al.  Organization in Vision: Essays on Gestalt Perception , 1979 .

[13]  E. Leeuwenberg,et al.  Coding theory of visual pattern completion. , 1981, Journal of experimental psychology. Human perception and performance.

[14]  I. Eggleton Intuitive Time-Series Extrapolation , 1982 .

[15]  G. Keren Cultural differences in the misperception of exponential growth , 1983, Perception & psychophysics.

[16]  D. Broadbent,et al.  On the Relationship between Task Performance and Associated Verbalizable Knowledge , 1984 .

[17]  B. Brehmer,et al.  Learning and hypothesis testing in probabilistic inference tasks , 1985 .

[18]  I. Biederman Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[19]  Simon Peyton Jones,et al.  The Implementation of Functional Programming Languages (Prentice-hall International Series in Computer Science) , 1987 .

[20]  H Pashler,et al.  Familiarity and visual change detection , 1988, Perception & psychophysics.

[21]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[22]  Paul B. Andreassen,et al.  Judgmental extrapolation and the salience of change , 1990 .

[23]  A. Robin Forrest,et al.  Interactive interpolation and approximation by Bézier polynomials , 1990, Comput. Aided Des..

[24]  D. Meyer,et al.  Function learning: induction of continuous stimulus-response relations. , 1991, Journal of experimental psychology. Learning, memory, and cognition.

[25]  David J. Field,et al.  Contour integration by the human visual system: Evidence for a local “association field” , 1993, Vision Research.

[26]  P. Goodwin,et al.  Improving judgmental time series forecasting: A review of the guidance provided by research , 1993 .

[27]  R. Nosofsky,et al.  Rule-plus-exception model of classification learning. , 1994, Psychological review.

[28]  Eunhee Byun,et al.  Interaction between prior knowledge and type of nonlinear relationship on function learning , 1995 .

[29]  F. Bolger,et al.  Graphs versus tables: Effects of data presentation format on judgemental forecasting , 1996 .

[30]  Nigel Harvey Teresa Ewart Robert West,et al.  Effects of data noise on statistical judgement , 1997 .

[31]  M. McDaniel,et al.  Extrapolation: the sine qua non for abstraction in function learning. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[32]  Zoubin Ghahramani,et al.  Modular decomposition in visuomotor learning , 1997, Nature.

[33]  Nigel Harvey,et al.  Heuristics and biases in judgmental forecasting , 1998 .

[34]  H. Simon,et al.  Expert chess memory: revisiting the chunking hypothesis. , 1998, Memory.

[35]  J R Flanagan,et al.  Composition and Decomposition of Internal Models in Motor Learning under Altered Kinematic and Dynamic Environments , 1999, The Journal of Neuroscience.

[36]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[37]  A. Walden,et al.  Wavelet Methods for Time Series Analysis , 2000 .

[38]  Peter Sollich Gaussian Process Regression with Mismatched Models , 2001, NIPS.

[39]  Michael R. Chernick,et al.  Wavelet Methods for Time Series Analysis , 2001, Technometrics.

[40]  M. Kalish,et al.  Simplified learning in complex situations: knowledge partitioning in function learning. , 2002, Journal of experimental psychology. General.

[41]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[42]  P. Juslin,et al.  Exemplar effects in categorization and multiple-cue judgment. , 2003, Journal of experimental psychology. General.

[43]  Lewis Bott,et al.  Nonmonotonic extrapolation in function learning. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[44]  Stephan Lewandowsky,et al.  Population of linear experts: knowledge partitioning and function learning. , 2004, Psychological review.

[45]  J. Busemeyer,et al.  Learning Functional Relations Based on Experience With Input-Output Pairs by Humans and Artificial Neural Networks , 2005 .

[46]  M. McDaniel,et al.  The conceptual basis of function learning and extrapolation: Comparison of rule-based and associative-based models , 2005, Psychonomic bulletin & review.

[47]  Tai Sing Lee,et al.  Efficient Coding of Visual Scenes by Grouping and Segmentation: Theoretical Predictions and Biological Evidence , 2006 .

[48]  J. Tenenbaum,et al.  Optimal Predictions in Everyday Cognition , 2006, Psychological science.

[49]  A. Neal,et al.  Why people underestimate y when extrapolating in linear functions. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[50]  Thomas L. Griffiths,et al.  Language Evolution by Iterated Learning With Bayesian Agents , 2007, Cogn. Sci..

[51]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[52]  T. Griffiths,et al.  Iterated learning: Intergenerational knowledge transmission reveals inductive biases , 2007, Psychonomic bulletin & review.

[53]  Thomas L. Griffiths,et al.  Modeling human function learning with Gaussian processes , 2008, NIPS.

[54]  Thomas L. Griffiths,et al.  A Rational Analysis of Rule-Based Concept Learning , 2008, Cogn. Sci..

[55]  M. McDaniel,et al.  Predicting transfer performance: a comparison of competing function learning models. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[56]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[57]  Daniel R. Little,et al.  Simplicity Bias in the Estimation of Causal Functions , 2009 .

[58]  J. Tenenbaum,et al.  Structured statistical models of inductive reasoning. , 2009, Psychological review.

[59]  Adam N. Sanborn,et al.  Uncovering mental representations with Markov chain Monte Carlo , 2010, Cognitive Psychology.

[60]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[61]  J. Tenenbaum,et al.  Probabilistic models of cognition: exploring representations and inductive biases , 2010, Trends in Cognitive Sciences.

[62]  N. Turk-Browne,et al.  Mutual Interference Between Statistical Summary Perception and Statistical Learning , 2011, Psychological science.

[63]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[64]  Jeffrey N. Rouder,et al.  How to measure working memory capacity in the change detection paradigm , 2011, Psychonomic bulletin & review.

[65]  Timothy F. Brady,et al.  A review of visual memory capacity: Beyond individual items and toward structured representations. , 2011, Journal of vision.

[66]  M. Goodale,et al.  The role of vision in detecting and correcting fingertip force errors during object lifting. , 2011, Journal of vision.

[67]  F. Mathy,et al.  What’s magic about magic numbers? Chunking and data compression in short-term memory , 2012, Cognition.

[68]  George Kachergis,et al.  Gaussian Process Regression for Trajectory Analysis , 2012, CogSci.

[69]  Charles Kemp,et al.  Exploring the conceptual universe. , 2012, Psychological review.

[70]  Timothy F. Brady,et al.  A probabilistic model of visual working memory: Incorporating higher order regularities into working memory capacity estimates. , 2013, Psychological review.

[71]  Georg M. Goerg Forecastable Component Analysis , 2013, ICML.

[72]  Andrew Gordon Wilson,et al.  Gaussian Process Kernels for Pattern Discovery and Extrapolation , 2013, ICML.

[73]  M. Kalish Learning and extrapolating a periodic function , 2013, Memory & cognition.

[74]  Joshua B. Tenenbaum,et al.  Structure Discovery in Nonparametric Regression through Compositional Kernel Search , 2013, ICML.

[75]  R. Jacobs,et al.  A probabilistic clustering theory of the organization of visual short-term memory. , 2013, Psychological review.

[76]  Pablo Montero,et al.  TSclust: An R Package for Time Series Clustering , 2014 .

[77]  Joshua B. Tenenbaum,et al.  Assessing the Perceived Predictability of Functions , 2015, CogSci.

[78]  Bradley C. Love,et al.  Active learning as a means to distinguish among prominent decision strategies , 2015, CogSci.

[79]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[80]  Rob J. Hyndman,et al.  Large-Scale Unusual Time Series Detection , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[81]  O. Blanke,et al.  Learning to integrate contradictory multisensory self-motion cue pairings. , 2015, Journal of vision.

[82]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[83]  Andrew Gordon Wilson,et al.  The Human Kernel , 2015, NIPS.

[84]  Christopher G. Lucas,et al.  A rational model of function learning , 2015, Psychonomic Bulletin & Review.

[85]  E. Vul,et al.  Ensemble clustering in visual working memory biases location memories and reduces the Weber noise of relative positions. , 2015, Journal of vision.

[86]  Jiaying Zhao,et al.  Statistical regularities reduce perceived numerosity , 2016, Cognition.

[87]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[88]  Noah D. Goodman,et al.  The logical primitives of thought: Empirical foundations for compositional cognitive models. , 2016, Psychological review.

[89]  Samuel J. Gershman,et al.  Discovering hierarchical motion structure , 2016, Vision Research.

[90]  T. Poggio,et al.  Deep vs. shallow networks : An approximation theory perspective , 2016, ArXiv.

[91]  Jaesik Choi,et al.  Automatic Construction of Nonparametric Relational Regression Models for Multiple Time Series , 2016, ICML.

[92]  Jonathan D. Nelson,et al.  Exploration and generalization in vast spaces 1 , 2017 .

[93]  M. Speekenbrink,et al.  Putting bandits into context: How function learning supports decision making , 2016, bioRxiv.

[94]  Samuel J. Gershman,et al.  Structured Representations of Utility in Combinatorial Domains , 2017 .

[95]  N. Daw,et al.  Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework , 2017, Annual review of psychology.

[96]  M. Speekenbrink,et al.  Putting bandits into context: How function learning supports decision making , 2016, bioRxiv.

[97]  Xiao Wang,et al.  Statistical Efficiency of Compositional Nonparametric Prediction , 2018, AISTATS.

[98]  Andreas Krause,et al.  A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions , 2016, bioRxiv.

[99]  M. Manosevitz,et al.  High-Speed Scanning in Human Memory , 2022 .