How to Grow a Mind: Statistics, Structure, and Abstraction

In coming to understand the world—in learning concepts, acquiring language, and grasping causal relations—our minds make inferences that appear to go far beyond the data available. How do we do it? This review describes recent approaches to reverse-engineering human learning and cognitive development and, in parallel, engineering more humanlike machine learning systems. Computational models that perform probabilistic inference over hierarchies of flexibly structured representations can address some of the deepest questions about the nature and origins of human thought: How does abstract knowledge guide learning and reasoning from sparse data? What forms does our knowledge take, across different domains and tasks? And how is that abstract knowledge itself acquired?

[1]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[2]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[3]  Noam Chomsky,et al.  Rules and representations , 1980, Behavioral and Brain Sciences.

[4]  S. Carey Conceptual Change in Childhood , 1985 .

[5]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[7]  Noam Chomsky,et al.  Language and problems of knowledge : the Managua lectures , 1990 .

[8]  E. Markman Categorization and naming in children , 1989 .

[9]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[10]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization. , 1991 .

[11]  H. Wellman,et al.  Cognitive development: foundational theories of core domains. , 1992, Annual review of psychology.

[12]  E. Spelke,et al.  Origins of knowledge. , 1992, Psychological review.

[13]  Daniel Kahneman,et al.  Probabilistic reasoning , 1993 .

[14]  Manfred K. Warmuth Proceedings of the seventh annual conference on Computational learning theory , 1994, COLT 1994.

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[17]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[18]  R. Shiffrin,et al.  A model for recognition memory: REM—retrieving effectively from memory , 1997, Psychonomic bulletin & review.

[19]  S. Atran Folk biology and the anthropology of science: Cognitive universals and cultural particulars , 1998, Behavioral and Brain Sciences.

[20]  G. Bao AIR FORCE OFFICE OF SCIENTIFIC RESEARCH , 1999 .

[21]  S. Pinker Words and Rules: The Ingredients of Language , 1999 .

[22]  S. Pinker How the Mind Works , 1999, Annals of the New York Academy of Sciences.

[23]  Swathi Vanniarajan,et al.  WORDS AND RULES: THE INGREDIENTS OF LANGUAGE , 2001 .

[24]  M. Minami How Children Learn the Meanings of Words , 2001 .

[25]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[26]  N. Chater,et al.  The probabilistic approach to human reasoning , 2001, Trends in Cognitive Sciences.

[27]  Dirk van Rijn,et al.  Proceedings of the 31st annual conference of the Cognitive Science Society , 2003 .

[28]  P. Godfrey‐Smith Theory and Reality , 2003 .

[29]  Thomas G. Dietterich,et al.  SRL2004 ICML 2004 Workshop on Statistical Relational Learning and its Connections to Other Fields , 2004 .

[30]  S. Carey The Origin of Concepts , 2000 .

[31]  David M. Sobel,et al.  A theory of causal learning in children: causal maps and Bayes nets. , 2004, Psychological review.

[32]  Joseph Y. Halpern,et al.  Proceedings of the 20th conference on Uncertainty in artificial intelligence , 2004, UAI 2004.

[33]  R. Baierlein Probability Theory: The Logic of Science , 2004 .

[34]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[35]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[36]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[37]  宁北芳,et al.  疟原虫var基因转换速率变化导致抗原变异[英]/Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A , 2005 .

[38]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[39]  Johanna D. Moore,et al.  Proceedings of the 28th Annual Conference of the Cognitive Science Society , 2005 .

[40]  Partha Niyogi,et al.  Book Reviews: The Computational Nature of Language Learning and Evolution, by Partha Niyogi , 2007, CL.

[41]  J. Tenenbaum,et al.  Optimal Predictions in Everyday Cognition , 2006, Psychological science.

[42]  A. Yuille,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Vision as Bayesian inference: analysis by synthesis? , 2022 .

[43]  Christopher D. Manning,et al.  Probabilistic models of language processing and acquisition , 2006, Trends in Cognitive Sciences.

[44]  T. Griffiths,et al.  Probabilistic inference in human semantic memory , 2006, Trends in Cognitive Sciences.

[45]  S. Pinker The Stuff of Thought: Language as a Window into Human Nature , 2007 .

[46]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[47]  J. Tenenbaum,et al.  From mere coincidences to meaningful discoveries , 2007, Cognition.

[48]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[49]  A. Gopnik,et al.  Causal learning : psychology, philosophy, and computation , 2007 .

[50]  J. Tenenbaum,et al.  Sensitivity to Sampling in Bayesian Word Learning We Thank Members of the Ubc Baby Cognition Lab for Their Help with Data Collection, And , 2022 .

[51]  J. Tenenbaum,et al.  Bayesian Special Section Learning Overhypotheses with Hierarchical Bayesian Models , 2022 .

[52]  Tevye R. Krynski,et al.  The role of causality in judgment under uncertainty. , 2007, Journal of experimental psychology. General.

[53]  Keith A. Markus,et al.  Making Things Happen: A Theory of Causal Explanation , 2007 .

[54]  Nick Chater,et al.  The probabilistic mind , 2008 .

[55]  Alastair J. Gill,et al.  Proceedings for the 30th Annual Meeting of the Cognitive Science Society , 2008 .

[56]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[57]  A. Yuille,et al.  Bayesian generic priors for causal learning. , 2008, Psychological review.

[58]  T. Griffiths,et al.  A Bayesian framework for word segmentation: Exploring the effects of context , 2009, Cognition.

[59]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[60]  Scott D. Brown,et al.  Detecting and predicting changes , 2009, Cognitive Psychology.

[61]  J. Tenenbaum,et al.  Structured statistical models of inductive reasoning. , 2009, Psychological review.

[62]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[63]  Thomas L. Griffiths,et al.  The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies , 2007, JACM.

[64]  J. Tenenbaum,et al.  Probabilistic models of cognition: exploring representations and inductive biases , 2010, Trends in Cognitive Sciences.

[65]  J. Tenenbaum,et al.  A probabilistic model of theory formation , 2010, Cognition.

[66]  James L. McClelland,et al.  Letting structure emerge: connectionist and dynamical systems approaches to cognition , 2010, Trends in Cognitive Sciences.

[67]  Noah D. Goodman,et al.  Learning a theory of causality. , 2011, Psychological review.

[68]  John K. Pate,et al.  Proceedings of the 33rd Annual Conference of the Cognitive Science Society , 2011 .