Learning deep taxonomic priors for concept learning from few positive examples

Human concept learning is surprisingly robust, allowing for precise generalizations given only a few positive examples. Bayesian formulations that account for this behavior require elaborate, pre-specified priors, leaving much of the learning process unexplained. More recent models of concept learning bootstrap from deep representations, but the deep neural networks are themselves trained using millions of positive and negative examples. In machine learning, recent progress in metalearning has provided large-scale learning algorithms that can learn new concepts from a few examples, but these approaches still assume access to implicit negative evidence. In this paper, we formulate a training paradigm that allows a meta-learning algorithm to solve the problem of concept learning from few positive examples. The algorithm discovers a taxonomic prior useful for learning novel concepts even from held-out supercategories and mimics human generalization behavior—the first to do so without hand-specified domain knowledge or negative examples of a novel concept.

[1]  Thomas L. Griffiths,et al.  Evaluating (and Improving) the Correspondence Between Deep Neural Networks and Human Representations , 2017, Cogn. Sci..

[2]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[3]  Thomas L. Griffiths,et al.  Evidence for the size principle in semantic and perceptual domains , 2017, CogSci.

[4]  Pietro Perona,et al.  A Bayesian approach to unsupervised one-shot learning of object categories , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[6]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[7]  J. Tenenbaum,et al.  Bayesian Special Section Learning Overhypotheses with Hierarchical Bayesian Models , 2022 .

[8]  Thomas L. Griffiths,et al.  of the Annual Meeting of the Cognitive Science Society Title Constructing a hypothesis space from the Web for large-scale Bayesian word learning , 2012 .

[9]  S. Carey The child as word learner , 1978 .

[10]  Ellen M. Markman,et al.  Categorization and Naming in Children: Problems of Induction , 1989 .

[11]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[12]  F. Ashby,et al.  Categorization as probability density estimation , 1995 .

[13]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[14]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[15]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[16]  Lauren A. Schmidt Meaning and compositionality as statistical induction of categories and constraints , 2009 .

[17]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[18]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[19]  Joshua B. Tenenbaum,et al.  Learning to Learn Visual Object Categories by Integrating Deep Learning with Hierarchical Bayes , 2017, CogSci.

[20]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[21]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[22]  Andreas Krause,et al.  Advances in Neural Information Processing Systems (NIPS) , 2014 .

[23]  Thomas L. Griffiths,et al.  Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels , 2018, CogSci.

[24]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[25]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[26]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[27]  Thomas L. Griffiths,et al.  Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies , 2013, NIPS.

[28]  J. Tenenbaum A Bayesian framework for concept learning , 1999 .

[29]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..