Learning to Make Analogies by Contrasting Abstract Relational Structure

Analogical reasoning has been a principal focus of various waves of AI research. Analogy is particularly challenging for machines because it requires relational structures to be represented such that they can be flexibly applied across diverse domains of experience. Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data. We find that the critical factor for inducing such a capacity is not an elaborate architecture, but rather, careful attention to the choice of data and the manner in which it is presented to the model. The most robust capacity for analogical reasoning is induced when networks learn analogies by contrasting abstract relational structures in their input domains, a training method that uses only the input data to force models to learn about important abstract features. Using this technique we demonstrate capacities for complex, visual and symbolic analogy making and generalisation in even the simplest neural network architectures.

[1]  H. Wildon Carr,et al.  I.—The Problem of Recognition , 1917 .

[2]  J. Raven,et al.  Manual for Raven's progressive matrices and vocabulary scales , 1962 .

[3]  Jaime G. Carbonell,et al.  A Computational Model of Analogical Problem Solving , 1981, IJCAI.

[4]  Ali M. Ali,et al.  The use of positive and negative examples during instruction , 1981 .

[5]  D. Gentner.  Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[6]  David J. Chalmers,et al.  High-level perception, representation, and analogy: a critique of artificial intelligence methodology , 1992, J. Exp. Theor. Artif. Intell..

[7]  Douglas R. Hofstadter,et al.  Fluid Concepts and Creative Analogies , 1995 .

[8]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[9]  Arthur B. Markman,et al.  Analogy just looks like high level perception: why a domain-general approach to analogical mapping is right , 1998, J. Exp. Theor. Artif. Intell..

[10]  Mary K. Hoard,et al.  Sex differences in spatial cognition, computational fluency, and arithmetical reasoning. , 2000, Journal of experimental child psychology.

[11]  K. Holyoak,et al.  Working-memory modularity in analogical reasoning , 2001 .

[12]  Bradley C. Love,et al.  CAB: Connectionist Analogy Builder , 2003, Cogn. Sci..

[13]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[14]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[15]  Noah A. Smith,et al.  Contrastive Estimation: Training Log-Linear Models on Unlabeled Data , 2005, ACL.

[16]  J. Qiu,et al.  The neural basis of analogical reasoning: An event-related potential study , 2008, Neuropsychologia.

[17]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[18]  Ting Li,et al.  Comparing machines and humans on a visual categorization test , 2011, Proceedings of the National Academy of Sciences.

[19]  Kenneth D. Forbus,et al.  Computational models of analogy. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[20]  Will Kanyusik.  The Problem of Recognition: The Disabled Male Veteran and Masculinity as Spectacle in William Wyler's The Best Years of Our Lives , 2012 .

[21]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[22]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[23]  Dedre Gentner,et al.  The Role of Difference-Detection in Learning Contrastive Categories , 2014, CogSci.

[24]  Noah D. Goodman,et al.  A rational account of pedagogical reasoning: Teaching by, and learning from, examples , 2014, Cognitive Psychology.

[25]  Georgiana Dinu,et al.  Hubness and Pollution: Delving into Cross-Space Mapping for Zero-Shot Learning , 2015, ACL.

[26]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yuting Zhang,et al.  Deep Visual Analogy-Making , 2015, NIPS.

[28]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[29]  Daan Wierstra,et al.  One-shot Learning with Memory-Augmented Neural Networks , 2016, ArXiv.

[30]  Annett Wechsler.  Analogy Making As Perception A Computer Model , 2016 .

[31]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[32]  Kenneth D. Forbus,et al.  Modeling Visual Problem Solving as Analogical Reasoning , 2017, Psychological review.

[33]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[34]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[35]  José Bento,et al.  Generative Adversarial Active Learning , 2017, ArXiv.

[36]  Marco Baroni,et al.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks , 2017, ICML.

[37]  Felix Hill,et al.  Measuring abstract reasoning in neural networks , 2018, ICML.

[38]  Alex Clarke,et al.  Integrated deep visual and semantic attractor neural networks predict fMRI pattern-information along the ventral object processing pathway , 2018 .