Environmental drivers of systematicity and generalization in a situated agent

The question of whether deep neural networks are good at generalising beyond their immediate training experience is of critical importance for learning-based approaches to AI. Here, we consider tests of out-of-sample generalisation that require an agent to respond to never-seen-before instructions by manipulating and positioning objects in a 3D Unity simulated room. We first describe a comparatively generic agent architecture that exhibits strong performance on these tests. We then identify three aspects of the training regime and environment that make a significant difference to its performance: (a) the number of object/word experiences in the training set; (b) the visual invariances afforded by the agent's perspective, or frame of reference; and (c) the variety of visual input inherent in the perceptual aspect of the agent's perception. Our findings indicate that the degree of generalisation that networks exhibit can depend critically on particulars of the environment in which a given task is instantiated. They further suggest that the propensity for neural networks to generalise in systematic ways may increase if, like human children, those networks have access to many frames of richly varying, multi-modal observations as they learn.

[1]  Brenden M. Lake,et al.  Compositional generalization through meta sequence-to-sequence learning , 2019, NeurIPS.

[2]  Jane X. Wang,et al.  Reinforcement Learning, Fast and Slow , 2019, Trends in Cognitive Sciences.

[3]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[4]  Ruosong Wang,et al.  Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks , 2019, ICML.

[5]  Marco Baroni,et al.  Human few-shot learning of compositional instructions , 2019, CogSci.

[6]  Yuanzhi Li,et al.  Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers , 2018, NeurIPS.

[7]  Aaron C. Courville,et al.  Systematic Generalization: What Is Required and Can It Be Learned? , 2018, ICLR.

[8]  Surya Ganguli,et al.  An analytic theory of generalization dynamics and transfer learning in deep linear networks , 2018, ICLR.

[9]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[10]  Yi Zhang,et al.  Stronger generalization bounds for deep nets via a compression approach , 2018, ICML.

[11]  Shane Legg,et al.  IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures , 2018, ICML.

[12]  Wei Xu,et al.  Interactive Grounded Language Acquisition and Generalization in a 2D World , 2018, ICLR.

[13]  Marco Baroni,et al.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks , 2017, ICML.

[14]  Murray Shanahan,et al.  SCAN: Learning Hierarchical Compositional Visual Concepts , 2017, ICLR.

[15]  Ruslan Salakhutdinov,et al.  Gated-Attention Architectures for Task-Oriented Language Grounding , 2017, AAAI.

[16]  Stephen Clark,et al.  Understanding Grounded Language Learning Agents , 2017, ArXiv.

[17]  Demis Hassabis,et al.  SCAN: Learning Abstract Hierarchical Compositional Visual Concepts , 2017, ArXiv.

[18]  James M. Rehg,et al.  Real-world visual statistics and infants' first-learned object names , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[19]  Dan Klein,et al.  Modular Multitask Reinforcement Learning with Policy Sketches , 2016, ICML.

[20]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[22]  Christian Lebiere,et al.  How Limited Systematicity Emerges: A Computational Cognitive Neuroscience Approach (Author's Manuscript) , 2014 .

[23]  P. Calvo,et al.  The Architecture of Cognition: Rethinking Fodor and Pylyshyn's Systematicity Challenge , 2014 .

[24]  Linda B. Smith,et al.  Statistical word learning at scale: the baby's view is better. , 2013, Developmental science.

[25]  Mark Steedman,et al.  Taking Scope - The Natural Semantics of Quantifiers , 2011 .

[26]  James L. McClelland,et al.  Letting structure emerge: connectionist and dynamical systems approaches to cognition , 2010, Trends in Cognitive Sciences.

[27]  Stefan L. Frank,et al.  Learn more by training less: systematicity in sentence processing by recurrent networks , 2006, Connect. Sci..

[28]  G. Harman,et al.  The Problem of Induction , 2006 .

[29]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[30]  Steven A. Sloman,et al.  The Problem of Induction , 2005 .

[31]  Michael L. Anderson.  Embodied Cognition: A field guide , 2003, Artif. Intell..

[32]  Randall C. O'Reilly,et al.  Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning , 2001, Neural Computation.

[33]  Mark Steedman,et al.  Connectionist sentence processing in perspective , 1999, Cogn. Sci..

[34]  David C. Plaut,et al.  Systematicity and specialization in semantics , 1999 .

[35]  G. Marcus.  Rethinking Eliminative Connectionism , 1998, Cognitive Psychology.

[36]  M. Arterberry,et al.  The Cradle of Knowledge: Development of Perception in Infancy , 1998 .

[37]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[38]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[39]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[40]  P. Anderson.  More is different. , 1972, Science.

[41]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .