The constructivist learning architecture: a model of cognitive development for robust autonomous robots

Autonomous robots are used more and more in remote and inaccessible places where they cannot be easily repaired if damaged or improperly programmed. A system is needed that allows these robots to repair themselves by recovering gracefully from damage and adapting to unforeseen changes. Newborn infants employ such a system to adapt to a new and dynamic world by building a hierarchical representation of their environment. This model allows them to respond robustly to changes by falling back to an earlier stage of knowledge, rather than failing completely. A computational model that replicates these phenomena in infants would afford a mobile robot the same adaptability and robustness that infants have. This dissertation presents such a model, the Constructivist Learning Architecture (CLA), that builds a hierarchical knowledge base using a set of interconnected self-organizing learning modules. The dissertation then demonstrates that CLA (1) replicates current studies in infant cognitive development, (2) builds sensorimotor schemas for robot control, (3) learns a goal-directed task from delayed rewards, and (4) can fall back and recover gracefully from damage. CLA is a new approach to robot control that allows robots to recover from damage or adapt to unforeseen changes in the environment. CLA is also a new approach to cognitive modeling that can be used to better understand how people learn for their environment in infancy and adulthood.

[1]  Mark B. Ring.  Continual learning in reinforcement environments , 1995, GMD-Bericht.

[2]  Harold H. Chaput.  Post-Piagetian Constructivism for Grounded Knowledge Acquisition , 2001 .

[3]  R. Baillargeon.  Object permanence in 3½- and 4½-month-old infants. , 1987 .

[4]  Thomas H. Schilling,et al.  Infants' Looking at Possible and Impossible Screen Rotations: The Role of Familiarization. , 2000, Infancy : the official journal of the International Society on Infant Studies.

[5]  Thomas G. Dietterich.  An Overview of MAXQ Hierarchical Reinforcement Learning , 2000, SARA.

[6]  Scott P. Johnson,et al.  ARTICLE WITH PEER COMMENTARIES AND RESPONSE Learning to perceive object unity: a connectionist account , 2002 .

[7]  R. Baillargeon,et al.  Object Individuation in Infancy: The Use of Featural Information in Reasoning about Occlusion Events , 1998, Cognitive Psychology.

[8]  Benjamin Kuipers,et al.  Map Learning with Uninterpreted Sensors and Effectors , 1995, Artif. Intell..

[9]  H. Furth.  Object permanence in five-month-old infants. , 1987, Cognition.

[10]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[11]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[12]  Drew McDermott,et al.  A critique of pure reason 1 , 1987, The Philosophy of Artificial Intelligence.

[13]  Bernhard Hengst,et al.  Discovering Hierarchy in Reinforcement Learning with HEXQ , 2002, ICML.

[14]  Minoru Asada,et al.  A Constructive Model of Mother-Infant Interaction towards Infant’s Vowel Articulation , 2003 .

[15]  A. Leslie.  Spatiotemporal Continuity and the Perception of Causality in Infants , 1984, Perception.

[16]  Y. Munakata,et al.  Hebbian learning and development. , 2004, Developmental science.

[17]  Risto Miikkulainen,et al.  Tilt Aftereffects in a Self-Organizing Model of the Primary Visual Cortex , 2000, Neural Computation.

[18]  Robert M. French,et al.  Asymmetric interference in 3- to 4-month-olds' sequential category learning , 2002, Cogn. Sci..

[19]  Harold H. Chaput and Benjamin Kuipers and Risto Miikkulainen.  Constructivist Learning: A Neural Implementation of the Schema Mechanism , 2003 .

[20]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[21]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[22]  Gary L. Drescher,et al.  Made-up minds - a constructivist approach to artificial intelligence , 1991 .

[23]  Peter Stone,et al.  Layered learning in multiagent systems - a winning approach to robotic soccer , 2000, Intelligent robotics and autonomous agents.

[24]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[25]  David I. Anderson,et al.  Travel Broadens the Mind. , 2000, Infancy : the official journal of the International Society on Infant Studies.

[26]  Ping Li,et al.  A self-organizing neural network model of the acquisition of word meaning , 2001 .

[27]  J. Lewis,et al.  A Connectionist Investigtion of Linguistic Arguments from the Poverty of the Stimulus: Learning the Unlearnable , 2001 .

[28]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[29]  L. Cohen,et al.  Infant object segregation implies information integration. , 2001, Journal of experimental child psychology.

[30]  E. Hartmann.  Perceptual Development: Visual, Auditory, and Speech Perception in Infancy , 1999 .

[31]  Thomas R. Shultz,et al.  Artificial grammer learning by infants: an auto‐associator perspective , 2000 .

[32]  Thomas R. Shultz,et al.  Knowledge-based cascade-correlation: Using knowledge to speed learning , 2001, Connect. Sci..

[33]  José E. Náñez,et al.  Young infant's perception of object unity in two-dimensional displays , 1995 .

[34]  Alan Slater,et al.  Newborn and older infants' perception of partly occluded objects☆ , 1990 .

[35]  Risto Miikkulainen,et al.  Self-Organization of Innate Face Preferences: Could Genetics Be Expressed through Learning? , 2000, AAAI/IAAI.

[36]  Richard T. Vaughan,et al.  The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems , 2003 .

[37]  Manuela M. Veloso,et al.  Fast and inexpensive color image segmentation for interactive robots , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[38]  T. Shultz.  Computational Developmental Psychology , 2003 .

[39]  Polina N. Kyriakides.  MRI Physics for Radiologists: A Visual Approach. 3rd ed , 1995 .

[40]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[41]  Leslie B. Cohen,et al.  Connectionist models of infant perceptual and cognitive development , 2002 .

[42]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[43]  Peter C M Molenaar,et al.  Modeling developmental transitions in adaptive resonance theory. , 2004, Developmental science.

[44]  Benjamin Kuipers,et al.  The Spatial Semantic Hierarchy , 2000, Artif. Intell..

[45]  T. Shultz,et al.  Rethinking innateness, learning, and constructivism: Connectionist perspectives on development , 1997 .

[46]  E. Spelke,et al.  Origins of knowledge. , 1992, Psychological review.

[47]  Linda B. Smith.  Teleology in connectionism , 2002 .

[48]  Yuko Munakata,et al.  Modeling infants’ perception of object unity: what have we learned? , 2002 .

[49]  Harold H. Chaput and Leslie B. Cohen.  A Model of infant Causal Perception and its Development , 2001 .

[50]  R. C. Oldfield.  THE PERCEPTION OF CAUSALITY , 1963 .

[51]  J R Saffran,et al.  The acquisition of language by children , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[52]  Matthew Schlesinger,et al.  Evolving agents as a metaphor for the developing child. , 2004, Developmental science.

[53]  E. Spelke,et al.  Perception of partly occluded objects in infancy , 1983, Cognitive Psychology.

[54]  R. Matthew Kretchmar,et al.  Improved Automatic Discovery of Subgoals for Options in Hierarchical Reinforcement Learning , 2003 .

[55]  L. Cohen,et al.  How infants perceive a simple causal event , 1993 .

[56]  Mark Witkowski,et al.  Dynamic Expectancy: An Approach to Behaviour Shaping Using a New Method of Reinforcement Learning , 1998 .

[57]  D. Parisi,et al.  The Agent-Based Approach: A New Direction for Computational Models of Development , 2001 .

[58]  Sean A. Spence,et al.  Descartes' Error: Emotion, Reason and the Human Brain , 1995 .

[59]  Shie Mannor,et al.  Dynamic abstraction in reinforcement learning via clustering , 2004, ICML.

[60]  S. Sirois.  Autoassociator networks: insights into infant cognition. , 2004, Developmental science.

[61]  John Field,et al.  Language and the mind , 1968 .

[62]  Thomas R. Shultz,et al.  Neural Network Simulation of Infant Familiarization to Artificial Sentences: Rule-Like Behavior Without Explicit Rules and Variables. , 2001, Infancy : the official journal of the International Society on Infant Studies.

[63]  P. L. Adams.  THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[64]  Melissa A. Redford,et al.  The Development of Infant Causal Perception , 1998 .

[65]  D. Mareschal,et al.  From Parts to Wholes: Mechanisms of Development in Infant Visual Object Processing. , 2004, Infancy : the official journal of the International Society on Infant Studies.

[66]  Cara H. Cashon,et al.  A constructivist model of infant cognition , 2002 .

[67]  A. Barto,et al.  Optimal Control Methods for Simulating the Perception of Causality in Young Infants , 2020, Proceedings of the Twenty First Annual Conference of the Cognitive Science Society.

[68]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[69]  Barbara A. Younger,et al.  Infant perception of angular relations , 1984 .

[70]  L. Cohen,et al.  Precursors to infants' perception of the causality of a simple event , 1998 .

[71]  Kasper Stoy,et al.  Player Robot Server , 2000 .

[72]  Cara H. Cashon,et al.  Eight-Month-Old Infants' Perception of Possible and Impossible Events. , 2000, Infancy : the official journal of the International Society on Infant Studies.

[73]  L. Steels.  Evolving grounded communication for robots , 2003, Trends in Cognitive Sciences.

[74]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[75]  Doina Precup,et al.  Learning Options in Reinforcement Learning , 2002, SARA.

[76]  Yuko Munakata,et al.  Computational cognitive neuroscience of early memory development , 2004 .

[77]  K. H. Grobman,et al.  Artificial life and Piaget , 2003, Trends in Cognitive Sciences.

[78]  Leslie B. Cohen,et al.  Do 7-month-old infants process independent features or facial configurations? , 2001 .

[79]  J. Piaget.  The construction of reality in the child , 1954 .

[80]  A. Leslie,et al.  Do six-month-old infants perceive causality? , 1987, Cognition.

[81]  R. Bogartz,et al.  Object Permanence in Five-and-a-Half-Month-Old Infants? , 2000, Infancy : the official journal of the International Society on Infant Studies.

[82]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[83]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[84]  Alfred L. Horowitz.  MRI Physics for Radiologists: A Visual Approach , 1993 .

[85]  Stuart J. Russell,et al.  Reinforcement Learning with Hierarchies of Machines , 1997, NIPS.

[86]  Gary F. Marcus.  The modules behind the learning , 2002 .

[87]  L. Cohen,et al.  Eight-month-old infants' perception of possible and impossible events , 1998 .