Modular and hierarchical brain organization to understand assimilation, accommodation and their relation to autism in reaching tasks: a developmental robotics hypothesis

By assimilation children embody sensorimotor experiences into already built mental structures. Conversely, by accommodation these structures are changed according to the child’s new experience. Despite the intuitive power of these concepts to trace the course of sensorimotor development, they have gradually lost ground in psychology. This likely due to the lack of brain-related views capturing the dynamic mechanisms underlying them. Here we propose that brain modular and hierarchical organization is crucial to understanding assimilation/accommodation. We devise an experiment where a bio-inspired modular and hierarchical mixture-of-experts model guides a simulated robot to learn different reaching tasks by trial-and-error. The model gives a novel interpretation of assimilation/accommodation based on the functional organization of the experts allocated through learning. Assimilation occurs when the model adapts a copy of the expert trained for solving a task, to face another task requiring similar sensorimotor mappings. Experts storing similar sensorimotor mappings belong to the same functional module. Accommodation occurs when the model uses non-trained experts to face tasks requiring different sensorimotor mappings (generating a new functional group of experts). The model also provides a new theoretical framework to investigate assimilation/accommodation impairment, and proposes that such impairment might be related to autism spectrum disorder.

[1]  G. Edelman Neural Darwinism: The Theory Of Neuronal Group Selection , 1989 .

[2]  Patrick Dupont,et al.  Graph analysis of functional brain networks for cognitive control of action in traumatic brain injury. , 2012, Brain : a journal of neurology.

[3]  R. K. Clifton,et al.  Is visually guided reaching in early infancy a myth? , 1993, Child development.

[4]  Giovanni Pezzulo,et al.  A spiking neuron model of the cortico-basal ganglia circuits for goal-directed and habitual action learning. , 2013, Neural networks : the official journal of the International Neural Network Society.

[5]  Margaret Wilson,et al.  The mirror reflects both ways: Action influences perception of others , 2010, Brain and Cognition.

[6]  A. Barto,et al.  Approximate optimal control as a model for motor learning. , 2005, Psychological review.

[7]  N. Berthier,et al.  Visual information and object size in infant reaching. , 2010, Infant behavior & development.

[8]  Gianluca Baldassarre,et al.  A modular neural-network model of the basal ganglia’s role in learning and selecting motor behaviours , 2002, Cognitive Systems Research.

[9]  Angelo Cangelosi,et al.  An open-source simulator for cognitive robotics research: the prototype of the iCub humanoid robot simulator , 2008, PerMIS.

[10]  S. Baron-Cohen,et al.  Task-related functional connectivity in autism spectrum conditions: an EEG study using wavelet transform coherence , 2013, Molecular Autism.

[11]  K. H. Grobman,et al.  Artificial life and Piaget , 2003, Trends in Cognitive Sciences.

[12]  Giulio Sandini,et al.  The iCub Cognitive Humanoid Robot: An Open-System Research Platform for Enactive Cognition , 2006, 50 Years of Artificial Intelligence.

[13]  E. Thelen,et al.  The transition to reaching: mapping intention and intrinsic dynamics. , 1993, Child development.

[14]  Francesco Mannella,et al.  The "Mechatronic Board": A Tool to Study Intrinsic Motivations in Humans, Monkeys, and Humanoid Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[15]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[16]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  P. V. Geert,et al.  A dynamic systems model of basic developmental mechanisms: Piaget, Vygotsky, and beyond , 1998 .

[19]  W. Penfield,et al.  SOMATIC MOTOR AND SENSORY REPRESENTATION IN THE CEREBRAL CORTEX OF MAN AS STUDIED BY ELECTRICAL STIMULATION , 1937 .

[20]  Tom Schaul,et al.  The two-dimensional organization of behavior , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[21]  C. Hofsten An action perspective on motor development , 2004, Trends in Cognitive Sciences.

[22]  Marco Mirolli,et al.  Computational and Robotic Models of the Hierarchical Organization of Behavior , 2013, Springer Berlin Heidelberg.

[23]  J. Fuster The Prefrontal Cortex—An Update Time Is of the Essence , 2001, Neuron.

[24]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[25]  A. J. Rasheed,et al.  Generalization Aspect of Neural Networks on Upgrading Assimilation Structure into Accommodating Scheme , 2009 .

[26]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[27]  Fumiya Iida,et al.  50 Years of Artificial Intelligence, Essays Dedicated to the 50th Anniversary of Artificial Intelligence , 2007, 50 Years of Artificial Intelligence.

[28]  Gianluca Baldassarre,et al.  What are intrinsic motivations? A biological perspective , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[29]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[30]  S. Kastner,et al.  Complex organization of human primary motor cortex: a high-resolution fMRI study. , 2008, Journal of neurophysiology.

[31]  Francesco Mannella,et al.  The Hierarchical Organisation of Cortical and Basal-Ganglia Systems: A Computationally-Informed Review and Integrated Hypothesis , 2013, Computational and Robotic Models of the Hierarchical Organization of Behavior.

[32]  Douglas S. Blank,et al.  Introduction to developmental robotics , 2006, Connect. Sci..

[33]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. I. A new functional anatomy , 2001, Biological Cybernetics.

[34]  D. Parisi,et al.  TRoPICALS: a computational embodied neuroscience model of compatibility effects. , 2010, Psychological review.

[35]  Afdc Hamilton,et al.  The motor hierarchy: from kinematics to goals and intentions , 2007 .

[36]  P. Strick,et al.  Basal ganglia and cerebellar loops: motor and cognitive circuits , 2000, Brain Research Reviews.

[37]  Stephen Grossberg,et al.  Adaptive Resonance Theory , 2010, Encyclopedia of Machine Learning.

[38]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[39]  Deborah J. Serrien,et al.  Decisions of Voluntary Action: What vs When , 2010, Frontiers in neuroscience.

[40]  M. Graziano,et al.  Complex Movements Evoked by Microstimulation of Precentral Cortex , 2002, Neuron.

[41]  P. Strick,et al.  Cerebellum and nonmotor function. , 2009, Annual review of neuroscience.

[42]  P. Strick,et al.  The temporal lobe is a target of output from the basal ganglia. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[44]  A. Graybiel The Basal Ganglia and Chunking of Action Repertoires , 1998, Neurobiology of Learning and Memory.

[45]  Michael P. Wellman,et al.  Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems , 2006, AAMAS 2006.

[46]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[47]  G. Rizzolatti,et al.  Impairment of actions chains in autism and its possible role in intention understanding , 2007, Proceedings of the National Academy of Sciences.

[48]  Manuela M. Veloso,et al.  Probabilistic policy reuse in a reinforcement learning agent , 2006, AAMAS '06.

[49]  Stefano Nolfi,et al.  Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems , 1998, Neural Networks.

[50]  Loukia D. Loukopoulos,et al.  Planning reaches by evaluating stored postures. , 1995, Psychological review.

[51]  J. Vercher,et al.  Target and hand position information in the online control of goal-directed arm movements , 2003, Experimental Brain Research.

[52]  Tony Charman,et al.  Development and autism: lessons from developmental psychopathology , 2001 .

[53]  Francesco Mannella,et al.  Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. , 2013, Neural networks : the official journal of the International Neural Network Society.

[54]  Benjamin J. Southwell,et al.  Human Object Recognition Using Colour and Depth Information from an RGB-D Kinect Sensor , 2013 .

[55]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[56]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[57]  T. Shultz Computational Developmental Psychology , 2003 .

[58]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[59]  Alan C. Evans,et al.  Revealing modular architecture of human brain structural networks by using cortical thickness from MRI. , 2008, Cerebral cortex.

[60]  Mitsuo Kawato,et al.  MOSAIC for Multiple-Reward Environments , 2008 .

[61]  Jun Nakanishi,et al.  Learning Movement Primitives , 2005, ISRR.

[62]  Sue Fletcher-Watson,et al.  A review of methods in the study of attention in autism , 2010 .

[63]  Paolo Tommasino,et al.  Reinforcement learning algorithms that assimilate and accommodate skills with multiple tasks , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[64]  Marco C. Bettoni,et al.  Made-Up Minds: A Constructivist Approach to Artificial Intelligence , 1993, IEEE Expert.

[65]  M. Clearfield Learning to walk changes infants' social interactions. , 2011, Infant behavior & development.

[66]  T. Charman,et al.  The Development of Autism : Perspectives From Theory and Research , 2001 .

[67]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[68]  S. Morgan,et al.  Autism and Piaget's theory: Are the two compatible? , 1986, Journal of autism and developmental disorders.

[69]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[70]  Graeme D. Ruxton,et al.  Modelling Perception with Artificial Neural Networks: General themes , 2010 .

[71]  T. Ziemke,et al.  Theories and computational models of affordance and mirror systems: An integrative review , 2013, Neuroscience & Biobehavioral Reviews.

[72]  C. Von Hofsten An action perspective on motor development. , 2004, Trends in cognitive sciences.

[73]  Eugenio Guglielmelli,et al.  The Role of Learning and Kinematic Features in Dexterous Manipulation: A Comparative Study with Two Robotic Hands , 2013 .

[74]  Eugenio Guglielmelli,et al.  A reinforcement learning model of reaching integrating kinematic and dynamic control in a simulated arm robot , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[75]  Yong He,et al.  Graph theoretical modeling of brain connectivity. , 2010, Current opinion in neurology.

[76]  G. Baldassarre,et al.  Learning to select targets within targets in reaching tasks , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[77]  Jun Tani,et al.  Development of hierarchical structures for actions and motor imagery: a constructivist view from synthetic neuro-robotics study , 2009, Psychological research.

[78]  Domenico Parisi,et al.  A Bioinspired Hierarchical Reinforcement Learning Architecture for Modeling Learning of Multiple Skills with Continuous States and Actions , 2010, EpiRob.

[79]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[80]  G. Baldassarre,et al.  Modelling Perception with Artificial Neural Networks: The interplay of Pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat , 2010 .

[81]  Bruno Castro da Silva,et al.  Learning Parameterized Skills , 2012, ICML.

[82]  Angelo Cangelosi,et al.  The iCub Humanoid Robot Simulator , 2008 .

[83]  N. Berthier Learning to reach: A mathematical model. , 1996 .

[84]  G. Humphreys,et al.  The role of motor simulation in action perception: a neuropsychological case study , 2009, Psychological research.

[85]  D. Parisi,et al.  Integrating reinforcement learning, equilibrium points, and minimum variance to understand the development of reaching: a computational model. , 2014, Psychological review.

[86]  G. Wagner,et al.  What does it take to evolve behaviorally complex organisms? , 2003, Bio Systems.

[87]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[88]  Judith W. Rosenthal,et al.  A comparison of cognitive development in normal and psychotic children in the first two years of life from home movies , 1980, Journal of autism and developmental disorders.

[89]  Michael S C Thomas,et al.  Mechanisms of developmental regression in autism and the broader phenotype: a neural network modeling approach. , 2011, Psychological review.

[90]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[91]  F. Volkmar,et al.  Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and Asperger syndrome. , 2000, Archives of general psychiatry.

[92]  M. Gernsbacher,et al.  Infant and toddler oral- and manual-motor skills predict later speech fluency in autism. , 2008, Journal of child psychology and psychiatry, and allied disciplines.

[93]  P. Maruff,et al.  The link between motor impairment level and motor imagery ability in children with developmental coordination disorder. , 2008, Human movement science.

[94]  Angelo Bifone,et al.  Community structure in networks of functional connectivity: Resolving functional organization in the rat brain with pharmacological MRI , 2009, NeuroImage.

[95]  G. Baldassarre,et al.  Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot , 2007, 2007 IEEE 6th International Conference on Development and Learning.

[96]  N. Chater,et al.  Computational models and Rethinking innateness , 1999, Journal of Child Language.

[97]  Loredana Zollo,et al.  Hierarchical reinforcement learning and central pattern generators for modeling the development of rhythmic manipulation skills , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[98]  J. Gittins,et al.  A dynamic allocation index for the discounted multiarmed bandit problem , 1979 .

[99]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[100]  J C Houk,et al.  Action selection and refinement in subcortical loops through basal ganglia and cerebellum , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[101]  Gary L. Drescher,et al.  Made-up minds - a constructivist approach to artificial intelligence , 1991 .

[102]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour , 2001, Biological Cybernetics.

[103]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.

[104]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[105]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[106]  Graeme S. Halford,et al.  Developing Cognitive Competence : New Approaches To Process Modeling , 2015 .

[107]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[108]  S. Grossberg,et al.  Neural dynamics of autistic behaviors: cognitive, emotional, and timing substrates. , 2006, Psychological review.

[109]  Claes von Hofsten,et al.  Action in development. , 2007, Developmental science.

[110]  Marco Mirolli,et al.  Evolving Childhood's Length and Learning Parameters in an Intrinsically Motivated Reinforcement Learning Robot , 2007 .

[111]  P. L. Adams THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[112]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[113]  G. Pezzulo,et al.  Neuroscience and Biobehavioral Reviews the Contribution of Brain Sub-cortical Loops in the Expression and Acquisition of Action Understanding Abilities , 2022 .

[114]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[115]  D. Caligiore,et al.  Vision, action and language unified through embodiment , 2013, Psychological research.

[116]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[117]  Kenneth Smith,et al.  Encyclopaedia of Cognitive Science , 2002 .

[118]  Jan Peters,et al.  Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics , 2022 .

[119]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[120]  Cornelis J. Stam,et al.  Disrupted Functional Brain Networks in Autistic Toddlers , 2013, Brain Connect..

[121]  Domenico Formica,et al.  A mechatronic platform for behavioral analysis on nonhuman primates. , 2012, Journal of integrative neuroscience.

[122]  N. Berthier,et al.  Vision and precision reaching in 15-month-old infants. , 2008, Infant behavior & development.

[123]  Bruno Castro da Silva,et al.  Learning parameterized motor skills on a humanoid robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[124]  T. Aflalo,et al.  Organization of the macaque extrastriate visual cortex re-examined using the principle of spatial continuity of function. , 2011, Journal of neurophysiology.

[125]  Claes von Hofsten,et al.  Action in development. , 2007 .

[126]  Edward T. Bullmore,et al.  Modular and Hierarchically Modular Organization of Brain Networks , 2010, Front. Neurosci..

[127]  J. Piaget Play, dreams and imitation in childhood , 1951 .

[128]  Marco Mirolli,et al.  Evolution and Learning in an Intrinsically Motivated Reinforcement Learning Robot , 2007, ECAL.

[129]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[130]  A. Hamilton,et al.  Motor Abilities in Autism: A Review Using a Computational Context , 2013, Journal of autism and developmental disorders.