Computational and Robotic Models of the Hierarchical Organization of Behavior: An Overview

The hierarchical organisation of behaviour is a fundamental means through which robots and organisms can acquire and produce sophisticated and flexible behaviours that allow them to solve multiple tasks in multiple conditions. Recently, the research on this topic has been receiving increasing attention. On the one hand, machine learning and robotics are recognising the fundamental importance of the hierarchical organisation of behaviour for building robots that scale up to solve complex tasks, possibly in a cumulative fashion. On the other hand, research in psychology and neuroscience is finding increasing evidence that modularity and hierarchy are pivotal organisation principles of behaviour and of the brain. This book reviews the state of the art in computational and robotic models of the hierarchical organisation of behaviour. Each contribution reviews the main works of the authors on this subject, the open challenges, and promising research directions. Together, the contributions give a good coverage of the most important models, findings, and challenges of the field. This introductory chapter presents the general aims and scope of the book and briefly summarises the contents of each chapter.

[1]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[2]  R. French.  Catastrophic Forgetting in Connectionist Networks , 2006 .

[3]  Edward T. Bullmore,et al.  Modular and Hierarchically Modular Organization of Brain Networks , 2010, Front. Neurosci..

[4]  Stephen Hart,et al.  Learning Generalizable Control Programs , 2011, IEEE Transactions on Autonomous Mental Development.

[5]  J. Fuster.  The Prefrontal Cortex—An Update Time Is of the Essence , 2001, Neuron.

[6]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[7]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[8]  K. Fischer.  A theory of cognitive development: The control and construction of hierarchies of skills. , 1980 .

[9]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[10]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[11]  Gordon D. Logan,et al.  Hierarchical control of cognitive processes: switching tasks in sequences. , 2006 .

[12]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[13]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[14]  A. Graybiel.  The Basal Ganglia and Chunking of Action Repertoires , 1998, Neurobiology of Learning and Memory.

[15]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[16]  Marco Mirolli,et al.  What are the Key Open Challenges for Understanding Autonomous Cumulative Learning of Skills ? , 2010 .

[17]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[18]  Satinder Singh.  Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[19]  M. Graziano.  The organization of behavioral repertoire in motor cortex. , 2006, Annual review of neuroscience.

[20]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[21]  Bram Bakker,et al.  Hierarchical Reinforcement Learning Based on Subgoal Discovery and Subpolicy Specialization , 2003 .

[22]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.