Chapter 1 Hierarchies for embodied action perception

During social interactions, humans are capable of initiating and responding to rich and complex social actions despite having incomplete world knowledge as well as physical, perceptual and computational constraints. This capability relies on action perception mechanisms, which exploit regularities in observed goal-oriented behaviours to generate robust predictions, and reduce the workload of sensing systems. To achieve this essential capability, we argue that the following three factors are fundamental. Firstly, human knowledge is frequently hierarchically structured, both in the perceptual and execution domains. Secondly, human perception is an active process driven by current task requirements and context. This is particularly important when the perceptual input is complex (e.g. human motion) and the agent has to operate under embodiment constraints. Thirdly, learning is at the heart of action perception mechanisms, underlying the agent’s ability to add new behaviours to its repertoire. Based on these factors, we review multiple instantiations of a hierarchically-organised biologically-inspired framework for embodied action perception, demonstrating its flexibility in addressing the rich computational contexts of action perception and learning in robotic platforms.

[1]  W. R. Hess,et al.  The functional organization of the diencephalon , 1957 .

[2]  Austin Tate,et al.  Generating Project Networks , 1977, IJCAI.

[3]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[4]  M. Jeannerod Intersegmental coordination during reaching at natural visual objects , 1981 .

[5]  R. Bajcsy Active perception , 1988, Proc. IEEE.

[6]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[7]  HERBERT A. SIMON,et al.  The Architecture of Complexity , 1991 .

[8]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[9]  M. Jeannerod The representing brain: Neural correlates of motor intention and imagery , 1994, Behavioral and Brain Sciences.

[10]  Rajesh P. N. Rao,et al.  An Active Vision Architecture Based on Iconic Representations , 1995, Artif. Intell..

[11]  G. Rizzolatti,et al.  Motor facilitation during action observation: a magnetic stimulation study. , 1995, Journal of neurophysiology.

[12]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[13]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[14]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[15]  A. Goldman,et al.  Mirror neurons and the simulation theory of mind-reading , 1998, Trends in Cognitive Sciences.

[16]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[17]  I. Biederman,et al.  Localizing the cortical region mediating visual awareness of object identity. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[18]  John Demiris,et al.  Movement imitation mechanisms in robots and humans , 1999 .

[19]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[20]  Pat Langley,et al.  Learning Context-Free Grammars with a Simplicity Bias , 2000, ECML.

[21]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[23]  Dario Floreano,et al.  An evolutionary active-vision system , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[24]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[25]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[26]  Á. Pascual-Leone,et al.  Phase-specific modulation of cortical motor output during movement observation , 2001, Neuroreport.

[27]  G. Hesslow Conscious thought as simulation of behaviour and perception , 2002, Trends in Cognitive Sciences.

[28]  Chrystopher L. Nehaniv,et al.  Imitation as a Dual-Route Process Featuring Predictive and Learning Components: A Biologically Plausible Computational Model , 2002 .

[29]  K. Dautenhahn,et al.  The correspondence problem , 2002 .

[30]  Daniel M. Wolpert,et al.  Hierarchical MOSAIC for movement generation , 2003 .

[31]  Yiannis Demiris,et al.  Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning , 2003, Connect. Sci..

[32]  Yiannis Demiris,et al.  Abstraction in Recognition to Solve the Correspondence Problem for Robot Imitation , 2004 .

[33]  Á. Pascual-Leone,et al.  Modulation of premotor mirror neuron activity during observation of unpredictable grasping movements , 2004, The European journal of neuroscience.

[34]  Leslie Pack Kaelbling,et al.  Representing hierarchical POMDPs as DBNs for multi-scale robot localization , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[35]  Rick Grush,et al.  The emulation theory of representation: Motor control, imagery, and perception , 2004, Behavioral and Brain Sciences.

[36]  R. Passingham,et al.  Action observation and acquired motor skills: an FMRI study with expert dancers. , 2005, Cerebral cortex.

[37]  Yiannis Demiris,et al.  Learning Forward Models for Robots , 2005, IJCAI.

[38]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Leila Reddy,et al.  Coding of visual objects in the ventral stream , 2006, Current Opinion in Neurobiology.

[40]  Dario Floreano,et al.  Evolutionary Active Vision Toward Three Dimensional Landmark-Navigation , 2006, SAB.

[41]  Jake K. Aggarwal,et al.  Recognition of Composite Human Activities through Context-Free Grammar Based Representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  Yiannis Demiris,et al.  Content-based control of goal-directed attention during human action perception , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[43]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[44]  Raymond H. Cuijpers,et al.  Goals and means in action observation: A computational approach , 2006, Neural Networks.

[45]  Yiannis Demiris,et al.  Object Grasping using the Minimum Variance Model , 2006, Biological Cybernetics.

[46]  B. Hommel,et al.  Intentional control of attention: action planning primes action-related stimulus dimensions , 2007, Psychological research.

[47]  Shimon Ullman,et al.  Semantic Hierarchies for Recognizing Objects and Parts , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Yiannis Demiris,et al.  Prediction of intent in robotics and multi-agent systems , 2007, Cognitive Processing.

[49]  E. Miller,et al.  Response to Comment on "Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices" , 2007, Science.

[50]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[51]  Scott T. Grafton,et al.  Evidence for a distributed hierarchy of action representation in the brain. , 2007, Human movement science.

[52]  A. P. Dawid,et al.  Generative or Discriminative? Getting the Best of Both Worlds , 2007 .

[53]  C. Keysers,et al.  The Observation and Execution of Actions Share Motor and Somatosensory Voxels in all Tested Subjects: Single-Subject Analyses of Unsmoothed fMRI Data , 2008, Cerebral cortex.

[54]  C. Keysers,et al.  Social Neuroscience: Mirror Neurons Recorded in Humans , 2010, Current Biology.

[55]  Giovanni Pezzulo,et al.  How can bottom-up information shape learning of top-down attention-control skills? , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[56]  George L. Malcolm,et al.  Combining top-down processes to guide eye movements during real-world scene search. , 2010, Journal of vision.

[57]  G. Pezzulo,et al.  When affordances climb into your mind: Advantages of motor simulation in a memory task performed by novice and expert rock climbers , 2010, Brain and Cognition.

[58]  Yiannis Demiris,et al.  Towards One Shot Learning by imitation for humanoid robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[59]  T. Nichols,et al.  The decerebrate cat generates the essential features of the force constraint strategy. , 2010, Journal of neurophysiology.

[60]  A. Goldman,et al.  Simulation theory. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[61]  Geoffrey E. Hinton Learning to represent visual input , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[62]  Yiannis Demiris,et al.  Towards incremental learning of task-dependent action sequences using probabilistic parsing , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[63]  Angelo Cangelosi,et al.  The Mechanics of Embodiment: A Dialog on Embodiment and Computational Modeling , 2011, Front. Psychology.

[64]  Giovanni Pezzulo,et al.  Learning to Grasp Information with Your Own Hands , 2011, TAROS.

[65]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[66]  Yiannis Demiris,et al.  Towards an open-source social middleware for humanoid robots , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[67]  Yiannis Demiris,et al.  Learning reusable task components using hierarchical activity grammars with uncertainties , 2012, 2012 IEEE International Conference on Robotics and Automation.

[68]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.