Parsing of action sequences: A neural dynamics approach

Abstract Parsing of action sequences is the process of segmenting observed behavior into individual actions. In robotics, this process is critical for imitation learning from observation and for representing an observed behavior in a form that may be communicated to a human. In this paper, we develop a model for action parsing, based on our understanding of principles of grounded cognitive processes, such as perceptual decision making, behavioral organization, and memory formation.We present a neural-dynamic architecture, in which action sequences are parsed using a mathematical and conceptual framework for embodied cognition—the Dynamic Field Theory. In this framework, we introduce a novel mechanism, which allows us to detect and memorize actions that are extended in time and are parametrized by the target object of an action. The core properties of the architecture are demonstrated in a set of simple, proof-of-concept experiments.

[1]  J. Munkres.  ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[2]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[3]  Yulia Sandamirskaya,et al.  Neural dynamics of hierarchically organized sequences: A robotic implementation , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[4]  Christian Faubel,et al.  Learning to recognize objects on the fly: A neurally based dynamic field approach , 2008, Neural Networks.

[5]  Stephan K. U. Zibner,et al.  Using Dynamic Field Theory to extend the embodiment stance toward higher cognition , 2013 .

[6]  Yulia Sandamirskaya,et al.  Dynamic neural fields as a step toward cognitive neuromorphic architectures , 2014, Front. Neurosci..

[7]  Yiannis Demiris,et al.  Towards incremental learning of task-dependent action sequences using probabilistic parsing , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[8]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[9]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[10]  Christian Faubel,et al.  The Counter-Change Model of Motion Perception: An Account Based on Dynamic Field Theory , 2012, ICANN.

[11]  Yiannis Aloimonos,et al.  The syntax of human actions and interactions , 2012, Journal of Neurolinguistics.

[12]  Gregor Schöner,et al.  Serial order in an acting system: A multidimensional dynamic neural fields implementation , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[13]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[14]  Stephan K. U. Zibner,et al.  Dynamic Neural Fields as Building Blocks of a Cortex-Inspired Architecture for Robotic Scene Representation , 2011, IEEE Transactions on Autonomous Mental Development.

[15]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[16]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[17]  Gregor Sch,et al.  Dynamical Systems Approaches to Cognition , 2008 .

[18]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[19]  Yulia Sandamirskaya,et al.  Neural-dynamic architecture for looking: Shift from visual to motor target representation for memory saccades , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[20]  Yiannis Aloimonos,et al.  Sensory grammars for sensor networks , 2009, J. Ambient Intell. Smart Environ..

[21]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[22]  Gregor Schöner,et al.  A neural-dynamic architecture for behavioral organization of an embodied agent , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[23]  Martin A. Giese,et al.  Learning Representations of Animated Motion Sequences - A Neural Model , 2014, Top. Cogn. Sci..

[24]  Gregor Schöner,et al.  A robotic architecture for action selection and behavioral organization inspired by human cognition , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Gregor Schöner,et al.  An embodied account of serial order: How instabilities drive sequence generation , 2010, Neural Networks.

[26]  V. Caggiano,et al.  Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions , 2013, The Journal of Neuroscience.

[27]  Ales Ude,et al.  A Simple Ontology of Manipulation Actions Based on Hand-Object Relations , 2013, IEEE Transactions on Autonomous Mental Development.

[28]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .