Cross-modal and scale-free action representations through enaction

Embodied action representation and action understanding are the first steps to understand what it means to communicate. We present a biologically plausible mechanism to the representation and the recognition of actions in a neural network with spiking neurons based on the learning mechanism of spike-timing-dependent plasticity (STDP). We show how grasping is represented through the multi-modal integration between the vision and tactile maps across multiple temporal scales. The network evolves into a small-world organization with scale-free dynamics promoting efficient inter-modal binding of the neural assemblies with accurate timing. Finally, it acquires the qualitative properties of the mirror neuron system to trigger an observed action performed by someone else.

[1]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[2]  Toshio Aoyagi,et al.  Synchrony-Induced Switching Behavior of Spike Pattern Attractors Created by Spike-Timing-Dependent Plasticity , 2007, Neural Computation.

[3]  Ichiro Tsuda,et al.  Chaotic itinerancy as a mechanism of irregular changes between synchronization and desynchronization in a neural network. , 2004, Journal of integrative neuroscience.

[4]  C. Hofsten,et al.  Infants predict other people's action goals , 2006, Nature Neuroscience.

[5]  G. Ermentrout Dynamic patterns: The self-organization of brain and behavior , 1997 .

[6]  Gerald M. Edelman,et al.  A Universe of Consciousness , 2000 .

[7]  P. Rochat Five levels of self-awareness as they unfold early in life , 2003, Consciousness and Cognition.

[8]  P. Rochat Self-perception and action in infancy , 1998, Experimental Brain Research.

[9]  P. Rochat,et al.  Perceived self in infancy , 2000 .

[10]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[11]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[12]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  Yiannis Demiris,et al.  Perceiving the unusual: Temporal properties of hierarchical motor representations for action perception , 2006, Neural Networks.

[14]  G. Buzsáki Rhythms of the brain , 2006 .

[15]  C. Heyes Causes and consequences of imitation , 2001, Trends in Cognitive Sciences.

[16]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[17]  Frank E. Pollick,et al.  Neural Substrates for Action Understanding at Different Description Levels in the Human Brain , 2008, Journal of Cognitive Neuroscience.

[18]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[19]  R. Miall,et al.  Connecting mirror neurons and forward models. , 2003, Neuroreport.

[20]  Pierre Kornprobst,et al.  From Light to Spikes: a Large-Scale Retina Simulator , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[21]  Yasuo Kuniyoshi,et al.  A deformable and deformation sensitive tactile distribution sensor , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[22]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB , 2004, Neural Networks.

[23]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[24]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[25]  Yasuo Kuniyoshi,et al.  A highly stretchable tactile distribution sensor for smooth surfaced humanoids , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[26]  G. Edelman Neural Darwinism: The Theory Of Neuronal Group Selection , 1989 .

[27]  J. Changeux,et al.  The Physiology of Truth: Neuroscience and Human Knowledge , 2004 .

[28]  Robert W. Mitchell,et al.  Self-Awareness in Animals and Humans: Developmental Perspectives , 1994 .

[29]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[30]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[31]  A. Opstal Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[32]  Kazuo Hiraki Detecting contingency: A key to understanding development of self and social cognition , 2006 .

[33]  E. Thelen,et al.  Motor constraints on the development of perception-action matching in infant reaching , 2000 .

[34]  Masayuki Inaba,et al.  From visuo-motor self learning to early imitation-a neural architecture for humanoid learning , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[35]  A. Berthoz,et al.  Le sens du mouvement , 1997 .

[36]  Christopher G. Prince,et al.  Synching models with infants: a perceptual-level model of infant audio-visual synchrony detection , 2005, Cognitive Systems Research.

[37]  Zoubin Ghahramani,et al.  Perspectives and problems in motor learning , 2001, Trends in Cognitive Sciences.

[38]  V. Gallese,et al.  Embodied simulation: From neurons to phenomenal experience , 2005 .

[39]  Sandra Blakeslee,et al.  Phantoms in the Brain , 1998 .

[40]  Patricia Zukow-Goldring Action to Language via the Mirror Neuron System: Assisted imitation: affordances, effectivities, and the mirror system in early language development , 2006 .

[41]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[42]  Koen Lamberts,et al.  The encoding–retrieval relationship: retrieval as mental simulation , 2008, Trends in Cognitive Sciences.

[43]  D. Perrett,et al.  Opinion TRENDS in Cognitive Sciences Vol.8 No.11 November 2004 Demystifying social cognition: a Hebbian perspective , 2022 .

[44]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[45]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[46]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[47]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[48]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[49]  A. Meltzoff,et al.  Imitation of Facial and Manual Gestures by Human Neonates , 1977, Science.

[50]  I. Tsuda Chaotic itinerancy as a dynamical basis of hermeneutics in brain and mind , 1991 .

[51]  J. Nadel,et al.  Experiencing contingency and agency : First step toward self-understanding in making a mind? , 2005 .

[52]  M. Brass,et al.  Imitation: is cognitive neuroscience solving the correspondence problem? , 2005, Trends in Cognitive Sciences.

[53]  John S. Watson Self-awareness in animals and humans: Detection of self: The perfect algorithm , 1994 .

[54]  A. Selverston,et al.  Dynamical principles in neuroscience , 2006 .