Learning Temporal Intervals in Neural Dynamics

Storing and reproducing temporal intervals is an important component of perception, action generation, and learning. How temporal intervals can be represented in neuronal networks is thus an important research question both in study of biological organisms and artificial neuromorphic systems. Here, we introduce a neural-dynamic computing architecture for learning temporal durations of actions. The architecture uses a dynamic neural fields (DNFs) representation of the elapsed time and a memory trace dynamics to store the experienced action duration. Interconnected dynamical nodes signal beginning of an action, its successful accomplishment, or failure, and activate formation of the memory trace that corresponds to the action’s duration. The accumulated memory trace influences the competition between the dynamical nodes in such a way that the failure node gains a competitive advantage earlier if the stored duration is shorter. The model uses neurally based DNF dynamics and is a process model of how temporal durations may be stored in neural systems, both biological and artificial ones. The focus of this paper is on the mechanism to store and use duration in artificial neuronal systems. The model is validated in closed-loop experiments with a simulated robot.

[1]  Richard B Ivry,et al.  Temporal Control and Coordination: The Multiple Timer Model , 2002, Brain and Cognition.

[2]  S. Lisberger,et al.  The Cerebellum: A Neuronal Learning Machine? , 1996, Science.

[3]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[4]  M. Shadlen,et al.  Exploring the Neurophysiology of Decisions , 1998, Neuron.

[5]  W. Meck.  Neuropsychology of timing and time perception , 2005, Brain and Cognition.

[6]  Geoffrey M. Ghose,et al.  Temporal Production Signals in Parietal Cortex , 2012, PLoS biology.

[7]  Estela Bicho,et al.  Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning , 2006, Robotics Auton. Syst..

[8]  Jochen J. Steil,et al.  Solving the Distal Reward Problem with Rare Correlations , 2013, Neural Computation.

[9]  D. Buonomano,et al.  Population clocks: motor timing with neural dynamics , 2010, Trends in Cognitive Sciences.

[10]  José Carlos Príncipe,et al.  Special issue on echo state networks and liquid state machines , 2007, Neural Networks.

[11]  Javier F. Medina,et al.  Timing Mechanisms in the Cerebellum: Testing Predictions of a Large-Scale Computer Simulation , 2000, The Journal of Neuroscience.

[12]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[13]  John E. Schlerf,et al.  Dedicated and intrinsic models of time perception , 2008, Trends in Cognitive Sciences.

[14]  E. Thelen,et al.  The dynamics of embodiment: A field theory of infant perseverative reaching , 2001, Behavioral and Brain Sciences.

[15]  N. Donegan,et al.  A model of Pavlovian eyelid conditioning based on the synaptic organization of the cerebellum. , 1997, Learning & memory.

[16]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[17]  Christian Faubel.  Fast learning to recognize objects : Dynamic Fields in label-feature spaces , 2006 .

[18]  D. Buonomano,et al.  Temporal Perceptual Learning , 2014 .

[19]  S. Grossberg,et al.  The Hippocampus and Cerebellum in Adaptively Timed Learning, Recognition, and Movement , 1996, Journal of Cognitive Neuroscience.

[20]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[21]  G. Schoner,et al.  Target position estimation, target acquisition, and obstacle avoidance , 1997, ISIE '97 Proceeding of the IEEE International Symposium on Industrial Electronics.

[22]  S. Keele,et al.  The cognitive and neural architecture of sequence representation. , 2003, Psychological review.

[23]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[24]  Stephen Grossberg,et al.  A neural model of timed response learning in the cerebellum , 1994, Neural Networks.

[25]  R. Romo,et al.  Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. , 2003, Cerebral cortex.

[26]  Stephen Grossberg,et al.  Neural dynamics of adaptive timing and temporal discrimination during associative learning , 1989, Neural Networks.

[27]  E. Thelen,et al.  Using dynamic field theory to rethink infant habituation. , 2006, Psychological review.

[28]  Yulia Sandamirskaya,et al.  Neural dynamics of hierarchically organized sequences: A robotic implementation , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[29]  U. Karmarkar,et al.  Timing in the Absence of Clocks: Encoding Time in Neural Network States , 2007, Neuron.

[30]  Gregor Schöner,et al.  The time course of saccadic decision making: Dynamic field theory , 2006, Neural Networks.

[31]  Gregor Schöner,et al.  Dynamic Thinking : A Primer on Dynamic Field Theory , 2015 .

[32]  H. Eichenbaum.  Time cells in the hippocampus: a new dimension for mapping memories , 2014, Nature Reviews Neuroscience.

[33]  Yulia Sandamirskaya,et al.  Simultaneous planning and action: neural-dynamic sequencing of elementary behaviors in robot navigation , 2015, Adapt. Behav..

[34]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Anne R. Schutte,et al.  Testing the dynamic field theory: working memory for locations becomes more spatially precise over development. , 2003, Child development.

[36]  R. Ivry,et al.  The neural representation of time , 2004, Current Opinion in Neurobiology.

[37]  Dean V Buonomano,et al.  Book Review: How Do We Tell Time? , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[38]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[39]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[40]  Stephan K. U. Zibner,et al.  Using Dynamic Field Theory to extend the embodiment stance toward higher cognition , 2013 .

[41]  Dean V. Buonomano,et al.  Timing as an intrinsic property of neural networks: evidence from in vivo and in vitro experiments , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[42]  Tadashi Yamazaki,et al.  A Computational Mechanism for Unified Gain and Timing Control in the Cerebellum , 2012, PloS one.

[43]  Anne R. Schutte,et al.  Generalizing the dynamic field theory of spatial cognition across real and developmental time scales , 2008, Brain Research.

[44]  A. Nobre,et al.  The Cerebellum Predicts the Timing of Perceptual Events , 2008, The Journal of Neuroscience.

[45]  R. Ivry.  The representation of temporal information in perception and motor control , 1996, Current Opinion in Neurobiology.

[46]  P. Balsam,et al.  Memory Reconsolidation: Time to Change Your Mind , 2013, Current Biology.

[47]  Yulia Sandamirskaya,et al.  Dynamic neural fields as a step toward cognitive neuromorphic architectures , 2014, Front. Neurosci..

[48]  Ann M Graybiel,et al.  Neural representation of time in cortico-basal ganglia circuits , 2009, Proceedings of the National Academy of Sciences.

[49]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[50]  Gregor Schöner,et al.  A neural-dynamic architecture for behavioral organization of an embodied agent , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[51]  Edward W. Large,et al.  Perceiving temporal regularity in music , 2002, Cogn. Sci..

[52]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[53]  Estela Bicho,et al.  Target Representation on an Autonomous Vehicle with Low-Level Sensors , 2000, Int. J. Robotics Res..

[54]  Daniel Durstewitz,et al.  Self-Organizing Neural Integrator Predicts Interval Times through Climbing Activity , 2003, The Journal of Neuroscience.

[55]  Jürgen Schmidhuber,et al.  Autonomous reinforcement of behavioral sequences in neural dynamics , 2012, The 2013 International Joint Conference on Neural Networks (IJCNN).

[56]  Giacomo Indiveri,et al.  Memory and Information Processing in Neuromorphic Systems , 2015, Proceedings of the IEEE.

[57]  J. Tanji,et al.  Interval time coding by neurons in the presupplementary and supplementary motor areas , 2009, Nature Neuroscience.

[58]  Melissa J. Allman,et al.  Pathophysiological distortions in time perception and timed performance. , 2012, Brain : a journal of neurology.

[59]  Julien Vitay,et al.  Timing and expectation of reward: a neuro-computational model of the afferents to the ventral tegmental area , 2014, Front. Neurorobot..

[60]  D V Buonomano,et al.  Decoding Temporal Information: A Model Based on Short-Term Synaptic Plasticity , 2000, The Journal of Neuroscience.

[61]  Karen Cheng,et al.  Learning of Temporal Motor Patterns: An Analysis of Continuous Versus Reset Timing , 2011, Front. Integr. Neurosci..

[62]  David Whitaker,et al.  Duration channels mediate human time perception , 2011, Proceedings of the Royal Society B: Biological Sciences.

[63]  Dean V Buonomano,et al.  The biology of time across different scales. , 2007, Nature chemical biology.

[64]  G. Schöner,et al.  Dynamic Field Theory of Movement Preparation , 2022 .

[65]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[66]  R. Knight,et al.  Cortical Networks Underlying Mechanisms of Time Perception , 1998, The Journal of Neuroscience.

[67]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[68]  Gregor Schöner,et al.  A robotic architecture for action selection and behavioral organization inspired by human cognition , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[69]  Gregor Schöner,et al.  An embodied account of serial order: How instabilities drive sequence generation , 2010, Neural Networks.

[70]  Estela Bicho,et al.  The dynamic neural field approach to cognitive robotics , 2006, Journal of neural engineering.

[71]  Giacomo Indiveri,et al.  Synthesizing cognition in neuromorphic electronic systems , 2013, Proceedings of the National Academy of Sciences.

[72]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[73]  Gregor Schöner,et al.  Serial order in an acting system: A multidimensional dynamic neural fields implementation , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[74]  M. Nicolelis,et al.  Decoding of temporal intervals from cortical ensemble activity. , 2008, Journal of neurophysiology.

[75]  Gregor Schöner,et al.  A Dynamic Field Architecture for the Generation of Hierarchically Organized Sequences , 2012, ICANN.