Facilitating neural dynamics for delay compensation and prediction in evolutionary neural networks

Delay in the nervous system is a serious issue for an organism that needs to act in real time. For example, during the time a signal travels from a peripheral sensor to the central nervous system, a moving object in the environment can cover a significant distance which can lead to critical errors in the effect of the corresponding motor output. This paper proposes that facilitating synapses which show a dynamic sensitivity to the changing input may play an important role in compensating for neural delays, through extrapolation. The idea was tested in a modified 2D pole-balancing problem which included sensory delays. Within this domain, we tested the behavior of recurrent neural networks with facilitatory neural dynamics trained via neuroevolution. Analysis of the performance and the evolved network parameters showed that, under various forms of delay, networks utilizing extrapolatory dynamics are at a significant competitive advantage compared to networks without such dynamics. In sum, facilitatory (or extrapolatory) dynamics can be used to compensate for delay at a single-neuron level, thus allowing a developing nervous system to stay in touch with the present environmental state.

[1]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[2]  K. Gegenfurtner,et al.  Neuronal Processing Delays Are Compensated in the Sensorimotor Branch of the Visual System , 2003, Current Biology.

[3]  Wolfram Erlhagen,et al.  The role of action plans and other cognitive factors in motion extrapolation: A modelling study , 2004 .

[4]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[5]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[6]  Yoonsuck Choe,et al.  Delay Compensation Through Facilitating Synapses and STDP: A Neural Basis for Orientation Flash-Lag Effect , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[7]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[8]  T J Sejnowski,et al.  Motion integration and postdiction in visual awareness. , 2000, Science.

[9]  Romi Nijhawan,et al.  Motion extrapolation in catching , 1994, Nature.

[10]  B. Sakmann,et al.  Transmitter release modulation by intracellular Ca2+ buffers in facilitating and depressing nerve terminals of pyramidal cells in layer 2/3 of the rat neocortex indicates a target cell‐specific difference in presynaptic calcium dynamics , 2001, The Journal of physiology.

[11]  Ralf Möller.  Perception through Anticipation –- An Approach to Behaviour-based Perception , 1997 .

[12]  William H. Press,et al.  Book-Review - Numerical Recipes in Pascal - the Art of Scientific Computing , 1989 .

[13]  I. Murakami,et al.  Latency difference, not spatial extrapolation , 1998, Nature Neuroscience.

[14]  D. Wolpert,et al.  Central cancellation of self-produced tickle sensation , 1998, Nature Neuroscience.

[15]  Stefan Schaal,et al.  Forward models in visuomotor control. , 2002, Journal of neurophysiology.

[16]  Yoonsuck Choe,et al.  The role of temporal parameters in a thalamocortical model of analogy , 2004, IEEE Transactions on Neural Networks.

[17]  Wulfram Gerstner,et al.  Hebbian learning of pulse timing in the Barn Owl auditory system , 1999 .

[18]  Y Dan,et al.  Motion-Induced Perceptual Extrapolation of Blurred Visual Targets , 2001, The Journal of Neuroscience.

[19]  Jack D. Cowan,et al.  DYNAMICS OF SELF-ORGANIZED DELAY ADAPTATION , 1999 .

[20]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning Through Evolving Neural Network Topologies , 2002, GECCO.

[21]  Horst-Michael Groß,et al.  Generative character of perception: a neural architecture for sensorimotor anticipation , 1999, Neural Networks.

[22]  Risto Miikkulainen,et al.  2-D Pole Balancing with Recurrent Evolutionary Networks , 1998 .

[23]  J. Elman.  Distributed Representations, Simple Recurrent Networks, And Grammatical Structure , 1991 .

[24]  Henry Markram,et al.  Coding of temporal information by activity-dependent synapses. , 2002, Journal of neurophysiology.

[25]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[26]  Yoonsuck Choe,et al.  Facilitatory neural activity compensating for neural delays as a potential cause of the flash-lag effect , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[27]  E. Fortune,et al.  Short-term synaptic plasticity as a temporal filter , 2001, Trends in Neurosciences.

[28]  T. Tsumoto,et al.  Change of conduction velocity by regional myelination yields constant latency irrespective of distance between thalamus and cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[30]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[31]  DeLiang Wang,et al.  Temporal pattern processing , 1998 .

[32]  D. Wolpert,et al.  Motor prediction , 2001, Current Biology.

[33]  H. Markram,et al.  Synaptic dynamics control the timing of neuronal excitation in the activated neocortical microcircuit , 2004, The Journal of physiology.

[34]  S. Thorpe,et al.  Seeking Categories in the Brain , 2001, Science.

[35]  B. Webb.  Neural mechanisms for prediction: do insects have forward models? , 2004, Trends in Neurosciences.

[36]  W. Press,et al.  Numerical Recipes in Fortran: The Art of Scientific Computing.@@@Numerical Recipes in C: The Art of Scientific Computing. , 1994 .

[37]  Theodore W. Berger,et al.  Dynamic synapse: Harnessing the computing power of synaptic dynamics , 1999, Neurocomputing.

[38]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.