Neurodynamics of mental exploration

Thinking allows an animal to take an effective action in a novel situation based on a mental exploration of possibilities and previous knowledge. We describe a model animal, with a neural system based loosely on the rodent hippocampus, which performs mental exploration to find a useful route in a spatial world it has previously learned. It then mentally recapitulates the chosen route, and this intent is converted to motor acts that move the animal physically along the route. The modeling is based on spiking neurons with spike-frequency adaptation. Adaptation causes the continuing evolution in the pattern of neural activity that is essential to mental exploration. A successful mental exploration is remembered through spike-timing-dependent synaptic plasticity. The system is also an episodic memory for an animal chiefly concerned with locations.

[1]  M. Wilson,et al.  Temporally Structured Replay of Awake Hippocampal Ensemble Activity during Rapid Eye Movement Sleep , 2001, Neuron.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  S. Nelson,et al.  Structural and Functional Properties of Hippocampal Neurons , 2006 .

[4]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[5]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[6]  H. Berg,et al.  Temporal comparisons in bacterial chemotaxis. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[7]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[8]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[9]  E. Tulving Episodic memory: from mind to brain. , 2002, Annual review of psychology.

[10]  H. Eichenbaum,et al.  The Hippocampus, Memory, and Place Cells Is It Spatial Memory or a Memory Space? , 1999, Neuron.

[11]  J. P. Seward An experimental analysis of latent learning. , 1949, Journal of experimental psychology.

[12]  E. Izhikevich Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[13]  Emilio Kropff,et al.  Place cells, grid cells, and the brain's spatial representation system. , 2008, Annual review of neuroscience.

[14]  John J. Hopfield,et al.  Searching for Memories, Sudoku, Implicit Check Bits, and the Iterative Use of Not-Always-Correct Rapid Neural Computation , 2006, Neural Computation.

[15]  K. Zhang,et al.  Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  G. Buzsáki,et al.  Forward and reverse hippocampal place-cell sequences during ripples , 2007, Nature Neuroscience.

[17]  O. John Hippocampal Neurophysiology in the Behaving Animal , 2006 .

[18]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.