Modeling the effects of neuromodulation on internal brain areas: Serotonin and dopamine

The effects of neuromodulator, such as serotonin and dopamine, on individual neurons in the brain have been known qualitatively. However, it is challenging to computationally model such effects in an emergent network, as the elements of internal representations do not have a static, task-specific meaning. Weng and coworkers modeled the effects of serotonin and dopamine on only motor neurons in emergent networks. In this work, we extend the effects of serotonin and dopamine to all neurons inside the emergent network. Our new theory is that although serotonin and dopamine indicate events of different natures (aversive and appetitive), they produce similar effects on internal non-motor neurons in that they increase their learning rates from the cases without serotonin and dopamine. This is because the presence of serotonin and dopamine indicates a higher importance of the event compared with baseline cases. Experimentally, we show that the enhanced developmental network learns faster under a limited resource.

[1]  Juyang Weng,et al.  Modeling dopamine and serotonin systems in a visual recognition network , 2011, The 2011 International Joint Conference on Neural Networks.

[2]  Juyang Weng,et al.  Dually Optimal Neuronal Layers: Lobe Component Analysis , 2009, IEEE Transactions on Autonomous Mental Development.

[3]  Jeffrey L. Krichmar,et al.  Neuromodulation as a robot controller , 2009, IEEE Robotics & Automation Magazine.

[4]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[5]  DayanPeter,et al.  Opponent interactions between serotonin and dopamine , 2002 .

[6]  J. Vasterling,et al.  Attention, learning, and memory performances and intellectual resources in Vietnam veterans: PTSD and no disorder comparisons. , 2002, Neuropsychology.

[7]  Jonathan D. Cohen,et al.  Computational roles for dopamine in behavioural control , 2004, Nature.

[8]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[9]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  Sebastiano Cavallaro,et al.  Genomic analysis of serotonin receptors in learning and memory , 2008, Behavioural Brain Research.

[11]  G. Edelman,et al.  Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. , 1998, Cerebral cortex.

[12]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[13]  Olaf Sporns,et al.  Plasticity in Value Systems and its Role in Adaptive Behavior , 2000, Adapt. Behav..

[14]  Juyang Weng,et al.  Where-What Network 3: Developmental top-down attention for multiple foregrounds and complex backgrounds , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Juyang Weng,et al.  Where-what network 1: “Where” and “what” assist each other through top-down connections , 2008, 2008 7th IEEE International Conference on Development and Learning.

[17]  Ulrich Müller,et al.  Serotonin Modulates Striatal Responses to Fairness and Retaliation in Humans , 2013, The Journal of Neuroscience.

[18]  Ronald L. Davis,et al.  Distinct Traces for Appetitive versus Aversive Olfactory Memories in DPM Neurons of Drosophila , 2012, Current Biology.

[19]  Francesco Mannella,et al.  Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. , 2013, Neural networks : the official journal of the International Neural Network Society.

[20]  Juyang Weng,et al.  WWN-2: A biologically inspired neural network for concurrent visual attention and recognition , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  P. W. Jones,et al.  Bandit Problems, Sequential Allocation of Experiments , 1987 .

[23]  Juyang Weng,et al.  Where-What Network 5: Dealing with scales for objects in complex backgrounds , 2011, The 2011 International Joint Conference on Neural Networks.

[24]  M. Paradiso,et al.  Neuroscience: Exploring the Brain , 1996 .

[25]  Juyang Weng,et al.  2008 Special issue , 2008 .

[26]  P. Pini.  Addiction , 1996, The Lancet.

[27]  S. Hyman,et al.  Addiction, Dopamine, and the Molecular Mechanisms of Memory , 2000, Neuron.

[28]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[29]  Peter Salovey,et al.  Mood Regulation and Memory: Repairing Sad Moods with Happy Memories , 1996 .

[30]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[31]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[32]  Juyang Weng.  Brain Model For Developmental Robots : The Spatial Brain for Any Temporal Lengths , 2010 .