Desynchronizing of noisy neuron networks using reinforcement learning

Mitigating pathological synchrony of neurons in basal ganglia networks was considered as one of the potential mechanisms of deep brain stimulation (DBS) in treating Parkinson's disease. Motivated by reducing the energy of external stimuli, optimal control strategies are presented to regulate DBS waveform so as to mitigate synchronous oscillations of neural networks with fewer energy expenditure. In this paper, the adaptive optimal control of DBS based on reinforcement learning (RL) is designed to desynchronizing phase models of neural populations in the presence of noise. Numerical simulations show the effectiveness of the proposed method. Moreover, the influence of noise intensity on the control performance of the controller is analyzed.

[1]  J. Fell,et al.  The role of phase synchronization in memory processes , 2011, Nature Reviews Neuroscience.

[2]  W. Singer,et al.  Dynamic predictions: Oscillations and synchrony in top–down processing , 2001, Nature Reviews Neuroscience.

[3]  A. Alonso,et al.  Noise from voltage-gated ion channels may influence neuronal dynamics in the entorhinal cortex. , 1998, Journal of neurophysiology.

[4]  Warren M. Grill,et al.  Selection of stimulus parameters for deep brain stimulation , 2004, Clinical Neurophysiology.

[5]  Csaba Szepesv.  Reinforcement Learning Algorithms for MDPs , 2010 .

[6]  Idan Segev,et al.  Ion Channel Stochasticity May Be Critical in Determining the Reliability and Precision of Spike Timing , 1998, Neural Computation.

[7]  G. Ermentrout,et al.  Phase transition and other phenomena in chains of coupled oscilators , 1990 .

[8]  Eugenio Rodriguez,et al.  Neural synchrony and the development of cortical networks , 2010, Trends in Cognitive Sciences.

[9]  Ali Nabi,et al.  Single input optimal control for globally coupled neuron networks , 2011, Journal of neural engineering.

[10]  P. Krack,et al.  Deep-brain stimulation of the subthalamic nucleus or the pars interna of the globus pallidus in Parkinson's disease. , 2001, The New England journal of medicine.

[11]  S J Schiff,et al.  Model-based rational feedback controller design for closed-loop deep brain stimulation of Parkinson's disease , 2013, Journal of neural engineering.

[12]  Michael Rudolph,et al.  An Extended Analytic Expression for the Membrane Potential Distribution of Conductance-Based Synaptic Noise , 2005, Neural Computation.

[13]  F. Varela,et al.  Perception's shadow: long-distance synchronization of human brain activity , 1999, Nature.

[14]  C. Elger,et al.  Human memory formation is accompanied by rhinal–hippocampal coupling and decoupling , 2001, Nature Neuroscience.

[15]  G. Buzsáki,et al.  Neuronal Oscillations in Cortical Networks , 2004, Science.

[16]  Eric Shea-Brown,et al.  On the Phase Reduction and Response Dynamics of Neural Oscillator Populations , 2004, Neural Computation.

[17]  Xiao-Jiang Feng,et al.  Optimal deep brain stimulation of the subthalamic nucleus—a computational study , 2007, Journal of Computational Neuroscience.

[18]  J. White,et al.  Channel noise in neurons , 2000, Trends in Neurosciences.

[19]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[20]  J. Mink,et al.  Deep brain stimulation. , 2006, Annual review of neuroscience.

[21]  Adrian G. Bors,et al.  Introduction of the Radial Basis Function (RBF) Networks , 2000 .

[22]  Xile Wei,et al.  Effects of couplings on the optimal desynchronizing control of neuronal networks , 2016, Neurocomputing.

[23]  D. Hansel,et al.  Phase Dynamics for Weakly Coupled Hodgkin-Huxley Neurons , 1993 .

[24]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[25]  A Schnitzler,et al.  Review: Deep brain stimulation in Parkinson’s disease , 2009, Therapeutic advances in neurological disorders.