Bayesian adaptive dual control of deep brain stimulation in a computational model of Parkinson’s disease

In this paper, we present a novel Bayesian adaptive dual controller (ADC) for autonomously programming deep brain stimulation devices. We evaluated the Bayesian ADC’s performance in the context of reducing beta power in a computational model of Parkinson’s disease, in which it was tasked with finding the set of stimulation parameters which optimally reduced beta power as fast as possible. Here, the Bayesian ADC has dual goals: (a) to minimize beta power by exploiting the best parameters found so far, and (b) to explore the space to find better parameters, thus allowing for better control in the future. The Bayesian ADC is composed of two parts: an inner parameterized feedback stimulator and an outer parameter adjustment loop. The inner loop operates on a short time scale, delivering stimulus based upon the phase and power of the beta oscillation. The outer loop operates on a long time scale, observing the effects of the stimulation parameters and using Bayesian optimization to intelligently select new parameters to minimize the beta power. We show that the Bayesian ADC can efficiently optimize stimulation parameters, and is superior to other optimization algorithms. The Bayesian ADC provides a robust and general framework for tuning stimulation parameters, can be adapted to use any feedback signal, and is applicable across diseases and stimulator designs.

[1]  Bonnie Nijhof,et al.  A New Fiji-Based Algorithm That Systematically Quantifies Nine Synaptic Parameters Provides Insights into Drosophila NMJ Morphometry , 2016, PLoS Comput. Biol..

[2]  Theoden I. Netoff,et al.  The Sliding Windowed Infinite Fourier Transform [Tips & Tricks] , 2017, IEEE Signal Processing Magazine.

[3]  Jeffrey A. Herron,et al.  Kinematic Adaptive Deep Brain Stimulation for Resting Tremor in Parkinson's Disease , 2016, Movement disorders : official journal of the Movement Disorder Society.

[4]  M. Rosenblum,et al.  Delayed feedback control of collective synchrony: an approach to suppression of pathological brain rhythms. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  HighWire Press The journal of neuroscience : the official journal of the Society for Neuroscience. , 1981 .

[6]  P. Brown,et al.  Reduction in subthalamic 8–35 Hz oscillatory activity correlates with clinical improvement in Parkinson's disease , 2006, The European journal of neuroscience.

[7]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[8]  Rens Verhagen,et al.  Directional steering , 2014, Neurology.

[9]  J. Volkmann,et al.  Introduction to the programming of deep brain stimulators , 2002, Movement disorders : official journal of the Movement Disorder Society.

[10]  Peter Brown,et al.  Oscillations in the Basal Ganglia: The good, the bad, and the unexpected , 2005 .

[11]  K. Miller,et al.  Exaggerated phase–amplitude coupling in the primary motor cortex in Parkinson disease , 2013, Proceedings of the National Academy of Sciences.

[12]  Allison T. Connolly,et al.  Modulations in Oscillatory Frequency and Coupling in Globus Pallidus with Increasing Parkinsonian Severity , 2015, The Journal of Neuroscience.

[13]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[14]  Sacha Jennifer van Albada,et al.  Mean-field modeling of the basal ganglia-thalamocortical system. II Dynamics of parkinsonian oscillations. , 2009, Journal of theoretical biology.

[15]  G Bard Ermentrout,et al.  Phase-response curves and synchronized neural networks , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[16]  Richard Kempter,et al.  State-dependencies of learning across brain scales , 2015, Front. Comput. Neurosci..

[17]  P. Brown,et al.  Adaptive Deep Brain Stimulation In Advanced Parkinson Disease , 2013, Annals of neurology.

[18]  A. Lozano,et al.  Directional deep brain stimulation: an intraoperative double-blind pilot study. , 2014, Brain : a journal of neurology.

[19]  David Hansel,et al.  Late emergence of synchronized oscillatory activity in the pallidum during progressive parkinsonism , 2007, The European journal of neuroscience.

[20]  Zongfu Yu,et al.  Measurement of time-dependent CP asymmetries in Bjavax.xml.bind.JAXBElement@6d610b0 → D(*)±π± decays and constraints on sin(2β + γ) , 2004 .

[21]  Prabhat,et al.  Scalable Bayesian Optimization Using Deep Neural Networks , 2015, ICML.

[22]  W. Grill,et al.  Closed-Loop Control of Deep Brain Stimulation: A Simulation Study , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Daniel K. Leventhal,et al.  Basal Ganglia Beta Oscillations Accompany Cue Utilization , 2012, Neuron.

[24]  Timothy Denison,et al.  Computationally efficient, configurable, causal, real-time phase detection applied to local field potential oscillations , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[25]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[26]  Bard Ermentrout,et al.  Type I Membranes, Phase Resetting Curves, and Synchrony , 1996, Neural Computation.

[27]  Günther Deuschl,et al.  Clinical outcome of deep brain stimulation for Parkinson's disease. , 2013, Handbook of clinical neurology.

[28]  M. Rosenblum,et al.  Controlling synchronization in an ensemble of globally coupled oscillators. , 2004, Physical review letters.

[29]  P. Brown,et al.  Adaptive Deep Brain Stimulation for Movement Disorders: The Long Road to Clinical Therapy , 2017, Movement disorders : official journal of the Movement Disorder Society.

[30]  Nando de Freitas,et al.  A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot , 2009, Auton. Robots.

[31]  Jeff Moehlis,et al.  Phasic Burst Stimulation: A Closed-Loop Approach to Tuning Deep Brain Stimulation Parameters for Parkinson’s Disease , 2016, PLoS Comput. Biol..

[32]  Andrea A. Kühn,et al.  High-Frequency Stimulation of the Subthalamic Nucleus Suppresses Oscillatory β Activity in Patients with Parkinson's Disease in Parallel with Improvement in Motor Performance , 2008, The Journal of Neuroscience.

[33]  Terry Coyne,et al.  A new biomarker for subthalamic deep brain stimulation for patients with advanced Parkinson’s disease—a pilot study , 2015, Journal of neural engineering.

[34]  Jonas Mockus,et al.  Application of Bayesian approach to numerical methods of global and stochastic optimization , 1994, J. Glob. Optim..

[35]  Aaron Klein,et al.  Bayesian Optimization with Robust Bayesian Neural Networks , 2016, NIPS.

[36]  Nicola J. Ray,et al.  Local field potential beta activity in the subthalamic nucleus of patients with Parkinson's disease is associated with improvements in bradykinesia after dopamine and deep brain stimulation , 2008, Experimental Neurology.

[37]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[38]  Hong Yu,et al.  Maximal subthalamic beta hypersynchrony of the local field potential in Parkinson's disease is located in the central region of the nucleus , 2011, Journal of Neurology, Neurosurgery & Psychiatry.

[39]  Sacha Jennifer van Albada,et al.  Mean-field modeling of the basal ganglia-thalamocortical system. I Firing rates in healthy and parkinsonian states. , 2009, Journal of theoretical biology.

[40]  M. Okun,et al.  Parkinson’s disease DBS: what, when, who and why? The time has come to tailor DBS targets , 2010, Expert review of neurotherapeutics.

[41]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[42]  P. Brown,et al.  The nature of tremor circuits in parkinsonian and essential tremor , 2014, Brain : a journal of neurology.

[43]  Hagai Bergman,et al.  Subthalamic span of beta oscillations predicts deep brain stimulation efficacy for patients with Parkinson's disease. , 2010, Brain : a journal of neurology.

[44]  Andrea A. Kühn,et al.  The relationship between local field potential and neuronal discharge in the subthalamic nucleus of patients with Parkinson's disease , 2005, Experimental Neurology.

[45]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[46]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[47]  P. Brown,et al.  Phase dependent modulation of tremor amplitude in essential tremor through thalamic stimulation , 2013, Brain : a journal of neurology.

[48]  Sara Marceglia,et al.  Adaptive deep brain stimulation in a freely moving parkinsonian patient , 2015, Movement disorders : official journal of the Movement Disorder Society.

[49]  Alireza Gharabaghi,et al.  Phase-dependent modulation as a novel approach for therapeutic brain stimulation , 2015, Front. Comput. Neurosci..

[50]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[51]  Sara Marceglia,et al.  Adaptive deep brain stimulation controls levodopa‐induced side effects in Parkinsonian patients , 2017, Movement disorders : official journal of the Movement Disorder Society.

[52]  B. Wittenmark Adaptive dual control , 2002 .

[53]  Simon Streltsov,et al.  A Non-myopic Utility Function for Statistical Global Optimization Algorithms , 1999, J. Glob. Optim..

[54]  E. Vaadia,et al.  Spike Synchronization in the Cortex-Basal Ganglia Networks of Parkinsonian Primates Reflects Global Dynamics of the Local Field Potentials , 2004, The Journal of Neuroscience.

[55]  P. Brown,et al.  Event-related beta desynchronization in human subthalamic nucleus correlates with motor performance. , 2004, Brain : a journal of neurology.

[56]  Pedram Afshar,et al.  Chronic cortical and electromyographic recordings from a fully implantable device: preclinical experience in a nonhuman primate , 2014, Journal of neural engineering.

[57]  Nicole C. Swann,et al.  Therapeutic deep brain stimulation reduces cortical phase-amplitude coupling in Parkinson's disease , 2015, Nature Neuroscience.

[58]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[59]  Karl J. Friston,et al.  Stimulating at the right time: phase-specific deep brain stimulation , 2016, Brain : a journal of neurology.

[60]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[61]  A. Graybiel,et al.  Synchronous, Focally Modulated β-Band Oscillations Characterize Local Field Potential Activity in the Striatum of Awake Behaving Monkeys , 2003, The Journal of Neuroscience.

[62]  Hayriye Cagnan,et al.  Bilateral adaptive deep brain stimulation is effective in Parkinson's disease , 2015, Journal of Neurology, Neurosurgery & Psychiatry.

[63]  Peter Brown,et al.  Brain state-dependency of coherent oscillatory activity in the cerebral cortex and basal ganglia of the rat. , 2004, Journal of neurophysiology.

[64]  P. Brown Abnormal oscillatory synchronisation in the motor system leads to impaired movement , 2007, Current Opinion in Neurobiology.

[65]  Andrea A. Kühn,et al.  Pathological synchronisation in the subthalamic nucleus of patients with Parkinson's disease relates to both bradykinesia and rigidity , 2009, Experimental Neurology.

[66]  Marco Locatelli,et al.  Bayesian Algorithms for One-Dimensional Global Optimization , 1997, J. Glob. Optim..

[67]  Theoden I. Netoff,et al.  Origins and suppression of oscillations in a computational model of Parkinson’s disease , 2014, Journal of Computational Neuroscience.