Toward Building Hybrid Biological/in silico Neural Networks for Motor Neuroprosthetic Control

In this article, we introduce the Bioinspired Neuroprosthetic Design Environment (BNDE) as a practical platform for the development of novel brain–machine interface (BMI) controllers, which are based on spiking model neurons. We built the BNDE around a hard real-time system so that it is capable of creating simulated synapses from extracellularly recorded neurons to model neurons. In order to evaluate the practicality of the BNDE for neuroprosthetic control experiments, a novel, adaptive BMI controller was developed and tested using real-time closed-loop simulations. The present controller consists of two in silico medium spiny neurons, which receive simulated synaptic inputs from recorded motor cortical neurons. In the closed-loop simulations, the recordings from the cortical neurons were imitated using an external, hardware-based neural signal synthesizer. By implementing a reward-modulated spike timing-dependent plasticity rule, the controller achieved perfect target reach accuracy for a two-target reaching task in one-dimensional space. The BNDE combines the flexibility of software-based spiking neural network (SNN) simulations with powerful online data visualization tools and is a low-cost, PC-based, and all-in-one solution for developing neurally inspired BMI controllers. We believe that the BNDE is the first implementation, which is capable of creating hybrid biological/in silico neural networks for motor neuroprosthetic control and utilizes multiple CPU cores for computationally intensive real-time SNN simulations.

[1]  Steve M. Potter,et al.  Closed-Loop, Multichannel Experimentation Using the Open-Source NeuroRighter Electrophysiology Platform , 2013, Front. Neural Circuits.

[2]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[3]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[4]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[5]  Justin C. Sanchez,et al.  Using Reinforcement Learning to Provide Stable Brain-Machine Interface Control Despite Neural Input Reorganization , 2014, PloS one.

[6]  J. Tepper,et al.  Basal ganglia macrocircuits. , 2007, Progress in brain research.

[7]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[8]  Karen A. Moxon,et al.  Decoding Hindlimb Movement for a Brain Machine Interface after a Complete Spinal Transection , 2012, PloS one.

[9]  David J. Christini,et al.  Real-time Experiment Interface for biological control applications , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[10]  Aaron C. Koralek,et al.  Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills , 2012, Nature.

[11]  M L Boninger,et al.  Ten-dimensional anthropomorphic arm control in a human brain−machine interface: difficulties, solutions, and limitations , 2015, Journal of neural engineering.

[12]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[13]  John A. White,et al.  GenNet: A Platform for Hybrid Network Experiments , 2011, Front. Neuroinform..

[14]  José Carlos Príncipe,et al.  Ascertaining the importance of neurons to develop better brain-machine interfaces , 2004, IEEE Transactions on Biomedical Engineering.

[15]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[16]  Anatol C. Kreitzer,et al.  Striatal Plasticity and Basal Ganglia Circuit Function , 2008, Neuron.

[17]  Justin C. Sanchez,et al.  Towards autonomous neuroprosthetic control using Hebbian reinforcement learning , 2013, Journal of neural engineering.

[18]  Robert E. Hampson,et al.  A Hippocampal Cognitive Prosthesis: Multi-Input, Multi-Output Nonlinear Modeling and VLSI Implementation , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  A. Aertsen,et al.  Spike synchronization and rate modulation differentially involved in motor cortical function. , 1997, Science.

[20]  L. Paninski,et al.  Information about movement direction obtained from synchronous activity of motor cortical neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Emery N Brown,et al.  Differential Involvement of Excitatory and Inhibitory Neurons of Cat Motor Cortex in Coincident Spike Activity Related to Behavioral Context , 2010, The Journal of Neuroscience.

[22]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[23]  M S Lewicki,et al.  A review of methods for spike sorting: the detection and classification of neural action potentials. , 1998, Network.

[24]  Alan D Dorval,et al.  Hybrid neuronal network studies under dynamic clamp. , 2007, Methods in molecular biology.

[25]  José Carlos Príncipe,et al.  Coadaptive Brain–Machine Interface via Reinforcement Learning , 2009, IEEE Transactions on Biomedical Engineering.

[26]  Y. Frégnac,et al.  Bidirectional control of a one-dimensional robotic actuator by operant conditioning of a single unit in rat motor cortex , 2014, Front. Neurosci..

[27]  张静,et al.  Banana Ovate family protein MaOFP1 and MADS-box protein MuMADS1 antagonistically regulated banana fruit ripening , 2015 .

[28]  J. Kerr,et al.  Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity , 2008, The Journal of Neuroscience.

[29]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[30]  Ethem Alpaydin Introduction to machine learning, 2rd ed , 2014 .

[31]  William H. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[32]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[33]  P. Mantegazza,et al.  RTAI: Real Time Application Interface , 2000 .

[34]  Hagai Bergman,et al.  Inducing Gamma Oscillations and Precise Spike Synchrony by Operant Conditioning via Brain-Machine Interface , 2013, Neuron.

[35]  James S. Sochacki,et al.  A Picard-Maclaurin theorem for initial value PDEs , 2000 .

[36]  W. Schultz Book Review: Reward Signaling by Dopamine Neurons , 2001, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[37]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[38]  Joseph T. Francis,et al.  Toward an Autonomous Brain Machine Interface: Integrating Sensorimotor Reward Modulation and Reinforcement Learning , 2015, The Journal of Neuroscience.

[39]  S. Royer,et al.  Conservation of total synaptic weight through balanced synaptic depression and potentiation , 2003, Nature.

[40]  Steve M. Potter,et al.  Closed-Loop, Open-Source Electrophysiology , 2010, Front. Neurosci..

[41]  Wulfram Gerstner,et al.  Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail , 2009, PLoS Comput. Biol..

[42]  Jiping He,et al.  Selection and parameterization of cortical neurons for neuroprosthetic control , 2006, Journal of neural engineering.

[43]  Dragan F. Dimitrov,et al.  Reversible large-scale modification of cortical networks during neuroprosthetic control , 2011, Nature Neuroscience.

[44]  Wulfram Gerstner,et al.  A History of Spike-Timing-Dependent Plasticity , 2011, Front. Syn. Neurosci..

[45]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[46]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[47]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[48]  D. Kleinfeld,et al.  Variability of extracellular spike waveforms of cortical neurons. , 1996, Journal of neurophysiology.

[49]  G. Turrigiano Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same , 1999, Trends in Neurosciences.

[50]  Wyeth Bair,et al.  Spiking neural network simulation: numerical integration with the Parker-Sochacki method , 2009, Journal of Computational Neuroscience.

[51]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[52]  Paolo Del Giudice,et al.  A neuro-inspired model-based closed-loop neuroprosthesis for the substitution of a cerebellar learning function in anesthetized rats , 2015, Scientific Reports.

[53]  J. Carmena,et al.  Active Sensing of Target Location Encoded by Cortical Microstimulation , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Alexa Riehle,et al.  Spike synchronization and firing rate in a population of motor cortical neurons in relation to movement direction and reaction time , 2003, Biological Cybernetics.

[55]  Jacob G. Bernstein,et al.  Millisecond-Timescale Optical Control of Neural Dynamics in the Nonhuman Primate Brain , 2009, Neuron.

[56]  Henning Sprekeler,et al.  Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity , 2010, The Journal of Neuroscience.

[57]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[58]  W. Lytton,et al.  Reinforcement Learning of Targeted Movement in a Spiking Neuronal Model of Motor Cortex , 2012, PloS one.

[59]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[60]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[61]  Jan Grewe,et al.  A Bottom-up Approach to Data Annotation in Neurophysiology , 2011, Front. Neuroinform..

[62]  J. Carmena Advances in Neuroprosthetic Learning and Control , 2013, PLoS biology.

[63]  Kwabena Boahen,et al.  A Brain-Machine Interface Operating with a Real-Time Spiking Neural Network Control Algorithm , 2011, NIPS.

[64]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[65]  M. Nicolelis,et al.  Reconstructing the Engram: Simultaneous, Multisite, Many Single Neuron Recordings , 1997, Neuron.

[66]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[67]  William W. Lytton,et al.  Towards a real-time interface between a biomimetic model of sensorimotor cortex and a robotic arm , 2014, Pattern Recognit. Lett..

[68]  J. Wickens,et al.  Neural mechanisms of reward-related motor learning , 2003, Current Opinion in Neurobiology.

[69]  William W. Lytton,et al.  Reinforcement Learning of Two-Joint Virtual Arm Reaching in a Computer Model of Sensorimotor Cortex , 2013, Neural Computation.

[70]  Justin C. Sanchez,et al.  A Symbiotic Brain-Machine Interface through Value-Based Decision Making , 2011, PloS one.

[71]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[72]  E. Izhikevich Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[73]  S. Grillner,et al.  Mechanisms for selection of basic motor programs – roles for the striatum and pallidum , 2005, Trends in Neurosciences.