Design and validation of a real-time spiking-neural-network decoder for brain–machine interfaces

OBJECTIVE Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. APPROACH One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). MAIN RESULTS Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. SIGNIFICANCE These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.

[1]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[2]  Yongji Wang,et al.  Spiking Neural Networks for Cortical Neuronal Spike Train Decoding , 2010, Neural Computation.

[3]  V. Gilja,et al.  Spiking neural network decoder for brain-machine interfaces , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[4]  M. Laubach,et al.  Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task , 2022 .

[5]  Kwabena Boahen,et al.  A Brain-Machine Interface Operating with a Real-Time Spiking Neural Network Control Algorithm , 2011, NIPS.

[6]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[8]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[9]  Teresa H. Y. Meng,et al.  HermesE: A 96-Channel Full Data Rate Direct Neural Interface in 0.13 $\mu$ m CMOS , 2012, IEEE Journal of Solid-State Circuits.

[10]  R.R. Harrison,et al.  A Low-Power Integrated Circuit for a Wireless 100-Electrode Neural Recording System , 2006, IEEE Journal of Solid-State Circuits.

[11]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[12]  Krishna V. Shenoy,et al.  Challenges and Opportunities for Next-Generation Intracortically Based Neural Prostheses , 2011, IEEE Transactions on Biomedical Engineering.

[13]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[14]  R.R. Harrison,et al.  HermesC: Low-Power Wireless Neural Recording System for Freely Moving Primates , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[16]  Reid R. Harrison,et al.  A low-power, low-noise CMOS amplifier for neural recording applications , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[17]  L. Shupe,et al.  The Neurochip-2: An Autonomous Head-Fixed Computer for Recording and Stimulating in Freely Behaving Monkeys , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  R. R. Harrison,et al.  A low-power low-noise CMOS amplifier for neural recording applications , 2003, IEEE J. Solid State Circuits.

[19]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[20]  Reid R. Harrison,et al.  The Design of Integrated Circuits to Observe Brain Activity , 2008, Proceedings of the IEEE.

[21]  Teresa H. Y. Meng,et al.  HermesD: A High-Rate Long-Range Wireless Transmission System for Simultaneous Multichannel Neural Recording Applications , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[22]  C. Eliasmith,et al.  Higher-Dimensional Neurons Explain the Tuning and Dynamics of Working Memory Cells , 2006, The Journal of Neuroscience.

[23]  Kwabena Boahen,et al.  Silicon-Neuron Design: A Dynamical Systems Approach , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[24]  Dragan F. Dimitrov,et al.  Reversible large-scale modification of cortical networks during neuroprosthetic control , 2011, Nature Neuroscience.

[25]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[26]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[27]  L. Miller,et al.  Restoration of grasp following paralysis through brain-controlled stimulation of muscles , 2012, Nature.

[28]  Kathie L. Olsen,et al.  Neurotech for Neuroscience: Unifying Concepts, Organizing Principles, and Emerging Tools , 2007, The Journal of Neuroscience.

[29]  Chris Eliasmith,et al.  How to build a brain: from function to implementation , 2007, Synthese.

[30]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[31]  Chris Eliasmith,et al.  A Unified Approach to Building and Controlling Spiking Attractor Networks , 2005, Neural Computation.

[32]  R. Normann,et al.  Thermal Impact of an Active 3-D Microelectrode Array Implanted in the Brain , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Joseph E. O’Doherty,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[34]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[35]  Teresa H. Y. Meng,et al.  HermesB: A Continuous Neural Recording System for Freely Behaving Primates , 2007, IEEE Transactions on Biomedical Engineering.

[36]  David Sussillo,et al.  A recurrent neural network for closed-loop intracortical brain–machine interface decoders , 2012, Journal of neural engineering.

[37]  Teresa H. Y. Meng,et al.  A programmable pulse UWB transmitter with 34% energy efficiency for multichannel neuro-recording systems , 2010, IEEE Custom Integrated Circuits Conference 2010.

[38]  P. Wolf Thermal Considerations for the Design of an Implanted Cortical Brain–Machine Interface (BMI) , 2008 .

[39]  E. Fetz,et al.  Long-term motor cortex plasticity induced by an electronic neural implant , 2006, Nature.

[40]  K. Boahen Neuromorphic Microchips. , 2005, Scientific American.

[41]  A. Schwartz,et al.  Behavioral and neural correlates of visuomotor adaptation observed through a brain-computer interface in primary motor cortex. , 2012, Journal of neurophysiology.

[42]  Arto Nurmikko,et al.  An implantable wireless neural interface for recording cortical circuit dynamics in moving primates , 2013, Journal of neural engineering.

[43]  Vikash Gilja,et al.  Autonomous head-mounted electrophysiology systems for freely behaving primates , 2010, Current Opinion in Neurobiology.

[44]  R. Wm Thermal Considerations for the Design of an Implanted Cortical Brain–Machine Interface (BMI) -- Indwelling Neural Implants: Strategies for Contending with the In Vivo Environment , 2008 .

[45]  K.V. Shenoy,et al.  Power feasibility of implantable digital spike sorting circuits for neural prosthetic systems , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[46]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[47]  Boris Murmann,et al.  A 96-channel full data rate direct neural interface in 0.13µm CMOS , 2011, 2011 Symposium on VLSI Circuits - Digest of Technical Papers.

[48]  Peter J. Ifft,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[49]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[50]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .