Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices. Able-bodied monkeys have used a neural interface system to control a robotic arm, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

[1]  D R Humphrey,et al.  Predicting Measures of Motor Performance from Multiple Cortical Spike Trains , 1970, Science.

[2]  J. Gerring A case study , 2011, Technology and Society.

[3]  William K. Durfee,et al.  IEEE/RSJ/GI International Conference on Intelligent Robots and Systems , 1994 .

[4]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[5]  C. Mehring,et al.  Inference of hand movements from local field potentials in monkey motor cortex , 2003, Nature Neuroscience.

[6]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[7]  J.P. Donoghue,et al.  Reliability of signals from a chronically implanted, silicon-based electrode array in non-human primate primary motor cortex , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Andrew B. Schwartz,et al.  Brain-Controlled Interfaces: Movement Restoration with Neural Prosthetics , 2006, Neuron.

[9]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[10]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[11]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[12]  Alin Albu-Schäffer,et al.  The DLR lightweight robot: design and control concepts for robots in human environments , 2007, Ind. Robot.

[13]  Michael J. Black,et al.  Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[14]  Y. Amit,et al.  Encoding of Movement Fragments in the Motor Cortex , 2007, The Journal of Neuroscience.

[15]  Eran Stark,et al.  Predicting Movement from Multiunit Activity , 2007, The Journal of Neuroscience.

[16]  Robert D. Lipschutz,et al.  Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study , 2007, The Lancet.

[17]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[18]  John P. Donoghue,et al.  Bridging the Brain to the World: A Perspective on Neural Interface Systems , 2008, Neuron.

[19]  Alin Albu-Schaffer,et al.  Soft robotics , 2008, IEEE Robotics Autom. Mag..

[20]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[21]  S. Scott Inconvenient Truths about neural processing in primary motor cortex , 2008, The Journal of physiology.

[22]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[23]  Robert E Kass,et al.  Functional network reorganization during learning in a brain-computer interface paradigm , 2008, Proceedings of the National Academy of Sciences.

[24]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[25]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[26]  Hong Liu,et al.  Multisensory five-finger dexterous hand: The DLR/HIT Hand II , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Steven M Chase,et al.  Control of a brain–computer interface without spike sorting , 2009, Journal of neural engineering.

[28]  S. Solla,et al.  Toward the Restoration of Hand Use to a Paralyzed Monkey: Brain-Controlled Functional Electrical Stimulation of Forearm Muscles , 2009, PloS one.

[29]  Miguel A. L. Nicolelis,et al.  Principles of neural ensemble physiology underlying the operation of brain–machine interfaces , 2009, Nature Reviews Neuroscience.

[30]  R. Quian Quiroga What is the real shape of extracellular spikes? , 2009, Journal of Neuroscience Methods.

[31]  M. Nicolelis,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[32]  Alin Albu-Schäffer,et al.  Requirements for Safe Robots: Measurements, Analysis and New Insights , 2009, Int. J. Robotics Res..

[33]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[34]  G. Rizzolatti,et al.  Seven Years of Recording from Monkey Cortex with a Chronically Implanted Multiple Microelectrode , 2010, Front. Neuroeng..

[35]  Yali Amit,et al.  Encoding of Coordinated Grasp Trajectories in Primary Motor Cortex , 2010, The Journal of Neuroscience.

[36]  L. Resnik,et al.  Research update: VA study to optimize DEKA arm. , 2010, Journal of rehabilitation research and development.

[37]  Michael J. Black,et al.  Decoding Complete Reach and Grasp Actions from Local Primary Motor Cortex Populations , 2010, The Journal of Neuroscience.

[38]  Nitish V. Thakor,et al.  Neural Decoding of Finger Movements Using Skellam-Based Maximum-Likelihood Decoding , 2010, IEEE Transactions on Biomedical Engineering.

[39]  Nicholas G Hatsopoulos,et al.  Incorporating Feedback from Multiple Sensory Modalities Enhances Brain–Machine Interface Control , 2010, The Journal of Neuroscience.

[40]  Krishna V. Shenoy,et al.  Challenges and Opportunities for Next-Generation Intracortically Based Neural Prostheses , 2011, IEEE Transactions on Biomedical Engineering.

[41]  J D Simeral,et al.  Continuous neuronal ensemble control of simulated arm reaching by a human with tetraplegia , 2011, Journal of neural engineering.

[42]  Stefan Mihalas,et al.  Does Afferent Heterogeneity Matter in Conveying Tactile Feedback Through Peripheral Nerve Stimulation? , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[43]  Michael J. Black,et al.  Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Michael J. Black,et al.  Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array , 2011 .

[45]  J. Kalaska,et al.  Learning to Move Machines with the Mind , 2022 .

[46]  Arjun K. Bansal,et al.  Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. , 2011, Journal of neurophysiology.

[47]  Vikash Gilja,et al.  Long-term Stability of Neural Prosthetic Control Signals from Silicon Cortical Arrays in Rhesus Macaque Motor Cortex , 2010 .

[48]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.