Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates

Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

[1]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[2]  E. Fetz,et al.  Operant Conditioning of Specific Patterns of Neural and Muscular Activity , 1971, Science.

[3]  E. Fetz,et al.  Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles. , 1973, Journal of neurophysiology.

[4]  F. Mussa-Ivaldi,et al.  Do neurons in the motor cortex encode movement direction? An alternative hypothesis , 1988, Neuroscience Letters.

[5]  J. T. Massey,et al.  Mental rotation of the neuronal population vector. , 1989, Science.

[6]  G E Alexander,et al.  Neural representations of the target (goal) of visually guided arm movements in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[7]  E. Fetz Movement control: Are movement parameters recognizably coded in the activity of single neurons? , 1992 .

[8]  Division on Earth Guide for the Care and Use of Laboratory Animals , 1996 .

[9]  M. Tanaka,et al.  Coding of modified body schema during tool use by macaque postcentral neurones. , 1996, Neuroreport.

[10]  Paul B. Johnson,et al.  Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. , 1997, Annual review of neuroscience.

[11]  G E Alexander,et al.  Preferential representation of instructed target location versus limb trajectory in dorsal premotor area. , 1997, Journal of neurophysiology.

[12]  G W Plant,et al.  Long-Distance Axonal Regeneration in the Transected Adult Rat Spinal Cord Is Promoted by Olfactory Ensheathing Glia Transplants , 1998, The Journal of Neuroscience.

[13]  A. Nobunaga,et al.  Recent demographic and injury trends in people served by the Model Spinal Cord Injury Care Systems. , 1999, Archives of physical medicine and rehabilitation.

[14]  D. Hoffman,et al.  Muscle and movement representations in the primary motor cortex. , 1999, Science.

[15]  F. Lacquaniti,et al.  Parieto-frontal coding of reaching: an integrated framework , 1999, Experimental Brain Research.

[16]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[17]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[18]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[19]  I. Weissman,et al.  Direct isolation of human central nervous system stem cells. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[20]  B. King,et al.  Dynamic Interplay of Neural Signals during the Emergence of Cursor Related Cortex in a Human Implanted with the Neurotrophic , 2000 .

[21]  E. Todorov Direct cortical control of muscle activation in voluntary arm movements: a model , 2000, Nature Neuroscience.

[22]  J. Kalaska,et al.  Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task. , 2000, Journal of neurophysiology.

[23]  Miguel A. L. Nicolelis,et al.  Actions from thoughts , 2001, Nature.

[24]  Mary Bartlett Bunge,et al.  Book Review: Bridging Areas of Injury in the Spinal Cord , 2001 .

[25]  S. Wise,et al.  Tuning for the orientation of spatial attention in dorsal premotor cortex , 2001, The European journal of neuroscience.

[26]  M B Bunge,et al.  Bridging areas of injury in the spinal cord. , 2001, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[27]  Carolyn R. Mason,et al.  Central processes for the multiparametric control of arm movements in primates , 2001, Current Opinion in Neurobiology.

[28]  S. Scott,et al.  Dissociation between hand motion and population vectors from neural activity in motor cortex , 2022 .

[29]  P. Caroni,et al.  Spinal axon regeneration evoked by replacing two growth cone proteins in adult neurons , 2001, Nature Neuroscience.

[30]  E. Bizzi,et al.  Neuronal Correlates of Motor Performance and Motor Learning in the Primary Motor Cortex of Monkeys Adapting to an External Force Field , 2001, Neuron.

[31]  Shennan A. Weiss,et al.  Rat navigation guided by remote control , 2002 .

[32]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[33]  Nicholas G. Hatsopoulos,et al.  Brain-machine interface: Instant neural control of a movement signal , 2002, Nature.

[34]  Sidarta Ribeiro,et al.  Multielectrode recordings: the next steps , 2002, Current Opinion in Neurobiology.

[35]  Bijan Pesaran,et al.  Temporal structure in neuronal activity during working memory in macaque parietal cortex , 2000, Nature Neuroscience.

[36]  M. Schwab Repairing the Injured Spinal Cord , 2002, Science.

[37]  John P. Donoghue,et al.  Connecting cortex to machines: recent advances in brain interfaces , 2002, Nature Neuroscience.

[38]  S. Meagher Instant neural control of a movement signal , 2002 .

[39]  Jerald D. Kralik,et al.  Chronic, multisite, multielectrode recordings in macaque monkeys , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces to restore motor function and probe neural circuits , 2003, Nature Reviews Neuroscience.

[41]  N. Turner PLOS Biology , 2004, BMJ : British Medical Journal.

[42]  E. Fetz,et al.  Correlations between activity of motor cortex cells and arm muscles during operantly conditioned response patterns , 1975, Experimental Brain Research.

[43]  M. Nicolelis,et al.  Optimizing a Linear Algorithm for Real-Time Robotic Control using Chronic Cortical Ensemble Recordings in Monkeys , 2004, Journal of Cognitive Neuroscience.