Decoding a Wide Range of Hand Configurations from Macaque Motor, Premotor, and Parietal Cortices

Despite recent advances in decoding cortical activity for motor control, the development of hand prosthetics remains a major challenge. To reduce the complexity of such applications, higher cortical areas that also represent motor plans rather than just the individual movements might be advantageous. We investigated the decoding of many grip types using spiking activity from the anterior intraparietal (AIP), ventral premotor (F5), and primary motor (M1) cortices. Two rhesus monkeys were trained to grasp 50 objects in a delayed task while hand kinematics and spiking activity from six implanted electrode arrays (total of 192 electrodes) were recorded. Offline, we determined 20 grip types from the kinematic data and decoded these hand configurations and the grasped objects with a simple Bayesian classifier. When decoding from AIP, F5, and M1 combined, the mean accuracy was 50% (using planning activity) and 62% (during motor execution) for predicting the 50 objects (chance level, 2%) and substantially larger when predicting the 20 grip types (planning, 74%; execution, 86%; chance level, 5%). When decoding from individual arrays, objects and grip types could be predicted well during movement planning from AIP (medial array) and F5 (lateral array), whereas M1 predictions were poor. In contrast, predictions during movement execution were best from M1, whereas F5 performed only slightly worse. These results demonstrate for the first time that a large number of grip types can be decoded from higher cortical areas during movement preparation and execution, which could be relevant for future neuroprosthetic devices that decode motor plans.

[1]  Maarten J. IJzerman,et al.  Survey of the needs of patients with spinal cord injury: impact and priority for improvement in hand function in tetraplegics , 2004, Spinal Cord.

[2]  H. Scherberger,et al.  Musculoskeletal Representation of a Large Repertoire of Hand Grasping Actions in Primates , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Silvestro Micera,et al.  Continuous decoding of grasping tasks for a prospective implantable cortical neuroprosthesis , 2012, Journal of NeuroEngineering and Rehabilitation.

[4]  Aaron M. Dollar,et al.  Grasp Frequency and Usage in Daily Household and Machine Shop Tasks , 2013, IEEE Transactions on Haptics.

[5]  H. Sakata,et al.  Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. , 2000, Journal of neurophysiology.

[6]  R. Paul Robot manipulators : mathematics, programming, and control : the computer control of robot manipulators , 1981 .

[7]  Maria C Romero,et al.  Coding of Shape Features in the Macaque Anterior Intraparietal Area , 2014, The Journal of Neuroscience.

[8]  Division on Earth Guidelines for the Care and Use of Mammals in Neuroscience and Behavioral Research , 2003 .

[9]  G. Rizzolatti,et al.  Object representation in the ventral premotor cortex (area F5) of the monkey. , 1997, Journal of neurophysiology.

[10]  G. Rizzolatti,et al.  The Cortical Motor System , 2001, Neuron.

[11]  M. Schieber Individuated finger movements of rhesus monkeys: a means of quantifying the independence of the digits. , 1991, Journal of neurophysiology.

[12]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[13]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[14]  Michael J. Black,et al.  Decoding Complete Reach and Grasp Actions from Local Primary Motor Cortex Populations , 2010, The Journal of Neuroscience.

[15]  R. Quian Quiroga,et al.  Unsupervised Spike Detection and Sorting with Wavelets and Superparamagnetic Clustering , 2004, Neural Computation.

[16]  Maria Alessandra Umiltà,et al.  Cortical control of grasp in non-human primates , 2007, Current Opinion in Neurobiology.

[17]  Scott L. Delp,et al.  A Model of the Upper Extremity for Simulating Musculoskeletal Surgery and Analyzing Neuromuscular Control , 2005, Annals of Biomedical Engineering.

[18]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[19]  A. Murata,et al.  Cortical connections of the macaque anterior intraparietal (AIP) area. , 2008, Cerebral cortex.

[20]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[21]  R. Andersen,et al.  Cortical Local Field Potential Encodes Movement Intentions in the Posterior Parietal Cortex , 2005, Neuron.

[22]  L. Fogassi,et al.  Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. , 2006, Journal of neurophysiology.

[23]  Marc H Schieber,et al.  State-based decoding of hand and finger kinematics using neuronal ensemble and LFP activity during dexterous reach-to-grasp movements. , 2013, Journal of neurophysiology.

[24]  Hansjörg Scherberger,et al.  Context-Specific Grasp Movement Representation in the Macaque Anterior Intraparietal Area , 2009, The Journal of Neuroscience.

[25]  Nicholas G Hatsopoulos,et al.  The science of neural interface systems. , 2009, Annual review of neuroscience.

[26]  Michael J. Black,et al.  Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[28]  Peter Janssen,et al.  Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area. , 2012, Journal of neurophysiology.

[29]  Erk Subasi,et al.  In search of more robust decoding algorithms for neural prostheses, a data driven approach , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[30]  H. Scherberger,et al.  Reach and Gaze Representations in Macaque Parietal and Premotor Grasp Areas , 2013, The Journal of Neuroscience.

[31]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[32]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[33]  Hansjörg Scherberger,et al.  Neural Prostheses for Reaching , 2009 .

[34]  Marzio Gerbella,et al.  Multimodal architectonic subdivision of the rostral part (area F5) of the macaque ventral premotor cortex , 2009, The Journal of comparative neurology.

[35]  M. Schieber,et al.  How somatotopic is the motor cortex hand area? , 1993, Science.

[36]  L. Fogassi,et al.  Decoding the activity of grasping neurons recorded from the ventral premotor area F5 of the macaque monkey , 2011, Neuroscience.

[37]  Marzio Gerbella,et al.  Projections of the hand field of the macaque ventral premotor area F5 to the brainstem and spinal cord , 2010, The Journal of comparative neurology.

[38]  P. Strick,et al.  Subdivisions of primary motor cortex based on cortico-motoneuronal cells , 2009, Proceedings of the National Academy of Sciences.

[39]  P. Strick,et al.  Frontal Lobe Inputs to the Digit Representations of the Motor Areas on the Lateral Surface of the Hemisphere , 2005, The Journal of Neuroscience.

[40]  S Schaffelhofer,et al.  A new method of accurate hand- and arm-tracking for small primates. , 2012, Journal of neural engineering.

[41]  Krishna V. Shenoy,et al.  Challenges and Opportunities for Next-Generation Intracortically Based Neural Prostheses , 2011, IEEE Transactions on Biomedical Engineering.

[42]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[43]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[44]  Hansjörg Scherberger,et al.  Context-Specific Grasp Movement Representation in Macaque Ventral Premotor Cortex , 2010, The Journal of Neuroscience.

[45]  Erk Subasi,et al.  Grasp Movement Decoding from Premotor and Parietal Cortex , 2011, The Journal of Neuroscience.

[46]  F. Raab,et al.  Magnetic Position and Orientation Tracking System , 1979, IEEE Transactions on Aerospace and Electronic Systems.

[47]  Robert D. Lipschutz,et al.  Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. , 2009, JAMA.

[48]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[49]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[50]  A. Murata,et al.  Largely segregated parietofrontal connections linking rostral intraparietal cortex (areas AIP and VIP) and the ventral premotor cortex (areas F5 and F4) , 1999, Experimental Brain Research.

[51]  Arjun K. Bansal,et al.  Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials. , 2012, Journal of neurophysiology.

[52]  R. Andersen,et al.  Cognitive neural prosthetics. , 2010, Annual review of psychology.

[53]  Pierpaolo Pani,et al.  Selectivity for Three-Dimensional Shape and Grasping-Related Activity in the Macaque Ventral Premotor Cortex , 2012, The Journal of Neuroscience.