Decoding of standard and non-standard visuomotor associations from parietal cortex

Objective Neural signals can be decoded and used to move neural prostheses with the purpose of restoring motor function in patients with mobility impairments. Such patients typically have intact eye movement control and visual function, suggesting that cortical visuospatial signals could be used to guide external devices. Neurons in parietal cortex mediate sensory-motor transformations, encode the spatial coordinates for reaching goals, hand position and movements, and other spatial variables. We studied how spatial information is represented at the population level, and the possibility to decode not only the position of visual targets and the plans to reach them, but also conditional, non-spatial motor responses. Approach The animals first fixated one of nine targets in 3D space and then, after the target changed color, either reached toward it, or performed a non-spatial motor response (lift hand from a button). Spiking activity of parietal neurons was recorded in monkeys during two tasks. We then decoded different task related parameters. Main results We first show that a maximum-likelihood estimation (MLE) algorithm trained separately in each task transformed neural activity into accurate metric predictions of target location. Furthermore, by combining MLE with a Naïve Bayes classifier, we decoded the monkey's motor intention (reach or hand lift) and the different phases of the tasks. These results show that, although V6A encodes the spatial location of a target during a delay period, the signals they carry are updated around the movement execution in an intention/motor specific way. Significance These findings show the presence of multiple levels of information in parietal cortex that could be decoded and used in brain machine interfaces to control both goal-directed movements and more cognitive visuomotor associations.

[1]  Silvio P. Sabatini,et al.  Fix Your Eyes in the Space You Could Reach: Neurons in the Macaque Medial Parietal Cortex Prefer Gaze Positions in Peripersonal Space , 2011, PloS one.

[2]  J. Culham,et al.  Neural coding within human brain areas involved in actions , 2015, Current Opinion in Neurobiology.

[3]  F. Bremmer,et al.  Decoding Target Distance and Saccade Amplitude from Population Activity in the Macaque Lateral Intraparietal Area (LIP) , 2016, Front. Integr. Neurosci..

[4]  David J. Freedman,et al.  Independent Category and Spatial Encoding in Parietal Cortex , 2013, Neuron.

[5]  Il Memming Park,et al.  Encoding and decoding in parietal cortex during sensorimotor decision-making , 2014, Nature Neuroscience.

[6]  David J. Freedman,et al.  Interaction between Spatial and Feature Attention in Posterior Parietal Cortex , 2016, Neuron.

[7]  Kenneth F. Valyear,et al.  Decoding Action Intentions from Preparatory Brain Activity in Human Parieto-Frontal Networks , 2011, The Journal of Neuroscience.

[8]  Patrizia Fattori,et al.  Multiple Aspects of Neural Activity during Reaching Preparation in the Medial Posterior Parietal Area V6A , 2014, Journal of Cognitive Neuroscience.

[9]  Stefano Fusi,et al.  Why neurons mix: high dimensionality for higher cognition , 2016, Current Opinion in Neurobiology.

[10]  Adam P. Morris,et al.  A Stable Visual World in Primate Primary Visual Cortex , 2019, Current Biology.

[11]  C. Galletti,et al.  The cortical visual area V6: brain location and visual topography , 1999, The European journal of neuroscience.

[12]  C. Galletti,et al.  Common neural substrate for processing depth and direction signals for reaching in the monkey medial posterior parietal cortex. , 2014, Cerebral cortex.

[13]  Jason P Gallivan,et al.  The dorsal "action" pathway. , 2018, Handbook of clinical neurology.

[14]  R. Andersen,et al.  Early planning activity in frontal and parietal cortex in a simplified task. , 2015, Journal of neurophysiology.

[15]  S. Scott,et al.  Cortical control of reaching movements , 1997, Current Opinion in Neurobiology.

[16]  P Fattori,et al.  Reduced neural representation of arm/hand actions in the medial posterior parietal cortex , 2019, Scientific Reports.

[17]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[18]  Maureen A. Hagan,et al.  Mixed Spatial and Movement Representations in the Primate Posterior Parietal Cortex , 2019, Front. Neural Circuits.

[19]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[20]  R. Andersen,et al.  Partially Mixed Selectivity in Human Posterior Parietal Association Cortex , 2017, Neuron.

[21]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[22]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[23]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[24]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[25]  Stephen I. Ryu,et al.  A High-Performance Neural Prosthesis Incorporating Discrete State Selection With Hidden Markov Models , 2017, IEEE Transactions on Biomedical Engineering.

[26]  C. Galletti,et al.  Spatial tuning of reaching activity in the medial parieto‐occipital cortex (area V6A) of macaque monkey , 2005, The European journal of neuroscience.

[27]  Michela Gamberini,et al.  ‘Arm‐reaching’ neurons in the parietal area V6A of the macaque monkey , 2001, The European journal of neuroscience.

[28]  David J. Freedman,et al.  Dynamic population coding of category information in inferior temporal and prefrontal cortex. , 2008, Journal of neurophysiology.

[29]  Frank Bremmer,et al.  Eye-Position Signals in the Dorsal Visual System Are Accurate and Precise on Short Timescales , 2013, The Journal of Neuroscience.

[30]  Elaine Astrand,et al.  Selective visual attention to drive cognitive brain–machine interfaces: from concepts to neurofeedback and rehabilitation applications , 2014, Front. Syst. Neurosci..

[31]  Martin I. Sereno,et al.  The human homologue of macaque area V6A , 2012, NeuroImage.

[32]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[33]  R. Andersen,et al.  Intentional maps in posterior parietal cortex. , 2002, Annual review of neuroscience.

[34]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[35]  Frank Bremmer,et al.  Dynamics of Eye-Position Signals in the Dorsal Visual System , 2012, Current Biology.

[36]  Jay A. Hennig,et al.  Signal Multiplexing and Single-Neuron Computations in Lateral Intraparietal Area During Decision-Making , 2013, The Journal of Neuroscience.

[37]  Byron M. Yu,et al.  Detecting neural-state transitions using hidden Markov models for motor cortical prostheses. , 2008, Journal of neurophysiology.

[38]  R. Andersen,et al.  Coding of intention in the posterior parietal cortex , 1997, Nature.

[39]  Patrizia Fattori,et al.  The Dorsomedial Pathway Is Not Just for Reaching: Grasping Neurons in the Medial Parieto-Occipital Cortex of the Macaque Monkey , 2010, The Journal of Neuroscience.

[40]  Gerald E Loeb,et al.  Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands , 2012, Proceedings of the National Academy of Sciences.

[41]  C. Galletti,et al.  Brain location and visual topography of cortical area V6A in the macaque monkey , 1999, The European journal of neuroscience.

[42]  Ziv M. Williams,et al.  A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design , 2013, PloS one.

[43]  Nitish V. Thakor,et al.  Decoding Kinematics Using Task-Independent Movement-Phase-Specific Encoding Models , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  R. Andersen,et al.  Intention, Action Planning, and Decision Making in Parietal-Frontal Circuits , 2009, Neuron.

[45]  R. Andersen,et al.  Cognitive neural prosthetics. , 2010, Annual review of psychology.

[46]  R. Andersen,et al.  Neural prosthetic control signals from plan activity , 2003, Neuroreport.

[47]  B. Pesaran,et al.  Cognitive neural prosthetics , 2004, Trends in Cognitive Sciences.

[48]  Gaspare Galati,et al.  Resting-state connectivity and functional specialization in human medial parieto-occipital cortex , 2014, Brain Structure and Function.

[49]  Nicholas G. Hatsopoulos,et al.  2009 Special Issue: Exploiting multiple sensory modalities in brain-machine interfaces , 2009 .

[50]  M. Goldberg,et al.  Attention, intention, and priority in the parietal lobe. , 2010, Annual review of neuroscience.

[51]  P. Fattori,et al.  Three-dimensional eye position signals shape both peripersonal space and arm movement activity in the medial posterior parietal cortex , 2012, Front. Integr. Neurosci..

[52]  Frank Bremmer,et al.  The Dorsal Visual System Predicts Future and Remembers Past Eye Position , 2016, Front. Syst. Neurosci..

[53]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[54]  Patrizia Fattori,et al.  Temporal stability of reference frames in monkey area V6A during a reaching task in 3D space , 2017, Brain Structure and Function.

[55]  Katja Fiehler,et al.  Decoding Movement Goals from the Fronto-Parietal Reach Network , 2017, Front. Hum. Neurosci..

[56]  V. Mountcastle,et al.  Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. , 1975, Journal of neurophysiology.

[57]  S.I. Ryu,et al.  Cortical Neural Prosthesis Performance Improves When Eye Position Is Monitored , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[58]  Michela Gamberini,et al.  Vision for Prehension in the Medial Parietal Cortex , 2015, Cerebral cortex.

[59]  R. Breveglieri,et al.  Decoding Information for Grasping from the Macaque Dorsomedial Visual Stream , 2017, The Journal of Neuroscience.

[60]  Bruno B Averbeck,et al.  Rapid Sequences of Population Activity Patterns Dynamically Encode Task-Critical Spatial Information in Parietal Cortex , 2010, The Journal of Neuroscience.

[61]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[62]  J. Culham,et al.  The role of parietal cortex in visuomotor control: What have we learned from neuroimaging? , 2006, Neuropsychologia.

[63]  C. Galletti,et al.  Reference frames for reaching when decoupling eye and target position in depth and direction , 2016, Scientific Reports.

[64]  Jody C. Culham,et al.  Neuroimaging reveals enhanced activation in a reach-selective brain area for objects located within participants’ typical hand workspaces , 2011, Neuropsychologia.

[65]  J. Duhamel,et al.  Differential Dynamics of Spatial Attention, Position, and Color Coding within the Parietofrontal Network , 2015, The Journal of Neuroscience.

[66]  R. Andersen,et al.  Posterior Parietal Cortex Encodes Autonomously Selected Motor Plans , 2007, Neuron.

[67]  H. Scherberger,et al.  Reach and Gaze Representations in Macaque Parietal and Premotor Grasp Areas , 2013, The Journal of Neuroscience.

[68]  C. Galletti,et al.  Role of the medial parieto-occipital cortex in the control of reaching and grasping movements , 2003, Experimental Brain Research.

[69]  Patrizia Fattori,et al.  Prediction of Reach Goals in Depth and Direction from the Parietal Cortex. , 2018, Cell reports.

[70]  K. Mazurek,et al.  Mirror Neuron Populations Represent Sequences of Behavioral Epochs During Both Execution and Observation , 2018, The Journal of Neuroscience.

[71]  Richard A Andersen,et al.  Temporal Analysis of Reference Frames in Parietal Cortex Area 5d during Reach Planning , 2014, The Journal of Neuroscience.

[72]  Silvio P. Sabatini,et al.  Eye Position Encoding in Three-Dimensional Space: Integration of Version and Vergence Signals in the Medial Posterior Parietal Cortex , 2012, The Journal of Neuroscience.