Single-trial decoding of movement intentions using functional ultrasound neuroimaging

Brain-machine interfaces (BMI) are powerful devices for restoring function to people living with paralysis. Leveraging significant advances in neurorecording technology, computational power, and understanding of the underlying neural signals, BMI have enabled severely paralyzed patients to control external devices, such as computers and robotic limbs. However, high-performance BMI currently require highly invasive recording techniques, and are thus only available to niche populations. Here, we show that a minimally invasive neuroimaging approach based on functional ultrasound (fUS) imaging can be used to detect and decode movement intention signals usable for BMI. We trained non-human primates to perform memory-guided movements while using epidural fUS imaging to record changes in cerebral blood volume from the posterior parietal cortex – a brain area important for spatial perception, multisensory integration, and movement planning. Using hemodynamic signals acquired during movement planning, we classified left-cued vs. right-cued movements, establishing the feasibility of ultrasonic BMI. These results demonstrate the ability of fUS-based neural interfaces to take advantage of the excellent spatiotemporal resolution, sensitivity, and field of view of ultrasound without breaching the dura or physically penetrating brain tissue.

[1]  J. Kamiya,et al.  The control of electroencephalographic alpha rhythms through auditory feedback and the associated mental activity. , 1970, Psychophysiology.

[2]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[3]  D. Pandya,et al.  Intrinsic connections and architectonics of posterior parietal cortex in the rhesus monkey , 1982, The Journal of comparative neurology.

[4]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[5]  C. Colby,et al.  Spatial representations for action in parietal cortex. , 1996, Brain research. Cognitive brain research.

[6]  J. Hopfield,et al.  Electrical microstimulation suggests two different forms of representation of head-centered space in the intraparietal sulcus of rhesus monkeys. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[7]  R. Andersen,et al.  Coding of intention in the posterior parietal cortex , 1997, Nature.

[8]  R. Andersen,et al.  Change in motor plan, without a change in the spatial locus of attention, modulates activity in posterior parietal cortex. , 1998, Journal of neurophysiology.

[9]  R. Andersen,et al.  Electrical microstimulation distinguishes distinct saccade-related areas in the posterior parietal cortex. , 1998, Journal of neurophysiology.

[10]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[11]  R. Normann,et al.  Chronic recording capability of the Utah Intracortical Electrode Array in cat sensory cortex , 1998, Journal of Neuroscience Methods.

[12]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[13]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[14]  R. Andersen,et al.  Intention-related activity in the posterior parietal cortex: a review , 2000, Vision Research.

[15]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[16]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[17]  Anthony R. Dickinson,et al.  Non-spatial, motor-specific activation in posterior parietal cortex , 2002, Nature Neuroscience.

[18]  R. Andersen,et al.  Intentional maps in posterior parietal cortex. , 2002, Annual review of neuroscience.

[19]  Tirin Moore,et al.  Complex movements evoked by microstimulation of the ventral intraparietal area , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[20]  R. Andersen,et al.  Memory related motor planning activity in posterior parietal cortex of macaque , 1988, Experimental Brain Research.

[21]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[22]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[23]  R. M. Siegel,et al.  Neurons of area 7 activated by both visual stimuli and oculomotor behavior , 2004, Experimental Brain Research.

[24]  G. Pfurtscheller,et al.  Conversion of EEG activity into cursor movement by a brain-computer interface (BCI) , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  P. Tresco,et al.  Response of brain tissue to chronically implanted neural electrodes , 2005, Journal of Neuroscience Methods.

[26]  R. Andersen,et al.  Movement Intention Is Better Predicted than Attention in the Posterior Parietal Cortex , 2006, The Journal of Neuroscience.

[27]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[28]  M. Goldberg,et al.  Saccades, salience and attention: the role of the lateral intraparietal area in visual behavior. , 2006, Progress in brain research.

[29]  Reinhold Scherer,et al.  A fully on-line adaptive BCI , 2006, IEEE Transactions on Biomedical Engineering.

[30]  Zoran Nenadic,et al.  Approximate information discriminant analysis: A computationally simple heteroscedastic feature extraction technique , 2008, Pattern Recognit..

[31]  Daniel Baldauf,et al.  The Posterior Parietal Cortex Encodes in Parallel Both Goals for Double-Reach Sequences , 2008, The Journal of Neuroscience.

[32]  R. Andersen,et al.  Intention, Action Planning, and Decision Making in Parietal-Frontal Circuits , 2009, Neuron.

[33]  Zoran Nenadic,et al.  An efficient discriminant-based solution for small sample size problem , 2009, Pattern Recognit..

[34]  Xuedong Chen,et al.  Decoding human motor activity from EEG single trials for a discrete two-dimensional cursor control , 2009, Journal of neural engineering.

[35]  Jonathan Westley Peirce,et al.  Neuroinformatics Original Research Article Generating Stimuli for Neuroscience Using Psychopy , 2022 .

[36]  M. Fink,et al.  Coherent plane-wave compounding for very high frame rate ultrasonography and transient elastography , 2009, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[37]  R. Andersen,et al.  Space representation for eye movements is more contralateral in monkeys than in humans , 2010, Proceedings of the National Academy of Sciences.

[38]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[39]  An H. Do,et al.  Brain-Computer Interface Controlled Functional Electrical Stimulation System for Ankle Movement , 2011, Journal of NeuroEngineering and Rehabilitation.

[40]  J Bercoff,et al.  Ultrafast compound doppler imaging: providing full blood flow characterization , 2011, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[41]  M. Fink,et al.  Functional ultrasound imaging of the brain , 2011, Nature Methods.

[42]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[43]  R. Andersen,et al.  Functional imaging reveals rapid reorganization of cortical activity after parietal inactivation in monkeys , 2012, Proceedings of the National Academy of Sciences.

[44]  Zoran Nenadic,et al.  Brain-computer interface controlled robotic gait orthosis , 2012, Journal of NeuroEngineering and Rehabilitation.

[45]  Lawrence H Snyder,et al.  The representations of reach endpoints in posterior parietal cortex depend on which hand does the reaching. , 2012, Journal of neurophysiology.

[46]  Gregor Schöner,et al.  Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making , 2012, PLoS Comput. Biol..

[47]  Naotaka Fujii,et al.  Decoding continuous three-dimensional hand trajectories from epidural electrocorticographic signals in Japanese macaques , 2012, Journal of neural engineering.

[48]  Jonathan R. Wolpaw,et al.  Brain–Computer InterfacesPrinciples and Practice , 2012 .

[49]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[50]  Zoran Nenadic,et al.  Self-paced brain–computer interface control of ambulation in a virtual reality environment , 2012, Journal of neural engineering.

[51]  Gerald E Loeb,et al.  Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands , 2012, Proceedings of the National Academy of Sciences.

[52]  Andrew J. Woolley,et al.  Chronic intracortical microelectrode arrays induce non-uniform, depth-related tissue responses , 2013, Journal of neural engineering.

[53]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[54]  J. Donoghue,et al.  Failure mode analysis of silicon-based intracortical microelectrode arrays in non-human primates , 2013, Journal of neural engineering.

[55]  M. Fink,et al.  Functional ultrasound imaging of the brain: theory and basic principles , 2013, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[56]  Mickaël Tanter,et al.  Functional ultrasound imaging reveals different odor-evoked patterns of vascular activity in the main olfactory bulb and the anterior piriform cortex , 2014, NeuroImage.

[57]  Arnulf B. A. Graf,et al.  Brain–machine interface for eye movements , 2014, Proceedings of the National Academy of Sciences.

[58]  An H. Do,et al.  The feasibility of a brain-computer interface functional electrical stimulation system for the restoration of overground walking after paraplegia , 2015, Journal of NeuroEngineering and Rehabilitation.

[59]  Charlie Demené,et al.  Spatiotemporal Clutter Filtering of Ultrafast Ultrasound Data Highly Increases Doppler and fUltrasound Sensitivity , 2015, IEEE Transactions on Medical Imaging.

[60]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[61]  Minkyu Ahn,et al.  Journal of Neuroscience Methods , 2015 .

[62]  Yundi Shi,et al.  A diffusion tensor MRI atlas of the postmortem rhesus macaque brain , 2015, NeuroImage.

[63]  Elodie Tiran,et al.  EEG and functional ultrasound imaging in mobile rats , 2015, Nature Methods.

[64]  R. Andersen,et al.  Inactivation of Parietal Reach Region Affects Reaching But Not Saccade Choices in Internally Guided Decisions , 2015, The Journal of Neuroscience.

[65]  G. Montaldo,et al.  Real-time imaging of brain activity in freely moving rats using functional ultrasound , 2015, Nature Methods.

[66]  Arnulf B. A. Graf,et al.  Predicting oculomotor behaviour from correlated populations of posterior parietal neurons , 2014, Nature Communications.

[67]  R. Kümmerli,et al.  Quorum sensing triggers the stochastic escape of individual cells from Pseudomonas putida biofilms , 2015, Nature Communications.

[68]  Mickaël Tanter,et al.  Transcranial functional ultrasound imaging of the brain using microbubble-enhanced ultrasensitive Doppler , 2016, NeuroImage.

[69]  F. Bremmer,et al.  Decoding Target Distance and Saccade Amplitude from Population Activity in the Macaque Lateral Intraparietal Area (LIP) , 2016, Front. Integr. Neurosci..

[70]  B. Forstmann,et al.  Frontosubthalamic Circuits for Control of Action and Cognition , 2016, The Journal of Neuroscience.

[71]  Charlie Demené,et al.  4D microvascular imaging based on ultrafast Doppler tomography , 2016, NeuroImage.

[72]  Mickael Tanter,et al.  Intraoperative Functional Ultrasound Imaging of Human Brain Activity , 2017, Scientific Reports.

[73]  Krishna V Shenoy,et al.  Augmenting intracortical brain-machine interface with neurally driven error detectors , 2017, Journal of neural engineering.

[74]  Pieter Kruizinga,et al.  Compressive 3D ultrasound imaging using a single sensor , 2017, Science Advances.

[75]  Francis R. Willett,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[76]  M. Tanter,et al.  Functional ultrasound imaging of brain activity in human newborns , 2017, Science Translational Medicine.

[77]  E. Pnevmatikakis,et al.  NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data , 2017, Journal of Neuroscience Methods.

[78]  Thomas Deffieux,et al.  Functional ultrasound neuroimaging: a review of the preclinical and clinical state of the art , 2018, Current Opinion in Neurobiology.

[79]  M. Tanter,et al.  A large aperture row column addressed probe for in vivo 4D ultrafast doppler ultrasound imaging. , 2018, Physics in medicine and biology.

[80]  J. Wolpaw,et al.  Controlling pre-movement sensorimotor rhythm can improve finger extension after stroke , 2018, Journal of neural engineering.

[81]  Richard Rau,et al.  3D functional ultrasound imaging of pigeons , 2018, NeuroImage.

[82]  S. Shamma,et al.  Multi-scale mapping along the auditory hierarchy using high-resolution functional UltraSound in the awake ferret , 2018, bioRxiv.

[83]  S. Charpak,et al.  Vascular Compartmentalization of Functional Hyperemia from the Synapse to the Pia , 2018, Neuron.

[84]  A large aperture row column addressed probe for in vivo 4D ultrafast doppler ultrasound imaging. , 2018, Physics in medicine and biology.

[85]  Spencer Kellis,et al.  From thought to action: The brain–machine interface in posterior parietal cortex , 2019, Proceedings of the National Academy of Sciences.

[86]  Zoran Nenadic,et al.  A benchtop system to assess the feasibility of a fully independent and implantable brain-machine interface , 2019, Journal of neural engineering.

[87]  P. Sachs,et al.  SMARCAD1 ATPase activity is required to silence endogenous retroviruses in embryonic stem cells , 2019, Nature Communications.

[88]  J. Sahel,et al.  Functional ultrasound imaging of the brain reveals propagation of task-related brain activity in behaving primates , 2019, Nature Communications.

[89]  M. Shanechi Brain–machine interfaces from motor to mood , 2019, Nature Neuroscience.

[90]  A. Benabid,et al.  An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration , 2019, The Lancet Neurology.

[91]  R. Andersen,et al.  Intrinsic Variable Learning for Brain-Machine Interface Control by Human Anterior Intraparietal Cortex , 2019, Neuron.

[92]  M. Tanter,et al.  4D functional ultrasound imaging of whole-brain activity in rodents , 2019, Nature Methods.

[93]  S. Koekkoek,et al.  Functional Ultrasound (fUS) During Awake Brain Surgery: The Clinical Potential of Intra-Operative Functional and Vascular Brain Mapping , 2020, Frontiers in Neuroscience.

[94]  D. Le Bihan,et al.  Mesoscopic and microscopic imaging of sensory responses in the same animal , 2019, Nature Communications.

[95]  Thomas Payen,et al.  Acoustic biomolecules enhance hemodynamic functional ultrasound imaging of neural activity , 2020, NeuroImage.

[96]  Sangjin Yoo,et al.  Ultrasound Technologies for Imaging and Modulating Neural Activity , 2020, Neuron.

[97]  Meijun Ye,et al.  Longitudinal neural and vascular structural dynamics produced by chronic microelectrode implantation. , 2020, Biomaterials.

[98]  Clément Brunner,et al.  A Platform for Brain-wide Volumetric Functional Ultrasound Imaging and Analysis of Circuit Dynamics in Awake Mice , 2020, Neuron.

[99]  Mason McGill,et al.  A map of object space in primate inferotemporal cortex , 2020, Nature.

[100]  Thomas Deffieux,et al.  Functional ultrasound imaging of deep visual cortex in awake nonhuman primates , 2020, Proceedings of the National Academy of Sciences.