A fast intracortical brain–machine interface with patterned optogenetic feedback

OBJECTIVE The development of brain-machine interfaces (BMIs) brings new prospects to patients with a loss of autonomy. By combining online recordings of brain activity with a decoding algorithm, patients can learn to control a robotic arm in order to perform simple actions. However, in contrast to the vast amounts of somatosensory information channeled by limbs to the brain, current BMIs are devoid of touch and force sensors. Patients must therefore rely solely on vision and audition, which are maladapted to the control of a prosthesis. In contrast, in a healthy limb, somatosensory inputs alone can efficiently guide the handling of a fragile object, or ensure a smooth trajectory. We have developed a BMI in the mouse that includes a rich artificial somatosensory-like cortical feedback. APPROACH Our setup includes online recordings of the activity of multiple neurons in the whisker primary motor cortex (vM1) and delivers feedback simultaneously via a low-latency, high-refresh-rate, spatially structured photo-stimulation of the whisker primary somatosensory cortex (vS1), based on a mapping obtained by intrinsic imaging. MAIN RESULTS We demonstrate the operation of the loop and show that mice can detect the neuronal spiking in vS1 triggered by the photo-stimulations. Finally, we show that the mice can learn a behavioral task relying solely on the artificial inputs and outputs of the closed-loop BMI. SIGNIFICANCE This is the first motor BMI that includes a short-latency, intracortical, somatosensory-like feedback. It will be a useful platform to discover efficient cortical feedback schemes towards future human BMI applications.

[1]  T. Kim,et al.  The effect of delayed visual feedback on telerobotic surgery , 2005, Surgical Endoscopy And Other Interventional Techniques.

[2]  F. Helmchen,et al.  Behaviour-dependent recruitment of long-range projection neurons in somatosensory cortex , 2013, Nature.

[3]  Ernst Bamberg,et al.  Spectral characteristics of the photocycle of channelrhodopsin-2 and its implication for channel function. , 2008, Journal of molecular biology.

[4]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[5]  C. Petersen,et al.  Optogenetic Stimulation of Cortex to Map Evoked Whisker Movements in Awake Head-Restrained Mice , 2018, Neuroscience.

[6]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[7]  A. Djourno,et al.  De l'excitation électrique du nerfcochléaire chez l'homme, par induction à distance, a l'aide d'un micro bobinage inclus à demeure. , 1957 .

[8]  Luca Citi,et al.  Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses , 2014, Science Translational Medicine.

[9]  E. Bamberg,et al.  Channelrhodopsin-2, a directly light-gated cation-selective membrane channel , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[10]  W. Penfield,et al.  SOMATIC MOTOR AND SENSORY REPRESENTATION IN THE CEREBRAL CORTEX OF MAN AS STUDIED BY ELECTRICAL STIMULATION , 1937 .

[11]  V. Ego-Stengel,et al.  Representation of Tactile Scenes in the Rodent Barrel Cortex , 2018, Neuroscience.

[12]  Zengcai V. Guo,et al.  Erratum: Procedures for Behavioral Experiments in Head-Fixed Mice (PLoS ONE (2014) 9, 2 (e88678) DOI:10.1371/journal.pone.0088678) , 2014 .

[13]  A. E. Casale,et al.  Motor Cortex Feedback Influences Sensory Processing by Modulating Network State , 2013, Neuron.

[14]  D. Shulz,et al.  The Matrix: A new tool for probing the whisker-to-barrel system with natural stimuli , 2010, Journal of Neuroscience Methods.

[15]  Fritjof Helmchen,et al.  Chronic imaging of cortical sensory map dynamics using a genetically encoded calcium indicator , 2012, The Journal of physiology.

[16]  Shen Wang,et al.  Optogenetic spatial and temporal control of cortical circuits on a columnar scale. , 2016, Journal of neurophysiology.

[17]  K. Svoboda,et al.  Long-term, high-resolution imaging in the mouse neocortex through a chronic cranial window , 2009, Nature Protocols.

[18]  Celine Mateo,et al.  Motor Control by Sensory Cortex , 2010, Science.

[19]  L. Carin,et al.  Relationship between intracortical electrode design and chronic recording function. , 2013, Biomaterials.

[20]  M. Popovic,et al.  Restoration of Reaching and Grasping Functions in Hemiplegic Patients with Severe Arm Paralysis , 2003 .

[21]  R. S. Johansson,et al.  Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects , 2004, Experimental Brain Research.

[22]  Keehoon Kim,et al.  Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees. , 2011, Brain : a journal of neurology.

[23]  P. Rossini,et al.  Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans , 2016, eLife.

[24]  N. Farah,et al.  Patterned optical activation of Channelrhodopsin II expressing retinal ganglion cells , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[25]  W. H. Dobelle.  Artificial vision for the blind by connecting a television camera to the visual cortex. , 2000, ASAIO journal.

[26]  Miguel A. L. Nicolelis,et al.  A Brain-Machine Interface Instructed by Direct Intracortical Microstimulation , 2009, Front. Integr. Neurosci..

[27]  Peter J. Ifft,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[28]  Lauren N Ayton,et al.  Progress in the clinical development and utilization of vision prostheses: an update , 2016, Eye and brain.

[29]  Miguel A L Nicolelis,et al.  Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis , 2016, The Journal of Neuroscience.

[30]  R. Reid,et al.  Direct Activation of Sparse, Distributed Populations of Cortical Neurons by Electrical Microstimulation , 2009, Neuron.

[31]  C. S. Ajay Kumar,et al.  A Brain Machine Interface for an Paralysis Patient , 2015 .

[32]  David Kleinfeld,et al.  Precision mapping of the vibrissa representation within murine primary somatosensory cortex , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  Justin C. Williams,et al.  Chronic neural recording using silicon-substrate microelectrode arrays implanted in cerebral cortex , 2004, IEEE Transactions on Biomedical Engineering.

[34]  Stephen T. Foldes,et al.  Intracortical microstimulation of human somatosensory cortex , 2016, Science Translational Medicine.

[35]  T. Wiesel,et al.  Functional architecture of cortex revealed by optical imaging of intrinsic signals , 1986, Nature.

[36]  F. Haiss,et al.  Spatiotemporal Dynamics of Cortical Sensorimotor Integration in Behaving Mice , 2007, Neuron.

[37]  Joseph E O'Doherty,et al.  A learning–based approach to artificial sensory feedback leads to optimal integration , 2014, Nature Neuroscience.

[38]  S. Arber,et al.  Degradation of mouse locomotor pattern in the absence of proprioceptive sensory feedback , 2014, Proceedings of the National Academy of Sciences.

[39]  Zengcai V. Guo,et al.  Procedures for Behavioral Experiments in Head-Fixed Mice , 2014, PloS one.

[40]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[41]  Allan R. Jones,et al.  A toolbox of Cre-dependent optogenetic transgenic mice for light-induced activation and silencing , 2012, Nature Neuroscience.

[42]  R L Sainburg,et al.  Control of limb dynamics in normal subjects and patients without proprioception. , 1995, Journal of neurophysiology.

[43]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[44]  David A. McCormick,et al.  Competing Neural Ensembles in Motor Cortex Gate Goal-Directed Motor Output , 2015, Neuron.

[45]  D. Shulz,et al.  Spatial structure of multiwhisker receptive fields in the barrel cortex is stimulus dependent. , 2011, Journal of neurophysiology.

[46]  S. Bensmaia,et al.  Behavioral assessment of sensitivity to intracortical microstimulation of primate somatosensory cortex , 2015, Proceedings of the National Academy of Sciences.

[47]  M. Mladejovsky,et al.  Artificial Vision for the Blind: Electrical Stimulation of Visual Cortex Offers Hope for a Functional Prosthesis , 1974, Science.

[48]  Max Ortiz-Catalan,et al.  An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs , 2014, Science Translational Medicine.

[49]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[50]  F. Werblin,et al.  Differential Targeting of Optical Neuromodulators to Ganglion Cell Soma and Dendrites Allows Dynamic Control of Center-Surround Antagonism , 2011, Neuron.

[51]  Nico Van de Weghe,et al.  REPRESENTATION OF MOVING OBJECTS ALONG A ROAD NETWORK , 2004 .

[52]  Benoit P. Delhaye,et al.  The neural basis of perceived intensity in natural and artificial touch , 2016, Science Translational Medicine.

[53]  M L Boninger,et al.  Ten-dimensional anthropomorphic arm control in a human brain−machine interface: difficulties, solutions, and limitations , 2015, Journal of neural engineering.

[54]  M. Carandini,et al.  Long Term Recordings with Immobile Silicon Probes in the Mouse Cortex , 2015, bioRxiv.

[55]  Trygve B. Leergaard,et al.  Brain-wide map of efferent projections from rat barrel cortex , 2014, Front. Neuroinform..

[56]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[57]  Allan M Smith,et al.  The effects of digital anesthesia on force control using a precision grip. , 2003, Journal of neurophysiology.

[58]  Yves Frégnac,et al.  “Master” Neurons Induced by Operant Conditioning in Rat Motor Cortex during a Brain-Machine Interface Task , 2013, The Journal of Neuroscience.

[59]  D. Huber,et al.  Rapid Integration of Artificial Sensory Feedback during Operant Conditioning of Motor Cortex Neurons , 2017, Neuron.

[60]  Vincent Jacob,et al.  Emergent Properties of Tactile Scenes Selectively Activate Barrel Cortex Neurons , 2008, Neuron.

[61]  C. Petersen.  The Functional Organization of the Barrel Cortex , 2007, Neuron.

[62]  Steven S. Hsiao,et al.  Multimodal Interactions between Proprioceptive and Cutaneous Signals in Primary Somatosensory Cortex , 2015, Neuron.

[63]  A DJOURNO,et al.  [Auditory prosthesis by means of a distant electrical stimulation of the sensory nerve with the use of an indwelt coiling]. , 1957, La Presse medicale.

[64]  Jessica A. Cardin,et al.  Targeted optogenetic stimulation and recording of neurons in vivo using cell-type-specific expression of Channelrhodopsin-2 , 2010, Nature Protocols.

[65]  E. Welker,et al.  Organization of feedback and feedforward projections of the barrel cortex: a PHA-L study in the mouse , 2004, Experimental Brain Research.

[66]  G. Fleming,et al.  Somatic Motor and Sensory Representation in the Cerebral Cortex of Man as Studied by Electrical Stimulation. (Brain, vol. lx, p. 389, Dec., 1937.) Penfield, W., and Boldrey, E. , 1938 .

[67]  Y. Frégnac,et al.  Bidirectional control of a one-dimensional robotic actuator by operant conditioning of a single unit in rat motor cortex , 2014, Front. Neurosci..

[68]  Patrick J Drew,et al.  Representation of moving wavefronts of whisker deflection in rat somatosensory cortex. , 2007, Journal of neurophysiology.

[69]  Luc Estebanez,et al.  Parvalbumin-Expressing GABAergic Neurons in Primary Motor Cortex Signal Reaching , 2017, Cell reports.

[70]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.

[71]  R. J. Vogelstein,et al.  Restoring the sense of touch with a prosthetic hand through a brain interface , 2013, Proceedings of the National Academy of Sciences.

[72]  Vinzenz H. Schönfelder,et al.  Transformation of Perception from Sensory to Motor Cortex , 2017, Current Biology.

[73]  E. Fetz.  Operant Conditioning of Cortical Unit Activity , 1969, Science.

[74]  G. Brindley,et al.  The sensations produced by electrical stimulation of the visual cortex , 1968, The Journal of physiology.

[75]  F. Clippinger,et al.  A sensory feedback system for an upper-limb amputation prosthesis. , 1974, Bulletin of prosthetics research.