Brain Co-Processors: Using AI to Restore and Augment Brain Function

Brain-computer interfaces (BCIs) use decoding algorithms to control prosthetic devices based on brain signals for restoration of lost function. Computer-brain interfaces (CBIs), on the other hand, use encoding algorithms to transform external sensory signals into neural stimulation patterns for restoring sensation or providing sensory feedback for closed-loop prosthetic control. In this article, we introduce brain co-processors, devices that combine decoding and encoding in a unified framework using artificial intelligence (AI) to supplement or augment brain function. Brain co-processors can be used for a range of applications, from inducing Hebbian plasticity for rehabilitation after brain injury to reanimating paralyzed limbs and enhancing memory. A key challenge is simultaneous multi-channel neural decoding and encoding for optimization of external behavioral or task-related goals. We describe a new framework for developing brain co-processors based on artificial neural networks, deep learning and reinforcement learning. These "neural co-processors" allow joint optimization of cost functions with the nervous system to achieve desired behaviors. By coupling artificial neural networks with their biological counterparts, neural co-processors offer a new way of restoring and augmenting the brain, as well as a new scientific tool for brain research. We conclude by discussing the potential applications and ethical implications of brain co-processors.

[1]  Yuxiao Yang,et al.  Mood variations decoded from multi-site intracranial human brain activity , 2018, Nature Biotechnology.

[2]  Joseph E O'Doherty,et al.  A learning–based approach to artificial sensory feedback leads to optimal integration , 2014, Nature Neuroscience.

[3]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[4]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[5]  Rajesh P. N. Rao,et al.  Generalized neural decoders for transfer learning across participants and recording modalities , 2020, bioRxiv.

[6]  Rajesh P. N. Rao,et al.  Playing 20 Questions with the Mind: Collaborative Problem Solving by Humans Using a Brain-to-Brain Interface , 2015, PloS one.

[7]  Rajesh P. N. Rao,et al.  Towards neural co-processors for the brain: combining decoding and encoding in brain–computer interfaces , 2018, Current Opinion in Neurobiology.

[8]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[9]  Krishna V Shenoy,et al.  ERAASR: an algorithm for removing electrical stimulation artifacts from multielectrode array recordings , 2017, bioRxiv.

[10]  Lee E Miller,et al.  Toward a Proprioceptive Neural Interface that Mimics Natural Cortical Activity. , 2016, Advances in experimental medicine and biology.

[11]  Rajesh P. N. Rao Brain-Computer Interfacing: An Introduction , 2010 .

[12]  Rajesh P. N. Rao,et al.  Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans , 2019, Scientific Reports.

[13]  E Donchin,et al.  The truth will out: interrogative polygraphy ("lie detection") with event-related brain potentials. , 1991, Psychophysiology.

[14]  Dustin J Tyler,et al.  Neural interfaces for somatosensory feedback: bringing life to a prosthesis. , 2015, Current opinion in neurology.

[15]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[16]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[17]  E. Donchin,et al.  Brain-computer interface research at the university of south Florida cognitive psychophysiology laboratory: the P300 speller , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Rajesh P. N. Rao,et al.  A Direct Brain-to-Brain Interface in Humans , 2014, PloS one.

[19]  Rajesh P. N. Rao,et al.  Signal recovery from stimulation artifacts in intracranial recordings with dictionary learning , 2020, Journal of neural engineering.

[20]  Sina Tafazoli,et al.  Learning to control the brain through adaptive closed-loop patterned stimulation , 2020, bioRxiv.

[21]  Miguel A. L. Nicolelis,et al.  Building an organic computing device with multiple interconnected brains , 2015, Scientific Reports.

[22]  Matthew T Rich,et al.  Plasticity at Thalamo-amygdala Synapses Regulates Cocaine-Cue Memory Formation and Extinction. , 2019, Cell reports.

[23]  M. Nicolelis,et al.  Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. , 2017, Physiological reviews.

[24]  Benjamin C. Johnson,et al.  Toward true closed-loop neuromodulation: artifact-free recording during stimulation , 2018, Current Opinion in Neurobiology.

[25]  Jing Wu,et al.  Task-Specific Somatosensory Feedback via Cortical Stimulation in Humans , 2016, IEEE Transactions on Haptics.

[26]  Spencer Kellis,et al.  A cognitive neuroprosthetic that uses cortical stimulation for somatosensory feedback , 2014, Journal of neural engineering.

[27]  Rafael Yuste,et al.  On the Necessity of Ethical Guidelines for Novel Neurotechnologies , 2016, Cell.

[28]  Á. Pascual-Leone,et al.  Conscious Brain-to-Brain Communication in Humans Using Non-Invasive Technologies , 2014, Brain Stimulation.

[29]  Kevin A. Johnson,et al.  Detecting Deception Using Functional Magnetic Resonance Imaging , 2005, Biological Psychiatry.

[30]  Wolfram Burgard,et al.  New Perspectives on Neuroengineering and Neurotechnologies: NSF-DFG Workshop Report , 2016, IEEE Transactions on Biomedical Engineering.

[31]  Albert Rothenberg,et al.  Physical Control of the Mind: Toward a Psychocivilized Society , 1970, The Yale Journal of Biology and Medicine.

[32]  M. Nuttin,et al.  Asynchronous non-invasive brain-actuated control of an intelligent wheelchair , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[33]  J. Meditch,et al.  Applied optimal control , 1972, IEEE Transactions on Automatic Control.

[34]  Francis R. Willett,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[35]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Theodore W Berger,et al.  A cortical neural prosthesis for restoring and enhancing memory , 2011, Journal of neural engineering.

[37]  B. Wilson,et al.  Cochlear Implants: Principles & Practices , 2000 .

[38]  Rajesh P. N. Rao,et al.  Navigating a 2D Virtual World Using Direct Brain Stimulation , 2016, Front. Robot. AI.

[39]  A. Lozano,et al.  Fornical Closed-Loop Stimulation for Alzheimer’s Disease , 2018, Trends in Neurosciences.

[40]  Qin,et al.  A Brain–Spinal Interface Alleviating Gait Deficits after Spinal Cord Injury in Primates , 2017 .

[41]  Alan Rubel,et al.  Four ethical priorities for neurotechnologies and AI , 2017, Nature.

[42]  Howard Jay Chizeck,et al.  Cortical Brain–Computer Interface for Closed-Loop Deep Brain Stimulation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[43]  Robert E. Hampson,et al.  A cognitive prosthesis for memory facilitation by closed-loop functional ensemble stimulation of hippocampal neurons in primate brain , 2017, Experimental Neurology.

[44]  Laehyun Kim,et al.  Non-invasive transmission of sensorimotor information in humans using an EEG/focused ultrasound brain-to-brain interface , 2017, PloS one.

[45]  Rajesh P. N. Rao,et al.  Brain–computer interfaces: a powerful tool for scientific inquiry , 2014, Current Opinion in Neurobiology.

[46]  J. Weiland,et al.  Retinal Prosthesis , 2014, IEEE Transactions on Biomedical Engineering.

[47]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.

[48]  R. J. Vogelstein,et al.  Restoring the sense of touch with a prosthetic hand through a brain interface , 2013, Proceedings of the National Academy of Sciences.

[49]  David J. Guggenmos,et al.  Restoration of function after brain damage using a neural prosthesis , 2013, Proceedings of the National Academy of Sciences.

[50]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[51]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[52]  M. Nicolelis,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[53]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[54]  Michael S. Beauchamp,et al.  Electrical Stimulation of Visual Cortex: Relevance for the Development of Visual Cortical Prosthetics. , 2017, Annual review of vision science.

[55]  Rajesh P. N. Rao,et al.  When Two Brains Connect , 2014 .

[56]  S. Meagher Instant neural control of a movement signal , 2002 .

[57]  Wentai Liu,et al.  Retinal Prosthesis , 2018, Essentials in Ophthalmology.

[58]  Rajesh P. N. Rao,et al.  Control of a humanoid robot by a noninvasive brain–computer interface in humans , 2008, Journal of neural engineering.

[59]  L. Miller,et al.  Restoration of grasp following paralysis through brain-controlled stimulation of muscles , 2012, Nature.

[60]  Jing Wang,et al.  A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information , 2013, Scientific Reports.

[61]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[62]  Stephen T. Foldes,et al.  Intracortical microstimulation of human somatosensory cortex , 2016, Science Translational Medicine.

[63]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[64]  John P. Cunningham,et al.  Electrical stimulus artifact cancellation and neural spike detection on large multi-electrode arrays , 2016, bioRxiv.

[65]  Michael L. Boninger,et al.  Intracortical Microstimulation as a Feedback Source for Brain-Computer Interface Users , 2017, Brain-Computer Interface Research.

[66]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[67]  Rajesh P. N. Rao,et al.  Automatic extraction of command hierarchies for adaptive brain-robot interfacing , 2012, 2012 IEEE International Conference on Robotics and Automation.

[68]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[69]  E. Fetz,et al.  Long-term motor cortex plasticity induced by an electronic neural implant , 2006, Nature.

[70]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[71]  Rajesh P. N. Rao,et al.  BrainNet: A Multi-Person Brain-to-Brain Interface for Direct Collaboration Between Brains , 2018, ArXiv.

[72]  M. Nuttin,et al.  A brain-actuated wheelchair: Asynchronous and non-invasive Brain–computer interfaces for continuous control of robots , 2008, Clinical Neurophysiology.

[73]  Miguel A. L. Nicolelis,et al.  A Brain-Machine Interface Instructed by Direct Intracortical Microstimulation , 2009, Front. Integr. Neurosci..

[74]  Peter J. Ifft,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.