Cortical visual prostheses: from microstimulation to functional percept

Cortical visual prostheses are intended to restore vision by targeted electrical stimulation of the visual cortex. The perception of spots of light, called phosphenes, resulting from microstimulation of the visual pathway, suggests the possibility of creating meaningful percept made of phosphenes. However, to date electrical stimulation of V1 has still not resulted in perception of phosphenated images that goes beyond punctate spots of light. In this review, we summarize the clinical and experimental progress that has been made in generating phosphenes and modulating their associated perceptual characteristics in human and macaque primary visual cortex (V1). We focus specifically on the effects of different microstimulation parameters on perception and we analyse key challenges facing the generation of meaningful artificial percepts. Finally, we propose solutions to these challenges based on the application of supervised learning of population codes for spatial stimulation of visual cortex.

[1]  Ljubomir Manola,et al.  Anodal vs cathodal stimulation of motor cortex: A modeling study , 2007, Clinical Neurophysiology.

[2]  Amy M. Ni,et al.  Microstimulation Reveals Limits in Detecting Different Signals from a Local Cortical Region , 2010, Current Biology.

[3]  Emily A. Tobey,et al.  Cross‐modal plasticity and the sensitive period for cochlear implantation , 2006 .

[4]  J. Rauschecker Compensatory plasticity and sensory substitution in the cerebral cortex , 1995, Trends in Neurosciences.

[5]  K. Najafi,et al.  A Modular 32-site wireless neural stimulation microsystem , 2004, IEEE Journal of Solid-State Circuits.

[6]  N Parikh,et al.  Performance of visually guided tasks using simulated prosthetic vision and saliency-based cues , 2013, Journal of neural engineering.

[7]  A. Sher,et al.  Photovoltaic Retinal Prosthesis with High Pixel Density , 2012, Nature Photonics.

[8]  K. Svoboda,et al.  Sparse optical microstimulation in barrel cortex drives learned behaviour in freely moving mice , 2008, Nature.

[9]  Richard A. Normann,et al.  Simulation of a phosphene-based visual field: Visual acuity in a pixelized vision system , 2006, Annals of Biomedical Engineering.

[10]  Richard T. Born,et al.  Cortical magnification plus cortical plasticity equals vision? , 2015, Vision Research.

[11]  C. Schwarz,et al.  Spatiotemporal effects of microstimulation in rat neocortex: a parametric study using multielectrode recordings. , 2003, Journal of neurophysiology.

[12]  T. Wiesel,et al.  Receptive field dynamics in adult primary visual cortex , 1992, Nature.

[13]  Warren M Slocum,et al.  What delay fields tell us about striate cortex. , 2007, Journal of neurophysiology.

[14]  Y. Yamane,et al.  Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns , 2001, Nature Neuroscience.

[15]  M. Hallett,et al.  Neural networks for Braille reading by the blind. , 1998 .

[16]  G. S. Brindley,et al.  Properties of Cortical Electrical Phosphenes , 1978 .

[17]  Antonio Martínez-Álvarez,et al.  A neuroengineering suite of computational tools for visual prostheses , 2007, Neurocomputing.

[18]  Eduardo Fernández,et al.  Erratum: Toward the development of a cortically based visual neuroprosthesis (Journal of Neural Engineering (2009) 6 (035001)) , 2009 .

[19]  J. Kaas,et al.  Rapid reorganization of cortical maps in adult cats following restricted deafferentation in retina , 1992, Vision Research.

[20]  W. Lytton,et al.  Cortical Plasticity Induced by Spike-Triggered Microstimulation in Primate Somatosensory Cortex , 2013, PloS one.

[21]  K. Mathieson,et al.  Performance of photovoltaic arrays in-vivo and characteristics of prosthetic vision in animals with retinal degeneration , 2015, Vision Research.

[22]  Bahman Tahayori,et al.  Modelling extracellular electrical stimulation: III. Derivation and interpretation of neural tissue equations , 2014, Journal of neural engineering.

[23]  E. J. Tehovnik Electrical stimulation of neural tissue to evoke behavioral responses , 1996, Journal of Neuroscience Methods.

[24]  F. Rattay,et al.  The basic mechanism for the electrical stimulation of the nervous system , 1999, Neuroscience.

[25]  D. Pascolini,et al.  Global estimates of visual impairment: 2010 , 2011, British Journal of Ophthalmology.

[26]  E. Seidemann,et al.  Dynamics of Depolarization and Hyperpolarization in the Frontal Cortex and Saccade Goal , 2002, Science.

[27]  E Jankowska,et al.  Direct and indirect activation of nerve cells by electrical pulses applied extracellularly. , 1976, The Journal of physiology.

[28]  E. J. Tehovnik,et al.  Phosphene induction and the generation of saccadic eye movements by striate cortex. , 2005, Journal of neurophysiology.

[29]  M. Raichle,et al.  Adaptive changes in early and late blind: a fMRI study of Braille reading. , 2002, Journal of neurophysiology.

[30]  J. Rauschecker,et al.  A Positron Emission Tomographic Study of Auditory Localization in the Congenitally Blind , 2000, The Journal of Neuroscience.

[31]  Spencer C. Chen,et al.  Simulating prosthetic vision: II. Measuring functional capacity , 2009, Vision Research.

[32]  F. Haiss,et al.  Spatiotemporal Dynamics of Cortical Sensorimotor Integration in Behaving Mice , 2007, Neuron.

[33]  Christopher C. Pack,et al.  Hierarchical processing of complex motion along the primate dorsal visual pathway , 2012, Proceedings of the National Academy of Sciences.

[34]  Anthony J. Maeder,et al.  Visual prostheses for the blind : a framework for information presentation , 2007 .

[35]  M. Stryker,et al.  Development and Plasticity of the Primary Visual Cortex , 2012, Neuron.

[36]  Sébastien Joucla,et al.  Modeling extracellular electrical neural stimulation: From basic understanding to MEA-based applications , 2012, Journal of Physiology-Paris.

[37]  E. Jankowska,et al.  An electrophysiological demonstration of the axonal projections of single spinal interneurones in the cat , 1972, The Journal of physiology.

[38]  S. Hillyard,et al.  Improved auditory spatial tuning in blind humans , 1999, Nature.

[39]  DK Murphey,et al.  Perceiving Electrical Stimulation of Identified Human Visual Areas , 2009, NeuroImage.

[40]  S I Helms Tillery,et al.  Computational modeling of direct neuronal recruitment during intracortical microstimulation in somatosensory cortex , 2013, Journal of neural engineering.

[41]  E. Chichilnisky,et al.  High-Resolution Electrical Stimulation of Primate Retina for Epiretinal Implant Design , 2008, The Journal of Neuroscience.

[42]  C. Duffy MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[43]  Kechen Zhang,et al.  A Sparse Object Coding Scheme in Area V4 , 2011, Current Biology.

[44]  Shelley Fried,et al.  The Retinal Response to Sinusoidal Electrical Stimulation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  M. Ptito,et al.  rTMS of the occipital cortex abolishes Braille reading and repetition priming in blind subjects , 2007, Neurology.

[46]  Arthur James Lowery,et al.  Restoration of vision in blind individuals using bionic devices: A review with a focus on cortical visual prostheses , 2015, Brain Research.

[47]  C. Kufta,et al.  Feasibility of a visual prosthesis for the blind based on intracortical microstimulation of the visual cortex , 1996 .

[48]  David Tsai,et al.  Current steering in retinal stimulation via a quasimonopolar stimulation paradigm. , 2013, Investigative ophthalmology & visual science.

[49]  M. Hallett,et al.  Functional relevance of cross-modal plasticity in blind humans , 1997, Nature.

[50]  Á. Pascual-Leone,et al.  The metamodal organization of the brain. , 2001, Progress in brain research.

[51]  Joseph F. Rizzo,et al.  Psychophysical testing of visual prosthetic devices: a call to establish a multi-national joint task force , 2014, Journal of neural engineering.

[52]  Mohamad Sawan,et al.  A Highly Flexible System for Microstimulation of the Visual Cortex: Design and Implementation , 2007, IEEE Transactions on Biomedical Circuits and Systems.

[53]  Peter H. Schiller,et al.  The ON and OFF channels of the visual system , 1992, Trends in Neurosciences.

[54]  F. Rattay,et al.  Which elements of the mammalian central nervous system are excited by low current stimulation with microelectrodes? , 2010, Neuroscience.

[55]  J. Weiland,et al.  Selective labeling of retinal ganglion cells with calcium indicators by retrograde loading in vitro , 2009, Journal of Neuroscience Methods.

[56]  R. Kiani,et al.  Microstimulation of inferotemporal cortex influences face categorization , 2006, Nature.

[57]  John H. R. Maunsell,et al.  Topographic organization of the middle temporal visual area in the macaque monkey: Representational biases and the relationship to callosal connections and myeloarchitectonic boundaries , 1987, The Journal of comparative neurology.

[58]  G. Loeb,et al.  Visual sensations produced by intracortical microstimulation of the human occipital cortex , 1990, Medical and Biological Engineering and Computing.

[59]  R. Doty,et al.  Laminar variation in threshold for detection of electrical excitation of striate cortex by macaques. , 2005, Journal of neurophysiology.

[60]  Tetsuya Yagi,et al.  A programmable controller for spatio-temporal pattern stimulation of cortical visual prosthesis , 2016, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[61]  Frank Rattay,et al.  Correction: Energy-Optimal Electrical-Stimulation Pulses Shaped by the Least-Action Principle , 2014, PLoS ONE.

[62]  M. Sahin,et al.  Non-rectangular waveforms for neural stimulation with practical electrodes , 2007, Journal of neural engineering.

[63]  W. H. Dobelle Artificial vision for the blind by connecting a television camera to the visual cortex. , 2000, ASAIO journal.

[64]  A. Fuchs,et al.  Eye movements evoked by stimulation of frontal eye fields. , 1969, Journal of neurophysiology.

[65]  L. Merabet,et al.  The plastic human brain cortex. , 2005, Annual review of neuroscience.

[66]  E. J. Tehovnik,et al.  Saccadic eye movements evoked by microstimulation of striate cortex , 2003, The European journal of neuroscience.

[67]  C. Kufta,et al.  Visuotopic mapping through a multichannel stimulating implant in primate V1. , 2005, Journal of neurophysiology.

[68]  R. Malach,et al.  Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind , 2003, Nature Neuroscience.

[69]  Mark S Humayun,et al.  Electrical Stimulation of the Retina to Produce Artificial Vision. , 2016, Annual review of vision science.

[70]  Á. Pascual-Leone,et al.  Alexia for Braille following bilateral occipital stroke in an early blind woman , 2000, NeuroReport.

[71]  Geoffrey L. Kendall,et al.  New methods devised specify the size and color of the spots monkeys see when striate cortex (area V1) is electrically stimulated , 2011, Proceedings of the National Academy of Sciences.

[72]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[73]  N. Logothetis,et al.  Lack of long-term cortical reorganization after macaque retinal lesions , 2005, Nature.

[74]  J. S. Lee,et al.  Deafness: Cross-modal plasticity and cochlear implants , 2001, Nature.

[75]  T S Davis,et al.  Spatial and temporal characteristics of V1 microstimulation during chronic implantation of a microelectrode array in a behaving macaque , 2012, Journal of neural engineering.

[76]  J. B. Ranck,et al.  Which elements are excited in electrical stimulation of mammalian central nervous system: A review , 1975, Brain Research.

[77]  Daniel C Millard,et al.  Voltage-sensitive Dye Imaging Reveals Improved Topographic Activation of Cortex in Response to Manipulation of Thalamic Microstimulation Parameters , 2011 .

[78]  J. Maunsell,et al.  The Effect of Perceptual Learning on Neuronal Responses in Monkey Visual Area V4 , 2004, The Journal of Neuroscience.

[79]  Guy A Orban,et al.  Higher order visual processing in macaque extrastriate cortex. , 2008, Physiological reviews.

[80]  Arthur R. Houweling,et al.  Behavioural report of single neuron stimulation in somatosensory cortex , 2008, Nature.

[81]  D. Whitteridge,et al.  The representation of the visual field on the cerebral cortex in monkeys , 1961, The Journal of physiology.

[82]  Leslie G. Ungerleider,et al.  Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys , 1982, Behavioural Brain Research.

[83]  Eduardo Fernandez,et al.  Toward the development of a cortically based visual neuroprosthesis , 2009, Journal of neural engineering.

[84]  E. J. Tehovnik,et al.  Mapping Cortical Activity Elicited with Electrical Microstimulation Using fMRI in the Macaque , 2005, Neuron.

[85]  S. Dumoulin,et al.  The Relationship between Cortical Magnification Factor and Population Receptive Field Size in Human Visual Cortex: Constancies in Cortical Architecture , 2011, The Journal of Neuroscience.

[86]  A. Arnold,et al.  Further study on the excitation of pyramidal tract cells by intracortical microstimulation , 1976, Experimental Brain Research.

[87]  W. Newsome,et al.  Microstimulation in visual area MT: effects on direction discrimination performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[88]  A. Sher,et al.  Photovoltaic retinal prosthesis: implant fabrication and performance , 2012, Journal of neural engineering.

[89]  T. Smith,et al.  The C.I.E. colorimetric standards and their use , 1931 .

[90]  W. Newsome,et al.  Microstimulation in visual area MT: effects of varying pulse amplitude and frequency , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[91]  M. Ito,et al.  Electrophysiological analysis of the vestibulospinal reflex pathway of rabbit. I. Classification of tract cells , 1973, Experimental Brain Research.

[92]  Lee E. Miller,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[93]  Gislin Dagnelie,et al.  Detection, eye–hand coordination and virtual mobility performance in simulated vision for a cortical visual prosthesis device , 2009, Journal of neural engineering.

[94]  James D. Weiland,et al.  Visual Prosthesis , 2008, Proceedings of the IEEE.

[95]  G. Brindley,et al.  The sensations produced by electrical stimulation of the visual cortex , 1968, The Journal of physiology.

[96]  Joseph F Rizzo,et al.  Selective activation of neuronal targets with sinusoidal electric stimulation. , 2010, Journal of neurophysiology.

[97]  Garrett B. Stanley,et al.  Behavioral and Electrophysiological Effects of Cortical Microstimulation Parameters , 2013, PloS one.

[98]  Amy M. Ni,et al.  Insights into cortical mechanisms of behavior from microstimulation experiments , 2013, Progress in Neurobiology.

[99]  W. D. Thompson,et al.  Excitation of pyramidal tract cells by intracortical microstimulation: effective extent of stimulating current. , 1968, Journal of neurophysiology.

[100]  Amgad G. Habib,et al.  Quasi-monopolar electrical stimulation of the retina: a computational modelling study , 2014, Journal of neural engineering.

[101]  D. Bavelier,et al.  Cross-modal plasticity: where and how? , 2002, Nature Reviews Neuroscience.

[102]  Warren M. Grill,et al.  Selective Microstimulation of Central Nervous System Neurons , 2000, Annals of Biomedical Engineering.

[103]  M. Mladejovsky,et al.  Artificial Vision for the Blind: Electrical Stimulation of Visual Cortex Offers Hope for a Functional Prosthesis , 1974, Science.

[104]  A. Arnold,et al.  Spinal branching of corticospinal axons in the cat , 1976, Experimental Brain Research.

[105]  M. Hallett,et al.  Period of susceptibility for cross‐modal plasticity in the blind , 1999, Annals of neurology.

[106]  N. Logothetis,et al.  The effects of electrical microstimulation on cortical signal propagation , 2010, Nature Neuroscience.

[107]  E. J. Tehovnik,et al.  Microstimulation of visual cortex to restore vision. , 2009, Progress in brain research.

[108]  Frans W Cornelissen,et al.  Large-scale remapping of visual cortex is absent in adult humans with macular degeneration , 2011, Nature Neuroscience.

[109]  E. J. Tehovnik,et al.  Microstimulation of macaque V1 disrupts target selection: effects of stimulation polarity , 2002, Experimental Brain Research.

[110]  L. Merabet,et al.  What blindness can tell us about seeing again: merging neuroplasticity and neuroprostheses , 2005, Nature Reviews Neuroscience.

[111]  Christopher C. Pack,et al.  The Contribution of Area MT to Visual Motion Perception Depends on Training , 2016, Neuron.

[112]  Philip M. Lewis,et al.  Electrical stimulation of the brain and the development of cortical visual prostheses: An historical perspective , 2016, Brain Research.

[113]  Barry B. Lee,et al.  Psychophysics of electrical stimulation of striate cortex in macaques. , 2005, Journal of neurophysiology.

[114]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[115]  M. Sawan,et al.  Wireless Smart Implants Dedicated to Multichannel Monitoring and Microstimulation , 2005, The IEEE/ACS International Conference on Pervasive Services.

[116]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[117]  David Bradley,et al.  A model for intracortical visual prosthesis research. , 2003, Artificial organs.

[118]  M. Carandini,et al.  Adaptation maintains population homeostasis in primary visual cortex , 2013, Nature Neuroscience.

[119]  C. Gilbert,et al.  Topographic reorganization in the striate cortex of the adult cat and monkey is cortically mediated , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[120]  L. Merabet,et al.  Development of a cortical visual neuroprosthesis for the blind: the relevance of neuroplasticity , 2005, Journal of neural engineering.

[121]  F. Helmchen,et al.  In vivo calcium imaging of neural network function. , 2007, Physiology.

[122]  R. B. Freeman,et al.  Line length selective masking , 1976, Brain Research.

[123]  M. Hallett,et al.  Activation of the primary visual cortex by Braille reading in blind subjects , 1996, Nature.

[124]  M. Mladejovsky,et al.  ‘Braille’ reading by a blind volunteer by visual cortex stimulation , 1976, Nature.

[125]  John H.R. Maunsell,et al.  Behavioral Detection of Electrical Microstimulation in Different Cortical Visual Areas , 2007, Current Biology.

[126]  J D Loudin,et al.  Optoelectronic retinal prosthesis: system design and performance , 2007, Journal of neural engineering.

[127]  T S Davis,et al.  Multiple factors may influence the performance of a visual prosthesis based on intracortical microstimulation: nonhuman primate behavioural experimentation , 2011, Journal of neural engineering.

[128]  R. Reid,et al.  Direct Activation of Sparse, Distributed Populations of Cortical Neurons by Electrical Microstimulation , 2009, Neuron.

[129]  F. Rösler,et al.  Speech processing activates visual cortex in congenitally blind humans , 2002, The European journal of neuroscience.

[130]  Arthur James Lowery Introducing the Monash vision group's cortical prosthesis , 2013, 2013 IEEE International Conference on Image Processing.

[131]  E. J. Tehovnik,et al.  Phosphene induction by microstimulation of macaque V1 , 2007, Brain Research Reviews.

[132]  W. Dobelle,et al.  Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind , 1974, The Journal of physiology.

[133]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[134]  R. Doty,et al.  Electrical stimulation of the brain in behavioral context. , 1969, Annual review of psychology.

[135]  E. Chichilnisky,et al.  Spatially Patterned Electrical Stimulation to Enhance Resolution of Retinal Prostheses , 2014, The Journal of Neuroscience.

[136]  J. Kaas,et al.  Reorganization of retinotopic cortical maps in adult mammals after lesions of the retina. , 1990, Science.

[137]  C. Kufta,et al.  Estimating Phosphene Maps for Psychophysical Experiments used in Testing a Cortical Visual Prosthesis Device , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[138]  Giles S. Brindley,et al.  Sensory Effects of Electrical Stimulation of the Visual and Paravisual Cortex in Man , 1973 .

[139]  Gislin Dagnelie,et al.  Real and virtual mobility performance in simulated prosthetic vision , 2007, Journal of neural engineering.

[140]  Gislin Dagnelie,et al.  Psychophysical evaluation for visual prosthesis. , 2008, Annual review of biomedical engineering.

[141]  Yong Gu,et al.  Causal Links between Dorsal Medial Superior Temporal Area Neurons and Multisensory Heading Perception , 2012, The Journal of Neuroscience.

[142]  E. R Gizewski,et al.  Cross-modal plasticity for sensory and motor activation patterns in blind subjects , 2003, NeuroImage.