Visuomotor learning from postdictive motor error

Sensorimotor learning adapts motor output to maintain movement accuracy. For saccadic eye movements, learning also alters space perception, suggesting a dissociation between the performed saccade and its internal representation derived from corollary discharge (CD). This is critical since learning is commonly believed to be driven by CD-based visual prediction error. We estimate the internal saccade representation through pre- and trans-saccadic target localization, showing that it decouples from the actual saccade during learning. We present a model that explains motor and perceptual changes by collective plasticity of spatial target percept, motor command, and a forward dynamics model that transforms CD from motor into visuospatial coordinates. We show that learning does not follow visual prediction error but instead a postdictive update of space after saccade landing. We conclude that trans-saccadic space perception guides motor learning via CD-based postdiction of motor error under the assumption of a stable world.

[1]  M. Morrone,et al.  Vision During Saccadic Eye Movements. , 2018, Annual review of vision science.

[2]  Marcus Nyström,et al.  Sampling frequency and eye-tracking measures: how speed affects durations, latencies, and more , 2010 .

[3]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[4]  Markus Lappe,et al.  Saccadic adaptation shapes visual space in macaques. , 2014, Journal of neurophysiology.

[5]  Thérèse Collins,et al.  The relative importance of retinal error and prediction in saccadic adaptation. , 2012, Journal of neurophysiology.

[6]  D. E. Irwin,et al.  Causal Inference for Spatial Constancy across Saccades , 2016, PLoS Comput. Biol..

[7]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[8]  M. Goldberg,et al.  Spatial processing in the monkey frontal eye field. I. Predictive visual responses. , 1997, Journal of neurophysiology.

[9]  A. Fuchs,et al.  The characteristics and neuronal substrate of saccadic eye movement plasticity , 2004, Progress in Neurobiology.

[10]  Robert H Wurtz,et al.  Corollary Discharge Contributions to Perceptual Continuity Across Saccades. , 2018, Annual review of vision science.

[11]  E. Holst,et al.  Das Reafferenzprinzip , 2004, Naturwissenschaften.

[12]  Katharina Havermann,et al.  Fine-Scale Plasticity of Microscopic Saccades , 2014, The Journal of Neuroscience.

[13]  Markus Lappe,et al.  Adaptation and mislocalization fields for saccadic outward adaptation in humans , 2010 .

[14]  J. Yelnik,et al.  Involvement of the cerebellar thalamus in human saccade adaptation , 2001, The European journal of neuroscience.

[15]  A. Hall,et al.  Adaptive Switching Circuits , 2016 .

[16]  Mark W Greenlee,et al.  Differential cortical activation during saccadic adaptation. , 2012, Journal of neurophysiology.

[17]  P. P. Battaglini,et al.  Parietal neurons encoding spatial locations in craniotopic coordinates , 2004, Experimental Brain Research.

[18]  Martin Rolfs,et al.  A generative learning model for saccade adaptation , 2019, PLoS computational biology.

[19]  G. Remington,et al.  Psychopharmacology for the Clinician , 2014 .

[20]  M. Lappe,et al.  Instability of visual error processing for sensorimotor adaptation in schizophrenia , 2017, European Archives of Psychiatry and Clinical Neuroscience.

[21]  M. Goldberg,et al.  The Role of the Lateral Intraparietal Area of the Monkey in the Generation of Saccades and Visuospatial Attention , 2002, Annals of the New York Academy of Sciences.

[22]  Bart Krekelberg,et al.  Postsaccadic visual references generate presaccadic compression of space , 2000, Nature.

[23]  J. Izawa,et al.  The cerebro-cerebellum: Could it be loci of forward models? , 2016, Neuroscience Research.

[24]  Wilsaan M Joiner,et al.  Quantifying the spatial extent of the corollary discharge benefit to transsaccadic visual perception. , 2016, Journal of neurophysiology.

[25]  Wilsaan M. Joiner,et al.  Adaptive Control of Saccades via Internal Feedback , 2008, The Journal of Neuroscience.

[26]  Maria Concetta Morrone,et al.  Spatiotopic neural representations develop slowly across saccades , 2013, Current Biology.

[27]  Masahiko Fujita,et al.  Selective and delay adaptation of human saccades. , 2002, Brain research. Cognitive brain research.

[28]  J. Müsseler,et al.  Perceptual judgment and saccadic behavior in a spatial distortion with briefly presented stimuli. , 2010, Advances in cognitive psychology.

[29]  R. Wurtz,et al.  A circuit for saccadic suppression in the primate brain. , 2017, Journal of neurophysiology.

[30]  E Zimmermann,et al.  Perception during double-step saccades , 2018, Scientific Reports.

[31]  L. Matin,et al.  The influence of saccade length on the saccadic suppression of displacement detection , 1990, Perception & psychophysics.

[32]  W. Becker The neurobiology of saccadic eye movements. Metrics. , 1989, Reviews of oculomotor research.

[33]  Robert H. Wurtz,et al.  Influence of the thalamus on spatial visual processing in frontal cortex , 2006, Nature.

[34]  David A. Robinson,et al.  Models of the saccadic eye movement control system , 1973, Kybernetik.

[35]  D. B. Henson,et al.  Investigation into corrective saccadic eye movements for refixation amplitudes of 10 degrees and below , 1979, Vision Research.

[36]  C. Harris,et al.  Does saccadic undershoot minimize saccadic flight-time? A Monte-Carlo study , 1995, Vision Research.

[37]  J Douglas Crawford,et al.  TMS over human frontal eye fields disrupts trans-saccadic memory of multiple objects. , 2010, Cerebral cortex.

[38]  Konrad Paul Kording,et al.  The dynamics of memory as a consequence of optimal adaptation to a changing body , 2007, Nature Neuroscience.

[39]  Clayton E Curtis,et al.  Obligatory adaptation of saccade gains. , 2008, Journal of neurophysiology.

[40]  Daniel M Wolpert,et al.  Computational principles of sensorimotor control that minimize uncertainty and variability , 2007, The Journal of physiology.

[41]  Martin Rolfs,et al.  Failure to use corollary discharge to remap visual target locations is associated with psychotic symptom severity in schizophrenia. , 2015, Journal of neurophysiology.

[42]  R. Wurtz,et al.  What the brain stem tells the frontal cortex. I. Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus. , 2004, Journal of neurophysiology.

[43]  R. Shadmehr,et al.  Cerebellar Contributions to Adaptive Control of Saccades in Humans , 2009, The Journal of Neuroscience.

[44]  Clayton E. Curtis,et al.  Secondary adaptation of memory-guided saccades , 2010, Experimental Brain Research.

[45]  R. Shadmehr,et al.  Spontaneous recovery of motor memory during saccade adaptation. , 2008, Journal of neurophysiology.

[46]  S. Petersen,et al.  Responses of pulvinar neurons to real and self-induced stimulus movement , 1985, Brain Research.

[47]  Jan Drewes,et al.  Smaller Is Better: Drift in Gaze Measurements due to Pupil Dynamics , 2014, PloS one.

[48]  R. Sperry Neural basis of the spontaneous optokinetic response produced by visual inversion. , 1950, Journal of comparative and physiological psychology.

[49]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[50]  Marc A Sommer,et al.  The frontal eye field as a prediction map. , 2008, Progress in brain research.

[51]  Edward G Freedman,et al.  The Locus of Motor Activity in the Superior Colliculus of the Rhesus Monkey Is Unaltered during Saccadic Adaptation , 2010, The Journal of Neuroscience.

[52]  R. Wurtz,et al.  Organization of Corollary Discharge Neurons in Monkey Medial Dorsal Thalamus , 2020, The Journal of Neuroscience.

[53]  Scott E. Bevans,et al.  Effect of visual error size on saccade adaptation in monkey. , 2003, Journal of neurophysiology.

[54]  Peter Thier,et al.  False perception of motion in a patient who cannot compensate for eye movements , 1997, Nature.

[55]  D. Pélisson,et al.  Sensorimotor adaptation of saccadic eye movements , 2010, Neuroscience & Biobehavioral Reviews.

[56]  A. Fuchs,et al.  Extraocular muscle afferents to the cerebellum of the cat , 1969, The Journal of physiology.

[57]  David W. Franklin,et al.  Computational Mechanisms of Sensorimotor Control , 2011, Neuron.

[58]  Randolph Blake,et al.  Pupil size dynamics during fixation impact the accuracy and precision of video-based gaze estimation , 2016, Vision Research.

[59]  Bruce Bridgeman,et al.  Failure to detect displacement of the visual world during saccadic eye movements , 1975, Vision Research.

[60]  Heiner Deubel,et al.  Relative mislocalization of briefly presented stimuli in the retinal periphery , 1999, Perception & psychophysics.

[61]  Reza Shadmehr,et al.  Encoding of error and learning to correct that error by the Purkinje cells of the cerebellum , 2018, Nature Neuroscience.

[62]  I Feinberg,et al.  Efference copy and corollary discharge: implications for thinking and its disorders. , 1978, Schizophrenia bulletin.

[63]  Markus Lappe,et al.  Motor space structures perceptual space: Evidence from human saccadic adaptation , 2007, Brain Research.

[64]  F. Bremmer,et al.  Saccadic suppression of displacement in face of saccade adaptation , 2011, Vision Research.

[65]  L. Stark,et al.  Ocular proprioception and efference copy in registering visual direction , 1991, Vision Research.

[66]  Mathieu Koppen,et al.  Causal inference for spatial constancy across whole body motion. , 2019, Journal of neurophysiology.

[67]  A. Fuchs,et al.  Characteristics of saccadic gain adaptation in rhesus macaques. , 1997, Journal of neurophysiology.

[68]  J. Ford,et al.  Neurophysiological evidence of corollary discharge dysfunction in schizophrenia. , 2001, The American journal of psychiatry.

[69]  J. Douglas Crawford,et al.  Optimal transsaccadic integration explains distorted spatial perception , 2003, Nature.

[70]  Markus Lappe,et al.  Eye Position Effects in Oculomotor Plasticity and Visual Localization , 2011, The Journal of Neuroscience.

[71]  Arnold Ziesche,et al.  Brain circuits underlying visual stability across eye movements—converging evidence for a neuro-computational model of area LIP , 2014, Front. Comput. Neurosci..

[72]  Mingsha Zhang,et al.  The proprioceptive representation of eye position in monkey primary somatosensory cortex , 2007, Nature Neuroscience.

[73]  R. Wurtz,et al.  Brain circuits for the internal monitoring of movements. , 2008, Annual review of neuroscience.

[74]  M. Goldberg,et al.  Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. , 1995, Journal of neurophysiology.

[75]  F. C. Volkmann,et al.  Time course of visual inhibition during voluntary saccades. , 1968, Journal of the Optical Society of America.

[76]  Reza Shadmehr,et al.  Changes in Control of Saccades during Gain Adaptation , 2008, The Journal of Neuroscience.

[77]  E. J. Tehovnik,et al.  Eye Movements Modulate Visual Receptive Fields of V4 Neurons , 2001, Neuron.

[78]  A. Opstal,et al.  Transfer of short-term adaptation in human saccadic eye movements , 2004, Experimental Brain Research.

[79]  Harold E. Bedell,et al.  Changes in oculocentric visual direction induced by the recalibration of saccades , 1988, Vision Research.

[80]  C. Curtis,et al.  Cortico-cerebellar network involved in saccade adaptation. , 2018, Journal of neurophysiology.

[81]  Kae Nakamura,et al.  Updating of the visual representation in monkey striate and extrastriate cortex during saccades , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[82]  Markus Lappe,et al.  Visual Space Constructed by Saccade Motor Maps , 2016, Front. Hum. Neurosci..

[83]  M. Sommer,et al.  Frontal Eye Field Neurons with Spatial Representations Predicted by Their Subcortical Input , 2009, The Journal of Neuroscience.

[84]  Markus Lappe,et al.  Post-saccadic eye position contributes to oculomotor error estimation in saccadic adaptation. , 2019, Journal of neurophysiology.

[85]  Wilsaan M. Joiner,et al.  Neuronal mechanisms for visual stability: progress and problems , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[86]  Patrick Cavanagh,et al.  Spatiotopic apparent motion reveals local variations in space constancy. , 2011, Journal of vision.

[87]  A. Opstal,et al.  Monkey Superior Colliculus Activity During Short-Term Saccadic Adaptation , 1997, Brain Research Bulletin.

[88]  A. Fuchs,et al.  Further properties of the human saccadic system: eye movements and correction saccades with and without visual fixation points. , 1969, Vision research.

[89]  M. Sommer,et al.  Corollary discharge circuits in the primate brain , 2008, Current Opinion in Neurobiology.

[90]  R. Ivry,et al.  Cerebellar and prefrontal cortex contributions to adaptation, strategies, and reinforcement learning. , 2014, Progress in brain research.

[91]  P. Thier,et al.  Saccadic Dysmetria and Adaptation after Lesions of the Cerebellar Cortex , 1999, The Journal of Neuroscience.

[92]  A. Fuchs,et al.  Saccadic gain modification: visual error drives motor adaptation. , 1998, Journal of neurophysiology.

[93]  Frank Bremmer,et al.  Dynamics of Eye-Position Signals in the Dorsal Visual System , 2012, Current Biology.

[94]  I Daum,et al.  The role of the human thalamus in processing corollary discharge. , 2005, Brain : a journal of neurology.

[95]  Denis Pélisson,et al.  Functional activation of the cerebral cortex related to sensorimotor adaptation of reactive and voluntary saccades , 2012, NeuroImage.

[96]  Ned Jenkinson,et al.  The Role of the Posterior Cerebellum in Saccadic Adaptation: A Transcranial Direct Current Stimulation Study , 2015, The Journal of Neuroscience.

[97]  Tatsuya Kimura,et al.  Cerebellar complex spikes encode both destinations and errors in arm movements , 1998, Nature.

[98]  Maria Concetta Morrone,et al.  Constructing Stable Spatial Maps of the Word , 2012, Perception.

[99]  O. White,et al.  Force field adaptation does not alter space representation , 2018, Scientific Reports.

[100]  Denis Pélisson,et al.  Behavioral evidence of separate adaptation mechanisms controlling saccade amplitude lengthening and shortening. , 2009, Journal of neurophysiology.

[101]  J. Ford,et al.  The function and failure of sensory predictions , 2018, Annals of the New York Academy of Sciences.

[102]  Christian Bellebaum,et al.  Cortical processing of saccade‐related efference copy signals in patients with cerebellar lesion , 2013, The European journal of neuroscience.

[103]  R. Wurtz Neuronal mechanisms of visual stability , 2008, Vision Research.

[104]  Stephan Heckers,et al.  Disrupted Saccadic Corollary Discharge in Schizophrenia , 2015, The Journal of Neuroscience.

[105]  P. Thier,et al.  Role of the Vermal Cerebellum in Visually Guided Eye Movements and Visual Motion Perception. , 2019, Annual review of vision science.

[106]  Reza Shadmehr,et al.  Estimating properties of the fast and slow adaptive processes during sensorimotor adaptation. , 2018, Journal of neurophysiology.

[107]  E Zimmermann,et al.  Impairment of saccade adaptation in a patient with a focal thalamic lesion. , 2015, Journal of neurophysiology.

[108]  Konrad Paul Kording,et al.  Relevance of error: what drives motor adaptation? , 2009, Journal of neurophysiology.

[109]  O. Grüsser,et al.  Afterimage movement during saccades in the dark , 1987, Vision Research.

[110]  R. Wurtz,et al.  What the brain stem tells the frontal cortex. II. Role of the SC-MD-FEF pathway in corollary discharge. , 2004, Journal of neurophysiology.

[111]  H Deubel,et al.  Adaptive gain control of saccadic eye movements. , 1986, Human neurobiology.

[112]  M. Banks,et al.  How does saccade adaptation affect visual perception? , 2008, Journal of vision.

[113]  F. Hamker,et al.  A Computational Model for the Influence of Corollary Discharge and Proprioception on the Perisaccadic Mislocalization of Briefly Presented Stimuli in Complete Darkness , 2011, The Journal of Neuroscience.

[114]  T. Collins Trade-off between spatiotopy and saccadic plasticity. , 2014, Journal of vision.

[115]  Vaibhav A. Diwadkar,et al.  Oculomotor Prediction: A Window into the Psychotic Mind , 2017, Trends in Cognitive Sciences.

[116]  Katharina Havermann,et al.  The influence of the consistency of postsaccadic visual errors on saccadic adaptation. , 2010, Journal of neurophysiology.

[117]  R. Wurtz,et al.  The Neurobiology of Saccadic Eye Movements , 1989 .

[118]  E Kowler,et al.  Illusory shifts in visual direction accompany adaptation of saccadic eye movements , 1999, Nature.

[119]  R. Wurtz,et al.  A Pathway in Primate Brain for Internal Monitoring of Movements , 2002, Science.

[120]  Robert H Wurtz,et al.  Saccadic Corollary Discharge Underlies Stable Visual Perception , 2016, The Journal of Neuroscience.

[121]  Christopher T. Noto,et al.  Characteristics of simian adaptation fields produced by behavioral changes in saccade size and direction. , 1999, Journal of neurophysiology.

[122]  Mark Shelhamer,et al.  Sensorimotor adaptation error signals are derived from realistic predictions of movement outcomes. , 2011, Journal of neurophysiology.

[123]  Robert L. White,et al.  Subthreshold microstimulation in frontal eye fields updates spatial memories , 2007, Experimental Brain Research.

[124]  C. Ploner,et al.  Human thalamus contributes to perceptual stability across eye movements , 2009, Proceedings of the National Academy of Sciences.

[125]  M. Lappe,et al.  Motor signals in visual localization. , 2010, Journal of vision.

[126]  S. C. Mclaughlin Parametric adjustment in saccadic eye movements , 1967 .

[127]  H. Deubel ADAPTIVITY OF GAIN AND DIRECTION IN OBLIQUE SACCADES1 , 1987 .

[128]  M. Lappe,et al.  Effects of saccadic adaptation on visual localization before and during saccades , 2008, Experimental Brain Research.

[129]  David C. Burr,et al.  Compression of visual space before saccades , 1997, Nature.

[130]  C. Colby,et al.  Trans-saccadic perception , 2008, Trends in Cognitive Sciences.

[131]  Markus Lappe,et al.  Effect of saccadic adaptation on localization of visual targets. , 2005, Journal of neurophysiology.

[132]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[133]  Kaoru Yoshida,et al.  Memory of Learning Facilitates Saccadic Adaptation in the Monkey , 2004, The Journal of Neuroscience.

[134]  Markus Lappe,et al.  Mislocalization of Flashed and Stationary Visual Stimuli after Adaptation of Reactive and Scanning Saccades , 2009, The Journal of Neuroscience.

[135]  Florian Ostendorf,et al.  Theta-burst stimulation over human frontal cortex distorts perceptual stability across eye movements. , 2012, Cerebral cortex.

[136]  D. Wolpert,et al.  Principles of sensorimotor learning , 2011, Nature Reviews Neuroscience.

[137]  C. Frith,et al.  The positive and negative symptoms of schizophrenia reflect impairments in the perception and initiation of action , 1987, Psychological Medicine.

[138]  Peter Thier,et al.  Modification of the filehne illusion by conditioning visual stimuli , 1996, Vision Research.

[139]  R. Shadmehr,et al.  Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning , 2006, PLoS biology.

[140]  M. Krebs,et al.  Correlates between neurological soft signs and saccadic parameters in schizophrenia , 2009, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[141]  M. Goldberg,et al.  Effect of short-term saccadic adaptation on saccades evoked by electrical stimulation in the primate superior colliculus. , 2002, Journal of neurophysiology.

[142]  K. Hoffmann,et al.  Neural Mechanisms of Saccadic Suppression , 2002, Science.

[143]  T Anstis,et al.  Saccadic plasticity: parametric adaptive control by retinal feedback. , 1981, Journal of experimental psychology. Human perception and performance.

[144]  B. Bridgeman,et al.  Postsaccadic target blanking prevents saccadic suppression of image displacement , 1996, Vision Research.

[145]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[146]  Heiner Deubel,et al.  Different effects of eyelid blinks and target blanking on saccadic suppression of displacement , 2004, Perception & psychophysics.

[147]  Michael E Goldberg,et al.  The lateral intraparietal area codes the location of saccade targets and not the dimension of the saccades that will be made to acquire them. , 2013, Journal of neurophysiology.

[148]  Heiner Deubel,et al.  Post-saccadic location judgments reveal remapping of saccade targets to non-foveal locations. , 2009, Journal of vision.

[149]  Yoshiko Kojima,et al.  Encoding of action by the Purkinje cells of the cerebellum , 2015, Nature.

[150]  P. Strick,et al.  Basal ganglia and cerebellar loops: motor and cognitive circuits , 2000, Brain Research Reviews.

[151]  Eileen Kowler,et al.  The control of saccadic adaptation: implications for the scanning of natural visual scenes , 2000, Vision Research.

[152]  P. Thier,et al.  Reduced saccadic resilience and impaired saccadic adaptation due to cerebellar disease , 2007, The European journal of neuroscience.

[153]  Patrick Cavanagh,et al.  Global saccadic adaptation , 2010, Vision Research.