Predictive coding of visual object position ahead of moving objects revealed by time-resolved EEG decoding

&NA; Due to the delays inherent in neuronal transmission, our awareness of sensory events necessarily lags behind the occurrence of those events in the world. If the visual system did not compensate for these delays, we would consistently mislocalize moving objects behind their actual position. Anticipatory mechanisms that might compensate for these delays have been reported in animals, and such mechanisms have also been hypothesized to underlie perceptual effects in humans such as the Flash‐Lag Effect. However, to date no direct physiological evidence for anticipatory mechanisms has been found in humans. Here, we apply multivariate pattern classification to time‐resolved EEG data to investigate anticipatory coding of object position in humans. By comparing the time‐course of neural position representation for objects in both random and predictable apparent motion, we isolated anticipatory mechanisms that could compensate for neural delays when motion trajectories were predictable. As well as revealing an early neural position representation (lag 80–90 ms) that was unaffected by the predictability of the object's trajectory, we demonstrate a second neural position representation at 140–150 ms that was distinct from the first, and that was pre‐activated ahead of the moving object when it moved on a predictable trajectory. The latency advantage for predictable motion was approximately 16 ± 2 ms. To our knowledge, this provides the first direct experimental neurophysiological evidence of anticipatory coding in human vision, revealing the time‐course of predictive mechanisms without using a spatial proxy for time. The results are numerically consistent with earlier animal work, and suggest that current models of spatial predictive coding in visual cortex can be effectively extended into the temporal domain. HighlightsEEG revealed preactivation of neural representations ahead of a moving object.There is a 16 ms latency advantage for predictable vs unpredictable trajectories.This predictive mechanism can partly compensate for neural processing delays.This constitutes the first direct neural evidence of predictive coding in humans.

[1]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[2]  J. G. Axford,et al.  Source locations of pattern-specific components of human visual evoked potentials. II. Component of extrastriate cortical origin , 2004, Experimental Brain Research.

[3]  E. Wagenmakers,et al.  An Introduction to Model-Based Cognitive Neuroscience , 2015, Springer New York.

[4]  Michael W. Spratling A review of predictive coding algorithms , 2017, Brain and Cognition.

[5]  Michael W. Spratling,et al.  Encyclopedia of Computational Neuroscience , 2013 .

[6]  Hinze Hogendoorn,et al.  Interpolation and extrapolation on the path of apparent motion , 2008, Vision Research.

[7]  Laurent Goffart,et al.  Does the Brain Extrapolate the Position of a Transient Moving Target? , 2015, The Journal of Neuroscience.

[8]  David Whitney,et al.  Motion distorts visual space: shifting the perceived position of remote stationary objects , 2000, Nature Neuroscience.

[9]  N. Ramsey,et al.  Predictive coding for motion stimuli in human early visual cortex , 2016, Brain Structure and Function.

[10]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[11]  Gregor Schöner,et al.  Shorter latencies for motion trajectories than for flashes in population responses of cat primary visual cortex , 2004, The Journal of physiology.

[12]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[13]  John H. R. Maunsell,et al.  Visual response latencies in striate cortex of the macaque monkey. , 1992, Journal of neurophysiology.

[14]  R. Grush Internal models and the construction of time: generalizing from state estimation to trajectory estimation to address temporal features of perception, including temporal illusions , 2005, Journal of neural engineering.

[15]  Terrence J Sejnowski,et al.  Motion signals bias localization judgments: a unified explanation for the flash-lag, flash-drag, flash-jump, and Frohlich illusions. , 2007, Journal of vision.

[16]  Peter Elias,et al.  Predictive coding-I , 1955, IRE Trans. Inf. Theory.

[17]  Kathleen Akins,et al.  Cognition and the Brain: The Philosophy and Neuroscience Movement , 2005 .

[18]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[19]  D. Dennett,et al.  The Nature of Consciousness , 2006 .

[20]  D. Dennett,et al.  Time and the Observer: the Where and When of Consciousness in the Brain Endnote 1 , 1995 .

[21]  Shinsuke Shimojo,et al.  Postdiction: its implications on visual awareness, hindsight, and sense of agency , 2014, Front. Psychol..

[22]  George L. Gerstein,et al.  Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex , 1994, Nature.

[23]  D. Jeffreys,et al.  Source locations of pattern-specific components of human visual evoked potentials. I. Component of striate cortical origin , 2004, Experimental Brain Research.

[24]  T J Sejnowski,et al.  Motion integration and postdiction in visual awareness. , 2000, Science.

[25]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[26]  Floris P. de Lange,et al.  Predictive Coding in Sensory Cortex , 2015 .

[27]  Patrick Cavanagh,et al.  Motion Extrapolation for Eye Movements Predicts Perceived Motion-Induced Position Shifts , 2018, The Journal of Neuroscience.

[28]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[29]  Patrick Cavanagh,et al.  Strikingly rapid neural basis of motion-induced position shifts revealed by high temporal-resolution EEG pattern classification , 2015, Vision Research.

[30]  Rick Grush,et al.  Cognition and the Brain: Brain Time and Phenomenological Time , 2005 .

[31]  Guillaume S. Masson,et al.  The Flash-Lag Effect as a Motion-Based Predictive Shift , 2017, PLoS Comput. Biol..

[32]  Jeroen B. J. Smeets,et al.  Visuomotor delays when hitting running spiders. , 1998 .

[33]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[34]  Romi Nijhawan,et al.  Motion extrapolation in catching , 1994, Nature.

[35]  Michael J. Berry,et al.  Anticipation of moving stimuli by the retina , 1999, Nature.

[36]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[37]  Patrick Cavanagh,et al.  The flash grab effect , 2012, Vision Research.

[38]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[39]  Y Dan,et al.  Motion-Induced Perceptual Extrapolation of Blurred Visual Targets , 2001, The Journal of Neuroscience.

[40]  Rick Grush,et al.  Temporal representation and dynamics , 2008 .

[41]  Jérome Fleuriet,et al.  Saccadic Interception of a Moving Visual Target after a Spatiotemporal Perturbation , 2012, The Journal of Neuroscience.

[42]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[43]  P Cavanagh,et al.  The Position of Moving Objects , 2000, Science.

[44]  M. Meister,et al.  Dynamic predictive coding by the retina , 2005, Nature.

[45]  David Whitney,et al.  Motion-Dependent Representation of Space in Area MT+ , 2013, Neuron.

[46]  Rajesh P. N. Rao,et al.  Predictive Coding , 2019, A Blueprint for the Hard Problem of Consciousness.

[47]  Michael J. Berry,et al.  Synchronized Firing among Retinal Ganglion Cells Signals Motion Reversal , 2007, Neuron.

[48]  Hinze Hogendoorn,et al.  Predictive Coding with Neural Transmission Delays: A Real-Time Temporal Alignment Hypothesis , 2018, eNeuro.