Working Title: Motion Extrapolation in Visual Processing: Lessons from 25 Years of Flash-Lag Debate

Because of the delays inherent in neural transmission, the brain needs time to process incoming visual information. If these delays were not somehow compensated, we would consistently mislocalize moving objects behind their physical positions. Twenty-five years ago, Nijhawan used a perceptual illusion he called the flash-lag effect (FLE) to argue that the brain's visual system solves this computational challenge by extrapolating the position of moving objects (Nijhawan, 1994). Although motion extrapolation had been proposed a decade earlier (e.g., Finke et al., 1986), the proposal that it caused the FLE and functioned to compensate for computational delays was hotly debated in the years that followed, with several alternative interpretations put forth to explain the effect. Here, I argue, 25 years later, that evidence from behavioral, computational, and particularly recent functional neuroimaging studies converges to support the existence of motion extrapolation mechanisms in the visual system, as well as their causal involvement in the FLE. First, findings that were initially argued to challenge the motion extrapolation model of the FLE have since been explained, and those explanations have been tested and corroborated by more recent findings. Second, motion extrapolation explains the spatial shifts observed in several FLE conditions that cannot be explained by alternative (temporal) models of the FLE. Finally, neural mechanisms that actually perform motion extrapolation have been identified at multiple levels of the visual system, in multiple species, and with multiple different methods. I outline key questions that remain, and discuss possible directions for future research.

[1]  Gerrit W. Maus,et al.  Motion Extrapolation Into the Blind Spot , 2008, Psychological science.

[2]  Timothy L. Hubbard,et al.  AUDITORY REPRESENTATIONAL MOMENTUM: SURFACE FORM, DIRECTION, AND VELOCITY EFFECTS , 1995 .

[3]  Elle van Heusden,et al.  Predictive coding of visual motion in both monocular and binocular human visual processing. , 2019, Journal of vision.

[4]  F. Chavane,et al.  Imaging cortical correlates of illusion in early visual cortex , 2004, Nature.

[5]  Shinsuke Shimojo,et al.  Changing objects lead briefly flashed ones , 2000, Nature Neuroscience.

[6]  Shinichi Kita,et al.  Our own faces: perceiving fluctuating asymmetry in the highly familiar objects , 2010 .

[7]  Stanley A. Klein,et al.  Extrapolation or attention shift? , 1995, Nature.

[8]  Wolfram Erlhagen,et al.  Space and Time in Perception and Action: Bridging the gap: a model of common neural mechanisms underlying the Fröhlich effect, the flash-lag effect, and the representational momentum effect , 2010 .

[9]  Gopathy Purushothaman,et al.  Moving ahead through differential visual latency , 1998, Nature.

[10]  Shinsuke Shimojo,et al.  Compensation of neural delays in visual‐motor behaviour: No evidence for shorter afferent delays for visual motion , 2004 .

[11]  Kuno Kirschfeld,et al.  Analogous Mechanisms Compensate for Neural Delays in the Sensory and the Motor Pathways Evidence from Motor Flash-Lag , 2003, Current Biology.

[12]  David Whitney,et al.  Motion-Dependent Representation of Space in Area MT+ , 2013, Neuron.

[13]  Markus Lappe,et al.  Temporal recruitment along the trajectory of moving objects and the perception of position , 1999, Vision Research.

[14]  Romi Nijhawan,et al.  Forward displacements of fading objects in motion: The role of transient signals in perceiving position , 2006, Vision Research.

[15]  Trevor J. Hine,et al.  Attention ‘capture’ by the flash-lag flash , 2006, Vision Research.

[16]  R. VanRullen,et al.  Conscious updating is a rhythmic process , 2012, Proceedings of the National Academy of Sciences.

[17]  Géry d'Ydewalle,et al.  Representational momentum and event course anticipation in the perception of implied periodical motions. , 1991, Journal of experimental psychology. Learning, memory, and cognition.

[18]  Heiko Hecht,et al.  The representational dynamics of remembered projectile locations. , 2013, Journal of experimental psychology. Human perception and performance.

[19]  E. Brenner,et al.  Motion extrapolation is not responsible for the flash–lag effect , 2000, Vision Research.

[20]  R. Goebel,et al.  Motion Displaces Population Receptive Fields in the Direction Opposite to Motion , 2019, bioRxiv.

[21]  M. Lappe,et al.  The position of moving objects. , 2000, Perception.

[22]  Patrick Cavanagh,et al.  Motion Extrapolation for Eye Movements Predicts Perceived Motion-Induced Position Shifts , 2018, The Journal of Neuroscience.

[23]  M. Corballis Wandering tales: evolutionary origins of mental time travel and language , 2013, Front. Psychol..

[24]  Zhuanghua Shi,et al.  Motion Extrapolation in the Central Fovea , 2012, PloS one.

[25]  David Whitney,et al.  Temporal facilitation for moving stimuli is independent of changes in direction , 2000, Vision Research.

[26]  M. Chappell,et al.  Turning the corner with the flash–lag illusion , 2014, Vision Research.

[27]  Megumi Hatori,et al.  The role of arrestin In melanopsin function , 2009 .

[28]  Serge O. Dumoulin,et al.  Visual motion transforms visual space representations similarly throughout the human visual hierarchy , 2016, NeuroImage.

[29]  Michael J. Berry,et al.  Anticipation of moving stimuli by the retina , 1999, Nature.

[30]  Zhuanghua Shi,et al.  Motion-induced positional biases in the flash-lag configuration , 2008, Cognitive neuropsychology.

[31]  J. Freyd,et al.  The mental representation of movement when static stimuli are viewed , 1983, Perception & psychophysics.

[32]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[33]  C. Reed,et al.  Conceptual effects on representational momentum. , 1996, Journal of experimental psychology. Human perception and performance.

[34]  G. Sperling,et al.  Episodic theory of the dynamics of spatial attention. , 1995 .

[35]  Patrick Cavanagh,et al.  Strikingly rapid neural basis of motion-induced position shifts revealed by high temporal-resolution EEG pattern classification , 2015, Vision Research.

[36]  Guillaume S. Masson,et al.  The Flash-Lag Effect as a Motion-Based Predictive Shift , 2017, PLoS Comput. Biol..

[37]  Knut Drewing,et al.  The buzz-lag effect , 2016, Experimental Brain Research.

[38]  I. Murakami,et al.  A flash-lag effect in random motion , 2001, Vision Research.

[39]  T J Sejnowski,et al.  Motion integration and postdiction in visual awareness. , 2000, Science.

[40]  Hinze Hogendoorn,et al.  Interpolation and extrapolation on the path of apparent motion , 2008, Vision Research.

[41]  R A Finke,et al.  Implied velocity and acceleration induce transformations of visual memory. , 1986, Journal of experimental psychology. General.

[42]  R. VanRullen Perceptual Cycles , 2016, Trends in Cognitive Sciences.

[43]  P Cavanagh,et al.  The Position of Moving Objects , 2000, Science.

[44]  Dirk Kerzel,et al.  Attention maintains mental extrapolation of target position: irrelevant distractors eliminate forward displacement after implied motion , 2003, Cognition.

[45]  Peter A. White,et al.  Is the perceived present a predictive model of the objective present? , 2018, Visual Cognition.

[46]  Timothy L. Hubbard,et al.  Representational momentum contributes to motion induced mislocalization of stationary objects , 2008 .

[47]  Neil W. Roach,et al.  Report a Forward Prediction of Spatial Pattern , 2016 .

[48]  T. Hubbard Representational momentum and related displacements in spatial memory: A review of the findings , 2005, Psychonomic bulletin & review.

[49]  T. Hubbard,et al.  Environmental invariants in the representation of motion: Implied dynamics and representational momentum, gravity, friction, and centripetal force , 1995, Psychonomic bulletin & review.

[50]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[51]  Hinze Hogendoorn,et al.  When predictions fail: Correction for extrapolation in the flash-grab effect. , 2019, Journal of vision.

[52]  Floris P de Lange,et al.  Time-compressed preplay of anticipated events in human primary visual cortex , 2017, Nature Communications.

[53]  David Whitney,et al.  Motion distorts visual space: shifting the perceived position of remote stationary objects , 2000, Nature Neuroscience.

[54]  Derek H. Arnold,et al.  Simple differential latencies modulate, but do not cause the flash-lag effect. , 2009, Journal of vision.

[55]  Romi Nijhawan,et al.  Motion extrapolation in catching , 1994, Nature.

[56]  Alex O. Holcombe,et al.  A delay in sampling information from temporally autocorrelated visual stimuli , 2019, Nature Communications.

[57]  Lihan Chen,et al.  Tactile flash lag effect: Taps with changing intensities lead briefly flashed taps , 2013, 2013 World Haptics Conference (WHC).

[58]  D. Kerzel Eye movements and visible persistence explain the mislocalization of the final position of a moving target , 2000, Vision Research.

[59]  S. McKee,et al.  Spatial configurations for visual hyperacuity , 1977, Vision Research.

[60]  Hinze Hogendoorn,et al.  Predictions drive neural representations of visual events ahead of incoming sensory information , 2020, Proceedings of the National Academy of Sciences.

[61]  T. Hubbard Do the flash-lag effect and representational momentum involve similar extrapolations? , 2013, Front. Psychol..

[62]  Hinze Hogendoorn,et al.  Predictive Coding with Neural Transmission Delays: A Real-Time Temporal Alignment Hypothesis , 2018, eNeuro.

[63]  Derek H. Arnold,et al.  Binocular rivalry: spreading dominance through complex images. , 2009, Journal of vision.

[64]  S. Klein,et al.  Evidence for an Attentional Component of the Perceptual Misalignment between Moving and Flashing Stimuli , 2002, Perception.

[65]  C. Koch,et al.  Is perception discrete or continuous? , 2003, Trends in Cognitive Sciences.

[66]  D. Mackay Perceptual Stability of a Stroboscopically Lit Visual Field containing Self-Luminous Objects , 1958, Nature.

[67]  Dirk Kerzel,et al.  Computational Theory and Cognition in Representational Momentum and Related Types of Displacement: A reply to Kerzel , 2006 .

[68]  Katsumi Watanabe,et al.  Object motion continuity and the flash-lag effect , 2013, Vision Research.

[69]  I. Murakami,et al.  Latency difference, not spatial extrapolation , 1998, Nature Neuroscience.

[70]  R. VanRullen,et al.  Visual Entrainment at 10 Hz Causes Periodic Modulation of the Flash Lag Illusion , 2019, bioRxiv.

[71]  K. Schneider The Flash-Lag, Fröhlich and Related Motion Illusions Are Natural Consequences of Discrete Sampling in the Visual System , 2018, Front. Psychol..

[72]  M. Chappell,et al.  Dividing attention in the flash-lag illusion , 2007, Vision Research.

[73]  Anthony N. Burkitt,et al.  Predictive coding of visual object position ahead of moving objects revealed by time-resolved EEG decoding , 2018, NeuroImage.

[74]  R Nijhawan,et al.  The Role of Attention in Motion Extrapolation: Are Moving Objects ‘Corrected’ or Flashed Objects Attentionally Delayed? , 2000, Perception.

[75]  Guillaume S Masson,et al.  Anticipatory responses along motion trajectories in awake monkey area V1 , 2020, bioRxiv.

[76]  N. Ramsey,et al.  Predictive coding for motion stimuli in human early visual cortex , 2016, Brain Structure and Function.

[77]  W. Metzger,et al.  Versuch einer gemeinsamen Theorie der Phänomene Fröhlichs und Hazelhoffs und Kritik ihrer Verfahren zur Messung der Empfindungszeit , 1932 .

[78]  T. Hubbard,et al.  Representational momentum and anisotropies in nearby visual space , 2018, Attention, perception & psychophysics.

[79]  Amiram Grinvald,et al.  Stimulus Localization by Neuronal Populations in Early Visual Cortex: Linking Functional Architecture to Perception , 2009 .

[80]  M. Lappe,et al.  Neuronal latencies and the position of moving objects , 2001, Trends in Neurosciences.

[81]  D. Burr,et al.  The “Flash-Lag” Effect Occurs in Audition and Cross-Modally , 2003, Current Biology.

[82]  Romi Nijhawan,et al.  Visual prediction: Psychophysics and neurophysiology of compensation for time delays , 2008, Behavioral and Brain Sciences.

[83]  Patrick Cavanagh,et al.  Default perception of high-speed motion , 2013, Proceedings of the National Academy of Sciences.

[84]  Shinsuke Shimojo,et al.  Shifts in perceived position of flashed stimuli by illusory object motion , 2002, Vision Research.

[85]  Patrick Cavanagh,et al.  The flash grab effect , 2012, Vision Research.

[86]  R. Nijhawan,et al.  Neural delays, visual motion and the flash-lag effect , 2002, Trends in Cognitive Sciences.

[87]  Marcus Vinícius C Baldo,et al.  The Modulation of the Flash-Lag Effect by Voluntary Attention , 2004, Perception.

[88]  Michael J. Berry,et al.  Synchronized Firing among Retinal Ganglion Cells Signals Motion Reversal , 2007, Neuron.

[89]  M. Ichikawa,et al.  Manual control of the visual stimulus reduces the flash-lag effect , 2006, Vision Research.

[90]  K. Gegenfurtner,et al.  Neuronal Processing Delays Are Compensated in the Sensorimotor Branch of the Visual System , 2003, Current Biology.

[91]  S. Kastner,et al.  A Rhythmic Theory of Attention , 2019, Trends in Cognitive Sciences.

[92]  Gregor Schöner,et al.  Shorter latencies for motion trajectories than for flashes in population responses of cat primary visual cortex , 2004, The Journal of physiology.

[93]  Alexander S. Ecker,et al.  Faster processing of moving compared to flashed bars in awake macaque V1 provides a neural correlate of the flash lag illusion , 2015, bioRxiv.

[94]  Timothy L Hubbard,et al.  The flash-lag effect and related mislocalizations: findings, properties, and theories. , 2014, Psychological bulletin.

[95]  Terrence J Sejnowski,et al.  Motion signals bias localization judgments: a unified explanation for the flash-lag, flash-drag, flash-jump, and Frohlich illusions. , 2007, Journal of vision.

[96]  Makoto Ichikawa,et al.  Reduction of the flash-lag effect in terms of active observation , 2010, Attention, perception & psychophysics.

[97]  R Nijhawan,et al.  The Flash-Lag Phenomenon: Object Motion and Eye Movements , 2001, Perception.

[98]  Géry d'Ydewalle,et al.  Representational momentum and event course anticipation in the perception of implied periodical motions , 1991 .