Laws of concatenated perception: Vision goes for novelty, decisions for perseverance

Every instant of perception depends on a cascade of brain processes calibrated to the history of sensory and decisional events. In the present work, we show that human visual perception is constantly shaped by two contrasting forces exerted by sensory adaptation and past decisions. In a series of experiments, we used multilevel modeling and cross-validation approaches to investigate the impact of previous stimuli and decisions on behavioral reports during adjustment and forced-choice tasks. Our results revealed that each perceptual report is permeated by opposite biases from a hierarchy of serially dependent processes: Low-level adaptation repels perception away from previous stimuli, whereas decisional traces attract perceptual reports toward the recent past. In this hierarchy of serial dependence, “continuity fields” arise from the inertia of decisional templates and not from low-level sensory processes. This finding is consistent with a Two-process model of serial dependence in which the persistence of readout weights in a decision unit compensates for sensory adaptation, leading to attractive biases in sequential perception. We propose a unified account of serial dependence in which functionally distinct mechanisms, operating at different stages, promote the differentiation and integration of visual information over time.

[1]  R Sekuler,et al.  Letter: Tilt aftereffect following very brief exposures. , 1974, Vision research.

[2]  C. Law,et al.  Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area , 2008, Nature Neuroscience.

[3]  M. Shadlen,et al.  Decision Making as a Window on Cognition , 2013, Neuron.

[4]  Timothy F. Brady,et al.  Hierarchical Encoding in Visual Working Memory , 2010, Psychological science.

[5]  A. V. Smirnov,et al.  Visualization of arc and plasma flow patterns for advanced material processing , 2015, J. Vis..

[6]  W. S. Verplanck,et al.  Nonindependence of successive responses in measurements of the visual threshold. , 1952, Journal of experimental psychology.

[7]  Kristjan Kalm,et al.  Visual recency bias is explained by a mixture model of internal representations. , 2018, Journal of vision.

[8]  Xiao-Jing Wang Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[9]  N. Prins Psychophysics: A Practical Introduction , 2009 .

[10]  Debashis Kushary,et al.  Bootstrap Methods and Their Application , 2000, Technometrics.

[11]  D. Burr,et al.  Visual aftereffects , 2009, Current Biology.

[12]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[13]  Isabelle Mareschal,et al.  Effects of contrast and size on orientation discrimination , 2004, Vision Research.

[14]  Floris P de Lange,et al.  Serial Dependence in Perceptual Decisions Is Reflected in Activity Patterns in Primary Visual Cortex , 2016, The Journal of Neuroscience.

[15]  John H. R. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[16]  A. Aron,et al.  Stimulus devaluation induced by stopping action. , 2014, Journal of experimental psychology. General.

[17]  Mark D'Esposito,et al.  Confidence Leak in Perceptual Decision Making , 2015, Psychological science.

[18]  D. Whitney,et al.  Serial dependence in position occurs at the time of perception , 2018, Psychonomic Bulletin & Review.

[19]  David Whitney,et al.  The perceived stability of scenes: serial dependence in ensemble representations , 2017, Scientific Reports.

[20]  Joonkoo Park,et al.  Attractive Serial Dependence in the Absence of an Explicit Task , 2018, Psychological science.

[21]  David Pascucci,et al.  Immediate Effect of Internal Reward on Visual Adaptation , 2013, Psychological science.

[22]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[23]  D. Whitney,et al.  Serial dependence in visual perception , 2011, Nature Neuroscience.

[24]  P. Mamassian,et al.  Predictive Properties of Visual Adaptation , 2012, Current Biology.

[25]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[26]  Tobias Teichert,et al.  The importance of decision onset. , 2016, Journal of neurophysiology.

[27]  Mary Hayhoe,et al.  Adaptation Mechanisms in Color and Brightness , 1991 .

[28]  A. Kristofferson,et al.  Response delays and the timing of discrete motor responses , 1973 .

[29]  Xiao-Jing Wang Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[30]  Bruce Cumming,et al.  Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron's causality? , 2010, Current Opinion in Neurobiology.

[31]  Angel R. Martinez,et al.  MATLAB Statistics Toolbox , 2001 .

[32]  Anna C Nobre,et al.  Tracking the changing feature of a moving object. , 2016, Journal of vision.

[33]  Yuka Sasaki,et al.  Perceptual learning: toward a comprehensive theory. , 2015, Annual review of psychology.

[34]  Daniel P. Bliss,et al.  Serial dependence is absent at the time of perception but increases in visual working memory , 2017, bioRxiv.

[35]  David Pascucci,et al.  Location transfer of perceptual learning: Passive stimulation and double training , 2015, Vision Research.

[36]  Xiao-Jing Wang Neural dynamics and circuit mechanisms of decision-making , 2012, Current Opinion in Neurobiology.

[37]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[38]  F. D. Lange,et al.  Opposite Effects of Recent History on Perception and Decision , 2017, Current Biology.

[39]  D. Whitney,et al.  Serial dependence promotes the stability of perceived emotional expression depending on face similarity , 2018, Attention, Perception, & Psychophysics.

[40]  Daniel P. Bliss,et al.  Serial Dependence across Perception, Attention, and Memory , 2017, Trends in Cognitive Sciences.

[41]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[42]  A. Karni,et al.  The time course of learning a visual skill , 1993, Nature.

[43]  Carly J. Leonard,et al.  Direct Evidence for Active Suppression of Salient-but-Irrelevant Sensory Inputs , 2015, Psychological science.

[44]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[45]  A. Fiorentini,et al.  Learning in grating waveform discrimination: Specificity for orientation and spatial frequency , 1981, Vision Research.

[46]  J. Gibson,et al.  Adaptation, after-effect and contrast in the perception of tilted lines. I. Quantitative studies , 1937 .

[47]  David W. Vinson,et al.  Decision contamination in the wild: Sequential dependencies in online review ratings , 2019, Behavior Research Methods.

[48]  Caspar M. Schwiedrzik,et al.  Untangling Perceptual Memory: Hysteresis and Adaptation Map into Separate Cortical Networks , 2012, Cerebral cortex.

[49]  E. Miller,et al.  Dynamics of neuronal sensitivity in visual cortex and local feature discrimination , 2002, Nature Neuroscience.

[50]  Estes Wk The problem of inference from curves based on group data. , 1956 .

[51]  Justin L. Gardner,et al.  Adaptable history biases in human perceptual decisions , 2016, Proceedings of the National Academy of Sciences.

[52]  Mark W. Greenlee,et al.  Tilt aftereffect following adaptation to translational Glass patterns , 2016, Scientific Reports.

[53]  Rachel N. Denison,et al.  Supra-optimality may emanate from suboptimality, and hence optimality is no benchmark in multisensory integration , 2018, Behavioral and Brain Sciences.

[54]  Talking heads or talking eyes? Effects of head orientation and sudden onset gaze cues on attention capture , 2018, Attention, perception & psychophysics.

[55]  K. Sakai,et al.  Autonomous Mechanism of Internal Choice Estimate Underlies Decision Inertia , 2014, Neuron.

[56]  D. Alais,et al.  Love at second sight: Sequential dependence of facial attractiveness in an on-line dating paradigm , 2016, Scientific Reports.

[57]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[58]  Oliver Braddick,et al.  Can speed be judged independent of direction? , 2017, Journal of vision.

[59]  D. Budescu,et al.  The dominance analysis approach for comparing predictors in multiple regression. , 2003, Psychological methods.

[60]  Edward F. Ester,et al.  Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory , 2015, Neuron.

[61]  J. Macke,et al.  Quantifying the effect of intertrial dependence on perceptual decisions. , 2014, Journal of vision.

[62]  M. Sur,et al.  Adaptation-Induced Plasticity of Orientation Tuning in Adult Visual Cortex , 2000, Neuron.

[63]  D L Hintzman,et al.  Violations of the independence assumption in process dissociation. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[64]  David Whitney,et al.  Ensemble perception: Summarizing the scene and broadening the limits of visual processing. , 2012 .

[65]  Cynthia S. Sahm,et al.  Past trials influence perception of ambiguous motion quartets through pattern completion. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Rachel T. A. Croson,et al.  The Gambler’s Fallacy and the Hot Hand: Empirical Data from Casinos , 2005 .

[67]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[68]  J. Gibson Adaptation with negative after-effect. , 1937 .

[69]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[70]  Yuhong Jiang,et al.  Proactive interference from items previously stored in visual working memory , 2008, Memory & cognition.

[71]  T. Pasternak,et al.  Directional Signals in the Prefrontal Cortex and in Area MT during a Working Memory for Visual Motion Task , 2006, The Journal of Neuroscience.

[72]  G. Campana,et al.  Where perception meets memory: A review of repetition priming in visual search tasks , 2010, Attention, perception & psychophysics.

[73]  D. Sagi Perceptual learning in Vision Research , 2011, Vision Research.

[74]  D. Whitney,et al.  Serial dependence promotes object stability during occlusion , 2016, Journal of vision.

[75]  M. Bethge,et al.  Inferring decoding strategies from choice probabilities in the presence of correlated variability , 2013, Nature Neuroscience.

[76]  Alexis Hervais-Adelman,et al.  Gating by induced Α–Γ asynchrony in selective attention , 2018, Human brain mapping.

[77]  L. M. Ward Mixed-modality psychophysical scaling: Inter- and intramodality sequential dependencies as a function of lag , 1985, Perception & psychophysics.

[78]  A. Tversky,et al.  BELIEF IN THE LAW OF SMALL NUMBERS , 1971, Pediatrics.

[79]  Jochen Triesch,et al.  What’s “up”? Working memory contents can bias orientation processing , 2013, Vision Research.

[80]  D. Ariely Seeing Sets: Representation by Statistical Properties , 2001, Psychological science.

[81]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[82]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[83]  M. Turatto,et al.  Permeability of priming of pop out to expectations. , 2012, Journal of vision.

[84]  David Alais,et al.  Different coding strategies for the perception of stable and changeable facial attributes , 2016, Scientific Reports.

[85]  Floris P de Lange,et al.  Prior expectations induce prestimulus sensory templates , 2017, Proceedings of the National Academy of Sciences.

[86]  Christopher J. Honey,et al.  Serial dependence transfers between perceptual objects , 2017, bioRxiv.

[87]  David Whitney,et al.  Serial dependence in the perception of attractiveness , 2015, Journal of vision.

[88]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.

[89]  Pradeep Shenoy,et al.  Strategic Impatience in Go/NoGo versus Forced-Choice Decision-Making , 2012, NIPS.

[90]  F. D. de Lange,et al.  Prior Expectations Bias Sensory Representations in Visual Cortex , 2013, The Journal of Neuroscience.

[91]  J. Jonides,et al.  Brain mechanisms of proactive interference in working memory , 2006, Neuroscience.

[92]  David C. Burr,et al.  Orientation discrimination depends on spatial frequency , 1991, Vision Research.

[93]  B. Dosher,et al.  An integrated reweighting theory of perceptual learning , 2013, Proceedings of the National Academy of Sciences.

[94]  L. M. Ward,et al.  Repeated magnitude estimations with a variable standard: Sequential effects and other properties , 1973 .

[95]  D. Burr,et al.  Compressive mapping of number to space reflects dynamic encoding mechanisms, not static logarithmic transform , 2014, Proceedings of the National Academy of Sciences.

[96]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[97]  D. Burr,et al.  Vision: Efficient Adaptive Coding , 2014, Current Biology.

[98]  G. Lockhead,et al.  Sequential effects in absolute judgments of loudness , 1968 .

[99]  Mriganka Sur,et al.  PLASTICITY OF ORIENTATION PROCESSING IN ADULT VISUAL CORTEX , 2004 .

[100]  Rick Gurnsey,et al.  Foveal and extra-foveal orientation discrimination , 2007, Experimental Brain Research.

[101]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[102]  David Whitney,et al.  Facilitating Stable Representations: Serial Dependence in Vision , 2011, PloS one.

[103]  D. Whitney,et al.  Serial Dependence in the Perception of Faces , 2014, Current Biology.

[104]  Kyriaki Mikellidou,et al.  Serial dependencies act directly on perception. , 2017, Journal of vision.

[105]  Rosalind Baker,et al.  Learning to predict: Exposure to temporal sequences facilitates prediction of future events , 2014, Vision Research.

[106]  D. Levi,et al.  Learning Optimizes Decision Templates in the Human Visual Cortex , 2013, Current Biology.

[107]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[108]  Z L Lu,et al.  Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[109]  Browne,et al.  Cross-Validation Methods. , 2000, Journal of mathematical psychology.

[110]  Caspar M. Schwiedrzik,et al.  How previous experience shapes perception in different sensory modalities , 2015, Front. Hum. Neurosci..

[111]  Denis Schluppeck,et al.  The role of early visual cortex in visual short-term memory and visual attention , 2009, Vision Research.

[112]  J. Nadal,et al.  Perceptual Decision-Making: Biases in Post-Error Reaction Times Explained by Attractor Network Dynamics , 2018, The Journal of Neuroscience.

[113]  C. Law,et al.  Reinforcement learning can account for associative and perceptual learning on a visual decision task , 2009, Nature Neuroscience.

[114]  Elise G. Rowe,et al.  Rapid Adaptation Induces Persistent Biases in Population Codes for Visual Motion , 2016, The Journal of Neuroscience.

[115]  C. Clifford,et al.  A functional angle on some after-effects in cortical vision , 2000, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[116]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[117]  D. Heeger,et al.  Neuronal Basis of the Motion Aftereffect Reconsidered , 2001, Neuron.

[118]  R. Sekuler,et al.  A specific and enduring improvement in visual motion discrimination. , 1982, Science.

[119]  S. Fernberger Interdependence of judgments within the series for the method of constant stimuli. , 1920 .

[120]  R. Ulrich,et al.  Trial-by-trial updating of an internal reference in discrimination tasks: Evidence from effects of stimulus order and trial sequence , 2012, Attention, Perception, & Psychophysics.

[121]  Joachim Vandekerckhove,et al.  Editorial: Bayesian methods for advancing psychological science , 2018, Psychonomic bulletin & review.

[122]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[123]  Floris P. de Lange,et al.  Prior expectations induce pre-stimulus sensory templates , 2017, bioRxiv.

[124]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[125]  D. M. Green,et al.  Sequential effects in judgments of loudness. , 1977, Journal of experimental psychology. Human perception and performance.

[126]  Nicholas Gaspelin,et al.  Inhibition as a potential resolution to the attentional capture debate. , 2019, Current opinion in psychology.

[127]  H. Seung,et al.  Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex. , 2005, Journal of neurophysiology.

[128]  G. Mather,et al.  The motion aftereffect reloaded , 2008, Trends in Cognitive Sciences.

[129]  Janneke F. M. Jehee,et al.  Sensory uncertainty decoded from visual cortex predicts behavior , 2015, Nature Neuroscience.

[130]  Nikolaos Smyrnis,et al.  Independent sources of anisotropy in visual orientation representation: a visual and a cognitive oblique effect , 2015, Experimental Brain Research.

[131]  S. Luck,et al.  Interactions between visual working memory representations , 2017, Attention, perception & psychophysics.

[132]  J. Gold,et al.  How mechanisms of perceptual decision-making affect the psychometric function , 2013, Progress in Neurobiology.

[133]  Joonkoo Park,et al.  Serial dependence in numerosity perception , 2018, Journal of vision.

[134]  N A Macmillan,et al.  Detection theory analysis of group data: estimating sensitivity from average hit and false-alarm rates. , 1985, Psychological bulletin.

[135]  David Alais,et al.  Linear Summation of Repulsive and Attractive Serial Dependencies: Orientation and Motion Dependencies Sum in Motion Perception , 2017, The Journal of Neuroscience.

[136]  Kyriaki Mikellidou,et al.  The functional role of serial dependence , 2018, Proceedings of the Royal Society B.

[137]  Gregory Ashby,et al.  On the Dangers of Averaging Across Subjects When Using Multidimensional Scaling or the Similarity-Choice Model , 1994 .

[138]  Frans A. J. Verstraten,et al.  The motion aftereffect , 1998, Trends in Cognitive Sciences.

[139]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[140]  G. Orban,et al.  Human perceptual learning in identifying the oblique orientation: retinotopy, orientation specificity and monocularity. , 1995, The Journal of physiology.

[141]  Peter E. Latham,et al.  Statistically Efficient Estimation Using Population Coding , 1998, Neural Computation.

[142]  Lawrence M. Ward,et al.  Response system processes in absolute judgment , 1971 .

[143]  E. Galanter,et al.  Psychophysical Scaling ' " , 2006 .

[144]  F. E. Grubbs Procedures for Detecting Outlying Observations in Samples , 1969 .

[145]  G. DeAngelis,et al.  Fine Discrimination Training Alters the Causal Contribution of Macaque Area MT to Depth Perception , 2008, Neuron.

[146]  Robert L. Goldstone,et al.  Definition , 1960, A Philosopher Looks at Sport.

[147]  L. Chelazzi,et al.  Desensitizing the Attention System to Distraction While Idling: A New Latent Learning Phenomenon in the Visual Attention Domain , 2018, Journal of experimental psychology. General.

[148]  R. Sekuler,et al.  Adaptation alters perceived direction of motion , 1976, Vision Research.

[149]  Arne Valberg,et al.  From Pigments to Perception , 1991, NATO ASI Series.

[150]  I. J. Myung,et al.  When a good fit can be bad , 2002, Trends in Cognitive Sciences.

[151]  David Aagten-Murphy,et al.  Adaptation to numerosity requires only brief exposures, and is determined by number of events, not exposure duration , 2016, Journal of vision.