An orthographic prediction error as the basis for efficient visual word recognition

Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations – typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to ‘explain away’ redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.

[1]  Matthew H. Davis,et al.  Temporal Predictive Codes for Spoken Words in Auditory Cortex , 2012, Current Biology.

[2]  Marc Brysbaert,et al.  Subtlex-UK: A New and Improved Word Frequency Database for British English , 2014, Quarterly journal of experimental psychology.

[3]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[4]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[5]  M. Brysbaert,et al.  Practice Effects in Large-Scale Visual Word Recognition Studies: A Lexical Decision Study on 14,000 Dutch Mono- and Disyllabic Words and Nonwords , 2010, Front. Psychology.

[6]  J. Hyönä,et al.  Dissociating spatial and letter-based word length effects observed in readers’ eye movement patterns , 2011, Vision Research.

[7]  Caspar M. Schwiedrzik,et al.  High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy , 2017, Neuron.

[8]  Michael Eickenberg,et al.  Machine learning for neuroimaging with scikit-learn , 2014, Front. Neuroinform..

[9]  J Grainger,et al.  Orthographic processing in visual word recognition: a multiple read-out model. , 1996, Psychological review.

[10]  DH Hubel,et al.  Segregation of form, color, and stereopsis in primate area 18 , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  Keith Rayner,et al.  Parafoveal processing in reading , 2011, Attention, Perception, & Psychophysics.

[12]  J. Catlin,et al.  On the word-frequency effect. , 1969 .

[13]  M. Sigman,et al.  The neural code for written words: a proposal , 2005, Trends in Cognitive Sciences.

[14]  A. Jacobs,et al.  The word frequency effect: a review of recent developments and implications for the choice of frequency estimates in German. , 2011, Experimental psychology.

[15]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[16]  Jonathan Grainger,et al.  Watching the Word Go by: On the Time-course of Component Processes in Visual Word Recognition , 2009, Lang. Linguistics Compass.

[17]  Reinhold Kliegl,et al.  SWIFT: a dynamical model of saccade generation during reading. , 2005, Psychological review.

[18]  Eliane Segers,et al.  A Special Font for People with Dyslexia: Does it Work and, if so, why? , 2016, Dyslexia.

[19]  Qiong Zhang,et al.  The Structures of Letters and Symbols throughout Human History Are Selected to Match Those Found in Objects in Natural Scenes , 2006, The American Naturalist.

[20]  S. Dehaene,et al.  The unique role of the visual word form area in reading , 2011, Trends in Cognitive Sciences.

[21]  C. Gilbert,et al.  Generation of end-inhibition in the visual cortex via interlaminar connections , 1986, Nature.

[22]  F. Hutzler,et al.  On forward inferences of fast and slow readers. An eye movement study , 2015, Scientific Reports.

[23]  C. Price,et al.  The Interactive Account of ventral occipitotemporal contributions to reading , 2011, Trends in Cognitive Sciences.

[24]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[25]  S Edelman,et al.  Representation is representation of similarities , 1996, Behavioral and Brain Sciences.

[26]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[27]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[28]  Christian J. Fiebach,et al.  Context-Based Facilitation in Visual Word Recognition: Evidence for Visual and Lexical But Not Pre-Lexical Contributions , 2019, eNeuro.

[29]  Ravi S. Menon,et al.  Effect of luminance contrast on BOLD fMRI response in human primary visual areas. , 1998, Journal of neurophysiology.

[30]  A. Jacobs,et al.  Frequency and predictability effects on event-related potentials during reading , 2006, Brain Research.

[31]  Katherine A. DeLong,et al.  Probabilistic word pre-activation during language comprehension inferred from electrical brain activity , 2005, Nature Neuroscience.

[32]  Simon P Liversedge,et al.  Preview benefit in English spaced compounds. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[33]  Marc Brysbaert,et al.  SUBTLEX-NL: A new measure for Dutch word frequency based on film subtitles , 2010, Behavior research methods.

[34]  Colin J Davis,et al.  The spatial coding model of visual word identification. , 2010, Psychological review.

[35]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[36]  Marco Zorzi,et al.  Nested incremental modeling in the development of computational theories: the CDP+ model of reading aloud. , 2007, Psychological review.

[37]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[38]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[39]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[40]  Marc Brysbaert,et al.  The French Lexicon Project: Lexical decision data for 38,840 French words and 38,840 pseudowords , 2010, Behavior research methods.

[41]  M Coltheart,et al.  DRC: a dual route cascaded model of visual word recognition and reading aloud. , 2001, Psychological review.

[42]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[43]  Mante S. Nieuwland,et al.  Large-scale replication study reveals a limit on probabilistic prediction in language comprehension , 2018, eLife.

[44]  E. Maris,et al.  Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study , 2011, The Journal of Neuroscience.

[45]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[46]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[47]  Christian J. Fiebach,et al.  The lexical categorization model: A computational model of left ventral occipito-temporal cortex activation in visual word recognition , 2020, bioRxiv.

[48]  Ralf Engbert,et al.  Length, frequency, and predictability effects of words on eye movements in reading , 2004 .

[49]  Marta Kutas,et al.  Interplay between computational models and cognitive electrophysiology in visual word recognition , 2007, Brain Research Reviews.

[50]  J. Changeux,et al.  A Neuronal Model of Predictive Coding Accounting for the Mismatch Negativity , 2012, The Journal of Neuroscience.

[51]  R. Shapley,et al.  LFP power spectra in V1 cortex: the graded effect of stimulus contrast. , 2005, Journal of neurophysiology.

[52]  Max Coltheart,et al.  Access to the internal lexicon , 1977 .

[53]  Shravan Vasishth,et al.  Are words pre-activated probabilistically during sentence comprehension? Evidence from new data and a Bayesian random-effects meta-analysis using publicly available data , 2020, Neuropsychologia.

[54]  Blair C. Armstrong,et al.  The what, when, where, and how of visual word recognition , 2014, Trends in Cognitive Sciences.

[55]  Manuel Carreiras,et al.  Converging evidence for functional and structural segregation within the left ventral occipitotemporal cortex in reading , 2018, Proceedings of the National Academy of Sciences.

[56]  Michael J Cortese,et al.  Visual word recognition of single-syllable words. , 2004, Journal of experimental psychology. General.

[57]  Reinhold Kliegl,et al.  Experimental Effects and Individual Differences in Linear Mixed Models: Estimating the Relationship between Spatial, Object, and Attraction Effects in Visual Attention , 2010, Front. Psychology.

[58]  G. Mangun,et al.  Luminance and spatial attention effects on early visual processing. , 1995, Brain research. Cognitive brain research.

[59]  Martin Luessi,et al.  MNE software for processing MEG and EEG data , 2014, NeuroImage.

[60]  A. Jacobs,et al.  Coregistration of eye movements and EEG in natural reading: analyses and review. , 2011, Journal of experimental psychology. General.

[61]  Benjamin Gagl,et al.  Blue hypertext is a good design decision: no perceptual disadvantage in reading and successful highlighting of relevant information , 2016, PeerJ.

[62]  D. Norris Models of visual word recognition , 2013, Trends in Cognitive Sciences.

[63]  Ross D. Markello,et al.  AtlasReader: A Python package to generate coordinate tables, region labels, and informative figures from statistical MRI images , 2019, J. Open Source Softw..

[64]  Roger Ratcliff,et al.  A diffusion model account of the lexical decision task. , 2004, Psychological review.

[65]  Marc Brysbaert,et al.  Lexique 2 : A new French lexical database , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[66]  Ralf Engbert,et al.  Tracking the mind during reading: the influence of past, present, and future words on fixation durations. , 2006, Journal of experimental psychology. General.

[67]  Michaël A. Stevens,et al.  The impact of word prevalence on lexical decision times: Evidence from the Dutch Lexicon Project 2. , 2016, Journal of experimental psychology. Human perception and performance.

[68]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[69]  Luc H. Arnal,et al.  Transitions in neural oscillations reflect prediction errors generated in audiovisual speech , 2011, Nature Neuroscience.

[70]  Erik D. Reichle,et al.  The E-Z Reader model of eye-movement control in reading: Comparisons to other models , 2003, Behavioral and Brain Sciences.

[71]  K. Rayner The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search , 2009, Quarterly journal of experimental psychology.

[72]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[73]  H. Akaike Maximum likelihood identification of Gaussian autoregressive moving average models , 1973 .

[74]  Thomas E. Nichols,et al.  Optimization of experimental design in fMRI: a general framework using a genetic algorithm , 2003, NeuroImage.

[75]  Wolfgang Huber,et al.  EBImage—an R package for image processing with applications to cellular phenotypes , 2010, Bioinform..

[76]  Jeffrey L. Elman,et al.  Large-Scale Modeling of Wordform Learning and Representation , 2008, Cogn. Sci..

[77]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[78]  D. Balota,et al.  Moving beyond Coltheart’s N: A new measure of orthographic similarity , 2008, Psychonomic bulletin & review.

[79]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[80]  Marc Brysbaert,et al.  The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words , 2011, Behavior Research Methods.

[81]  Fabio Richlan,et al.  Parafoveal preprocessing in reading revisited: evidence from a novel preview manipulation. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[82]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.