Computationally Rational Saccadic Control: An Explanation of Spillover Effects Based on Sampling from Noisy Perception and Memory

Eye-movements in reading exhibit frequency spillover effects: fixation durations on a word are affected by the frequency of the previous word. We explore the idea that this effect may be an emergent property of a computationally rational eyemovement strategy that is navigating a tradeoff between processing immediate perceptual input, and continued processing of past input based on memory. We present an adaptive eye-movement control model with a minimal capacity for such processing, based on a composition of thresholded sequential samplers that integrate information from noisy perception and noisy memory. The model is applied to the List Lexical Decision Task and shown to yield frequency spillover—a robust property of human eye-movements in this task, even with parafoveal masking. We show that spillover in the model emerges in approximately optimal control policies that sometimes process memory rather than perception. We compare this model with one that is able to give priority to perception over memory, and show that the perception-priority policies in such a model do not perform as well in a range of plausible noise settings. We explain how the frequency spillover arises from a counter-intuitive but fundamental property of sequenced thresholded samplers.

[1]  Bernice W. Polemis Nonparametric Statistics for the Behavioral Sciences , 1959 .

[2]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[3]  R. Schvaneveldt,et al.  Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. , 1971, Journal of experimental psychology.

[4]  Jean-Roger Vergnaud French relative clauses. , 1974 .

[5]  P. Suppes The Semantics of Children's Language , 1974 .

[6]  P. Lightbown,et al.  Imitation in language development: If, when, and why , 1974 .

[7]  Richard Montague,et al.  ENGLISH AS A FORMAL LANGUAGE , 1975 .

[8]  K. Rayner The perceptual span and peripheral cues in reading , 1975, Cognitive Psychology.

[9]  W. Chafe Givenness, contrastiveness, definiteness, subjects, topics, and point of view , 1976 .

[10]  M. Braine Children's First Word Combinations. , 1976 .

[11]  Roger W. Brown,et al.  A First Language: The Early Stages , 1974 .

[12]  S. Kuczaj The acquisition of regular and irregular past tense forms , 1977 .

[13]  Noam Chomsky Essays on Form and Interpretation , 1977 .

[14]  D. Swinney,et al.  Semantic facilitation across sensory modalities in the processing of individual words and sentences , 1979, Memory & cognition.

[15]  K. Rayner,et al.  Asymmetry of the effective visual field in reading , 1980, Perception & psychophysics.

[16]  T. Reinhart Pragmatics and Linguistics: an analysis of Sentence Topics , 1981, Philosophica.

[17]  T. Givón,et al.  Topic continuity in discourse : a quantitative cross-language study , 1983 .

[18]  C. Dunkel-schetter,et al.  Social Support and Cancer: Findings Based on Patient Interviews and Their Implications , 1984 .

[19]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[20]  Ellen Francik Referential choice and focus of attention in narratives , 1985 .

[21]  William D. Marslen-Wilson,et al.  Crossed and nested dependencies in German and Dutch , 1986 .

[22]  J. K. Bock Syntactic persistence in language production , 1986, Cognitive Psychology.

[23]  P. Tichý Constructions , 1986, Philosophy of Science.

[24]  R. Langacker Foundations of Cognitive Grammar: Volume I: Theoretical Prerequisites , 1987 .

[25]  L. Frazier,et al.  Filler driven parsing: A study of gap filling in dutch , 1989 .

[26]  Mira Ariel Accessing Noun-Phrase Antecedents , 1990 .

[27]  J. Henderson,et al.  Effects of foveal processing difficulty on the perceptual span in reading: Implications for attention and eye movement control. , 1990 .

[28]  J. Henderson,et al.  Effects of foveal processing difficulty on the perceptual span in reading: implications for attention and eye movement control. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[29]  B. MacWhinney The CHILDES project: tools for analyzing talk , 1992 .

[30]  M. Tomasello First Verbs: A Case Study of Early Grammatical Development , 1994 .

[31]  Bradley L. Pritchett Grammatical Competence and Parsing Performance , 1992 .

[32]  Philip Resnik,et al.  Left-Corner Parsing and Psychological Plausibility , 1992, COLING.

[33]  Noam Chomsky,et al.  The Minimalist Program , 1992 .

[34]  Jeanette K. Gundel,et al.  Cognitive Status and the Form of Referring Expressions in Discourse , 1993 .

[35]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[36]  Rosemary J. Stevenson,et al.  Thematic roles, focus and the representation of events , 1994 .

[37]  Richard S. Kayne The Antisymmetry of Syntax , 1994 .

[38]  Marilyn A. Walker,et al.  Japanese Discourse and the Process of Centering , 1994, Comput. Linguistics.

[39]  Venugopal V. Veeravalli,et al.  A sequential procedure for multihypothesis testing , 1994, IEEE Trans. Inf. Theory.

[40]  C. Clifton,et al.  Determinants of parafoveal preview benefit in high and low working memory capacity readers: implications for eye movement control. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[41]  S. Brennan Centering Attention in Discourse. , 1995 .

[42]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[43]  Megumi Kameyama,et al.  Indefeasible Semantics and Defeasible Pragmatics , 1994, ArXiv.

[44]  Aravind K. Joshi,et al.  A Processing Model for Free Word Order Languages , 1995, ArXiv.

[45]  Joan L. Bybee,et al.  Regular morphology and the lexicon. , 1995 .

[46]  J. Siskind A computational study of cross-situational techniques for learning word-to-meaning mappings , 1996, Cognition.

[47]  K. Rayner,et al.  Mindless reading revisited: Eye movements during reading and scanning are different , 1996, Perception & psychophysics.

[48]  Edward P. Stabler,et al.  Derivational Minimalism , 1996, LACL.

[49]  Ö. Dahl,et al.  Animacy in grammar and discourse , 1996 .

[50]  M. Tomasello,et al.  Young children's productivity with word order and verb morphology. , 1997, Developmental psychology.

[51]  Erik D. Reichle,et al.  Toward a model of eye movement control in reading. , 1998, Psychological review.

[52]  Jennifer E. Arnold Reference form and discourse patterns , 1998 .

[53]  E. Gibson Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[54]  C. Chambers,et al.  Structural Parallelism and Discourse Coherence: A Test of Centering Theory , 1998 .

[55]  M. Brysbaert,et al.  Eye Movement Control during Reading: Foveal Load and Parafoveal Processing , 1999, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[56]  M. Pickering,et al.  Syntactic priming in language production , 1999, Trends in Cognitive Sciences.

[57]  M. Tomasello Perceiving intentions and learning words in the second year of life , 2000 .

[58]  Zenzi M. Griffin,et al.  The persistence of structural priming: transient activation or implicit learning? , 2000, Journal of experimental psychology. General.

[59]  Alexandra A. Cleland,et al.  Syntactic co-ordination in dialogue , 2000, Cognition.

[60]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part II: Accurate asymptotic expansions for the expected sample size , 2000, IEEE Trans. Inf. Theory.

[61]  Kenneth Ward Church Empirical Estimates of Adaptation: The chance of Two Noriegas is closer to p/2 than p2 , 2000, COLING.

[62]  Jennifer E. Arnold,et al.  The Effect of Thematic Roles on Pronoun Use and Frequency of Reference Continuation , 2001 .

[63]  Suzanne Stevenson,et al.  Automatic Verb Classification Based on Statistical Distributions of Argument Structure , 2001, CL.

[64]  Jens Michaelis,et al.  Transforming Linear Context-Free Rewriting Systems into Minimalist Grammars , 2001, LACL.

[65]  Henk Harkema,et al.  A Characterization of Minimalist Languages , 2001, LACL.

[66]  Chris Brew,et al.  Inducing German Semantic Verb Classes from Purely Syntactic Subcategorisation Information , 2002, ACL.

[67]  Andrew Kehler,et al.  Coherence, reference, and the theory of grammar , 2002, CSLI lecture notes series.

[68]  Dan Klein,et al.  Fast Exact Inference with a Factored Model for Natural Language Parsing , 2002, NIPS.

[69]  M. Tomasello,et al.  A tale of two theories: response to Fisher , 2002, Cognition.

[70]  Cynthia Fisher,et al.  The role of abstract syntactic knowledge in language acquisition: a reply to Tomasello (2000) , 2002, Cognition.

[71]  Paul Kay,et al.  An Informal Sketch of a Formal Architecture for Construction Grammar , 2002, Grammars.

[72]  C. B. Taylor,et al.  Evaluation of an internet support group for women with primary breast cancer , 2003, Cancer.

[73]  M. Tomasello Constructing a Language: A Usage-Based Theory of Language Acquisition , 2003 .

[74]  Adele E. Goldberg Constructions: a new theoretical approach to language , 2003, Trends in Cognitive Sciences.

[75]  David Adger,et al.  Core Syntax: A Minimalist Approach , 2003 .

[76]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[77]  Daniel C. Richardson,et al.  Effects of merely local syntactic coherence on sentence processing , 2004 .

[78]  John Hale,et al.  Grammar, uncertainty and sentence processing , 2004 .

[79]  M. Pickering,et al.  The interactive-alignment model: Developments and refinements , 2004, Behavioral and Brain Sciences.

[80]  Julian M. Pine,et al.  Constructing a Language: A Usage-Based Theory of Language Acquisition. , 2004 .

[81]  S. Gries Syntactic Priming: A Corpus-based Approach , 2005, Journal of psycholinguistic research.

[82]  Qimei Chen,et al.  Internet Community Group Participation: Psychosocial Benefits for Women with Breast Cancer , 2005, J. Comput. Mediat. Commun..

[83]  Reinhold Kliegl,et al.  SWIFT: a dynamical model of saccade generation during reading. , 2005, Psychological review.

[84]  Sarah J. White,et al.  Eye movements and the modulation of parafoveal processing by foveal processing difficulty: A reexamination , 2005, Psychonomic bulletin & review.

[85]  Adele E. Goldberg,et al.  Constructions at Work , 2005 .

[86]  Juhani Järvikivi,et al.  Ambiguous Pronoun Resolution , 2005, Psychological science.

[87]  Dennis Norris,et al.  The Bayesian reader: explaining word recognition as an optimal Bayesian decision process. , 2006, Psychological review.

[88]  Johanna D. Moore,et al.  Priming of Syntactic Rules in Task-Oriented Dialogue and Spontaneous Conversation , 2006 .

[89]  Xiaojin Zhu,et al.  TagLDA: Bringing a document structure knowledge into topic models , 2006 .

[90]  Francisco Iacobelli,et al.  Computational Measures for Language Similarity Across Time in Online Communities , 2006, HLT-NAACL 2006.

[91]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[92]  Chien-Jer Charles Lin,et al.  GRAMMAR AND PARSING: A TYPOLOGICAL INVESTIGATION OF RELATIVE-CLAUSE PROCESSING , 2006 .

[93]  A. Goldberg Constructions at Work: The Nature of Generalization in Language , 2006 .

[94]  V. Ferreira,et al.  The functions of structural priming , 2006, Language and cognitive processes.

[95]  P. Healey Participation , Precedence and Co-ordination in Dialogue , 2006 .

[96]  Johanna D. Moore,et al.  Predicting Success in Dialogue , 2007, ACL.

[97]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[98]  T. Jaeger,et al.  Implicit Learning and Syntactic Persistence: Surprisal and Cumulativity , 2007 .

[99]  Mitchell P. Marcus,et al.  OntoNotes: A Unified Relational Semantic Representation , 2007, International Conference on Semantic Computing (ICSC 2007).

[100]  E. Conwell,et al.  Early syntactic productivity: Evidence from dative shift , 2007, Cognition.

[101]  J. Elman,et al.  Pronoun Interpretation as a Side Effect of Discourse Coherence , 2007 .

[102]  H. Cowles,et al.  Linguistic and cognitive prominence in anaphor resolution: topic, contrastive focus and pronouns , 2007 .

[103]  Gregory M. Kobele,et al.  An Automata-Theoretic Approach to Minimalism , 2007 .

[104]  Amanda Stent,et al.  Measuring Adaptation Between Dialogs , 2007, SIGdial.

[105]  A. Bambina Online Social Support: The Interplay of Social Networks and Computer-Mediated Communication , 2007 .

[106]  Stephani Foraker,et al.  The role of prominence in pronoun resolution: Active versus passive representations , 2007 .

[107]  Suzanne Stevenson,et al.  A Computational Model of Early Argument Structure Acquisition , 2008, Cogn. Sci..

[108]  David M. Blei,et al.  Syntactic Topic Models , 2008, NIPS.

[109]  M. Tanenhaus,et al.  Acquiring and processing verb argument structure: Distributional learning in a miniature language , 2008, Cognitive Psychology.

[110]  M. Pickering,et al.  Structural priming: a critical review. , 2008, Psychological bulletin.

[111]  Nancy Chang,et al.  A computational model of the emergence of early constructions , 2008 .

[112]  Christopher E. Beaudoin,et al.  Modeling the impact of online cancer resources on supporters of cancer patients , 2008, New Media Soc..

[113]  J. Trueswell,et al.  Interpreting pronouns and demonstratives in Finnish: Evidence for a form-specific approach to reference resolution , 2008 .

[114]  Darren Gergle,et al.  Linguistic mimicry and trust in text-based CMC , 2008, CSCW.

[115]  C. Fisher,et al.  Two-year-olds use distributional cues to interpret transitivity-alternating verbs , 2009, Language and cognitive processes.

[116]  S. Piantadosi,et al.  Refer efficiently : Use less informative expressions for more predictable meanings , 2009 .

[117]  Erik D. Reichle,et al.  Using E-Z reader to model the effects of higher level language processing on eye movements during reading , 2009, Psychonomic bulletin & review.

[118]  M. Tomasello,et al.  Modeling children's early grammatical knowledge , 2009, Proceedings of the National Academy of Sciences.

[119]  Simon Garrod,et al.  Joint Action, Interactive Alignment, and Dialog , 2009, Top. Cogn. Sci..

[120]  Michael Tomasello,et al.  Two-year-old children's production of multiword utterances: A usage-based analysis , 2009 .

[121]  R. Langacker A dynamic view of usage and language acquisition , 2009 .

[122]  Fuyun Wu Factors affecting relative clause processing in Mandarin , 2009 .

[123]  A. Verhagen The conception of constructions as complex signs: Emergence of structure and reduction to usage , 2009 .

[124]  A. Alishahi,et al.  Please Scroll down for Article Language and Cognitive Processes a Computational Model of Learning Semantic Roles from Child-directed Language a Computational Model of Learning Semantic Roles from Child-directed Language , 2022 .

[125]  M. Pickering,et al.  Linguistic alignment between people and computers , 2010 .

[126]  Jennifer E. Arnold,et al.  How Speakers Refer: The Role of Accessibility , 2010, Lang. Linguistics Compass.

[127]  J. Pine,et al.  Explaining Quantitative Variation in the Rate of Optional Infinitive Errors across Languages: a Comparison of Mosaic and the Variational Learning Model* , 2009 .

[128]  Suzanne Stevenson,et al.  Learning verb alternations in a usage-based Bayesian model , 2010 .

[129]  J. Tenenbaum,et al.  Variability, negative evidence, and the acquisition of verb argument constructions. , 2010, Journal of child language.

[130]  Susan T. Dumais,et al.  Mark my words!: linguistic style accommodation in social media , 2011, WWW.

[131]  John K. Pate,et al.  Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics , 2011 .

[132]  John Yen,et al.  Get Online Support, Feel Better -- Sentiment Analysis and Dynamics in an Online Cancer Survivor Community , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[133]  Johanna D. Moore,et al.  Explorer A Computational Cognitive Model of Syntactic Priming , 2017 .

[134]  Edward P. Stabler,et al.  Top-Down Recognizers for MCFGs and MGs , 2011, CMCL@ACL.

[135]  Edward P. Stabler,et al.  Computational Perspectives on Minimalism , 2011 .

[136]  Michael P. Kaschak,et al.  Structural priming as implicit learning: Cumulative priming effects and individual differences , 2011, Psychonomic bulletin & review.

[137]  Thomas Graf Locality and the Complexity of Minimalist Derivation Tree Languages , 2011, FG.

[138]  Michael P. Kaschak,et al.  Long-term cumulative structural priming persists for (at least) one week , 2011, Memory & cognition.

[139]  Cristian Danescu-Niculescu-Mizil,et al.  Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs , 2011, CMCL@ACL.

[140]  B. Ambridge,et al.  Semantics versus statistics in the retreat from locative overgeneralization errors , 2012, Cognition.

[141]  C. Frith,et al.  Coming to Terms , 2012, Psychological science.

[142]  Mark Steedman,et al.  A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings , 2012, EACL.

[143]  John Hale,et al.  Memory Resource Allocation in Top-Down Minimalist Parsing , 2013, FG.

[144]  Philip Resnik,et al.  Modeling topic control to detect influence in conversations using nonparametric topic models , 2014, Machine Learning.

[145]  Roger Levy,et al.  Memory and surprisal in human sentence comprehension , 2013 .

[146]  Greta E. Greer,et al.  Understanding topics and sentiment in an online cancer survivor community. , 2013, Journal of the National Cancer Institute. Monographs.

[147]  Jon M. Kleinberg,et al.  Characterizing and curating conversation threads: expansion, focus, volume, re-entry , 2013, WSDM.

[148]  Richard L. Lewis,et al.  The Adaptive Nature of Eye Movements in Linguistic Tasks: How Payoff and Architecture Shape Speed-Accuracy Trade-Offs , 2013, Top. Cogn. Sci..

[149]  Libby Barak,et al.  Acquisition of Desires before Beliefs: A Computional Investigation , 2013, CoNLL.

[150]  M. Kutas,et al.  Subject/object processing asymmetries in Korean relative clauses: Evidence from ERP data , 2013 .

[151]  Naomi Feldman,et al.  Quantifying the role of discourse topicality in speakers’ choices of referring expressions , 2014, CMCL@ACL.

[152]  John Yen,et al.  Finding influential users of online health communities: a new metric based on sentiment influence. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[153]  Franziska Frankfurter,et al.  Constructions: A construction grammar approach to argument structure: Adele E. Goldberg, Chicago, IL: The University of Chicago Press, 1995. xi + 265 pp , 1998 .