Tracking the Continuity of Language Comprehension: Computer Mouse Trajectories Suggest Parallel Syntactic Processing

Although several theories of online syntactic processing assume the parallel activation of multiple syntactic representations, evidence supporting simultaneous activation has been inconclusive. Here, the continuous and non-ballistic properties of computer mouse movements are exploited, by recording their streaming x, y coordinates to procure evidence regarding parallel versus serial processing. Participants heard structurally ambiguous sentences while viewing scenes with properties either supporting or not supporting the difficult modifier interpretation. The curvatures of the elicited trajectories revealed both an effect of visual context and graded competition between simultaneously active syntactic representations. The results are discussed in the context of 3 major groups of theories within the domain of sentence processing.

[1]  Michael J. Spivey,et al.  Gradiency and Visual Context in Syntactic Garden-Paths. , 2007, Journal of memory and language.

[2]  G. Rizzolatti,et al.  Orienting of attention and eye movements , 2004, Experimental Brain Research.

[3]  Irina A. Sekerina,et al.  The kindergarten-path effect: studying on-line sentence processing in young children , 1999, Cognition.

[4]  James S Magnuson,et al.  Moving hand reveals dynamics of thought. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[5]  K. Nakayama,et al.  Role of focal attention on latencies and trajectories of visually guided manual pointing. , 2006, Journal of vision.

[6]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[7]  M. A. Britt,et al.  The Interaction of Referential Ambiguity and Argument Structure in the Parsing of Prepositional Phrases , 1994 .

[8]  Michael J. Spivey,et al.  Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[9]  Julie C. Sedivy,et al.  Resolving attachment ambiguities with multiple constraints , 1995, Cognition.

[10]  M. Pickering,et al.  Adjunct attachment is not a form of lexical ambiguity resolution , 1998 .

[11]  Thomas G. Whiston,et al.  A National Survey , 1992 .

[12]  Michael J. Spivey,et al.  Graded motor responses in the time course of categorizing atypical exemplars , 2007, Memory & cognition.

[13]  C. Atkeson,et al.  Kinematic features of unrestrained vertical arm movements , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[15]  S. Brennan.  4 How Conversation Is Shaped by Visual and Spoken Evidence , 2005 .

[16]  Julie C. Sedivy,et al.  Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution , 2002, Cognitive Psychology.

[17]  R. Wurtz,et al.  Sequential activity of simultaneously recorded neurons in the superior colliculus during curved saccades. , 2003, Journal of neurophysiology.

[18]  W. Tabor,et al.  Evidence for self-organized sentence processing: digging-in effects. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[19]  S. Tipper,et al.  Selective Reaching to Grasp: Evidence for Distractor Interference Effects , 1997 .

[20]  Gabrielle A. Strouse,et al.  Age, Ethnicity, and Socioeconomic Patterns in Early Computer Use , 2005 .

[21]  Matthew W. Crocker,et al.  The Coordinated Interplay of Scene, Utterance, and World Knowledge: Evidence From Eye Tracking , 2006, Cogn. Sci..

[22]  M. Pickering,et al.  Evidence against competition during syntactic ambiguity resolution. , 2005 .

[23]  G. Miller,et al.  Cognitive science. , 1981, Science.

[24]  R. Walker,et al.  Curved saccade trajectories: Voluntary and reflexive saccades curve away from irrelevant distractors , 2001, Experimental Brain Research.

[25]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[26]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[27]  J. Driver,et al.  Control of Cognitive Processes: Attention and Performance XVIII , 2000 .

[28]  C. Prablanc,et al.  Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement , 1986, Nature.

[29]  Colin M. Brown,et al.  Early referential context effects in sentence processing: Evidence from event-related brain potentials , 1999 .

[30]  L. T. DeCarlo.  On the meaning and use of kurtosis. , 1997 .

[31]  Richard B. Darlington,et al.  Is Kurtosis Really “Peakedness?” , 1970 .

[32]  Wy Depa,et al.  A NATIONAL SURVEY , 1995 .

[33]  Michael J. Spivey,et al.  Continuous attraction toward phonological competitors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[34]  J. Gold,et al.  Representation of a perceptual decision in developing oculomotor commands , 2000, Nature.

[35]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[36]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[37]  D. Mitchell,et al.  Absence of real evidence against competition during syntactic ambiguity resolution , 2006 .

[38]  N J Pearlmutter,et al.  Distinguishing Serial and Parallel Parsing , 2000, Journal of psycholinguistic research.

[39]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[40]  M. Tanenhaus,et al.  Actions and affordances in syntactic ambiguity resolution. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[41]  Roger P. G. van Gompel,et al.  Reanalysis in Sentence Processing: Evidence against Current Constraint-Based and Two-Stage Models , 2001 .

[42]  Richard L. Lewis,et al.  Falsifying Serial and Parallel Parsing Models: Empirical Conundrums and An Overlooked Paradigm , 2000, Journal of psycholinguistic research.

[43]  J. Kalaska,et al.  Neural Correlates of Reaching Decisions in Dorsal Premotor Cortex: Specification of Multiple Direction Choices and Final Selection of Action , 2005, Neuron.

[44]  J. Trueswell,et al.  The developing constraints on parsing decisions: The role of lexical-biases and referential scenes in child and adult sentence processing , 2004, Cognitive Psychology.