Linguistic Complexity and Cognitive Load in a Dual-Task Context

In this work, we investigate the effect of linguistic complexity on cognitive load in a dual-task scenario, namely simultaneous driving and language use. To this end, we designed and implemented a psycholinguistic experiment where participants use a driving simulator while listening to spoken utterances and answering comprehension questions. On-line physiological measures of cognitive load, including the recently established Index of Cognitive Activity, as well as measures of performance in both tasks, have been collected. The resulting rich corpus of aligned fine-grained data streams can be used to test a vast array of different hypotheses about the relationship between performance, difficulty and cognitive load in dual tasks, at various levels of temporal resolution and linguistic structure. In this Master’s thesis, we present the theoretical motivation and background, the methodological and technical aspects of the experiment, the resulting corpus and some first interesting results from the data analysis.

[1]  Andrew L. Kun,et al.  Estimating cognitive load using remote eye tracking in a driving simulator , 2010, ETRA.

[2]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[3]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.

[4]  S. P. Marshall,et al.  The Index of Cognitive Activity: measuring cognitive workload , 2002, Proceedings of the IEEE 7th Conference on Human Factors and Power Plants.

[5]  Elena G. Patsenko,et al.  The Quarterly Journal of Experimental Psychology Pupillometry Reveals Processing Load during Spoken Language Comprehension , 2022 .

[6]  Marcela Charfuelan,et al.  The MARY TTS entry in the Blizzard Challenge 2008 , 2008 .

[7]  Sandra P. Marshall,et al.  Measuring cognitive workload across different eye tracking hardware platforms , 2012, ETRA.

[8]  E. Gibson.  The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[9]  Karel Brookhuis,et al.  Human Factors for Assistance and Automation , 2008 .

[10]  Frank A. Drews,et al.  Passenger and Cell-Phone Conversations in Simulated Driving , 2004 .

[11]  T. Jong.  Cognitive load theory, educational research, and instructional design: some food for thought , 2010 .

[12]  S. Sirois,et al.  Pupillometry , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[13]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[14]  Sandra P Marshall,et al.  Identifying cognitive state from eye metrics. , 2007, Aviation, space, and environmental medicine.

[15]  M. Dawson,et al.  The electrodermal system , 2007 .

[16]  Markus Bader,et al.  Subject-Object Ambiguities in German Embedded Clauses: An Across-the-Board Comparison , 1999 .

[17]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[18]  M. Just,et al.  The intensity dimension of thought: pupillometric indices of sentence processing. , 1993, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[19]  J. Dusek,et al.  Impact of Incremental Increases in Cognitive Workload on Physiological Arousal and Performance in Young Adult Drivers , 2009 .

[20]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[21]  Anne T McCartt,et al.  Cell Phones and Driving: Review of Research , 2006, Traffic injury prevention.

[22]  Joonwoo Son,et al.  Estimating Cognitive Load Complexity Using Performance and Physiological Data in a Driving Simulator , 2011 .

[23]  Fang Chen,et al.  Galvanic skin response (GSR) as an index of cognitive load , 2007, CHI Extended Abstracts.

[24]  Brian Roark,et al.  Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing , 2009, EMNLP.

[25]  Katja Kircher,et al.  Driver distraction : a review of the literature , 2007 .

[26]  D. Leutner,et al.  Direct Measurement of Cognitive Load in Multimedia Learning , 2003 .

[27]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[28]  M K Tanenhaus,et al.  Eye Movements and Lexical Access in Spoken-Language Comprehension: Evaluating a Linking Hypothesis between Fixations and Linguistic Processing , 2000, Journal of psycholinguistic research.

[29]  E Courchesne,et al.  Effects of rare non-target stimuli on brain electrophysiological activity and performance. , 1990, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  David L. Strayer,et al.  Driven to Distraction: Dual-Task Studies of Simulated Driving and Conversing on a Cellular Telephone , 2001, Psychological science.

[31]  N. Squires,et al.  Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. , 1975, Electroencephalography and clinical neurophysiology.

[32]  Paul Atchley,et al.  Potential Benefits and Costs of Concurrent Task Engagement to Maintain Vigilance , 2011, Hum. Factors.

[33]  Zhang Hua,et al.  Speech recognition interface design for in-vehicle system , 2010, AutomotiveUI.

[34]  Timothy L. Brown,et al.  Speech-Based Interaction with In-Vehicle Computers: The Effect of Speech-Based E-Mail on Drivers' Attention to the Roadway , 2001, Hum. Factors.

[35]  Vera Demberg,et al.  Syntactic Surprisal Affects Spoken Word Duration in Conversational Contexts , 2012, EMNLP.

[36]  Michael J Schwartz,et al.  Eye Metrics as an Objective Assessment of Surgical Skill , 2010, Annals of surgery.

[37]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[38]  F. Díaz,et al.  Effect of the relevance and position of the target stimuli on P300 and reaction time. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[39]  M. Just,et al.  Neuroindices of cognitive workload: Neuroimaging, pupillometric and event-related potential studies of brain work , 2003 .

[40]  David Crundall,et al.  Regulating Conversation During Driving: A Problem for Mobile Telephones? , 2005 .

[41]  Hiroshi Nittono,et al.  Event-related brain potentials during the standard autonomic-based concealed information test. , 2009, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[42]  M. Just,et al.  A decrease in brain activation associated with driving when listening to someone speak , 2008, Brain Research.

[43]  Frank Keller,et al.  A Computational Model of Prediction in Human Parsing: Unifying Locality and Surprisal Effects , 2009 .

[44]  George Yannis,et al.  Is it Risky to Talk, Eat, or Smoke While Driving? Findings from a Driving Simulator Experiment , 2011 .