Combined eyetracking and keystroke-logging methods for studying cognitive processes in text production

Writers typically spend a certain proportion of time looking back over the text that they have written. This is likely to serve a number of different functions, which are currently poorly understood. In this article, we present two systems, ScriptLog+TimeLine and EyeWrite, that adopt different and complementary approaches to exploring this activity by collecting and analyzing combined eye movement and keystroke data from writers composing extended texts. ScriptLog+TimeLine is a system that is based on an existing keystroke-logging program and uses heuristic, pattern-matching methods to identify reading episodes within eye movement data. EyeWrite is an integrated editor and analysis system that permits identification of the words that the writer fixates and their location within the developing text. We demonstrate how the methods instantiated within these systems can be used to make sense of the large amount of data generated by eyetracking and keystroke logging in order to inform understanding of the cognitive processes that underlie written text production.

[1]  Gert Rijlaarsdam,et al.  The dynamics of composing Modeling Writing Process Data. , 1996 .

[2]  Åsa Wengelin,et al.  Text Production in Adults with Reading and Writing Difficulties , 2002 .

[3]  Annie Piolat,et al.  Suppressing visual feedback in written composition: Effects on processing demands and coordination of the writing processes , 2002 .

[4]  Jana Holsanova,et al.  Picture viewing and picture description , 2003 .

[5]  B. A. Levy.  Proofreading familiar text: Constraints on visual processing , 1983, Memory & cognition.

[6]  C. Michael Levy,et al.  Computer-aided protocol analysis of writing processes , 1994 .

[7]  Victoria Johansson,et al.  Looking at the keyboard or the monitor: relationship with text production processes , 2010 .

[8]  R. Eklund.  Disfluency in Swedish human–human and human–machine travel booking dialogues , 2004 .

[9]  Jukka Hyönä,et al.  Perspective-driven text comprehension , 2008 .

[10]  Mariëlle Leijten,et al.  Writing and Speech Recognition : Observing Error Correction Strategies of Professional Writers , 2007 .

[11]  K. Rayner,et al.  Eye movements during reading: some current controversies , 2001, Trends in Cognitive Sciences.

[12]  Denis Alamargot,et al.  Through the Models of Writing , 2001 .

[13]  R. T. Kellogg.  Effects of topic knowledge on the allocation of processing time and cognitive effort to writing processes , 1987, Memory & cognition.

[14]  L. Chanquoy,et al.  Temporal management of short text writing by children and adults , 1990 .

[15]  A. Gordon,et al.  Eye Movements and Eye-Hand Coordination During Typing , 1997 .

[16]  R. L. C. Butsch.  Eye movements and the eye-hand span in typewriting. , 1932 .

[17]  Alexander Pollatsek,et al.  Eye Movements During Reading , 2008 .

[18]  Christine Ros,et al.  Eye and Pen: A new device for studying reading during writing , 2006, Behavior research methods.

[19]  M. J. Power,et al.  A technique for measuring processing load during speech production , 1986 .

[20]  Carolyn Plumb,et al.  Text revision: Detection and correction of errors. , 1994 .

[21]  S. Liversedge,et al.  Saccadic eye movements and cognition , 2000, Trends in Cognitive Sciences.

[22]  Sharon Pianko,et al.  A Description of the Composing Processes of College Freshman Writers. , 1979 .

[23]  Lucile Chanquoy,et al.  Revision : cognitive and instructional processes , 2004 .

[24]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[25]  J. Hayes.  A new framework for understanding cognition and affect in writing. , 1996 .

[26]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[27]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[28]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[29]  Kristyan Spelman Miller.  Academic writers on-line: investigating pausing in the production of text , 2000 .

[30]  W. Levelt.  Speaking: From Intention to Articulation , 1990 .

[31]  Erik D. Reichle,et al.  The E-Z Reader model of eye-movement control in reading: Comparisons to other models , 2003, Behavioral and Brain Sciences.

[32]  J. Schilperoord,et al.  How Hierarchical Text Structure Affects Retrieval Processes : Implications of Pause and Text Analysis , 1999 .

[33]  B. Rapp,et al.  Discreteness and interactivity in spoken word production. , 2000, Psychological review.

[34]  J. Hayes,et al.  The Pregnant Pause: An Inquiry into the Nature of Planning. , 1981 .

[35]  Victoria Johansson,et al.  Perceiving and Producing the Frog Story , 2004 .

[36]  Gert Rijlaarsdam,et al.  Chapter 9: The Dynamics of Idea Generation During Writing: An Online Study , 2007 .

[37]  Mark Torrance,et al.  The Teachability and Effectiveness of Cognitive Self-Regulation in Sixth-Grade Writers. , 2007 .

[38]  H. W. Dechert,et al.  Temporal variables in first and second language speech production , 1980 .

[39]  Victoria Johansson,et al.  Combining Keystroke Logging with Eye Tracking , 2006 .

[40]  H. H. Clark,et al.  Using uh and um in spontaneous speaking , 2002, Cognition.

[41]  C. Crain-Thoreson,et al.  Windows on comprehension: Reading comprehension processes as revealed by two think-aloud procedures. , 1997 .

[42]  Joost Schilperoord,et al.  It's about time: Temporal aspects of cognitive processes in text production , 1996 .

[43]  Martin Chodorow,et al.  Does the Effect of Familiarity on Proofreading Change With Encoding Task and Time? , 2006, The Journal of general psychology.

[44]  Joost Schilperoord,et al.  On the Cognitive Status of Pauses in Discourse Production , 2002 .

[45]  L. Wheeldon,et al.  Planning scope in spoken sentence production: the role of grammatical units. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[46]  An application of multilevel analysis on writing process data , 1996 .

[47]  Hayes.  identifying the organization of wi iiing processes , 1980 .

[48]  F. Goldman-Eisler.  Psycholinguistics: Experiments in spontaneous speech , 1968 .

[49]  Åsa Wengelin,et al.  Examining Pauses in Writing: Theory, Methods and Empirical Data , 2006, Computer Key-Stroke Logging and Writing.

[50]  Debra Myhill,et al.  More Than Just Error Correction , 2007 .

[51]  A. Postma.  Detection of errors during speech production: a review of speech monitoring models , 2000, Cognition.

[52]  Robert F. Lorch,et al.  Understanding Reading Comprehension: Current and Future Contributions of Cognitive Science , 1997 .

[53]  Eva Lindgren,et al.  Stimulated Recall as a Trigger for Increasing Noticing and Language Awareness in the L2 Writing Classroom: A Case Study of Two Young Female Writers , 2003 .

[54]  C. Michael Levy,et al.  Contemporary tools and techniques for studying writing , 2002 .

[55]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[56]  Ann Matsuhashi,et al.  Writing in real time : modelling production processes , 1988 .

[57]  T. Sanders,et al.  Identifying Writing Strategies Through Text Analysis , 1999 .

[58]  Jocelyne Bisaillon,et al.  Professional Editing Strategies Used by Six Editors , 2007 .

[59]  Huub van den Bergh,et al.  Effects of thinking aloud on writing processes , 1996 .

[60]  Luuk Van Waes,et al.  Inputlog: New Perspectives on the Logging of On-Line Writing Processes in a Windows Environment , 2006, Computer Key-Stroke Logging and Writing.

[61]  Ludo Verhoeven,et al.  Cross-linguistic perspectives on the development of text-production abilities: Speech and writing , 2002 .

[62]  J.-N. Foulin.  To what extent does pause location predict pause duration in adults' and children's writing ? , 1998 .