Reporting Writing Process Feedback in the Classroom. Using Keystroke Logging Data to Reflect on Writing Processes

Keystroke loggers facilitate researchers to collect fine-grained process data and offer support in analyzing these data. Keystroke logging has become popular in writing research, and study by study we are now paving the path to a better understanding of writing process data. However, few researchers have concentrated on how to bring keystroke logging to the classroom. Not because they are not convinced that writing development could benefit from a more process-oriented pedagogy, but because 'translating' complex and large data sets to an educational context is challenging. Therefore, we have developed a new function in Inputlog, specifically aiming to facilitate writing tutors in providing process feedback to their students. Based on an XMLlogfile, the so-called 'report' function automatically generates a pdf-file addressing different perspectives of the writing process: pausing, revision, source use, and fluency. These perspectives are reported either quantitatively or visually. Brief introductory texts explain the information presented. Inputlog provides a default feedback report, but users can also customize the report. This paper describes the process report and demonstrates the use of it in an intervention. We also present some additional pedagogical scenarios to actively use this type of feedback in writing classes.

[1]  Eva Lindgren,et al.  Stimulated Recall as a Trigger for Increasing Noticing and Language Awareness in the L2 Writing Classroom: A Case Study of Two Young Female Writers , 2003 .

[2]  Marie Stevenson,et al.  Supporting the Reflective Language Learner with Computer Keystroke Logging , 2008 .

[3]  D. Nicol,et al.  Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice , 2006 .

[4]  Mariëlle Leijten,et al.  MAPPING MASTER’S STUDENTS’ USE OF EXTERNAL SOURCES IN SOURCE-BASED WRITING IN L1 AND L2 , 2019, Studies in Second Language Acquisition.

[5]  Åsa Wengelin,et al.  Examining Pauses in Writing: Theory, Methods and Empirical Data , 2006, Computer Key-Stroke Logging and Writing.

[6]  Peggy A. Ertmer,et al.  Teacher beliefs and technology integration practices: A critical relationship , 2012, Comput. Educ..

[7]  Kirk P. H. Sullivan,et al.  Observing Writing , 2019 .

[8]  D. Laplane.  Thought and language. , 1992, Behavioural neurology.

[9]  Mireille Bétrancourt,et al.  'Digital authoring support for argumentative writing: what does it change? , 2020, Journal of Writing Research.

[10]  Robert Abbott,et al.  Effects of transcription ability and transcription mode on translation: , 2017, Journal of writing research.

[11]  Luuk Van Waes,et al.  Writing in the workplace: Constructing documents using multiple digital sources , 2014 .

[12]  Mo Zhang,et al.  Exploring the Feasibility of Using Writing Process Features to Assess Text Production Skills , 2015 .

[13]  S. Graham,et al.  An Examination of the Design Principles Underlying a Self-Regulated Strategy Development Study , 2018, Journal of Writing Research.

[14]  Cassie Berger,et al.  The Effect of a Word Processor as an Accommodation for Students with Learning Disabilities , 2013 .

[15]  Evgeny Chukharev-Hudilainen.  Empowering Automated Writing Evaluation with Keystroke Logging , 2019 .

[16]  Stephanie Link,et al.  Understanding Graduate Writers’ Interaction with and Impact of the Research Writing Tutor during Revision , 2020, Journal of Writing Research.

[17]  Saskia Brand-Gruwel,et al.  Electronic outlining as a writing strategy: Effects on students' writing products, mental effort and writing process , 2014, Comput. Educ..

[18]  Teresa Limpo,et al.  Relating beliefs in writing skill malleability to writing performance: The mediating role of achievement goals and self-efficacy , 2017 .

[19]  Evan F. Risko,et al.  Disfluency effects on lexical selection , 2017, Cognition.

[20]  M. Torrance,et al.  The online management of writing processes and their contribution to text quality in upper-primary students. , 2019, Psicothema.

[21]  Khaled Barkaoui,et al.  What and When Second-Language Learners Revise When Responding to Timed Writing Tasks on the Computer: The Roles of Task Type, Second Language Proficiency, and Keyboarding Skills. , 2016 .

[22]  Menno van Zaanen,et al.  Understanding the keystroke log: the effect of writing task on keystroke features , 2019, Reading and Writing.

[23]  Mariëlle Leijten,et al.  Keystroke Logging in Writing Research , 2013 .

[24]  K. Sullivan,et al.  Observing Writing and Website Browsing: Swedish Students Write L3 German , 2019, Observing Writing.

[25]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[26]  David Boud,et al.  The development of student feedback literacy: enabling uptake of feedback , 2018 .

[27]  Rianne Conijn,et al.  Human-Centered Design of a Dashboard on Students' Revisions During Writing , 2020, EC-TEL.

[28]  H. Van den Bergh,et al.  Effects of a Strategy-Focused Instructional Program on the Writing Quality of Upper Elementary Students in the Netherlands , 2018 .

[29]  Randy Elliot Bennett,et al.  Modeling Basic Writing Processes From Keystroke Logs , 2018, Journal of Educational Measurement.

[30]  Mirabelle Walker,et al.  An investigation into written comments on assignments: do students find them usable? , 2009 .

[31]  An application of multilevel analysis on writing process data , 1996 .

[32]  Veerle Baaijen,et al.  Aligning keystrokes with cognitive processes in writing , 2019 .

[33]  Karen Handley,et al.  From copying to learning: using exemplars to engage students with assessment criteria and feedback , 2011 .

[34]  Mark Torrance,et al.  COMBINED DEPLOYABLE KEYSTROKE LOGGING AND EYETRACKING FOR INVESTIGATING L2 WRITING FLUENCY , 2019, Studies in Second Language Acquisition.

[35]  Carl W. Swartz,et al.  Goals and Progress Feedback: Effects on Self-Efficacy and Writing Achievement , 1993 .

[36]  Eric Van Horenbeeck,et al.  Analysing Keystroke Logging Data from a Linguistic Perspective , 2019, Observing Writing.

[37]  Eva Lindgren.  The Uptake of Peer-Based Intervention in The Writing Classroom , 2005 .

[38]  Camilo Vieira,et al.  Visual learning analytics of educational data: A systematic literature review and research agenda , 2018, Comput. Educ..

[39]  Jim Ranalli,et al.  Exploring the potential of process-tracing technologies to support assessment for learning of L2 writing , 2018 .

[40]  Paul Deane,et al.  Prediction of Essay Scores From Writing Process and Product Features Using Data Mining Methods , 2019, Applied Measurement in Education.

[41]  Susan M. Gass,et al.  Second Language Research: Methodology and Design , 2021 .

[42]  Veerle M. Baaijen,et al.  Effects of writing beliefs and planning on writing performance , 2014 .

[43]  Veerle M. Baaijen,et al.  Discovery Through Writing: Relationships with Writing Processes and Text Quality , 2018 .

[44]  M. Braaksma,et al.  Observational learning and the effects of model-observer similarity. , 2002 .

[45]  Mariëlle Leijten,et al.  Typing competencies in Alzheimer's disease: An exploration of copy tasks , 2017, Comput. Hum. Behav..

[46]  Nigel Harwood,et al.  L2 writing in test and non-test situations: Process and product , 2015 .

[47]  Gert Rijlaarsdam,et al.  Writing process theory: A functional dynamic approach , 2006 .

[48]  Eva Lindgren,et al.  Supporting Learner Reflection in the Language Translation Class , 2009, Int. J. Inf. Commun. Technol. Hum. Dev..

[49]  Kathy A. Paulson Gjerde,et al.  The Impact of Process vs. Outcome Feedback on Student Performance and Perceptions. , 2017 .

[50]  Robert J. Mislevy,et al.  Automated Scoring with Validity in Mind , 2016 .

[51]  Steve Graham,et al.  A Revised Writer(s)-Within-Community Model of Writing , 2018, Educational Psychologist.

[52]  H. Bergh,et al.  Improving writing in primary schools through a comprehensive writing program , 2017 .

[53]  Marion Tillema,et al.  Writing in first and second language. Empirical studies on text quality and writing processes. , 2012 .

[54]  Linda Flower,et al.  The Dynamics of Composing : Making Plans and Juggling Constraints , 1980 .

[55]  R. Sternberg,et al.  The Power of Observational Learning Through Social Modeling , 2016 .

[56]  J. Aben,et al.  Feedback op de schrijfaanpak: de ontwikkeling van een schrijfprocesgericht feedbackrapport voor vwo-leerlingen , 2017 .

[57]  Simon Buckingham Shum,et al.  AcaWriter: A Learning Analytics Tool for Formative Feedback on Academic Writing , 2020, Journal of Writing Research.

[58]  Gert Rijlaarsdam,et al.  Strategy-focused writing instruction: just observing and reflecting on a model benefits 6th grade students , 2015 .

[59]  D. Sadler.  Formative assessment and the design of instructional systems , 1989 .

[60]  I. Schrijver,et al.  The impact of writing training on transediting in translation, analyzed from a product and process perspective , 2016 .

[61]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[62]  Åsa Wengelin,et al.  Keystroke logging in writing research : Analyzing online writing processes , 2015 .

[63]  G. Rijlaarsdam,et al.  Relations Between Writing Processes and Text Quality: When and How? , 1994 .

[64]  Avril Thomson,et al.  Rethinking feedback practices in higher education: a peer review perspective , 2014 .

[65]  Susan Armstrong,et al.  Constructive guidance and feedback for learning: the usefulness of exemplars, marking sheets and different types of feedback in a first year law subject , 2011 .

[66]  G. Rijlaarsdam,et al.  In search of an effective source use pattern for writing argumentative and informative synthesis texts , 2019, Reading and Writing.