Checking It Twice: Does Adding Spelling and Grammar Checkers Improve Essay Quality in an Automated Writing Tutor?

This study investigated the effect of incorporating spelling and grammar checking tools within an automated writing tutoring system, Writing Pal. High school students (n = 119) wrote and revised six persuasive essays. After initial drafts, all students received formative feedback about writing strategies. Half of the participants were also given access to spelling and grammar checking tools during the writing and revision periods. Linear mixed effects models revealed that essay quality for students in both conditions improved from initial draft to revision in terms of all aspects except essay unity. The availability of spelling and grammar checking yielded added improvements from initial draft to revision for several aspects of essay quality (i.e., conclusion, organization, voice, grammar/mechanics, and word choice), but other aspects were unaffected (i.e., introduction, body, unity, and sentence structure). The availability of spelling and grammar checking tools had no effect on holistic essay scores. These results indicate that automated spelling and grammar feedback contribute to modest, incremental improvements in writing quality that may complement automated strategy feedback.

[1]  Composition Errors and Essay Examination Grades Re-Examined , 1967 .

[2]  J. Marshall,et al.  WRITING NEATNESS, COMPOSITION ERRORS, AND ESSAY GRADES , 1969 .

[3]  Jill Fitzgerald,et al.  Research on Revision in Writing , 1987 .

[4]  R. Cooter,et al.  Gates-MacGinitie Reading Tests, Third Edition (Assessment). , 1989 .

[5]  W. Blanton,et al.  Spelling Instruction and Achievement in Six Classrooms , 1995, The Elementary School Journal.

[6]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[7]  L. Figueredo,et al.  Didn't You Run the Spell Checker? Effects of Type of Spelling Error and Use of a Spell Checker on Perceptions of the Author , 2005 .

[8]  Steve Graham,et al.  Strategy Instruction and the Teaching of Writing: A Meta-Analysis. , 2006 .

[9]  S. Graham,et al.  A meta-analysis of writing instruction for adolescent students. , 2007 .

[10]  R. T. Kellogg,et al.  Improving the writing skills of college students , 2007, Psychonomic bulletin & review.

[11]  S. Graham,et al.  Primary Grade Writing Instruction: A National Survey. , 2008 .

[12]  Dorothy Fuller,et al.  My New Teaching Partner? Using the Grammar Checker in Writing Instruction. , 2008 .

[13]  Trude Heift,et al.  Learner responses to corrective feedback for spelling errors in CALL , 2008 .

[14]  Lindy Crawford,et al.  Analysis of Student Revisions on a State Writing Test , 2008 .

[15]  Sharlene A. Kiuhara,et al.  Teaching writing to high school students: A national survey , 2009 .

[16]  Zoran Bosnić,et al.  Extending applications using an advanced approach to DLL injection and API hooking , 2010 .

[17]  Marcin Milkowski,et al.  Developing an open‐source, rule‐based proofreading tool , 2010, Softw. Pract. Exp..

[18]  Danielle S. McNamara,et al.  The Writing-Pal tutoring system: Development and design , 2010 .

[19]  Arthur C. Graesser,et al.  The Writing-Pal: Natural Language Algorithms to Support Intelligent Tutoring on Writing Strategies , 2012 .

[20]  Stephen Graham,et al.  Word processing programs and weaker writers/readers: a meta-analysis of research findings , 2012 .

[21]  Jill Burstein,et al.  Handbook of Automated Essay Evaluation Current Applications and New Directions , 2018 .

[22]  Danielle S. McNamara,et al.  Feedback and Revising in an Intelligent Tutoring System for Writing Strategies , 2013, AIED.

[23]  Rod D. Roscoe,et al.  Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom , 2013 .

[24]  Danielle S. McNamara,et al.  Using Automatic Scoring Models to Detect Changes in Student Writing in an Intelligent Tutoring System , 2013, FLAIRS Conference.

[25]  Laura K. Allen,et al.  Reading comprehension components and their relation to writing , 2014, L’Année psychologique.

[26]  Danielle S. McNamara,et al.  The Importance of Grammar and Mechanics in Writing Assessment and Instruction: Evidence from Data Mining , 2014, EDM.

[27]  Steve Graham,et al.  Teaching writing to middle school students: a national survey , 2013, Reading and Writing.

[28]  T. Santangelo,et al.  Does spelling instruction make students better spellers, readers, and writers? A meta-analytic review , 2014 .

[29]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[30]  Laura K. Allen,et al.  The Writing Pal Intelligent Tutoring System: Usability Testing and Development , 2014 .

[31]  Laura K. Allen,et al.  L2 writing practice: Game enjoyment as a key to engagement , 2014 .

[32]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[33]  Aek Phakiti,et al.  The effects of computer-generated feedback on the quality of writing , 2014 .

[34]  Laura K. Allen,et al.  A Hierarchical Classification Approach to Automated Essay Scoring. , 2015 .

[35]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[36]  Laura K. Allen,et al.  Automated Detection of Essay Revising Patterns: Applications for Intelligent Feedback in a Writing Tutor. , 2015 .

[37]  S. Graham,et al.  Evidence-based practice and writing instruction , 2015 .

[38]  Laura K. Allen,et al.  {ENTER}ing the Time Series {SPACE}: Uncovering the Writing Process through Keystroke Analyses , 2016, EDM.

[39]  Julie E. Boland,et al.  If You’re House Is Still Available, Send Me an Email: Personality Influences Reactions to Written Errors in Email Messages , 2016, PloS one.

[40]  Edward E. Bolls.  Development and Design , 2016 .

[41]  Xiaolong Wang,et al.  Designing Academic Writing Analytics for Civil Law Student Self-Assessment , 2018, International Journal of Artificial Intelligence in Education.

[42]  Laura K. Allen,et al.  The Writing Pal , 2016 .

[43]  Michelle Rose Cavaleri,et al.  You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students , 2016 .

[44]  Laura K. Allen,et al.  Computer-Based Writing Instruction. , 2016 .

[45]  Joshua Wilson,et al.  Automated essay evaluation software in English Language Arts classrooms: Effects on teacher feedback, student motivation, and writing quality , 2016, Comput. Educ..

[46]  Danielle S. McNamara,et al.  Adaptive Educational Technologies for Literacy Instruction , 2016 .

[47]  Laura K. Allen,et al.  Keystroke Dynamics Predict Essay Quality , 2017, CogSci.

[48]  Rod D. Roscoe,et al.  College student perceptions of writing errors, text quality, and author characteristics , 2017 .

[49]  Danielle S. McNamara,et al.  Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System , 2018, International Journal of Artificial Intelligence in Education.

[50]  C. Palermo,et al.  Teacher implementation of Self-Regulated Strategy Development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school students , 2018 .

[51]  Antje Proske,et al.  Digital support for academic writing: A review of technologies and pedagogies , 2019, Comput. Educ..