Automated Essay Scoring

The impacts of computers on writing have been widely studied for three decades. Even basic computers functions, i.e. word processing, have been of great assistance to writers in modifying their essays. The research on Automated Essay Scoring (AES) has revealed that computers have the capacity to function as a more effective cognitive tool (Attali, 2004). AES is defined as the computer technology that evaluates and scores the written prose (Shermis & Barrera, 2002; Shermis & Burstein, 2003; Shermis, Raymat, & Barrera, 2003). Revision and feedback are essential aspects of the writing process. Students need to receive feedback in order to increase their writing quality. However, responding to student papers can be a burden for teachers. Particularly if they have large number of students and if they assign frequent writing assignments, providing individual feedback to student essays might be quite time consuming. AES systems can be very useful because they can provide the student with a score as well as feedback within seconds (Page, 2003). Four types of AES systems, which are widely used by testing companies, universities, and public schools: Project Essay Grader (PEG), Intelligent Essay Assessor (IEA), E-rater, and IntelliMetric. AES is a developing technology. Many AES systems are used to overcome time, cost, and generalizability issues in writing assessment. The accuracy and reliability of these systems have been proven to be high. The search for excellence in machine scoring of essays is continuing and numerous studies are being conducted to improve the effectiveness of the AES systems.

[1]  Martin Chodorow,et al.  Beyond Essay Length: Evaluating e-rater[R]'s Performance on TOEFL[R] Essays. Research Reports. Report 73. RR-04-04. , 2004 .

[2]  Peter W. Foltz,et al.  Automated Essay Scoring: Applications to Educational Technology , 1999 .

[3]  Gregory K. W. K. Chung,et al.  Methodological Approaches to Online Scoring of Essays. , 1997 .

[4]  Karen Kukich,et al.  Beyond Automated Essay Scoring , 2000 .

[5]  Jill Burstein,et al.  The E-rater® scoring engine: Automated essay scoring with natural language processing. , 2003 .

[6]  Yigal Attali,et al.  Exploring the Feedback and Revision Features of Criterion , 2004 .

[7]  Fiona Hyland,et al.  The impact of teacher written feedback on individual writers , 1998 .

[8]  Lawrence M. Rudner,et al.  An Overview of Three Approaches to Scoring Written Essays by Computer. ERIC Digest. , 2001 .

[9]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[10]  Daniel Marcu,et al.  Benefits of Modularity in an Automated Essay Scoring System , 2000, COLING 2000.

[11]  M. Chodorow,et al.  BEYOND ESSAY LENGTH: EVALUATING E-RATER®'S PERFORMANCE ON TOEFL® ESSAYS , 2004 .

[12]  P. Nichols Evidence for the Interpretation and Use of Scores from an Automated Essay Scorer , 2004 .

[13]  Claudia Leacock,et al.  Automated evaluation of essays and short answers , 2001 .

[14]  Marti A. Hearst The debate on automated essay grading , 2000 .

[15]  Mark D. Shermis,et al.  Exit Assessments: Evaluating Writing Ability through Automated Essay Scoring , 2002 .

[16]  Bob Rehder,et al.  How Well Can Passage Meaning be Derived without Using Word Order? A Comparison of Latent Semantic Analysis and Humans , 1997 .

[17]  Martin Chodorow,et al.  Automated Essay Scoring for Nonnative English Speakers , 1999 .

[18]  Mark D. Shermis,et al.  Assessing Writing through the Curriculum with Automated Essay Scoring. , 2003 .

[19]  Jill Burstein,et al.  Automated Essay Scoring : A Cross-disciplinary Perspective , 2003 .

[20]  Abdel-Badeeh M. Salem The Potential Role of Artificial Intelligence Technology in Education. , 2000 .

[21]  Martin Chodorow,et al.  CriterionSM Online Essay Evaluation: An Application for Automated Evaluation of Student Essays , 2003, IAAI.

[22]  E. B. Page Project Essay Grade: PEG. , 2003 .

[23]  Peter W. Foltz,et al.  The Debate on Automated Essay Grading , 2000, IEEE Intell. Syst..

[24]  Benoît Lemaire,et al.  A System to Assess the Semantic Content of Student Essays , 2001 .