A Framework for Implementing Automated Scoring

Unpublished Work Copyright © 2009 by Educational Testing Service. All Rights Reserved. These materials are an unpublished, proprietary work of ETS. Any limited distribution shall not constitute publication. This work may not be reproduced or distributed to third parties without ETS's prior written consent. Submit all requests through www.ets.org/legal/index.html. Educational Testing Service, ETS, the ETS logo, and Listening. Learning. Leading. are registered trademarks of Educational Testing Service (ETS).

[1]  Brent Bridgeman,et al.  Considering Fairness and Validity in Evaluating Automated Scoring , 2009 .

[2]  E. B. Page,et al.  The use of the computer in analyzing student essays , 1968 .

[3]  Tom Mitchell,et al.  Towards robust computerised marking of free-text responses , 2002 .

[4]  Ruth Culham,et al.  6 + 1 Traits of Writing: The Complete Guide for the Primary Grades , 2005 .

[5]  Stephen B. Dunbar,et al.  Complex, Performance-Based Assessment: Expectations and Validation Criteria , 1991 .

[6]  Lawrence M. Rudner,et al.  An Evaluation of IntelliMetric™ Essay Scoring System , 2006 .

[7]  Xiaoming Xi,et al.  Automated Scoring of Spontaneous Speech Using SpeechRater? v1.0. Research Report. ETS RR-08-62. , 2008 .

[8]  Robert J. Mislevy,et al.  Automated scoring of complex tasks in computer-based testing , 2006 .

[9]  Stephen G. Pulman,et al.  Information Extraction and Machine Learning: Auto-Marking Short Free Text Responses to Science Questions , 2005, AIED.

[10]  Martin Chodorow,et al.  C-rater: Automated Scoring of Short-Answer Questions , 2003, Comput. Humanit..

[11]  Sylvain Chevalier.  Speech interaction with Saybot, a CALL software to help Chinese learners of English , 2007, SLaTE.

[12]  Ross E. Traub,et al.  On the Equivalence of Constructed- Response and Multiple-Choice Tests , 1977 .

[13]  Thomas Risse.  Testing and Assessing Mathematical Skills by a Script Based System , 2007 .

[14]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[15]  Jill Burstein,et al.  The E-rater® scoring engine: Automated essay scoring with natural language processing. , 2003 .

[16]  Standards for Education and Training in Psychological Assessment: Position of the Society for Personality Assessment , 2006, Journal of personality assessment.

[17]  Irvin R. Katz,et al.  Information and Communication Technology (ICT) Literacy: Integration and Assessment in Higher Education , 2007 .

[18]  Mary McGee Wood,et al.  A human-computer collaborative approach to the marking of free text answers , 2004 .

[19]  David M. Williamson,et al.  "Mental Model" Comparison of Automated and Human Scoring , 1999 .

[20]  Jill Burstein,et al.  Automated Essay Scoring : A Cross-disciplinary Perspective , 2003 .

[21]  Derrick Higgins,et al.  Evaluating the Construct-Coverage of the e-rater[R] Scoring Engine. Research Report. ETS RR-09-01. , 2009 .

[22]  Randy Elliot Bennett,et al.  VALIDATION AND EXTENSION OF THE MATHEMATICAL EXPRESSION RESPONSE TYPE: APPLICATIONS OF SCHEMA THEORY TO AUTOMATIC SCORING AND ITEM GENERATION IN MATHEMATICS , 1997 .

[23]  David B. Pisoni,et al.  Two Experiments on Automatic Scoring of Spoken Language Proficiency , 2000 .

[24]  Howard Wainer,et al.  ON THE RELATIVE VALUE OF MULTIPLE‐CHOICE, CONSTRUCTED‐RESPONSE, AND EXAMINEE‐SELECTED ITEMS ON TWO ACHIEVEMENT TESTS1 , 1993 .

[25]  David Callear,et al.  CAA of Short Non-MCQ Answers , 2001 .

[26]  William Wresch,et al.  The Imminence of Grading Essays by Computer-25 Years Later , 1993 .

[27]  E. B. Page.  Project Essay Grade: PEG. , 2003 .

[28]  Kristin Precoda,et al.  The SRI EduSpeak System: Recognition and Pronunciation Scoring for Language Learning , 2007 .

[29]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[30]  Klaus Zechner,et al.  Towards Automatic Scoring of Non-Native Spontaneous Speech , 2006, HLT-NAACL.