Task-Independent Features for Automated Essay Grading

Automated scoring of student essays is increasingly used to reduce manual grading effort. State-of-the-art approaches use supervised machine learning which makes it complicated to transfer a system trained on one task to another. We investigate which currently used features are task-independent and evaluate their transferability on English and German datasets. We find that, by using our task-independent feature set, models transfer better between tasks. We also find that the transfer works even better between tasks of the same type.

[1]  Jean-Marc Dewaele,et al.  Variation in the Contextuality of Language: An Empirical Measure , 2002 .

[2]  James L. Peterson,et al.  Computer-based readability indexes , 1982, ACM '82.

[3]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[4]  Salvatore Valenti,et al.  An Overview of Current Research on Automated Essay Grading , 2003, J. Inf. Technol. Educ..

[5]  R. Fisher FREQUENCY DISTRIBUTION OF THE VALUES OF THE CORRELATION COEFFIENTS IN SAMPLES FROM AN INDEFINITELY LARGE POPU;ATION , 1915 .

[6]  Oliver Ferschke,et al.  DKPro TC: A Java-based Framework for Supervised Learning Experiments on Textual Data , 2014, ACL.

[7]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[8]  Richard Taffler,et al.  Readability and Understandability: Different Measures of the Textual Complexity of Accounting Narrative , 1992 .

[9]  Sabine Bergler Conveying Attitude with Reported Speech , 2006, Computing Attitude and Affect in Text.

[10]  Manvi Mahana,et al.  Automated Essay Grading Using Machine Learning , 2012 .

[11]  Ben He,et al.  Automated Essay Scoring by Maximizing Human-Machine Agreement , 2013, EMNLP.

[12]  Claudia Leacock,et al.  Automated evaluation of essays and short answers , 2001 .

[13]  Peter W. Foltz,et al.  Automated Essay Scoring: Applications to Educational Technology , 1999 .

[14]  Wolfgang Lezius,et al.  TIGER: Linguistic Interpretation of a German Corpus , 2004 .

[15]  Semire Dikli,et al.  An Overview of Automated Scoring of Essays. , 2006 .

[16]  G. Harry McLaughlin,et al.  SMOG Grading - A New Readability Formula. , 1969 .

[17]  Martin Chodorow,et al.  Automated Scoring Using A Hybrid Feature Identification Technique , 1998, ACL.

[18]  Chi-Un Lei,et al.  Using Learning Analytics to Analyze Writing Skills of Students: A Case Study in a Technological Common Core Curriculum Course , 2014 .

[19]  Annelen Brunner,et al.  Automatic recognition of speech, thought, and writing representation in German narrative texts , 2013, Lit. Linguistic Comput..

[20]  Martin Chodorow,et al.  Automated Essay Scoring for Nonnative English Speakers , 1999 .

[21]  Jill Burstein,et al.  Automated Essay Scoring : A Cross-disciplinary Perspective , 2003 .

[22]  Ralf Krestel,et al.  Minding the Source: Automatic Tagging of Reported Speech in Newspaper Articles , 2008, LREC.

[23]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[24]  H. Breland,et al.  THE COLLEGE BOARD VOCABULARY STUDY , 1994 .

[25]  Robert Östling,et al.  Automated Essay Scoring for Swedish , 2013, BEA@NAACL-HLT.