Word Embedding for Response-To-Text Assessment of Evidence

Manually grading the Response to Text Assessment (RTA) is labor intensive. Therefore, an automatic method is being developed for scoring analytical writing when the RTA is administered in large numbers of classrooms. Our long-term goal is to also use this scoring method to provide formative feedback to students and teachers about students' writing quality. As a first step towards this goal, interpretable features for automatically scoring the evidence rubric of the RTA have been developed. In this paper, we present a simple but promising method for improving evidence scoring by employing the word embedding model. We evaluate our method on corpora of responses written by upper elementary students.

[1]  Martin Chodorow,et al.  Enriching Automated Essay Scoring Using Discourse Marking , 2001 .

[2]  Jill Burstein,et al.  Automated Essay Scoring : A Cross-disciplinary Perspective , 2003 .

[3]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[4]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[5]  Diane J. Litman,et al.  Assessing Students’ Use of Evidence and Organization in Response-to-Text Writing: Using Natural Language Processing for Rubric-Based Automated Scoring , 2017, International Journal of Artificial Intelligence in Education.

[6]  Diane J. Litman,et al.  Automatic Scoring of an Analytical Response-To-Text Assessment , 2014, Intelligent Tutoring Systems.

[7]  Klaus Zechner,et al.  Exploring Content Features for Automated Speech Scoring , 2012, HLT-NAACL.

[8]  Diane J. Litman,et al.  Ontology-Based Argument Mining and Automatic Essay Scoring , 2014, ArgMining@ACL.

[9]  Elaine Wang,et al.  Assessing Students' Skills at Writing Analytically in Response to Texts , 2013, The Elementary School Journal.

[10]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[11]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[12]  Ronan Cummins,et al.  Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays , 2016, BEA@NAACL-HLT.

[13]  M. de Rijke,et al.  Short Text Similarity with Word Embeddings , 2015, CIKM.

[14]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[15]  Stephen Clark,et al.  Specializing Word Embeddings for Similarity or Relatedness , 2015, EMNLP.

[16]  Annie Louis,et al.  Off-topic essay detection using short prompt texts , 2010 .

[17]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.