SEDNN: Shared and enhanced deep neural network model for cross-prompt automated essay scoring

Abstract Most existing studies on Automated Essay Scoring (AES) focused on a single prompt, and only a few studies have addressed the problem of cross-prompt AES. This paper addresses the key question of how to extract more transferable rating knowledge from multiple source prompts. Different from the common approach which extracts an invariant part among all prompts, we propose to fuse all the data from the source prompts and to extract their shared knowledge with the target prompt, including prompt-independent features and some prompt-dependent features. We show that more rating knowledge can be extracted and transferred to the target prompt using this strategy. The transferred model is then used to generate a set of pseudo training data on the target prompt, with which another enhanced model incorporating more prompt-dependent features is built for the target prompt. Experiments show that this enhanced model can further improve the final performance and it outperforms the state-of-the-art methods on cross-prompt AES.

[1]  Sinno Jialin Pan,et al.  Transition-based Adversarial Network for Cross-lingual Aspect Extraction , 2018, IJCAI.

[2]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[3]  Ben He,et al.  Automated Essay Scoring by Maximizing Human-Machine Agreement , 2013, EMNLP.

[4]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[5]  Yue Zhang,et al.  Automatic Features for Essay Scoring – An Empirical Study , 2016, EMNLP.

[6]  Chien-Liang Liu,et al.  An Unsupervised Automated Essay Scoring System , 2010, IEEE Intelligent Systems.

[7]  Sunil Kumar Kopparapu,et al.  An unsupervised approach to automated selection of good essays , 2011, 2011 IEEE Recent Advances in Intelligent Computational Systems.

[8]  Helen Yannakoudakis,et al.  Unsupervised Modeling of Topical Relevance in L2 Learner Text , 2016, BEA@NAACL-HLT.

[9]  Helen Yannakoudakis,et al.  Automatic Text Scoring Using Neural Networks , 2016, ACL.

[10]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[11]  Jill Burstein,et al.  Automated Essay Scoring : A Cross-disciplinary Perspective , 2003 .

[12]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[13]  Hwee Tou Ng,et al.  A Neural Approach to Automated Essay Scoring , 2016, EMNLP.

[14]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[15]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[16]  Xuanjing Huang,et al.  Cross-Domain Sentiment Classification with Target Domain Specific Information , 2018, ACL.

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Xia Li,et al.  Coherence-Based Automated Essay Scoring Using Self-attention , 2018, CCL.

[19]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[20]  Ben He,et al.  TDNN: A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring , 2018, ACL.

[21]  Leah S. Larkey,et al.  Automatic essay grading using text categorization techniques , 1998, SIGIR '98.

[22]  Hwee Tou Ng,et al.  Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression , 2015, EMNLP.

[23]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[25]  Lawrence M. Rudner,et al.  Automated Essay Scoring Using Bayes' Theorem , 2002 .

[26]  Siu Cheung Hui,et al.  SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring , 2017, AAAI.

[27]  Timothy Baldwin,et al.  What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training , 2018, NAACL.

[28]  Haoran Zhang,et al.  Co-Attention Based Neural Network for Source-Dependent Essay Scoring , 2018, BEA@NAACL-HLT.

[29]  Yue Zhang,et al.  Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring , 2017, CoNLL.

[30]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[31]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[32]  Neil T. Heffernan,et al.  A Memory-Augmented Neural Model for Automated Grading , 2017, L@S.

[33]  Laura K. Allen,et al.  A Hierarchical Classification Approach to Automated Essay Scoring. , 2015 .

[34]  Ted Briscoe,et al.  Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input , 2018, NAACL.

[35]  William Wresch,et al.  The Imminence of Grading Essays by Computer-25 Years Later , 1993 .

[36]  Xuanjing Huang,et al.  Automatic Essay Scoring Incorporating Rating Schema via Reinforcement Learning , 2018, EMNLP.

[37]  Helen Yannakoudakis,et al.  A New Dataset and Method for Automatically Grading ESOL Texts , 2011, ACL.