Modularized and Attention-Based Recurrent Convolutional Neural Network for Automatic Academic Paper Aspect Scoring

Thousands of academic papers are submitted at top venues each year. Manual audits are time-consuming and laborious. And the result may be influenced by human factors. This paper investigates a modularized and attention-based recurrent convolutional network model to represent academic paper and predict aspect scores. This model treats input text as module-document hierarchies, uses attention pooling CNN and LSTM to represent text, and outputs prediction with a linear layer. Empirical results on PeerRead data show that this model give the best performance among the baseline models.

[1]  Lawrence M. Rudner,et al.  Automated Essay Scoring Using Bayes' Theorem , 2002 .

[2]  Yue Zhang,et al.  Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring , 2017, CoNLL.

[3]  Yue Zhang,et al.  Automatic Features for Essay Scoring – An Empirical Study , 2016, EMNLP.

[4]  Helen Yannakoudakis,et al.  A New Dataset and Method for Automatically Grading ESOL Texts , 2011, ACL.

[5]  Leah S. Larkey,et al.  Automatic essay grading using text categorization techniques , 1998, SIGIR '98.

[6]  Hwee Tou Ng,et al.  Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression , 2015, EMNLP.

[7]  Pali U. K. De Silva,et al.  Preserving the Quality of Scientific Research: Peer Review of Research Articles , 2017 .

[8]  Bhavana Dalvi,et al.  A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications , 2018, NAACL.

[9]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[10]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[11]  Ben He,et al.  Automated Essay Scoring by Maximizing Human-Machine Agreement , 2013, EMNLP.

[12]  John Langford,et al.  The arbitrariness of reviews, and advice for school administrators , 2015, Commun. ACM.

[13]  Hwee Tou Ng,et al.  A Neural Approach to Automated Essay Scoring , 2016, EMNLP.

[14]  Daniel Jurafsky,et al.  A Hierarchical Neural Autoencoder for Paragraphs and Documents , 2015, ACL.

[15]  Helen Yannakoudakis,et al.  Automatic Text Scoring Using Neural Networks , 2016, ACL.

[16]  Torsten Zesch,et al.  Task-Independent Features for Automated Essay Grading , 2015, BEA@NAACL-HLT.