Scoring Persuasive Essays Using Opinions and their Targets

In this work, we investigate whether the analysis of opinion expressions can help in scoring persuasive essays. For this, we develop systems that predict holistic essay scores based on features extracted from opinion expressions, topical elements, and their combinations. Experiments on test taker essays show that essay scores produced using opinion features are indeed correlated with human scores. Moreover, we find that combining opinions with their targets (what the opinions are about) produces the best result when compared to using only opinions or only topics.

[1]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[2]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[3]  Chun Chen,et al.  Opinion Word Expansion and Target Extraction through Double Propagation , 2011, CL.

[4]  Semire Dikli,et al.  An Overview of Automated Scoring of Essays. , 2006 .

[5]  Anat Ben-Simon,et al.  Toward More Substantively Meaningful Automated Essay Scoring , 2007 .

[6]  Jun Zhao,et al.  Extracting Opinion Targets and Opinion Words from Online Reviews with Graph Co-ranking , 2014, ACL.

[7]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[8]  Laura L. Aull,et al.  Linguistic Markers of Stance in Early and Advanced Academic Writing , 2014 .

[9]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[10]  Eduard H. Hovy,et al.  The Automated Acquisition of Topic Signatures for Text Summarization , 2000, COLING.

[11]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[12]  David M. Williamson,et al.  A Framework for Evaluation and Use of Automated Scoring , 2012 .

[13]  Theresa Wilson.  Fine-grained subjectivity and sentiment analysis: recognizing the intensity, polarity, and attitudes of private states , 2008 .

[14]  Ming Zhou,et al.  Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification , 2014, ACL.

[15]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.

[16]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[17]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[18]  William Wresch,et al.  The Imminence of Grading Essays by Computer-25 Years Later , 1993 .

[19]  Paul Deane,et al.  On the relation between automated essay scoring and modern views of the writing construct , 2013 .

[20]  Lawrence M. Rudner,et al.  Automated Essay Scoring Using Bayes' Theorem , 2002 .

[21]  M. Chodorow,et al.  BEYOND ESSAY LENGTH: EVALUATING E-RATER®'S PERFORMANCE ON TOEFL® ESSAYS , 2004 .

[22]  Swapna Somasundaran,et al.  Lexical Chaining for Measuring Discourse Coherence Quality in Test-taker Essays , 2014, COLING.

[23]  Swapna Somasundaran,et al.  Recognizing Stances in Ideological On-Line Debates , 2010, HLT-NAACL 2010.

[24]  Swapna Somasundaran,et al.  Recognizing Stances in Online Debates , 2009, ACL.

[25]  Jill Burstein,et al.  The E-rater® scoring engine: Automated essay scoring with natural language processing. , 2003 .

[26]  Peter W. Foltz,et al.  The intelligent essay assessor: Applications to educational technology , 1999 .

[27]  Nitin Madnani,et al.  Identifying High-Level Organizational Elements in Argumentative Discourse , 2012, NAACL.

[28]  Johan Bollen,et al.  Modeling Public Mood and Emotion: Twitter Sentiment and Socio-Economic Phenomena , 2009, ICWSM.

[29]  Beata Beigman Klebanov,et al.  Sentiment profiles of multiword expressions in test-taker essays: The case of noun-noun compounds , 2013, TSLP.

[30]  Martin Chodorow,et al.  Enriching Automated Essay Scoring Using Discourse Marking , 2001 .

[31]  Ignacio Vázquez Orta,et al.  Writing with conviction: the use of boosters in modelling persuasion in academic discourses , 2009 .

[32]  Swapna Somasundaran,et al.  Finding the Sources and Targets of Subjective Expressions , 2008, LREC.

[33]  Beata Beigman Klebanov,et al.  Word Association Profiles and their Use for Automated Scoring of Essays , 2013, ACL.

[34]  Brent Bridgeman,et al.  Comparison of Human and Machine Scoring of Essays: Differences by Gender, Ethnicity, and Country , 2012 .

[35]  Tiejun Zhao,et al.  Target-dependent Twitter Sentiment Classification , 2011, ACL.

[36]  Iryna Gurevych,et al.  Identifying Argumentative Discourse Structures in Persuasive Essays , 2014, EMNLP.

[37]  Bing Liu,et al.  Opinion observer: analyzing and comparing opinions on the Web , 2005, WWW '05.

[38]  Steven Skiena,et al.  Large-Scale Sentiment Analysis for News and Blogs (system demonstration) , 2007, ICWSM.

[39]  Owen Rambow,et al.  Sentiment Analysis of Twitter Data , 2011 .

[40]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[41]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[42]  Beata Beigman Klebanov,et al.  Using Pivot-Based Paraphrasing and Sentiment Profiles to Improve a Subjectivity Lexicon for Essay Data , 2013, Transactions of the Association for Computational Linguistics.

[43]  Rob Malouf,et al.  A Preliminary Investigation into Sentiment Analysis of Informal Political Discourse , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[44]  Dan Klein,et al.  An Empirical Investigation of Statistical Significance in NLP , 2012, EMNLP.