Identifying Expressions of Emotion in Text

Finding emotions in text is an area of research with wide-ranging applications. We describe an emotion annotation task of identifying emotion category, emotion intensity and the words/phrases that indicate emotion in text. We introduce the annotation scheme and present results of an annotation agreement study on a corpus of blog posts. The average inter-annotator agreement on labeling a sentence as emotion or non-emotion was 0.76. The agreement on emotion categories was in the range 0.6 to 0.79; for emotion indicators, it was 0.66. Preliminary results of emotion classification experiments show the accuracy of 73.89%, significantly above the baseline.

[1]  Marshall S. Smith,et al.  The general inquirer: A computer approach to content analysis. , 1967 .

[2]  Gilad Mishne,et al.  Predicting Movie Sales from Blogger Sentiment , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[3]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[4]  Jeannett Martin,et al.  The Language of Evaluation: Appraisal in English , 2005 .

[5]  Henry Lieberman,et al.  A model of textual affect sensing using real-world knowledge , 2003, IUI '03.

[6]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[7]  Rebecca J. Passonneau,et al.  Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation , 2006, LREC.

[8]  Mitsuru Ishizuka,et al.  Analysis of affect expressed through the evolving language of online communication , 2007, IUI '07.

[9]  Casey Whitelaw.  Using Appraisal Taxonomies for Sentiment Analysis , 2005 .

[10]  Cecilia Ovesdotter Alm,et al.  Emotions from Text: Machine Learning for Text-based Emotion Prediction , 2005, HLT.

[11]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[12]  Hugo Liu,et al.  A Corpus-based Approach to Finding Happiness , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[13]  P. Ekman.  An argument for basic emotions , 1992 .

[14]  Philip J. Stone,et al.  Extracting Information. (Book Reviews: The General Inquirer. A Computer Approach to Content Analysis) , 1967 .

[15]  Jonathon Read,et al.  Recognising Affect in Text using Pointwise-Mutual Information , 2004 .

[16]  Jonathon Read,et al.  Annotating expressions of Appraisal in English , 2007, Language Resources and Evaluation.

[17]  Rebecca J. Passonneau.  Computing Reliability for Coreference Annotation , 2004, LREC.

[18]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[19]  Jan Svartvik,et al.  A __ comprehensive grammar of the English language , 1988 .

[20]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[21]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.