Evaluating Sentiment Analysis Systems in Russian

In this paper we describe our experience in conducting the first open sentiment analysis evaluations in Russian in 2011-2012. These initiatives took part within Russian Information Retrieval Seminar (ROMIP), which is an annual TREC-like competition in Russian. Several test and train collections were created for such tasks as sentiment classification in blogs and newswire, opinion retrieval. The paper describes the state of the art in sentiment analysis in Russian, collection characteristics, track tasks and evaluation metrics.

[1]  Peng Jin,et al.  SemEval-2010 task 18: disambiguating sentiment ambiguous adjectives , 2010, SemEval@ACL.

[2]  Roser Morante,et al.  *SEM 2012 Shared Task: Resolving the Scope and Focus of Negation , 2012, *SEMEVAL.

[3]  Hsin-Hsi Chen,et al.  Overview of Opinion Analysis Pilot Task at NTCIR-6 , 2007, NTCIR.

[4]  Josef Steinberger,et al.  Creating Sentiment Dictionaries via Triangulation , 2011, Decis. Support Syst..

[5]  Н.В. Лукашевич,et al.  Sentiment analysis track at ROMIP 2011 , 2012 .

[6]  Rada Mihalcea,et al.  Learning Multilingual Subjective Language via Cross-Lingual Projections , 2007, ACL.

[7]  Language independent approach to sentiment anaLysis ( Limsi participation in romip ’ 11 ) , 2012 .

[8]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[9]  Natalia V. Loukachevitch,et al.  Extraction of Russian Sentiment Lexicon for Product Meta-Domain , 2012, COLING.

[10]  Julio Gonzalo,et al.  Overview of RepLab 2012: Evaluating Online Reputation Management Systems , 2012, CLEF.

[11]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[12]  K. Bretonnel Cohen,et al.  Sentiment Analysis of Suicide Notes: A Shared Task , 2012, Biomedical informatics insights.

[13]  Verónica Pérez-Rosas,et al.  Learning Sentiment Lexicons in Spanish , 2012, LREC.

[14]  Iadh Ounis,et al.  Overview of the TREC 2008 Blog Track , 2008, TREC.

[15]  Gerhard Weikum,et al.  PolariCQ: polarity classification of political quotations , 2012, CIKM '12.

[16]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[17]  Iadh Ounis,et al.  Overview of the TREC 2010 Blog Track , 2010, TREC.

[18]  Anonymous Authors,et al.  Modeling Polarizing Topics: When Do Different Political Communities Respond Differently to the Same News? , 2012 .

[19]  John Carroll,et al.  Comparable English-Russian book review corpora for sentiment analysis , 2010 .

[20]  TaboadaMaite,et al.  Lexicon-based methods for sentiment analysis , 2011 .

[21]  Craig MacDonald,et al.  Overview of the TREC 2007 Blog Track , 2007, TREC.

[22]  Natalia V. Loukachevitch,et al.  Sentiment Analysis Track at ROMIP-2012 , 2013 .

[23]  Craig MacDonald,et al.  Overview of the TREC 2009 Blog Track , 2009, TREC.

[24]  Craig MacDonald,et al.  Overview of the TREC 2006 Blog Track , 2006, TREC.

[25]  TesTing rules for a senTimenT analysis sysTem , 2013 .

[26]  Muhammad Abdul-Mageed,et al.  Subjectivity and Sentiment Analysis of Modern Standard Arabic , 2011, ACL.