Open evaluation of sentiment-analysis systems based on the material of the Russian language

This paper describes the experience from carrying out open evaluation of the methods for the sentiment analysis in Russian based on the ROMIP seminar in 2011–2012. Several training collections, which are now in free access, have been created during the conduction of the track. This paper reviews the current state of the art in processing subjective texts in Russian and describes the major tasks and characteristics of collections, as well as quality-assessment measures.

[1]  Peng Jin,et al.  SemEval-2010 task 18: disambiguating sentiment ambiguous adjectives , 2010, SemEval@ACL.

[2]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[3]  Brendan T. O'Connor,et al.  Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics , 2011 .

[4]  Hsin-Hsi Chen,et al.  Overview of Opinion Analysis Pilot Task at NTCIR-6 , 2007, NTCIR.

[5]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[6]  Julio Gonzalo,et al.  Overview of RepLab 2012: Evaluating Online Reputation Management Systems , 2012, CLEF.

[7]  Patricio Martínez-Barco,et al.  Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis , 2011 .

[8]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[9]  Muhammad Abdul-Mageed,et al.  Subjectivity and Sentiment Analysis of Modern Standard Arabic , 2011, ACL.

[10]  Н.В. Лукашевич,et al.  Sentiment analysis track at ROMIP 2011 , 2012 .

[11]  Qiang Yang,et al.  Cross-domain sentiment classification via spectral feature alignment , 2010, WWW '10.

[12]  Verónica Pérez-Rosas,et al.  Learning Sentiment Lexicons in Spanish , 2012, LREC.

[13]  TesTing rules for a senTimenT analysis sysTem , 2013 .

[14]  Craig MacDonald,et al.  Overview of the TREC 2009 Blog Track , 2009, TREC.

[15]  Iadh Ounis,et al.  Overview of the TREC-2009 Blog Track | NIST , 2008 .

[16]  O Pestov,et al.  ReseaRch of lexical appRoach and machine leaRning methods foR sentiment analysis , 2013 .

[17]  Iadh Ounis,et al.  Overview of the TREC 2010 Blog Track , 2010, TREC.

[18]  K. Bretonnel Cohen,et al.  Sentiment Analysis of Suicide Notes: A Shared Task , 2012, Biomedical informatics insights.

[19]  Craig MacDonald,et al.  Overview of the TREC 2006 Blog Track , 2006, TREC.

[20]  Claire Cardie,et al.  Adapting a Polarity Lexicon using Integer Linear Programming for Domain-Specific Sentiment Classification , 2009, EMNLP.

[21]  John Carroll,et al.  Comparable English-Russian book review corpora for sentiment analysis , 2010 .

[22]  Iadh Ounis,et al.  Overview of the TREC 2008 Blog Track , 2008, TREC.

[23]  Roser Morante,et al.  *SEM 2012 Shared Task: Resolving the Scope and Focus of Negation , 2012, *SEMEVAL.

[24]  Rada Mihalcea,et al.  Learning Multilingual Subjective Language via Cross-Lingual Projections , 2007, ACL.

[25]  Natalia V. Loukachevitch,et al.  Sentiment Analysis Track at ROMIP-2012 , 2013 .

[26]  William W. Cohen,et al.  Modeling Polarizing Topics: When Do Different Political Communities Respond Differently to the Same News? , 2012, ICWSM.

[27]  Josef Steinberger,et al.  Creating Sentiment Dictionaries via Triangulation , 2011, Decis. Support Syst..

[28]  Natalia V. Loukachevitch,et al.  Extraction of Russian Sentiment Lexicon for Product Meta-Domain , 2012, COLING.

[29]  Craig MacDonald,et al.  Overview of the TREC 2007 Blog Track , 2007, TREC.

[30]  Carlo Strapparava,et al.  Proceedings of the 5th International Workshop on Semantic Evaluation , 2010 .

[31]  TaboadaMaite,et al.  Lexicon-based methods for sentiment analysis , 2011 .

[32]  Jonathan Ginzburg,et al.  Proceedings of COLING 2004 , 2004 .

[33]  Gerhard Weikum,et al.  PolariCQ: polarity classification of political quotations , 2012, CIKM '12.