Annotating Sentiment and Irony in the Online Italian Political Debate on #labuonascuola

In this paper we present the TWitterBuonaScuola corpus (TW-BS), a novel Italian linguistic resource for Sentiment Analysis, developed with the main aim of analyzing the online debate on the controversial Italian political reform “Buona Scuola” (Good school), aimed at reorganizing the national educational and training systems. We describe the methodologies applied in the collection and annotation of data. The collection has been driven by the detection of the hashtags mainly used by the participants to the debate, while the annotation has been focused on sentiment polarity and irony, but also extended to mark the aspects of the reform that were mainly discussed in the debate. An in-depth study of the disagreement among annotators is included. We describe the collection and annotation stages, and the in-depth analysis of disagreement made with Crowdflower, a crowdsourcing annotation platform.

[1]  Malvina Nissim,et al.  Overview of the Evalita 2014 SENTIment POLarity Classification Task , 2014 .

[2]  Cristina Bosco,et al.  Developing Corpora for Sentiment Analysis: The Case of Irony and Senti-TUT , 2013, IEEE Intelligent Systems.

[3]  Huan Liu,et al.  Identifying Users with Opposing Opinions in Twitter Debates , 2014, SBP.

[4]  Tommaso Caselli,et al.  State of the Art Language Technologies for Italian: The EVALITA 2014 Perspective , 2015, Intelligenza Artificiale.

[5]  Cristina Bosco,et al.  Debate on political reforms in Twitter: A hashtag-driven analysis of political polarization , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[6]  Ines Gloeckner,et al.  Relevance Communication And Cognition , 2016 .

[7]  Paolo Rosso,et al.  On the difficulty of automatically detecting irony: beyond a simple case of negation , 2014, Knowledge and Information Systems.

[8]  Cristina Bosco,et al.  Detecting Happiness in Italian Tweets: Towards an Evaluation Dataset for Sentiment Analysis in Felicittà , 2014 .

[9]  Malvina Nissim,et al.  Sentiment analysis on Italian tweets , 2013, WASSA@NAACL-HLT.

[10]  Deirdre Wilson,et al.  The pragmatics of verbal irony: Echo or pretence? , 2006 .

[11]  Stefano M. Iacus,et al.  Social media e sentiment analysis : l'evoluzione dei fenomeni sociali attraverso la rete , 2014 .

[12]  Michelangelo Conoscenti.  The Reframer: An Analysis of Barack Obama’s Political Discourse (2004-2010) , 2011 .

[13]  Cristina Bosco,et al.  Analyzing and annotating for sentiment analysis the socio-political debate on #labuonascuola , 2015 .

[14]  Jacob Ratkiewicz,et al.  Predicting the Political Alignment of Twitter Users , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[15]  Diana Maynard,et al.  Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis. , 2014, LREC.

[16]  Diana Maynard,et al.  Automatic Detection of Political Opinions in Tweets , 2011, #MSM.

[17]  Paolo Rosso,et al.  SemEval-2015 Task 11: Sentiment Analysis of Figurative Language in Twitter , 2015, *SEMEVAL.

[18]  C. Bosco,et al.  Building a Corpus on a Debate on Political Reform in Twitter , 2015 .

[19]  Nathalie Aussenac-Gilles,et al.  Towards a Contextual Pragmatic Model to Detect Irony in Tweets , 2015, ACL.

[20]  Johan Bos,et al.  Predicting the 2011 Dutch Senate Election Results with Twitter , 2012 .

[21]  Detlef Schoder,et al.  Web Science 2.0: Identifying Trends through Semantic Social Network Analysis , 2008, 2009 International Conference on Computational Science and Engineering.

[22]  Isabell M. Welpe,et al.  Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment , 2010, ICWSM.

[23]  Paolo Rosso,et al.  A multidimensional approach for detecting irony in Twitter , 2013, Lang. Resour. Evaluation.

[24]  Lois Ann Scheidt,et al.  It’s Complicated: The Social Lives of Networked Teens , 2015, New Media Soc..