Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ

In this paper we describe the main issues emerged within the application of a multi-layered scheme for the fine-grained annotation of irony (Karoui et al., 2017) on an Italian Twitter corpus, i.e. TWITTIRÒ, which is composed of about 1,500 tweets with various provenance. A discussion is proposed about the limits and advantages of the application of the scheme to Italian messages, supported by an analysis of the outcome of the annotation carried on by native Italian speakers in the development of the corpus. We present a quantitative and qualitative study both of the distribution of the labels for the different layers involved in the scheme which can shed some light on the process of human annotation for a validation of the annotation scheme on Italian irony-laden social media contents collected in the last years. This results in a novel gold standard for irony detection in Italian, enriched with fine-grained annotations, and in a language resource available to the community and exploitable in the crossand multi-lingual perspective which characterizes the work that inspired this research.

[1]  Cristina Bosco,et al.  Developing Corpora for Sentiment Analysis: The Case of Irony and Senti-TUT , 2013, IEEE Intelligent Systems.

[2]  Diana Maynard,et al.  Automatic Detection of Political Opinions in Tweets , 2011, #MSM.

[3]  Cynthia Van Hee Can machines sense irony? : exploring automatic irony detection on social media , 2017 .

[4]  Paolo Rosso,et al.  SemEval-2015 Task 11: Sentiment Analysis of Figurative Language in Twitter , 2015, *SEMEVAL.

[5]  Paolo Rosso,et al.  Irony Detection in Twitter , 2016, ACM Trans. Internet Techn..

[6]  Cristina Bosco,et al.  TWITTIRÒ: a Social Media Corpus with a Multi-layered Annotation for Irony , 2017, CLiC-it.

[7]  Cyril Grouin,et al.  Analyse d'opinion et langage figuratif dans des tweets : présentation et résultats du Défi Fouille de Textes DEFT2017 , 2017 .

[8]  Malvina Nissim,et al.  Overview of the Evalita 2014 SENTIment POLarity Classification Task , 2014 .

[9]  Paolo Rosso,et al.  Finding Humour in the Blogosphere: The Role of WordNet Resources , 2009 .

[10]  Ashley J. Llorens,et al.  Coarse-and Fine-Grained Sentiment Analysis of Social Media Text , 2011 .

[11]  Véronique Hoste,et al.  SemEval-2018 Task 3: Irony Detection in English Tweets , 2018, *SEMEVAL.

[12]  Cristina Bosco,et al.  PoSTWITA-UD: an Italian Twitter Treebank in Universal Dependencies , 2018, LREC.

[13]  Davide Buscaldi,et al.  From humor recognition to irony detection: The figurative language of social media , 2012, Data Knowl. Eng..

[14]  H. Grice Reasoning: Further Notes on Logic and Conversation , 2008 .

[15]  C. Burgers,et al.  Verbal Irony , 2012 .

[16]  Siobhan Chapman Logic and Conversation , 2005 .

[17]  Cristina Bosco,et al.  Annotating Italian Social Media Texts in Universal Dependencies , 2017, DepLing.

[18]  Nathalie Aussenac-Gilles,et al.  Towards a Contextual Pragmatic Model to Detect Irony in Tweets , 2015, ACL.

[19]  Véronique Hoste,et al.  Exploring the Realization of Irony in Twitter Data , 2016, LREC.

[20]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[21]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[22]  Nathalie Aussenac-Gilles,et al.  Exploring the Impact of Pragmatic Phenomena on Irony Detection in Tweets: A Multilingual Corpus Study , 2017, EACL.

[23]  Rada Mihalcea,et al.  Characterizing Humour: An Exploration of Features in Humorous Texts , 2009, CICLing.

[24]  Malvina Nissim,et al.  Overview of the Evalita 2016 SENTIment POLarity Classification Task , 2014, CLiC-it/EVALITA.

[25]  Cristina Bosco,et al.  Annotating Sentiment and Irony in the Online Italian Political Debate on #labuonascuola , 2016, LREC.

[26]  Paolo Rosso,et al.  Figurative messages and affect in Twitter: Differences between #irony, #sarcasm and #not , 2016, Knowl. Based Syst..