Detecting Happiness in Italian Tweets: Towards an Evaluation Dataset for Sentiment Analysis in Felicittà

This paper focuses on the development of a gold standard corpus for the validation of Felicitta, an online platform which uses Twitter as data source in order to estimate and interactively display the degree of happiness in the Italian cities. The ultimate goal is the creation of an Italian reference Twitter dataset for sentiment analysis that can be used in several frameworks aimed at detecting sentiment from big data sources. We will provide an overview of the reference corpus created for evaluating Felicitta, with a special focus on the issues raised from its development, on the inter-annotator agreement discussion and on implications for the further development of the corpus, considering that the assumption that a single right answer exists for each annotated instance cannot be done in several cases in the particular kind of data at issue.

[1]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[2]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[3]  Simone Paolo Ponzetto,et al.  BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network , 2012, Artif. Intell..

[4]  Hugo Liu,et al.  A Corpus-based Approach to Finding Happiness , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[5]  Cristina Bosco,et al.  Developing Corpora for Sentiment Analysis: The Case of Irony and Senti-TUT , 2013, IEEE Intelligent Systems.

[6]  Harith Alani,et al.  Evaluation Datasets for Twitter Sentiment Analysis: A survey and a new dataset, the STS-Gold , 2013, ESSEM@AI*IA.

[7]  Cristina Bosco,et al.  Felicittà: Visualizing and Estimating Happiness in Italian Cities from Geotagged Tweets , 2013, ESSEM@AI*IA.

[8]  Viviana Patti,et al.  Emotional Responses to Artworks in Online Collections , 2013, UMAP Workshops.

[9]  Ari Rappoport,et al.  Semi-Supervised Recognition of Sarcasm in Twitter and Amazon , 2010, CoNLL.

[10]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[11]  E. Diener,et al.  Subjective well-being. The science of happiness and a proposal for a national index. , 2000, The American psychologist.

[12]  Malvina Nissim,et al.  Sentiment analysis on Italian tweets , 2013, WASSA@NAACL-HLT.

[13]  Tommaso Caselli,et al.  Assigning Connotation Values to Events , 2012, LREC.

[14]  Cristina Baroglio,et al.  From Tags to Emotions: Ontology-driven Sentimental Analysis in the Social Semantic Web , 2011, DART@AI*IA.

[15]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[16]  Paolo Rosso,et al.  A multidimensional approach for detecting irony in Twitter , 2013, Lang. Resour. Evaluation.

[17]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[18]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[19]  Ron Artstein,et al.  Survey Article: Inter-Coder Agreement for Computational Linguistics , 2008, CL.

[20]  J. Helliwell,et al.  World happiness report , 2012 .

[21]  Johan Bollen,et al.  Twitter Mood as a Stock Market Predictor , 2011, Computer.

[22]  Tony Veale,et al.  An Ironic Fist in a Velvet Glove: Creative Mis-Representation in the Construction of Ironic Similes , 2010, Minds and Machines.

[23]  Daniele Quercia,et al.  Tracking "gross community happiness" from tweets , 2012, CSCW.

[24]  Rada Mihalcea,et al.  Sentiment Analysis , 2014, Encyclopedia of Social Network Analysis and Mining.

[25]  Yannick Versley.  Disagreement Dissected : Vagueness as a Source of Ambiguity in Nominal ( Co-) Reference , 2006 .

[26]  Timothy Baldwin,et al.  Social Media: Friend or Foe of Natural Language Processing? , 2012, PACLIC.

[27]  Christopher M. Danforth,et al.  The Geography of Happiness: Connecting Twitter Sentiment and Expression, Demographics, and Objective Characteristics of Place , 2013, PloS one.

[28]  Enrique Herrera-Viedma,et al.  Sentiment analysis: A review and comparative analysis of web services , 2015, Inf. Sci..

[29]  Andrea Bolioli,et al.  Social Media Monitoring in Real Life with Blogmeter Platform , 2013, ESSEM@AI*IA.

[30]  Nancy Ide,et al.  Distant Supervision for Emotion Classification with Discrete Binary Values , 2013, CICLing.

[31]  Ellen Riloff,et al.  Sarcasm as Contrast between a Positive Sentiment and Negative Situation , 2013, EMNLP.