Detecting Perceived Emotions in Hurricane Disasters

Natural disasters (e.g., hurricanes) affect millions of people each year, causing widespread destruction in their wake. People have recently taken to social media websites (e.g., Twitter) to share their sentiments and feelings with the larger community. Consequently, these platforms have become instrumental in understanding and perceiving emotions at scale. In this paper, we introduce HurricaneEmo, an emotion dataset of 15,000 English tweets spanning three hurricanes: Harvey, Irma, and Maria. We present a comprehensive study of fine-grained emotions and propose classification tasks to discriminate between coarse-grained emotion groups. Our best BERT model, even after task-guided pre-training which leverages unlabeled Twitter data, achieves only 68% accuracy (averaged across all groups). HurricaneEmo serves not only as a challenging benchmark for models but also as a valuable resource for analyzing emotions in disaster-centric domains.

[1]  Firoj Alam,et al.  Domain Adaptation with Adversarial Training and Graph Embeddings , 2018, ACL.

[2]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[3]  Jörg Tiedemann,et al.  Creating a Dataset for Multilingual Fine-grained Emotion-detection Using Gamification-based Annotation , 2018, WASSA@EMNLP.

[4]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[5]  Ron Artstein,et al.  Survey Article: Inter-Coder Agreement for Computational Linguistics , 2008, CL.

[6]  D. Alexander.  Natural Disasters , 1993 .

[7]  W Kinston,et al.  Disaster: effects on mental and physical state. , 1974, Journal of psychosomatic research.

[8]  Rebecca J. Passonneau.  Computing Reliability for Coreference Annotation , 2004, LREC.

[9]  Muhammad Imran,et al.  Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages , 2016, ISCRAM.

[10]  Paul Buitelaar,et al.  A Comparison Of Emotion Annotation Schemes And A New Annotated Data Set , 2018, LREC.

[11]  Erik Cambria,et al.  AFFECTIVE COMPUTI G AND SENTIMENT ANALYSIS Deep Learning-Based Document Modeling for Personality Detection from Text , 2017 .

[12]  Oren Etzioni,et al.  Named Entity Recognition in Tweets: An Experimental Study , 2011, EMNLP.

[13]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[14]  R. Plutchik.  Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice , 2016 .

[15]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[16]  Stan Szpakowicz,et al.  Identifying Expressions of Emotion in Text , 2007, TSD.

[17]  Shrey Desai,et al.  Evaluating Lottery Tickets Under Distributional Shifts , 2019, EMNLP.

[18]  A. Shapiro,et al.  National Consortium for the Study of Terrorism and Responses to Terrorism , 2010 .

[19]  Paul Schrodt,et al.  Making Sense of New Orleans Flood Trauma Recovery: Ethics, Research Design, and Policy Considerations for Future Disasters , 2006 .

[20]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[21]  Athena Vakali,et al.  Sentiment analysis leveraging emotions and word embeddings , 2017 .

[22]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[23]  Hassan Sajjad,et al.  Robust Classification of Crisis-Related Data on Social Networks Using Convolutional Neural Networks , 2017, ICWSM.

[24]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[25]  Axel Schulz,et al.  A fine-grained sentiment analysis approach for detecting crisis related microposts , 2013, ISCRAM.

[26]  P. Ekman.  An argument for basic emotions , 1992 .

[27]  R'emi Louf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[28]  Brooke Fisher Liu,et al.  Social media use during disasters: a review of the knowledge base and gaps. , 2012 .

[29]  L. Palen,et al.  Crisis informatics—New data for extraordinary times , 2016, Science.

[30]  Ilan Noy,et al.  NATURAL DISASTERS , 2011 .

[31]  Sarah Vieweg,et al.  Processing Social Media Messages in Mass Emergency , 2014, ACM Comput. Surv..

[32]  Cornelia Caragea,et al.  Towards Practical Usage of a Domain Adaptation Algorithm in the Early Hours of a Disaster , 2017, ISCRAM.

[33]  Cornelia Caragea,et al.  Fine-Grained Emotion Detection in Health-Related Online Posts , 2018, EMNLP.

[34]  Amit P. Sheth,et al.  Harnessing Twitter "Big Data" for Automatic Emotion Identification , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[35]  Susie Burke,et al.  Hope, despair and transformation: Climate change and the promotion of mental health and wellbeing , 2008, International journal of mental health systems.

[36]  Leysia Palen,et al.  Natural Language Processing to the Rescue? Extracting "Situational Awareness" Tweets During Mass Emergency , 2011, ICWSM.

[37]  Saif Mohammad,et al.  Using Hashtags to Capture Fine Emotion Categories from Tweets , 2015, Comput. Intell..

[38]  Felipe Bravo-Marquez,et al.  Meta-level sentiment models for big social data analysis , 2014, Knowl. Based Syst..

[39]  Svitlana Volkova,et al.  Inferring Perceived Demographics from User Emotional Tone and User-Environment Emotional Contrast , 2016, ACL.

[40]  Hassan Sajjad,et al.  Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks , 2016, ICWSM 2016.

[41]  Muhammad Abdul-Mageed,et al.  EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks , 2017, ACL.

[42]  Nikhil Ketkar,et al.  Convolutional Neural Networks , 2021, Deep Learning with Python.

[43]  Shady Elbassuoni,et al.  Practical extraction of disaster-relevant information from social media , 2013, WWW.

[44]  Saif Mohammad,et al.  #Emotional Tweets , 2012, *SEMEVAL.

[45]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[46]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[47]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[48]  Cornelia Caragea,et al.  Identifying informative messages in disaster events using Convolutional Neural Networks , 2016 .

[49]  Carlo Strapparava,et al.  A Parallel Corpus of Music and Lyrics Annotated with Emotions , 2012, LREC.

[50]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[51]  Carlo Strapparava,et al.  SemEval-2007 Task 14: Affective Text , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[52]  Charles E. Fritz,et al.  The NORC Studies of Human Behavior in Disaster , 1954 .

[53]  Richard Wicentowski,et al.  SWAT-MP:The SemEval-2007 Systems for Task 5 and Task 14 , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[54]  Jacob Eisenstein,et al.  Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling , 2019, EMNLP.