Improving classification of Adverse Drug Reactions through Using Sentiment Analysis and Transfer Learning

The availability of large-scale and real-time data on social media has motivated research into adverse drug reactions (ADRs). ADR classification helps to identify negative effects of drugs, which can guide health professionals and pharmaceutical companies in making medications safer and advocating patients’ safety. Based on the observation that in social media, negative sentiment is frequently expressed towards ADRs, this study presents a neural model that combines sentiment analysis with transfer learning techniques to improve ADR detection in social media postings. Our system is firstly trained to classify sentiment in tweets concerning current affairs, using the SemEval17-task4A corpus. We then apply transfer learning to adapt the model to the task of detecting ADRs in social media postings. We show that, in combination with rich representations of words and their contexts, transfer learning is beneficial, especially given the large degree of vocabulary overlap between the current affairs posts in the SemEval17-task4A corpus and posts about ADRs. We compare our results with previous approaches, and show that our model can outperform them by up to 3% F-score.

[1]  Chuhan Wu,et al.  Detecting Tweets Mentioning Drug Name and Adverse Drug Reaction with Hierarchical Tweet Representation and Multi-Head Self-Attention , 2018, EMNLP 2018.

[2]  Naomie Salim,et al.  Recognition of side effects as implicit-opinion words in drug reviews , 2016, Online Inf. Rev..

[3]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[4]  Christopher C. Yang,et al.  Social media mining for drug safety signal detection , 2012, SHB '12.

[5]  T. Kass-Hout,et al.  Social media in public health. , 2013, British medical bulletin.

[6]  Abeed Sarker,et al.  Portable automatic text classification for adverse drug reaction detection via multi-corpus training , 2015, J. Biomed. Informatics.

[7]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[8]  Sophia Ananiadou,et al.  Analysis of the effect of sentiment analysis on extracting adverse drug reactions from tweets and forum posts , 2016, J. Biomed. Informatics.

[9]  Danushka Bollegala,et al.  Social media and pharmacovigilance: A review of the opportunities and challenges. , 2015, British journal of clinical pharmacology.

[10]  Muhammad Abdul-Mageed,et al.  UBC-NLP at IEST 2018: Learning Implicit Emotion With an Ensemble of Language Models , 2018, WASSA@EMNLP.

[11]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[12]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[13]  Noah A. Smith,et al.  To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks , 2019, RepL4NLP@ACL.

[14]  Elena Tutubalina,et al.  Automated Detection of Adverse Drug Reactions from Social Media Posts with Machine Learning , 2017, AIST.

[15]  Alexandros Potamianos,et al.  An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models , 2019, NAACL.

[16]  Keyuan Jiang,et al.  Mining Twitter Data for Potential Drug Effects , 2013, ADMA.

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Bo Luo,et al.  Mining Adverse Drug Reactions from online healthcare forums using Hidden Markov Model , 2014, BMC Medical Informatics and Decision Making.

[20]  Fan Yu,et al.  Towards large-scale twitter mining for drug-related adverse events , 2012, SHB '12.

[21]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[22]  Emily Chia-Yu Su,et al.  Adverse Drug Reaction Post Classification with Imbalanced Classification Techniques , 2018, 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI).

[23]  Berry de Bruijn,et al.  NRC-Canada at SMM4H Shared Task: Classifying Tweets Mentioning Adverse Drug Reactions and Medication Intake , 2018, SMM4H@AMIA.

[24]  Abeed Sarker,et al.  Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features , 2015, J. Am. Medical Informatics Assoc..

[25]  Nikos Pelekis,et al.  DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis , 2017, *SEMEVAL.

[26]  Nazli Goharian,et al.  ADRTrace: Detecting Expected and Unexpected Adverse Drug Reactions from User Reviews on Social Media Sites , 2013, ECIR.

[27]  Yen S. Low,et al.  Text Mining for Adverse Drug Events: the Promise, Challenges, and State of the Art , 2014, Drug Safety.

[28]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[29]  Stefan M. Rüger,et al.  Adverse Drug Reaction Classification With Deep Neural Networks , 2016, COLING.

[30]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  Hsinchun Chen,et al.  A research framework for pharmacovigilance in health social media: Identification and evaluation of patient adverse drug event reports , 2015, J. Biomed. Informatics.

[32]  Sophia Ananiadou,et al.  Annotation and detection of drug effects in text for pharmacovigilance , 2018, Journal of Cheminformatics.