Spontaneously Generated Online Patient Experience of Modafinil: A Qualitative and NLP Analysis

Objective: To compare the findings from a qualitative and a natural language processing (NLP) based analysis of online patient experience posts on patient experience of the effectiveness and impact of the drug Modafinil. Methods: Posts (n = 260) from 5 online social media platforms where posts were publicly available formed the dataset/corpus. Three platforms asked posters to give a numerical rating of Modafinil. Thematic analysis: data was coded and themes generated. Data were categorized into PreModafinil, Acquisition, Dosage, and PostModafinil and compared to identify each poster's own view of whether taking Modafinil was linked to an identifiable outcome. We classified this as positive, mixed, negative, or neutral and compared this with numerical ratings. NLP: Corpus text was speech tagged and keywords and key terms extracted. We identified the following entities: drug names, condition names, symptoms, actions, and side-effects. We searched for simple relationships, collocations, and co-occurrences of entities. To identify causal text, we split the corpus into PreModafinil and PostModafinil and used n-gram analysis. To evaluate sentiment, we calculated the polarity of each post between −1 (negative) and +1 (positive). NLP results were mapped to qualitative results. Results: Posters had used Modafinil for 33 different primary conditions. Eight themes were identified: the reason for taking (condition or symptom), impact of symptoms, acquisition, dosage, side effects, other interventions tried or compared to, effectiveness of Modafinil, and quality of life outcomes. Posters reported perceived effectiveness as follows: 68% positive, 12% mixed, 18% negative. Our classification was consistent with poster ratings. Of the most frequent 100 keywords/keyterms identified by term extraction 88/100 keywords and 84/100 keyterms mapped directly to the eight themes. Seven keyterms indicated negation and temporal states. Sentiment was as follows 72% positive sentiment 4% neutral 24% negative. Matching of sentiment between the qualitative and NLP methods was accurate in 64.2% of posts. If we allow for one category difference matching was accurate in 85% of posts. Conclusions: User generated patient experience is a rich resource for evaluating real world effectiveness, understanding patient perspectives, and identifying research gaps. Both methods successfully identified the entities and topics contained in the posts. In contrast to current evidence, posters with a wide range of other conditions found Modafinil effective. Perceived causality and effectiveness were identified by both methods demonstrating the potential to augment existing knowledge.

[1]  Paloma Martínez,et al.  Pharmacovigilance through the development of text mining and natural language processing techniques , 2015, J. Biomed. Informatics.

[2]  J. Bensing,et al.  Bridging the gap. The separate worlds of evidence-based medicine and patient-centered medicine. , 2000, Patient education and counseling.

[3]  T. Driscoll,et al.  Pharmacological interventions for sleepiness and sleep disturbances caused by shift work , 2014, The Cochrane database of systematic reviews.

[4]  Zion Tsz Ho Tse,et al.  Ebola virus disease and social media: A systematic review. , 2016, American Journal of Infection Control.

[5]  X. Vidal,et al.  Psychostimulant drugs for cocaine dependence. , 2016, The Cochrane database of systematic reviews.

[6]  Sally Okun,et al.  Patient-reported Outcomes as a Source of Evidence in Off-Label Prescribing: Analysis of Data From PatientsLikeMe , 2011, Journal of medical Internet research.

[7]  T. Cribbin,et al.  Doing social media analytics , 2016 .

[8]  C. Pagliari,et al.  Mining social media data: How are research sponsors and researchers addressing the ethical challenges? , 2018 .

[9]  Yusuke Miyao,et al.  TwiMed: Twitter and PubMed Comparable Corpus of Drugs, Diseases, Symptoms, and Their Relations , 2017, JMIR public health and surveillance.

[10]  D Demner-Fushman,et al.  Aspiring to Unintended Consequences of Natural Language Processing: A Review of Recent Developments in Clinical and Consumer-Generated Text Processing , 2016, Yearbook of Medical Informatics.

[11]  R. C. Macridis.  A review , 1963 .

[12]  M. Steinberg,et al.  Efficacy and Safety of Modafinil in the Treatment of Cancer-Related Fatigue , 2009, The Annals of pharmacotherapy.

[13]  J. Ballon,et al.  A systematic review of modafinil: Potential clinical uses and mechanisms of action. , 2006, The Journal of clinical psychiatry.

[14]  M. Larsen,et al.  Ethics and Privacy in Social Media Research for Mental Health , 2020, Current Psychiatry Reports.

[15]  Yong Tan,et al.  The Consensus Effect on Shared Treatment Experience in Online Healthcare Communities , 2015 .

[16]  Jeffrey T. Hancock,et al.  Experimental evidence of massive-scale emotional contagion through social networks , 2014, Proceedings of the National Academy of Sciences.

[17]  Lina M. Sulieman,et al.  A systematic literature review of machine learning in online personal health data , 2019, J. Am. Medical Informatics Assoc..

[18]  Xiang Wang,et al.  Efficacy of Modafinil on Fatigue and Excessive Daytime Sleepiness Associated with Neurological Disorders: A Systematic Review and Meta-Analysis , 2013, PloS one.

[19]  I. Harris,et al.  A Systematic Review and Meta-Analysis of Randomized Controlled Trials , 2010 .

[20]  Philip E. Bourne,et al.  A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data , 2019, Int. J. Medical Informatics.

[21]  Alexandra Georgakopoulou-Nunes,et al.  The Sage Handbook of Social Media Research Methods , 2017 .

[22]  George Hripcsak,et al.  Detection of drug‐drug interactions through data mining studies using clinical sources, scientific literature and social media , 2018, Briefings Bioinform..

[23]  M. Daumer,et al.  HAGIL (Hamburg Vigil Study): a randomized placebo-controlled double-blind study with modafinil for treatment of fatigue in patients with multiple sclerosis , 2011, Multiple sclerosis.

[24]  Julie L. Andsager,et al.  Online Information Exchanges for Parents of Children With a Rare Health Condition: Key Findings From an Online Support Community , 2013, Journal of medical Internet research.

[25]  B. Sahakian,et al.  Modafinil augmentation therapy in unipolar and bipolar depression: a systematic review and meta-analysis of randomized controlled trials. , 2013, The Journal of clinical psychiatry.

[26]  Wessel Kraaij,et al.  Data-Driven Lexical Normalization for Medical SocialMedia , 2019 .

[27]  Desmond J Sheridan,et al.  Achievements and Limitations of Evidence-Based Medicine. , 2016, Journal of the American College of Cardiology.

[28]  Ming Yang,et al.  Filtering big data from social media - Building an early warning system for adverse drug reactions , 2015, J. Biomed. Informatics.

[29]  E. Mohammadi,et al.  Barriers and facilitators related to the implementation of a physiological track and trigger system: A systematic review of the qualitative evidence , 2017, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[30]  F. Elsner,et al.  Pharmacological treatments for fatigue associated with palliative care. , 2010, The Cochrane database of systematic reviews.

[31]  H. Witteman,et al.  Health Researchers’ Use of Social Media: Scoping Review , 2019, Journal of medical Internet research.

[32]  J. Frost,et al.  The case for using social media to aggregate patient experiences with off-label prescriptions , 2011, Expert review of pharmacoeconomics & outcomes research.

[33]  F. Greaves,et al.  Harnessing patient feedback data: A challenge for policy and service improvement , 2015, Digital health.

[34]  A. Makady,et al.  Use of Social Media in the Assessment of Relative Effectiveness: Explorative Review With Examples From Oncology , 2018, JMIR cancer.

[35]  N. Agrawal,et al.  Pharmacotherapy for chronic cognitive impairment in traumatic brain injury. , 2015, The Cochrane database of systematic reviews.

[36]  A. Beelen,et al.  Treatment for postpolio syndrome. , 2015, The Cochrane database of systematic reviews.

[37]  Graciela Gonzalez-Hernandez,et al.  Utilizing social media data for pharmacovigilance: A review , 2015, J. Biomed. Informatics.

[38]  M. Clanet,et al.  Modafinil for fatigue in MS , 2005, Neurology.

[39]  P. Brown,et al.  Interventions for preventing and ameliorating cognitive deficits in adults treated with cranial irradiation. , 2014, The Cochrane database of systematic reviews.

[40]  J. Lyon.  Chess Study Revives Debate Over Cognition-Enhancing Drugs. , 2017, JAMA.

[41]  Stephen H. Bell,et al.  A ?scoping review. , 2018, Sexual health.

[42]  Eric Gossett,et al.  Big Data: A Revolution That Will Transform How We Live, Work, and Think , 2015 .

[43]  C. Blandizzi,et al.  The usefulness of listening social media for pharmacovigilance purposes: a systematic review , 2018, Expert opinion on drug safety.

[44]  Adam D. I. Kramer,et al.  Detecting Emotional Contagion in Massive Social Networks , 2014, PloS one.

[45]  M. Delgado-Rodríguez,et al.  Systematic review and meta-analysis. , 2017, Medicina intensiva.

[46]  Jörg Ruof,et al.  Different Weights of the Evidence-Based Medicine Triad in Regulatory, Health Technology Assessment, and Clinical Decision Making , 2017, Pharmaceutical Medicine.

[47]  H. Kiene,et al.  The powerful placebo effect: fact or fiction? , 1997, Journal of clinical epidemiology.

[48]  G. Weinhouse,et al.  The Use of Modafinil in the Intensive Care Unit , 2016, Journal of intensive care medicine.

[49]  Kevin A Padrez,et al.  Twitter as a Tool for Health Research: A Systematic Review , 2017, American journal of public health.

[50]  T. Greenhalgh,et al.  Six ‘biases’ against patients and carers in evidence-based medicine , 2015, BMC Medicine.

[51]  James P. Bagrow,et al.  What we write about when we write about causality: Features of causal statements across large-scale social discourse , 2016, 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[52]  F. Blackhall,et al.  Modafinil for the treatment of fatigue in lung cancer: results of a placebo-controlled, double-blind, randomized trial. , 2014, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[53]  J. Merino,et al.  Exclusion of ‘nonRCT evidence’ in guidelines for chronic diseases – is it always appropriate? The Look AHEAD study , 2014, Current medical research and opinion.

[54]  Ruairidh M. Battleday,et al.  Modafinil for cognitive neuroenhancement in healthy non-sleep-deprived subjects: A systematic review , 2015, European Neuropsychopharmacology.

[55]  H. Beecher,et al.  The powerful placebo. , 1955, Journal of the American Medical Association.

[56]  Mark Sanderson,et al.  Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze, Introduction to Information Retrieval, Cambridge University Press 2008. ISBN-13 978-0-521-86571-5, xxi + 482 pages , 2010, Natural Language Engineering.

[57]  Elena Perrin,et al.  Advantages and limitations of online communities of patients for research on health products. , 2017, Therapie.

[58]  I. Sim.  Two Ways of Knowing: Big Data and Evidence-Based Medicine , 2016, Annals of Internal Medicine.

[59]  Lieven A. Schenk,et al.  The placebo effect: From concepts to genes , 2015, Neuroscience.

[60]  Pierre Zweigenbaum,et al.  Text mining applications in psychiatry: a systematic literature review , 2016, International journal of methods in psychiatric research.

[61]  D G Altman,et al.  The scandal of poor medical research , 1994, BMJ.

[62]  Kasturi Dewi Varathan,et al.  Using online social networks to track a pandemic: A systematic review , 2016, J. Biomed. Informatics.

[63]  Sandra Bringay,et al.  Mining social networks to improve suicide prevention: A scoping review , 2019, Journal of neuroscience research.

[64]  C. Gamble.  MENDS: The use of MElatonin in children with Neuro-developmental Disorders and impaired Sleep - a randomised, double-blind, placebo-controlled, parallel trial , 2011 .

[65]  J. Ramos-Quiroga,et al.  Amphetamines for Attention Deficit Hyperactivity Disorder (ADHD) in adults. , 2011, The Cochrane database of systematic reviews.

[66]  E. Şerefoğlu,et al.  On-demand Modafinil Improves Ejaculation Time and Patient-reported Outcomes in Men With Lifelong Premature Ejaculation. , 2016, Urology.

[67]  H. Berendse,et al.  Interventions for fatigue in Parkinson's disease. , 2015, The Cochrane database of systematic reviews.

[68]  Ian B. Wilkinson,et al.  Inverting the patient involvement paradigm: defining patient led research , 2018, Research Involvement and Engagement.

[69]  Michael J. Sateia.  Contemporary Reviews in Sleep MedicineInternational Classification of Sleep Disorders-Third Edition , 2014 .

[70]  V. Wilson.  Patient use of the internet for diabetes information. , 2013, Nursing times.

[71]  K. Beninger.  Social Media Users’ Views on the Ethics of Social Media Research , 2016 .

[72]  Caroline Jay,et al.  Web Use for Symptom Appraisal of Physical Health Conditions: A Systematic Review , 2017, Journal of medical Internet research.

[73]  Danushka Bollegala,et al.  Causality Patterns for Detecting Adverse Drug Reactions From Social Media: Text Mining Approach , 2018, JMIR public health and surveillance.

[74]  M. Keller,et al.  Patient Understanding of the Risks and Benefits of Biologic Therapies in Inflammatory Bowel Disease: Insights from a Large-scale Analysis of Social Media Platforms , 2017, Inflammatory bowel diseases.

[75]  Arun Ross,et al.  Social Media Analytics for Smart Health , 2014, IEEE Intelligent Systems.

[76]  S. Golder,et al.  Understanding Public Attitudes Toward Researchers Using Social Media for Detecting and Monitoring Adverse Events Data: Multi Methods Study , 2019, Journal of medical Internet research.

[77]  Graeme Hirst,et al.  Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review , 2018, BMC Medical Informatics and Decision Making.

[78]  Shion Guha,et al.  Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence? , 2017, J. Assoc. Inf. Sci. Technol..

[79]  Kai Cheng,et al.  A systematic review protocol , 2008 .

[80]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[81]  James P. Bagrow,et al.  What we talk about when we talk about causality : Features of causal statements across large-scale social discourse , 2016 .

[82]  John P. A. Ioannidis,et al.  Why Most Clinical Research Is Not Useful , 2016, PLoS medicine.

[83]  H. D. Koot,et al.  The Linguistic Expression of Causation , 2012 .

[84]  M. Olson,et al.  Patterns of Treatment Switching in Multiple Sclerosis Therapies in US Patients Active on Social Media: Application of Social Media Content Analysis to Health Outcomes Research , 2016, Journal of medical Internet research.

[85]  E. Chiauzzi,et al.  Digital Trespass: Ethical and Terms-of-Use Violations by Researchers Accessing Data From an Online Patient Community , 2019, Journal of medical Internet research.

[86]  J. Jankovic,et al.  Modafinil for daytime somnolence in Parkinson’s disease: double blind, placebo controlled parallel trial , 2005, Journal of Neurology, Neurosurgery & Psychiatry.

[87]  Antony K. Cooper,et al.  On the Ethics of Using Publicly-Available Data , 2020, I3E.

[88]  John Sinclair,et al.  Corpus, Concordance, Collocation , 1991 .

[89]  K. Davidson,et al.  N-of-1 randomized trials for psychological and health behavior outcomes: a systematic review protocol , 2015, Systematic Reviews.

[90]  Teresa Foo,et al.  : SYSTEMATIC LITERATURE REVIEW , 2004 .

[91]  K. Shadan,et al.  Available online: , 2012 .

[92]  Stefania Vicari,et al.  Health activism and the logic of connective action. A case study of rare disease patient organisations , 2016, Information, communication and society.

[93]  G Savova,et al.  Capturing the Patient’s Perspective: a Review of Advances in Natural Language Processing of Health-Related Text , 2017, Yearbook of Medical Informatics.

[94]  Andrew Booth,et al.  Attitudes Toward the Ethics of Research Using Social Media: A Systematic Review , 2017, Journal of medical Internet research.

[95]  Simon M. Lin,et al.  Collecting and Analyzing Patient Experiences of Health Care From Social Media , 2015, JMIR research protocols.

[96]  B. Ru,et al.  A Literature Review of Social Media-Based Data Mining for Health Outcomes Research , 2019, Social Web and Health Research.

[97]  Anastazia Zunic,et al.  Sentiment Analysis in Health and Well-Being: Systematic Review , 2020, JMIR medical informatics.