Predicting Social Anxiety Treatment Outcome Based on Therapeutic Email Conversations

Predicting therapeutic outcome in the mental health domain is of utmost importance to enable therapists to provide the most effective treatment to a patient. Using information from the writings of a patient can potentially be a valuable source of information, especially now that more and more treatments involve computer-based exercises or electronic conversations between patient and therapist. In this paper, we study predictive modeling using writings of patients under treatment for a social anxiety disorder. We extract a wealth of information from the text written by patients including their usage of words, the topics they talk about, the sentiment of the messages, and the style of writing. In addition, we study trends over time with respect to those measures. We then apply machine learning algorithms to generate the predictive models. Based on a dataset of 69 patients, we are able to show that we can predict therapy outcome with an area under the curve of 0.83 halfway through the therapy and with a precision of 0.78 when using the full data (i.e., the entire treatment period). Due to the limited number of participants, it is hard to generalize the results, but they do show great potential in this type of information.

[1]  Michelle G. Craske,et al.  Computer Therapy for the Anxiety and Depressive Disorders Is Effective, Acceptable and Practical Health Care: A Meta-Analysis , 2010, PloS one.

[2]  G. Andersson,et al.  Internet-Based Vs. Face-To-Face Cognitive Behaviour Therapy for Psychiatric and Somatic Disorders : a Systematic Review and Meta-Analysis , 2014 .

[3]  N. Jacobson,et al.  Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. , 1991, Journal of consulting and clinical psychology.

[4]  M. Zuckerman.  The development of an affect adjective check list for the measurement of anxiety. , 1960, Journal of consulting psychology.

[5]  Alan R. Aronson,et al.  Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program , 2001, AMIA.

[6]  T. Berger,et al.  Internet-based individually versus group guided self-help treatment for social anxiety disorder: protocol of a randomized controlled trial , 2014, BMC Psychiatry.

[7]  Per Carlbring,et al.  Guided Internet‐based vs. face‐to‐face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta‐analysis , 2014, World psychiatry : official journal of the World Psychiatric Association.

[8]  Stefan Conrad,et al.  Opinion Mining on a German Corpus of a Media Response Analysis , 2012, TSD.

[9]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[10]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[11]  Claire Cardie,et al.  Learning with Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis , 2008, EMNLP.

[12]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[13]  Peter Szolovits,et al.  Utilizing uncoded consultation notes from electronic medical records for predictive modeling of colorectal cancer , 2016, Artif. Intell. Medicine.

[14]  Sunghwan Sohn,et al.  Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications , 2010, J. Am. Medical Informatics Assoc..

[15]  J. Pennebaker,et al.  Language use of depressed and depression-vulnerable college students , 2004 .

[16]  I. Kohane,et al.  Electronic medical records for discovery research in rheumatoid arthritis , 2010, Arthritis care & research.

[17]  Simon Clematide,et al.  MLSA - A Multi-layered Reference Corpus for German Sentiment Analysis , 2012, LREC.

[18]  F. Caspar,et al.  Internet-based treatment for social phobia: a randomized controlled trial. , 2009, Journal of clinical psychology.

[19]  G. Wilson,et al.  Stepped care treatment for eating disorders. , 2000, Journal of consulting and clinical psychology.

[20]  Gerhard Heyer,et al.  SentiWS - A Publicly Available German-language Resource for Sentiment Analysis , 2010, LREC.

[21]  A. Leenaars,et al.  Suicide Note Classification Using Natural Language Processing: A Content Analysis , 2010, Biomedical informatics insights.

[22]  A. Wells.  A cognitive model of social phobia , 1995 .

[23]  S. L. Nielsen,et al.  A lab test and algorithms for identifying clients at risk for treatment failure. , 2005, Journal of clinical psychology.

[24]  Pim Cuijpers,et al.  Web-based depression treatment: associations of clients' word use with adherence and outcome. , 2014, Journal of affective disorders.

[25]  J. Boettcher,et al.  Does a pre-treatment diagnostic interview affect the outcome of internet-based self-help for social anxiety disorder? a randomized controlled trial. , 2012, Behavioural and cognitive psychotherapy.

[26]  Beatrice Santorini,et al.  Part-of-Speech Tagging Guidelines for the Penn Treebank Project (3rd Revision) , 1990 .

[27]  G. Andersson,et al.  Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. , 2011, Behaviour research and therapy.

[28]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[29]  Josep Maria Haro,et al.  Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys , 2007, The Lancet.

[30]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[31]  D. Haaga.  Introduction to the special section on stepped care models in psychotherapy. , 2000, Journal of consulting and clinical psychology.

[32]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[33]  M. Lambert,et al.  Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. , 2010, Journal of consulting and clinical psychology.

[34]  T. Heidenreich,et al.  Die Erfassung sozialer Phobie durch die Social Interaction Anxiety Scale (SIAS) und die Social Phobia Scale (SPS) , 1999 .

[35]  Peter Szolovits,et al.  Automatic lymphoma classification with sentence subgraph mining from pathology reports. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[36]  Thomas Berger,et al.  The Right View of Your Patient: A Computer-Assisted, Individualized Module for Psychotherapy Training , 2004 .

[37]  Scott T. Weiss,et al.  Extracting principal diagnosis, co-morbidity and smoking status for asthma research: evaluation of a natural language processing system , 2006, BMC Medical Informatics Decis. Mak..

[38]  Hans Kordy,et al.  Linguistic analyses of natural written language: unobtrusive assessment of cognitive style in eating disorders. , 2007, The International journal of eating disorders.

[39]  Mohammed Saeed,et al.  Risk Stratification of ICU Patients Using Topic Models Inferred from Unstructured Progress Notes , 2012, AMIA.

[40]  B. Ogles,et al.  The use of outcome measures by psychologists in clinical practice , 2004 .

[41]  T. Vos,et al.  Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010 , 2013, The Lancet.

[42]  Richard A. Olshen,et al.  CART: Classification and Regression Trees , 1984 .

[43]  M. Lambert.  Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care , 2007 .