Understanding Client Support Strategies to Improve Clinical Outcomes in an Online Mental Health Intervention

Online mental health interventions are increasingly important in providing access to, and supporting the effectiveness of, mental health treatment. While these technologies are effective, user attrition and early disengagement are key challenges. Evidence suggests that integrating a human supporter into such services mitigates these challenges, however, it remains under-studied how supporter involvement benefits client outcomes, and how to maximize such effects. We present our analysis of 234,735 supporter messages to discover how different support strategies correlate with clinical outcomes. We describe our machine learning methods for: (i) clustering supporters based on client outcomes; (ii) extracting and analyzing linguistic features from supporter messages; and (iii) identifying context-specific patterns of support. Our findings indicate that concrete, positive and supportive feedback from supporters that reference social behaviors are strongly associated with better outcomes; and show how their importance varies dependent on different client situations. We discuss design implications for personalized support and supporter interfaces.

[1]  Per Carlbring,et al.  Tailored vs. Standardized Internet-Based Cognitive Behavior Therapy for Depression and Comorbid Symptoms: A Randomized Controlled Trial , 2012, PloS one.

[2]  Susan M. Kaiser,et al.  Comparison of the Effects of Coaching and Receipt of App Recommendations on Depression, Anxiety, and Engagement in the IntelliCare Platform: Factorial Randomized Controlled Trial , 2019, Journal of medical Internet research.

[3]  Matjaz Gams,et al.  Continuous stress detection using a wrist device: in laboratory and real life , 2016, UbiComp Adjunct.

[4]  G. Andersson,et al.  The impact of telephone versus e-mail therapist guidance on treatment outcomes, therapeutic alliance and treatment engagement in Internet-delivered CBT for depression: A randomised pilot trial , 2014 .

[5]  L. Luborsky,et al.  Vocabulary measures for the evaluation of therapy outcome: re-studying transcripts from the penn psychotherapy project. , 1996, Psychotherapy research : journal of the Society for Psychotherapy Research.

[6]  Fanglin Chen,et al.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones , 2014, UbiComp.

[7]  Jorge E. Palacios,et al.  Internet-Delivered Cognitive Behaviour Therapy , 2017 .

[8]  Jonathan E. Cook,et al.  Working Alliance in Online Therapy as Compared to Face-to-Face Therapy: Preliminary Results , 2002, Cyberpsychology Behav. Soc. Netw..

[9]  R. Atun,et al.  Estimating the true global burden of mental illness. , 2016, The lancet. Psychiatry.

[10]  C. Martindale,et al.  Lexical differences between working and resistance sessions in psychoanalysis. , 1984, Journal of clinical psychology.

[11]  Bethany A. Teachman,et al.  Identification of Imminent Suicide Risk Among Young Adults using Text Messages , 2018, CHI.

[12]  W. Stiles,et al.  Responsiveness in psychotherapy. , 1998 .

[13]  G. Andersson,et al.  Therapeutic Alliance in Internet-Delivered Cognitive Behaviour Therapy for Depression or Generalized Anxiety. , 2017, Clinical psychology & psychotherapy.

[14]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[15]  P. Cuijpers,et al.  Supportive Accountability: A Model for Providing Human Support to Enhance Adherence to eHealth Interventions , 2011, Journal of medical Internet research.

[16]  N. Sampson,et al.  Mental Disorders, Comorbidity and Suicidal Behavior: Results from the National Comorbidity Survey Replication , 2009, Molecular Psychiatry.

[17]  Ambuj Tewari,et al.  Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health: Key Components and Design Principles for Ongoing Health Behavior Support , 2017, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[18]  Lei Zheng,et al.  DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection , 2017, KDD.

[19]  E. Bordin The generalizability of the psychoanalytic concept of the working alliance. , 1979 .

[20]  J. Sharry,et al.  A randomized controlled trial of an internet-delivered treatment: Its potential as a low-intensity community intervention for adults with symptoms of depression , 2015, European Psychiatry.

[21]  Kurt Hornik,et al.  Introduction to arules – A computational environment for mining association rules and frequent item sets , 2009 .

[22]  Jesse H. Wright,et al.  Computer-Assisted Cognitive-Behavior Therapy for Depression: A Systematic Review and Meta-Analysis. , 2019, The Journal of clinical psychiatry.

[23]  G. Andersson,et al.  Therapist Behaviours in Internet-Delivered Cognitive Behaviour Therapy: Analyses of E-Mail Correspondence in the Treatment of Generalized Anxiety Disorder , 2012, Behavioural and Cognitive Psychotherapy.

[24]  Nickolai Titov,et al.  Internet-delivered psychotherapy for depression in adults , 2011, Current opinion in psychiatry.

[25]  Sheana Salyers Bull,et al.  More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention , 2016, J. Biomed. Informatics.

[26]  Emotions and Primary Process in Guided Imagery Psychotherapy: Computerized Text-Analytic Measures , 2001 .

[27]  Jakob E. Bardram,et al.  Impact factor analysis: combining prediction with parameter ranking to reveal the impact of behavior on health outcome , 2015, Personal and Ubiquitous Computing.

[28]  Benjamin Recht,et al.  A step towards quantifying when an algorithm can and cannot predict an individual's wellbeing , 2017, UbiComp/ISWC Adjunct.

[29]  Daniel Jackson,et al.  Challenges for Designing new Technology for Health and Wellbeing in a Complex Mental Healthcare Context , 2016, CHI.

[30]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[31]  B. Löwe,et al.  Validation and Standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the General Population , 2008, Medical care.

[32]  Tanzeem Choudhury,et al.  In Situ Design for Mental Illness: Considering the Pathology of Bipolar Disorder in mHealth Design , 2015, MobileHCI.

[33]  Gerhard Andersson,et al.  Internet-based psychological treatments for depression , 2012, Expert review of neurotherapeutics.

[34]  K. Fitzpatrick,et al.  Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial , 2017, JMIR mental health.

[35]  Rebecca P. Ang,et al.  An introduction to association rule mining: An application in counseling and help-seeking behavior of adolescents , 2007, Behavior research methods.

[36]  D. Ståhl,et al.  A randomized controlled trial of internet-based cognitive-behavioural therapy for bulimia nervosa or related disorders in a student population , 2010, Psychological Medicine.

[37]  Evangelos Karapanos,et al.  Sustaining user engagement with behavior-change tools , 2015, Interactions.

[38]  Amit P. Sheth,et al.  Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media , 2017, ASONAM.

[39]  John Pestian,et al.  Using Natural Language Processing to Classify Suicide Notes , 2008, BioNLP.

[40]  George C. Anastassopoulos,et al.  A Contextual Data Mining Approach Toward Assisting the Treatment of Anxiety Disorders , 2010, IEEE Transactions on Information Technology in Biomedicine.

[41]  E. Goffman,et al.  Forms of talk , 1982 .

[42]  Patrick Olivier,et al.  Magic land: the design and evaluation of an interactive tabletop supporting therapeutic play with children , 2012, DIS '12.

[43]  Susan R. Fussell,et al.  Using linguistic features to measure presence in computer-mediated communication , 2006, CHI.

[44]  David Coyle,et al.  Computers in talk-based mental health interventions , 2007, Interact. Comput..

[45]  Stefan Rennick Egglestone,et al.  Health Technologies 'In the Wild': Experiences of Engagement with Computerised CBT , 2016, CHI.

[46]  Anind K. Dey,et al.  Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[47]  Daniel G. Goldstein,et al.  Manipulating and Measuring Model Interpretability , 2018, CHI.

[48]  G. Andrews,et al.  Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. , 2018, Journal of anxiety disorders.

[49]  T. Vos,et al.  Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010 , 2013, The Lancet.

[50]  David C. Atkins,et al.  "It's hard to argue with a computer": Investigating Psychotherapists' Attitudes towards Automated Evaluation , 2018, Conference on Designing Interactive Systems.

[51]  David Lo,et al.  Beyond support and confidence: Exploring interestingness measures for rule-based specification mining , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[52]  Jakob E. Bardram,et al.  Designing mobile health technology for bipolar disorder: a field trial of the monarca system , 2013, CHI.

[53]  Wei Luo,et al.  An integrated framework for suicide risk prediction , 2013, KDD.

[54]  G. Eysenbach The Law of Attrition , 2005, Journal of medical Internet research.

[55]  Adrian B. R. Shatte,et al.  Machine learning in mental health: a scoping review of methods and applications , 2019, Psychological Medicine.

[56]  Andreas Maercker,et al.  Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. , 2014, Journal of affective disorders.

[57]  Dinh Q. Phung,et al.  Using linguistic and topic analysis to classify sub-groups of online depression communities , 2015, Multimedia Tools and Applications.

[58]  R. Spitzer,et al.  The PHQ-9: validity of a brief depression severity measure. , 2001, Journal of general internal medicine.

[59]  Steven M. Drucker,et al.  Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models , 2019, CHI.

[60]  Ramesh Perera-Delcourt,et al.  Patient experience of supported computerized CBT in an inner-city IAPT service: a qualitative study , 2019, The Cognitive Behaviour Therapist.

[61]  Juan Manuel Górriz,et al.  Association rule-based feature selection method for Alzheimer's disease diagnosis , 2012, Expert Syst. Appl..

[62]  Daniella K. Villalba,et al.  Identifying Behavioral Phenotypes of Loneliness and Social Isolation with Passive Sensing: Statistical Analysis, Data Mining and Machine Learning of Smartphone and Fitbit Data , 2019, JMIR mHealth and uHealth.

[63]  David Coyle,et al.  Engagement with online mental health interventions: an exploratory clinical study of a treatment for depression , 2012, CHI.

[64]  Haniye Sadat Sajadi,et al.  Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017 , 2018, The Lancet.

[65]  Ran Gilad-Bachrach,et al.  PopTherapy: coping with stress through pop-culture , 2014, PervasiveHealth.

[66]  Britt Klein,et al.  A therapist-assisted cognitive behavior therapy internet intervention for posttraumatic stress disorder: pre-, post- and 3-month follow-up results from an open trial. , 2010, Journal of anxiety disorders.

[67]  Munmun De Choudhury,et al.  Modeling Stress with Social Media Around Incidents of Gun Violence on College Campuses , 2017, Proc. ACM Hum. Comput. Interact..

[68]  G. Andersson,et al.  Therapist behaviours in internet-based cognitive behaviour therapy (ICBT) for depressive symptoms , 2015, Internet interventions.

[69]  Ramakanth Kavuluru,et al.  Classification of Helpful Comments on Online Suicide Watch Forums , 2016, BCB.

[70]  Annie T. Chen,et al.  A multi-faceted approach to characterizing user behavior and experience in a digital mental health intervention , 2019, J. Biomed. Informatics.

[71]  Tanzeem Choudhury,et al.  Passive and In-Situ assessment of mental and physical well-being using mobile sensors , 2011, UbiComp '11.

[72]  Kurt Hornik,et al.  A CLUE for CLUster Ensembles , 2005 .

[73]  Munmun De Choudhury,et al.  Methodological Gaps in Predicting Mental Health States from Social Media: Triangulating Diagnostic Signals , 2019, CHI.

[74]  Stevie Chancellor,et al.  Computational Methods to Understand Deviant Mental Wellness Communities , 2018, CHI Extended Abstracts.

[75]  Hong Lai,et al.  Nonparametric bootstrapping for hierarchical data , 2010 .

[76]  Mohan S. Kankanhalli,et al.  Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda , 2018, CHI.

[77]  G. Andersson,et al.  Internet‐delivered psychological treatments: from innovation to implementation , 2019, World psychiatry : official journal of the World Psychiatric Association.

[78]  Luís Carriço,et al.  Fear therapy for children: a mobile approach , 2012, EICS '12.

[79]  Reeva Lederman,et al.  Moderated online social therapy: Designing and evaluating technology for mental health , 2014, TCHI.

[80]  A. Przeworski,et al.  A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? , 2011, Clinical psychology review.

[81]  D. Mohr,et al.  Integrating Human Support Into Behavioral Intervention Technologies: The Efficiency Model of Support , 2017 .

[83]  David Coyle,et al.  Clinical evaluations and collaborative design: developing new technologies for mental healthcare interventions , 2009, CHI.

[84]  W. Yule What Works with Children, Adolescents and Adults? A Review of Research on the Effectiveness of Psychotherapy , 2010 .

[85]  Alistair Morrison,et al.  Student Perspectives on Digital Phenotyping: The Acceptability of Using Smartphone Data to Assess Mental Health , 2019, CHI.

[86]  L. Gostin,et al.  Public Health Ethics: Mapping the Terrain , 2002, The Journal of law, medicine & ethics : a journal of the American Society of Law, Medicine & Ethics.

[87]  Munmun De Choudhury,et al.  Mental Health Support and its Relationship to Linguistic Accommodation in Online Communities , 2018, CHI.

[88]  Grigoris Antoniou,et al.  Mining Free-Text Medical Notes for Suicide Risk Assessment , 2018, SETN.

[89]  Mike Conway,et al.  Examining thematic similarity, difference, and membership in three online mental health communities from reddit: A text mining and visualization approach , 2018, Comput. Hum. Behav..

[90]  N. Titov,et al.  Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. , 2011, Behaviour research and therapy.

[91]  VenkateshSvetha,et al.  Using linguistic and topic analysis to classify sub-groups of online depression communities , 2017 .

[92]  J. Holmes,et al.  Digital phenotyping and sociological perspectives in a Brave New World , 2017, Addiction.

[93]  Amit Singhal,et al.  Pivoted document length normalization , 1996, SIGIR 1996.

[94]  Mark Bilandzic,et al.  Bringing Transparency Design into Practice , 2018, IUI.

[95]  Anind K. Dey,et al.  Modeling and Understanding Human Routine Behavior , 2016, CHI.

[96]  Jure Leskovec,et al.  Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health , 2016, TACL.

[97]  Kristina Höök,et al.  HCI and Affective Health: Taking stock of a decade of studies and charting future research directions , 2019, CHI.

[98]  L. Timulak,et al.  Client-identified helpful and hindering events in therapist-delivered vs. self-administered online cognitive-behavioural treatments for depression in college students , 2012 .

[99]  Lucy Yardley,et al.  Developing and Evaluating Digital Interventions to Promote Behavior Change in Health and Health Care: Recommendations Resulting From an International Workshop , 2017, Journal of medical Internet research.

[100]  Mary Czerwinski,et al.  Pocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy , 2018, CHI.

[101]  Amit Sharma,et al.  Moments of Change: Analyzing Peer-Based Cognitive Support in Online Mental Health Forums , 2019, CHI.