Attrition and adherence in smartphone-delivered interventions for mental health problems: A systematic and meta-analytic review.

OBJECTIVES Although the efficacy of smartphone-delivered interventions for mental health problems is emerging, randomized controlled trials (RCTs) of smartphone interventions are characterized by high rates of attrition and low adherence. High attrition and low adherence may threaten the validity of RCT findings, so a better understanding of these phenomena is needed. We examined attrition and adherence in 70 RCTs of smartphone interventions. METHOD Four online databases were searched for RCTs of mental health interventions delivered via smartphones. RESULTS The mean meta-analytic study attrition rate was 24.1% (95% CI [19.3, 29.6]) at short-term follow up and 35.5% (95% CI [26.7, 45.3]) at longer-term follow up. These rates varied according to target mental health condition. Attrition rates were significantly lower in trials that delivered an acceptance-based intervention, offered participants monetary compensation, and reminded participants to engage in the intervention, and were significantly higher in trials that used an online enrollment method (relative to telephone or in-person enrollment). No participant-level baseline characteristic reliably predicted attrition. Evidence of attrition bias came from many RCTs not conducting intention-to-treat analyses. However, the mean difference in the between-groups effect size on primary outcomes in trials that reported both per protocol an intention-to-treat analyses was only Δd = 0.18. Adherence rates were also suboptimal based on our qualitative synthesis; several participants failed to download the intervention, and intervention usage consistently declined over the course of the trial. CONCLUSION Study attrition and low adherence are common, problematic, and may undermine the validity of findings in RCTs of smartphone-delivered interventions for mental health problems. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

[1]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[2]  Per Carlbring,et al.  The efficacy of app‐supported smartphone interventions for mental health problems: a meta‐analysis of randomized controlled trials , 2019, World psychiatry : official journal of the World Psychiatric Association.

[3]  N. Rickard,et al.  Engagement with a cognitive behavioural therapy mobile phone app predicts changes in mental health and wellbeing: MoodMission , 2019, Australian Psychologist.

[4]  Yuh-Ming Hou,et al.  Home-delivered attention bias modification training via smartphone to improve attention control in sub-clinical generalized anxiety disorder: A randomized, controlled multi-session experiment. , 2019, Journal of affective disorders.

[5]  Michael E. Levin,et al.  A Randomized Controlled Trial of Multiple Versions of an Acceptance and Commitment Therapy Matrix App for Well-Being , 2019, Behavior modification.

[6]  J. Sneed,et al.  Computerized cognitive training in young adults with depressive symptoms: Effects on mood, cognition, and everyday functioning. , 2019, Journal of affective disorders.

[7]  K. Krysta,et al.  A telemedicine platform to improve clinical parameters in paranoid schizophrenia patients: Results of a one-year randomized study , 2019, Schizophrenia Research.

[8]  Andrew Steptoe,et al.  Mindfulness On-The-Go: Effects of a Mindfulness Meditation App on Work Stress and Well-Being , 2019, Journal of occupational health psychology.

[9]  N. Farb,et al.  Effects of a Mindfulness Meditation App on Subjective Well-Being: Active Randomized Controlled Trial and Experience Sampling Study , 2019, JMIR mental health.

[10]  Elvo K. L. Sou,et al.  An evaluation of a low intensity mHealth enhanced mindfulness intervention for Chinese university students: A randomized controlled trial , 2018, Psychiatry Research.

[11]  H. Hayne,et al.  Mobile Mindfulness Meditation: a Randomised Controlled Trial of the Effect of Two Popular Apps on Mental Health , 2018, Mindfulness.

[12]  S. Y. Yip,et al.  Efficacy and Moderation of Mobile App–Based Programs for Mindfulness-Based Training, Self-Compassion Training, and Cognitive Behavioral Psychoeducation on Mental Health: Randomized Controlled Noninferiority Trial , 2018, JMIR mental health.

[13]  C. Depp,et al.  Single-Session Mobile-Augmented Intervention in Serious Mental Illness: A Three-Arm Randomized Controlled Trial. , 2018, Schizophrenia bulletin.

[14]  David Bakker,et al.  A randomized controlled trial of three smartphone apps for enhancing public mental health. , 2018, Behaviour research and therapy.

[15]  Per Carlbring,et al.  Adding a smartphone app to internet-based self-help for social anxiety: A randomized controlled trial , 2018, Comput. Hum. Behav..

[16]  Y. A. Hong,et al.  Development and Feasibility Testing of an mHealth (Text Message and WeChat) Intervention to Improve the Medication Adherence and Quality of Life of People Living with HIV in China: Pilot Randomized Controlled Trial , 2018, JMIR mHealth and uHealth.

[17]  Steffen Moritz,et al.  A randomized controlled trial on a smartphone self-help application (Be Good to Yourself) to reduce depressive symptoms , 2018, Psychiatry Research.

[18]  Adrian B. R. Shatte,et al.  A Mobile App–Based Intervention for Depression: End-User and Expert Usability Testing Study , 2018, JMIR mental health.

[19]  Michael E. Levin,et al.  Tailoring Acceptance and Commitment Therapy Skill Coaching in the Moment Through Smartphones: Results from a Randomized Controlled Trial , 2018, Mindfulness.

[20]  John C. Sieverdes,et al.  Meditation Smartphone Application Effects on Prehypertensive Adults’ Blood Pressure: Dose-Response Feasibility Trial , 2018, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[21]  Michael E. Levin,et al.  Comparing Cognitive Defusion and Cognitive Restructuring Delivered Through a Mobile App for Individuals High in Self-Criticism , 2018, Cognitive Therapy and Research.

[22]  D. Wilfley,et al.  Dropout from interpersonal psychotherapy for mental health disorders: A systematic review and meta-analysis , 2018, Psychotherapy research : journal of the Society for Psychotherapy Research.

[23]  Oksana Zelenko,et al.  Efficacy and Outcomes of a Music-Based Emotion Regulation Mobile App in Distressed Young People: Randomized Controlled Trial , 2018, JMIR mHealth and uHealth.

[24]  Danielle A Schlosser,et al.  Efficacy of PRIME, a Mobile App Intervention Designed to Improve Motivation in Young People With Schizophrenia , 2018, Schizophrenia bulletin.

[25]  John Torous,et al.  Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements , 2018, Evidence Based Journals.

[26]  J. Potter,et al.  Quality of life among women diagnosed with breast Cancer: A randomized waitlist controlled trial of commercially available mobile app‐delivered mindfulness training , 2018, Psycho-oncology.

[27]  S. Westermann,et al.  A Mobile App for Social Anxiety Disorder: A Three-Arm Randomized Controlled Trial Comparing Mobile and PC-Based Guided Self-Help Interventions , 2018, Journal of consulting and clinical psychology.

[28]  C. Stonnington,et al.  Resilience Training for Work-Related Stress Among Health Care Workers: Results of a Randomized Clinical Trial Comparing In-Person and Smartphone-Delivered Interventions , 2018, Journal of occupational and environmental medicine.

[29]  Robert E Drake,et al.  Mobile Health (mHealth) Versus Clinic-Based Group Intervention for People With Serious Mental Illness: A Randomized Controlled Trial. , 2018, Psychiatric services.

[30]  J. Greeson,et al.  Effects of mindfulness training programmes delivered by a self-directed mobile app and by telephone compared with an education programme for survivors of critical illness: a pilot randomised clinical trial , 2018, Thorax.

[31]  M. Twohig,et al.  A meta-analysis of dropout rates in acceptance and commitment therapy. , 2018, Behaviour research and therapy.

[32]  Rika M. L. Meyer,et al.  Happier Healers: Randomized Controlled Trial of Mobile Mindfulness for Stress Management. , 2018, Journal of alternative and complementary medicine.

[33]  B. Verkuil,et al.  Effectiveness of a smartphone-based worry-reduction training for stress reduction: A randomized-controlled trial , 2018, Psychology & health.

[34]  Min-Sup Shin,et al.  Effects of smartphone-based memory training for older adults with subjective memory complaints: a randomized controlled trial , 2018, Aging & mental health.

[35]  S. Stjernswärd,et al.  Effectiveness and usability of a web-based mindfulness intervention for caregivers of people with mental or somatic illness. A randomized controlled trial , 2018, Internet interventions.

[36]  Marcos Economides,et al.  Improvements in Stress, Affect, and Irritability Following Brief Use of a Mindfulness-based Smartphone App: A Randomized Controlled Trial , 2018, Mindfulness.

[37]  J. Smyth,et al.  Acceptance lowers stress reactivity: Dismantling mindfulness training in a randomized controlled trial , 2018, Psychoneuroendocrinology.

[38]  M. Jung,et al.  Evaluation of an mHealth App (DeStressify) on University Students’ Mental Health: Pilot Trial , 2018, JMIR mental health.

[39]  HurJi-Won,et al.  A Scenario-Based Cognitive Behavioral Therapy Mobile App to Reduce Dysfunctional Beliefs in Individuals with Depression: A Randomized Controlled Trial. , 2018 .

[40]  N. Kazantzis,et al.  Systematic review and meta-analysis of dropout rates in individual psychotherapy for generalized anxiety disorder. , 2017, Journal of anxiety disorders.

[41]  Marina Christoforou,et al.  Two Novel Cognitive Behavioral Therapy–Based Mobile Apps for Agoraphobia: Randomized Controlled Trial , 2017, Journal of medical Internet research.

[42]  Kien Hoa Ly,et al.  A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods , 2017, Internet interventions.

[43]  John Torous,et al.  Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. , 2017, Journal of affective disorders.

[44]  Michelle E. Costanzo,et al.  An App a Day Keeps the Doctor Away: Guided Education and Training via Smartphones in Subthreshold Post Traumatic Stress Disorder , 2017, Cyberpsychology Behav. Soc. Netw..

[45]  M. Berking,et al.  Reducing procrastination using a smartphone-based treatment program: A randomized controlled pilot study , 2017, Internet interventions.

[46]  Michael E. Levin,et al.  The acceptance and commitment therapy matrix mobile app: A pilot randomized trial on health behaviors , 2017 .

[47]  T. Dennis-Tiwary,et al.  Salutary effects of an attention bias modification mobile application on biobehavioral measures of stress and anxiety during pregnancy , 2017, Biological Psychology.

[48]  Jaap Lancee,et al.  Efficacy of a Mindfulness-Based Mobile Application: a Randomized Waiting-List Controlled Trial , 2017, Mindfulness.

[49]  Simone Orlowski,et al.  Efficacy of a Web-Based Guided Recommendation Service for a Curated List of Readily Available Mental Health and Well-Being Mobile Apps for Young People: Randomized Controlled Trial , 2017, Journal of medical Internet research.

[50]  Derek J Smolenski,et al.  A Virtual Hope Box: Randomized Controlled Trial of a Smartphone App for Emotional Regulation and Coping With Distress. , 2017, Psychiatric services.

[51]  Corine H. G. Horsch,et al.  Mobile Phone-Delivered Cognitive Behavioral Therapy for Insomnia: A Randomized Waitlist Controlled Trial , 2017, Journal of medical Internet research.

[52]  Marcus A. Badgeley,et al.  The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit , 2017, Nature Biotechnology.

[53]  C. B. Taylor,et al.  A Randomized Controlled Trial of a Smartphone App for Posttraumatic Stress Disorder Symptoms , 2017, Journal of consulting and clinical psychology.

[54]  H. Christensen,et al.  Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial , 2017, BMJ Open.

[55]  David C. Atkins,et al.  The Use and Effectiveness of Mobile Apps for Depression: Results From a Fully Remote Clinical Trial , 2016, Journal of medical Internet research.

[56]  Kien Hoa Ly,et al.  Guided and unguided Acceptance and Commitment Therapy for social anxiety disorder and/or panic disorder provided via the Internet and a smartphone application: A randomized controlled trial. , 2016, Journal of anxiety disorders.

[57]  William H. Collinge,et al.  Post-9/11 Veterans and Their Partners Improve Mental Health Outcomes with a Self-directed Mobile and Web-based Wellness Training Program: A Randomized Controlled Trial , 2016, Journal of medical Internet research.

[58]  M. Hogan,et al.  A protocol for a randomised active-controlled trial to evaluate the effects of an online mindfulness intervention on executive control, critical thinking and key thinking dispositions in a university student sample , 2016, BMC psychology.

[59]  Evan M. Kleiman,et al.  A brief mobile app reduces nonsuicidal and suicidal self-injury: Evidence from three randomized controlled trials. , 2016, Journal of consulting and clinical psychology.

[60]  N. Rickard,et al.  Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments , 2016, JMIR mental health.

[61]  P. Cuijpers,et al.  Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial , 2016, Occupational and Environmental Medicine.

[62]  D. Ary,et al.  MoodHacker Mobile Web App With Email for Adults to Self-Manage Mild-to-Moderate Depression: Randomized Controlled Trial , 2016, JMIR mHealth and uHealth.

[63]  Julia E Hoffman,et al.  Feasibility, acceptability, and potential efficacy of the PTSD Coach app: A pilot randomized controlled trial with community trauma survivors. , 2016, Psychological trauma : theory, research, practice and policy.

[64]  H. Whiteford,et al.  Establishing and Governing e-Mental Health Care in Australia: A Systematic Review of Challenges and A Call For Policy-Focussed Research , 2016, Journal of medical Internet research.

[65]  E. Stuart,et al.  Multiple Imputation: A Flexible Tool for Handling Missing Data. , 2015, JAMA.

[66]  J. Swift,et al.  Meta-analysis of dropout from cognitive behavioral therapy: Magnitude, timing, and moderators. , 2015, Journal of consulting and clinical psychology.

[67]  A. Cooper,et al.  Dropout from individual psychotherapy for major depression: A meta-analysis of randomized clinical trials. , 2015, Clinical psychology review.

[68]  C. Ritz,et al.  Daily electronic self-monitoring in bipolar disorder using smartphones – the MONARCA I trial: a randomized, placebo-controlled, single-blind, parallel group trial , 2015, Psychological Medicine.

[69]  Peter Schulthess,et al.  Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy , 2015, Front. Psychol..

[70]  Dana Rosen,et al.  Mobile Technology Boosts the Effectiveness of Psychotherapy and Behavioral Interventions , 2015, Behavior modification.

[71]  Rebecca L. Schneider,et al.  The state of personalized treatment for anxiety disorders: A systematic review of treatment moderators. , 2015, Clinical psychology review.

[72]  J. McGonigal,et al.  Randomized Controlled Trial of SuperBetter, a Smartphone-Based/Internet-Based Self-Help Tool to Reduce Depressive Symptoms. , 2015, Games for health journal.

[73]  N. Lindefors,et al.  Living SMART — A randomized controlled trial of a guided online course teaching adults with ADHD or sub-clinical ADHD to use smartphones to structure their everyday life , 2015 .

[74]  Giuseppe Riva,et al.  Does a Meditation Protocol Supported by a Mobile Application Help People Reduce Stress? Suggestions from a Controlled Pragmatic Trial , 2015, Cyberpsychology Behav. Soc. Netw..

[75]  Gerhard Andersson,et al.  Stress management for middle managers via an acceptance and commitment-based smartphone application: A randomized controlled trial , 2014 .

[76]  G. Andersson,et al.  Cognitive behavior therapy versus interpersonal psychotherapy for social anxiety disorder delivered via smartphone and computer: a randomized controlled trial. , 2014, Journal of anxiety disorders.

[77]  S. Hofmann,et al.  Attention Bias Modification Training Via Smartphone to Reduce Social Anxiety: A Randomized, Controlled Multi-Session Experiment , 2014, Cognitive Therapy and Research.

[78]  N. Rüsch,et al.  What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies , 2014, Psychological Medicine.

[79]  Per Carlbring,et al.  Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: a randomised controlled trial , 2014, BMJ Open.

[80]  Helen Christensen,et al.  Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial , 2013, BMC Psychiatry.

[81]  Gavin Andrews,et al.  CBT for depression: a pilot RCT comparing mobile phone vs. computer , 2013, BMC Psychiatry.

[82]  O. Kristjansdottir,et al.  A Smartphone-Based Intervention With Diaries and Therapist-Feedback to Reduce Catastrophizing and Increase Functioning in Women With Chronic Widespread Pain: Randomized Controlled Trial , 2013, Journal of medical Internet research.

[83]  Lena Sanci,et al.  Self-monitoring Using Mobile Phones in the Early Stages of Adolescent Depression: Randomized Controlled Trial , 2012, Journal of medical Internet research.

[84]  Derek Richards,et al.  Computer-based psychological treatments for depression: a systematic review and meta-analysis. , 2012, Clinical psychology review.

[85]  J. Swift,et al.  Premature discontinuation in adult psychotherapy: a meta-analysis. , 2012, Journal of consulting and clinical psychology.

[86]  Leslie R. Martin,et al.  Improving patient adherence: a three-factor model to guide practice , 2012 .

[87]  Bruce Neal,et al.  A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies , 2011, Journal of medical Internet research.

[88]  Zarnie Khadjesari,et al.  Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials , 2011, Journal of medical Internet research.

[89]  A. Kazdin,et al.  Rebooting Psychotherapy Research and Practice to Reduce the Burden of Mental Illness , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[90]  Leanne M. Casey,et al.  Dropout from Internet-based treatment for psychological disorders. , 2010, The British journal of clinical psychology.

[91]  Vikram Patel,et al.  Global mental health: a new global health field comes of age. , 2010, JAMA.

[92]  J. Smyth,et al.  Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. , 2010, British journal of health psychology.

[93]  C. Ferguson An effect size primer: A guide for clinicians and researchers. , 2009 .

[94]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[95]  H. Christensen,et al.  Adherence in Internet Interventions for Anxiety and Depression: Systematic Review , 2009, Journal of medical Internet research.

[96]  G. Eysenbach The Law of Attrition Revisited – Author’s Reply , 2006, Journal of Medical Internet Research.

[97]  Chris Kelleher,et al.  Overcoming Depression on the Internet (ODIN) (2): A Randomized Trial of a Self-Help Depression Skills Program With Reminders , 2005, Journal of medical Internet research.

[98]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[99]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[100]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[101]  Alan E Kazdin,et al.  Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. , 2017, Behaviour research and therapy.

[102]  Julia E Hoffman,et al.  Using PTSD Coach in primary care with and without clinician support: a pilot randomized controlled trial. , 2016, General hospital psychiatry.