Analyzing the Impact of Mobile App Engagement on Mental Health Outcomes: Secondary Analysis of the Unwinding Anxiety Program

Background App-based interventions provide a promising avenue for mitigating the burden on mental health services by complimenting therapist-led treatments for anxiety. However, it remains unclear how specific systems’ use of app features may be associated with changes in mental health outcomes (eg, anxiety and worry). Objective This study was a secondary analysis of engagement data from a stage 1 randomized controlled trial testing the impact of the Unwinding Anxiety mobile app among adults with generalized anxiety disorder. The aims of this study were 2-fold: to investigate whether higher microengagement with the primary intervention feature (ie, educational modules) is associated with positive changes in mental health outcomes at 2 months (ie, anxiety, worry, interoceptive awareness, and emotional reactivity) and to investigate whether the use of adjunctive app features is also associated with changes in mental health outcomes. Methods We analyzed the intervention group during the stage 1 trial of the Unwinding Anxiety mobile app. The total use of specific mobile app features and the use specific to each feature were calculated. We used multivariate linear models with a priori significance of α=.05 to investigate the impact of cumulative app use on anxiety, worry, interoceptive awareness, and emotional regulation at 2 months, controlling for baseline scores, age, and education level in all models. Significant relationships between system use metrics and baseline participant characteristics were assessed for differences in use groupings using between-group testing (ie, 2-tailed t tests for continuous data and chi-square analyses for categorical data). Results The sample was primarily female (25/27, 93%), and the average age was 42.9 (SD 15.6) years. Educational module completion, the central intervention component, averaged 20.2 (SD 11.4) modules out of 32 for the total sample. Multivariate models revealed that completing >75% of the program was associated with an average 22.6-point increase in interoceptive awareness (b=22.6; SE 8.32; P=.01; 95% CI 5.3-39.8) and an 11.6-point decrease in worry (b=−11.6; SE 4.12; P=.01; 95% CI −20.2 to −3.1). In addition, a single log unit change in the total number of meditations was associated with a 0.62-point reduction in the Generalized Anxiety Disorder-7 scale scores (b=0.62; SE 0.27; P=.005; 95% CI −1.2 to −0.6), whereas a single log unit use of the stress meter was associated with an average of a 0.5-point increase in emotional regulation scores (Five Facet Mindfulness Questionnaire; b=0.5; SE 0.21; P=.03; 95% CI 0.1-0.9). Conclusions This study offers a clearer understanding of the impact of engagement with app features on broader engagement with the health outcomes of interest. This study highlights the importance of comprehensive investigations of engagement during the development of evidence-based mobile apps.

[1]  R. Sanderman,et al.  White Paper: Open Digital Health – accelerating transparent and scalable health promotion and treatment , 2022, Health psychology review.

[2]  L. Gega,et al.  Digital Interventions for Generalized Anxiety Disorder (GAD): Systematic Review and Network Meta-Analysis , 2021, Frontiers in Psychiatry.

[3]  Helen Christensen,et al.  Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis , 2021, Frontiers in Digital Health.

[4]  Esti Iturralde,et al.  Opportunities to Integrate Mobile App–Based Interventions Into Mental Health and Substance Use Disorder Treatment Services in the Wake of COVID-19 , 2021, American journal of health promotion : AJHP.

[5]  C. Botella,et al.  Internet- and mobile-based interventions for the treatment of specific phobia: A systematic review and preliminary meta-analysis , 2021, Internet interventions.

[6]  P. Anderson,et al.  Engagement with mobile health interventions for depression: A systematic review , 2021, Internet interventions.

[7]  M. Hudgens,et al.  Interpreting the Results of Intention-to-Treat, Per-Protocol, and As-Treated Analyses of Clinical Trials. , 2021, JAMA.

[8]  J. Brewer,et al.  Awareness drives changes in reward value which predict eating behavior change: Probing reinforcement learning using experience sampling from mobile mindfulness training for maladaptive eating , 2021, Journal of behavioral addictions.

[9]  L. Yardley,et al.  Adapting Behavioral Interventions for a Changing Public Health Context: A Worked Example of Implementing a Digital Intervention During a Global Pandemic Using Rapid Optimisation Methods , 2021, Frontiers in Public Health.

[10]  D. Mukamel,et al.  Barriers to and Facilitators of User Engagement With Digital Mental Health Interventions: Systematic Review , 2021, Journal of medical Internet research.

[11]  F. Gunning,et al.  Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement , 2021, npj Digital Medicine.

[12]  H. van Hell,et al.  The effects of the COVID-19 outbreak and measures in patients with a pre-existing psychiatric diagnosis: A cross-sectional study , 2021, Journal of Affective Disorders Reports.

[13]  Elizabeth A. Hoge,et al.  Clinical Efficacy and Psychological Mechanisms of an App-Based Digital Therapeutic for Generalized Anxiety Disorder: Randomized Controlled Trial , 2021, Journal of medical Internet research.

[14]  H. O'Brien,et al.  Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement , 2020, BJPsych Open.

[15]  H. Hahm,et al.  Evidence for elevated psychiatric distress, poor sleep, and quality of life concerns during the COVID-19 pandemic among U.S. young adults with suspected and reported psychiatric diagnoses , 2020, Psychiatry Research.

[16]  A. Powell,et al.  Health Information Technology Resources to Support Measurement-Based Care. , 2020, Child and adolescent psychiatric clinics of North America.

[17]  J. Leiferman,et al.  Mental Health During the COVID-19 Pandemic in the United States: Online Survey (Preprint) , 2020 .

[18]  B. Tran,et al.  Do psychiatric patients experience more psychiatric symptoms during COVID-19 pandemic and lockdown? A case-control study with service and research implications for immunopsychiatry , 2020, Brain, Behavior, and Immunity.

[19]  F. Naughton,et al.  Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review , 2020, Journal of medical Internet research.

[20]  J. Torous,et al.  A Narrative Review of Methods for Applying User Experience in the Design and Assessment of Mental Health Smartphone Interventions , 2020, International Journal of Technology Assessment in Health Care.

[21]  I. Vedel,et al.  Optimizing smartphone intervention features to improve chronic disease management: A rapid review , 2019, Health Informatics J..

[22]  M. V. van Velthoven,et al.  Mobile Apps for Health Behavior Change in Physical Activity, Diet, Drug and Alcohol Use, and Mental Health: Systematic Review , 2019, JMIR mHealth and uHealth.

[23]  H. Cole-Lewis,et al.  Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions , 2019, JMIR formative research.

[24]  Dori M. Steinberg,et al.  Dose-response research in digital health interventions: Concepts, considerations, and challenges. , 2019, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[25]  A. Hoffart,et al.  Psychometric Properties of the General Anxiety Disorder 7-Item (GAD-7) Scale in a Heterogeneous Psychiatric Sample , 2019, Front. Psychol..

[26]  J. Brewer,et al.  Physician Anxiety and Burnout: Symptom Correlates and a Prospective Pilot Study of App-Delivered Mindfulness Training , 2019, JMIR mHealth and uHealth.

[27]  C. Granell,et al.  Smartphone Apps for the Treatment of Mental Disorders: Systematic Review , 2019, JMIR mHealth and uHealth.

[28]  Douglas G. Altman,et al.  The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design , 2019, BMJ.

[29]  J. Brewer,et al.  Quitting starts in the brain: a randomized controlled trial of app-based mindfulness shows decreases in neural responses to smoking cues that predict reductions in smoking , 2019, Neuropsychopharmacology.

[30]  John Torous,et al.  User Engagement in Mental Health Apps: A Review of Measurement, Reporting, and Validity. , 2019, Psychiatric services.

[31]  L. Collins,et al.  Achieving the Goals of Translational Science in Public Health Intervention Research: The Multiphase Optimization Strategy (MOST). , 2019, American journal of public health.

[32]  C. Vandelanotte,et al.  Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies , 2018, Journal of medical Internet research.

[33]  Mattia Prosperi,et al.  A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. , 2018, Journal of psychiatric research.

[34]  D. Scheinost,et al.  Craving to Quit: A Randomized Controlled Trial of Smartphone app-based Mindfulness Training for Smoking Cessation. , 2018, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[35]  John Torous,et al.  Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements , 2018, Evidence Based Journals.

[36]  J. Brewer,et al.  Testing a mobile mindful eating intervention targeting craving-related eating: feasibility and proof of concept , 2018, Journal of Behavioral Medicine.

[37]  Mona Choi,et al.  Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review , 2018, BMC Medical Informatics and Decision Making.

[38]  P. Cuijpers,et al.  Anxiety: There is an app for that. A systematic review of anxiety apps , 2017, Depression and anxiety.

[39]  Ann Blandford,et al.  Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis , 2016, Translational behavioral medicine.

[40]  Kien Hoa Ly,et al.  Guided and unguided Acceptance and Commitment Therapy for social anxiety disorder and/or panic disorder provided via the Internet and a smartphone application: A randomized controlled trial. , 2016, Journal of anxiety disorders.

[41]  Ann Blandford,et al.  Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions. , 2016, American journal of preventive medicine.

[42]  B. Molinuevo,et al.  Internet-based self-help treatment for panic disorder: a randomized controlled trial comparing mandatory versus optional complementary psychological support , 2016, Cognitive behaviour therapy.

[43]  T. Joiner,et al.  Evaluating factors and interventions that influence help-seeking and mental health service utilization among suicidal individuals: A review of the literature. , 2015, Clinical psychology review.

[44]  Miranda Olff,et al.  Mobile mental health: a challenging research agenda , 2015, European journal of psychotraumatology.

[45]  Susie Skarl,et al.  Anxiety and Depression Association of America , 2015 .

[46]  J. Boettcher,et al.  Internet-based guided self-help for several anxiety disorders: a randomized controlled trial comparing a tailored with a standardized disorder-specific approach. , 2014, Psychotherapy.

[47]  Helen Christensen,et al.  Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial , 2013, BMC Psychiatry.

[48]  G. Andersson,et al.  Predictors of outcome in Internet-based cognitive behavior therapy for severe health anxiety. , 2013, Behaviour research and therapy.

[49]  K. Bucholz,et al.  DSM-5 criteria for substance use disorders: recommendations and rationale. , 2013, The American journal of psychiatry.

[50]  A. Stewart,et al.  The Multidimensional Assessment of Interoceptive Awareness (MAIA) , 2012, PloS one.

[51]  G. Andersson,et al.  Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. , 2011, Behaviour research and therapy.

[52]  A. Stewart,et al.  Body Awareness: Construct and Self-Report Measures , 2009, PloS one.

[53]  Gregory T. Smith,et al.  Construct Validity of the Five Facet Mindfulness Questionnaire in Meditating and Nonmeditating Samples , 2008, Assessment.

[54]  Patrick Monahan,et al.  Anxiety Disorders in Primary Care: Prevalence, Impairment, Comorbidity, and Detection , 2007, Annals of Internal Medicine.

[55]  B. Löwe,et al.  A brief measure for assessing generalized anxiety disorder: the GAD-7. , 2006, Archives of internal medicine.

[56]  Z. Segal,et al.  Mindfulness: A Proposed Operational Definition , 2004 .

[57]  Gregory T. Smith,et al.  Using Self-Report Assessment Methods to Explore Facets of Mindfulness , 2006, Assessment.

[58]  C. DeVane,et al.  Anxiety disorders in the 21st century: status, challenges, opportunities, and comorbidity with depression. , 2005, The American journal of managed care.

[59]  A. Afifi,et al.  Comparison of Stopping Rules in Forward “Stepwise” Regression , 1977 .

[60]  M. Lekander,et al.  Predictors in Internet-delivered cognitive behavior therapy and behavioral stress management for severe health anxiety. , 2015, Behaviour research and therapy.

[61]  D. Sheehan,et al.  The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. , 1998, The Journal of clinical psychiatry.

[62]  J. Kabat-Zinn,et al.  Full catastrophe living : using the wisdom of your body and mind to face stress, pain, and illness , 1990 .

[63]  T J Meyer,et al.  Development and validation of the Penn State Worry Questionnaire. , 1990, Behaviour research and therapy.