Patterns of engagement in a digital mental health service during COVID-19: a cohort study for children and young people

Introduction The COVID-19 pandemic increased public use of digital mental health technologies. However, little is known about changes in user engagement with these platforms during the pandemic. This study aims to assess engagement changes with a digital mental healthcare service during COVID-19. Methods A cohort study based on routinely collected service usage data from a digital mental health support service in the United Kingdom. Returning users aged 14–25 years that interacted for a maximum of two months were included. The study population was divided into pre-COVID and COVID cohorts. Demographic and usage information between cohorts were compared and usage clusters were identified within each cohort. Differences were tested using Chi-squared, two-sample Kolmogorov–Smirnov tests and logit regressions. Results Of the 624,103 users who joined the service between May 1, 2019, and October 1, 2021, 18,889 (32.81%) met the inclusion criteria: 5,048 in the pre-COVID cohort and 13,841 in the COVID cohort. The COVID cohort wrote more journals; maintained the same focus on messaging practitioners, posting discussions, commenting on posts, and having booked chats; and engaged less in writing journals, setting personal goals, posting articles, and having ad-hoc chats. Four usage profiles were identified in both cohorts: one relatively disengaged, one focused on contacting practitioners through chats/messages, and two broadly interested in writing discussions and comments within the digital community. Despite their broad similarities, usage patterns also exhibited differences between cohorts. For example, all four clusters had over 70% of users attempting to have ad-hoc chats with practitioners in the pre-COVID cohort, compared to one out of four clusters in the COVID cohort. Overall, engagement change patterns during the COVID-19 pandemic were not equal across clusters. Sensitivity analysis revealed varying strength of these defined clusters. Discussion Our study identified changes in user activity and engagement behavior within a digital mental healthcare service during the COVID-19 pandemic. These findings suggest that usage patterns within digital mental health services may be susceptible to change in response to external events such as a pandemic. Continuous monitoring of engagement patterns is important for informed design and personalized interventions.

[1]  D. de Leo,et al.  Evaluation of the Use of Digital Mental Health Platforms and Interventions: Scoping Review , 2022, International journal of environmental research and public health.

[2]  L. Bécares,et al.  Inequalities in mental health, self-rated health, and social support among sexual minority young adults during the COVID-19 pandemic: analyses from the UK Millennium Cohort Study , 2022, Social Psychiatry and Psychiatric Epidemiology.

[3]  L. Salhi,et al.  Alienated and unsafe: Experiences of the first national UK COVID‐19 lockdown for vulnerable young people (aged 11–24 years) as revealed in Web‐based therapeutic sessions with mental health professionals , 2022, Counselling and psychotherapy research.

[4]  R. Stewart,et al.  Changes in daily mental health service use and mortality at the commencement and lifting of COVID-19 ‘lockdown’ policy in 10 UK sites: a regression discontinuity in time design , 2021, BMJ Open.

[5]  Jiayuan Wang,et al.  A Review on Data Preprocessing Techniques Toward Efficient and Reliable Knowledge Discovery From Building Operational Data , 2021, Frontiers in Energy Research.

[6]  E. Stip,et al.  Disaster management of the psychological impact of the COVID-19 pandemic , 2021, International Journal of Emergency Medicine.

[7]  I. Scott,et al.  Impact of COVID-19 pandemic on utilisation of healthcare services: a systematic review , 2021, BMJ Open.

[8]  Richard Armitage.  Antidepressants, primary care, and adult mental health services in England during COVID-19 , 2021, The Lancet Psychiatry.

[9]  C. Chew‐Graham,et al.  Effects of the COVID-19 pandemic on primary care-recorded mental illness and self-harm episodes in the UK: a population-based cohort study , 2021, The Lancet. Public health.

[10]  J. Hemberg,et al.  Adolescents’ and young adults’ experiences of loneliness and their thoughts about its alleviation , 2021, International Journal of Adolescence and Youth.

[11]  L. Major,et al.  Generation COVID: emerging work and education inequalities , 2020 .

[12]  D. O'Connor,et al.  Mental health and well-being during the COVID-19 pandemic: longitudinal analyses of adults in the UK COVID-19 Mental Health & Wellbeing study , 2020, The British Journal of Psychiatry.

[13]  Karola V Kreitmair,et al.  Ethics of Digital Mental Health During COVID-19: Crisis and Opportunities , 2020, JMIR mental health.

[14]  Jorge E. Palacios,et al.  A Machine Learning Approach to Understanding Patterns of Engagement With Internet-Delivered Mental Health Interventions , 2020, JAMA network open.

[15]  Moon Joon Kim,et al.  Unintended consequences of healthcare reform in South Korea: evidence from a regression discontinuity in time design , 2019, Health Research Policy and Systems.

[16]  H. Christensen,et al.  Using Cluster Analysis to Explore Engagement and e-Attainment as Emergent Behavior in Electronic Mental Health , 2019, Journal of medical Internet research.

[17]  Johannes L. Schönberger,et al.  SciPy 1.0: fundamental algorithms for scientific computing in Python , 2019, Nature Methods.

[18]  T. Fleming,et al.  The Importance of User Segmentation for Designing Digital Therapy for Adolescent Mental Health: Findings From Scoping Processes , 2019, JMIR mental health.

[19]  P. Topham,et al.  Interaction and Engagement with an Anxiety Management App: Analysis Using Large-Scale Behavioral Data , 2018, JMIR mental health.

[20]  Stephen H. Bell,et al.  A ?scoping review. , 2018, Sexual health.

[21]  P. Stallard,et al.  Mental Health Mobile Apps for Preadolescents and Adolescents: A Systematic Review , 2017, Journal of medical Internet research.

[22]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[23]  R. Griffith.  What is Gillick competence? , 2016, Human vaccines & immunotherapeutics.

[24]  J. Clarke,et al.  What is a systematic review? , 2011, Evidence Based Nursing.

[25]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[26]  M. Greenacre,et al.  Multiple Correspondence Analysis and Related Methods , 2006 .

[27]  L. Brown,et al.  Interval Estimation for a Binomial Proportion , 2001 .

[28]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[29]  M. Stephens.  EDF Statistics for Goodness of Fit and Some Comparisons , 1974 .

[30]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[31]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[32]  Coronavirus: Impact on young people with mental health needs , 2020 .

[33]  Na Hyun Kim,et al.  Cross-sectional and longitudinal association between hemoglobin concentration and hypertension , 2016, Medicine.

[34]  Vijayan N. Nair,et al.  A REVIEW AND RECENT DEVELOPMENTS , 2005 .

[35]  J. L. Hodges,et al.  The significance probability of the smirnov two-sample test , 1958 .

[36]  Tackling the mental health impact of the COVID-19 crisis: An integrated, whole-of-society response , 2022, OECD Policy Responses to Coronavirus (COVID-19).