Methodological Challenges in Web-Based Trials: Update and Insights From the Relatives Education and Coping Toolkit Trial

There has been a growth in the number of web-based trials of web-based interventions, adding to an increasing evidence base for their feasibility and effectiveness. However, there are challenges associated with such trials, which researchers must address. This discussion paper follows the structure of the Down Your Drink trial methodology paper, providing an update from the literature for each key trial parameter (recruitment, registration eligibility checks, consent and participant withdrawal, randomization, engagement with a web-based intervention, retention, data quality and analysis, spamming, cybersquatting, patient and public involvement, and risk management and adverse events), along with our own recommendations based on designing the Relatives Education and Coping Toolkit randomized controlled trial for relatives of people with psychosis or bipolar disorder. The key recommendations outlined here are relevant for future web-based and hybrid trials and studies using iterative development and test models such as the Accelerated Creation-to-Sustainment model, both within general health research and specifically within mental health research for relatives. Researchers should continue to share lessons learned from conducting web-based trials of web-based interventions to benefit future studies. International Registered Report Identifier (IRRID) RR2-10.1136/bmjopen-2017-016965

[1]  A. Davies,et al.  Recommendations on the Use of Open Surgical and Endovascular Aneurysm Repair for the Management of Unruptured Abdominal Aortic Aneurysm from the Guideline Development Committee Appointed by the UK National Institute for Health and Care Excellence. , 2021, European journal of vascular and endovascular surgery : the official journal of the European Society for Vascular Surgery.

[2]  Steven H. Jones,et al.  Clinical effectiveness of a web-based peer-supported self-management intervention for relatives of people with psychosis or bipolar (REACT): online, observer-blind, randomised controlled superiority trial , 2020, BMC Psychiatry.

[3]  Steven H. Jones,et al.  Online randomised controlled trial to evaluate the clinical and cost-effectiveness of a web-based peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: Relatives’ Education And Coping Toolkit (REACT) , 2019 .

[4]  F. Lobban,et al.  A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach , 2018, JMIR mental health.

[5]  S. Michie,et al.  Predictors of Engagement, Response to Follow Up, and Extent of Alcohol Reduction in Users of a Smartphone App (Drink Less): Secondary Analysis of a Factorial Randomized Controlled Trial , 2018, JMIR mHealth and uHealth.

[6]  J. McClure,et al.  Participant Recruitment and Retention in Remote eHealth Intervention Trials: Methods and Lessons Learned From a Large Randomized Controlled Trial of Two Web-Based Smoking Interventions , 2018, Journal of medical Internet research.

[7]  D. Fitzpatrick-Lewis,et al.  Impact of Internet-Based Interventions on Caregiver Mental Health: Systematic Review and Meta-Analysis , 2018, Journal of medical Internet research.

[8]  E. Murray,et al.  Seven lessons for interdisciplinary research on interactive digital health interventions , 2018, Digital health.

[9]  V. Cornelius,et al.  eHealth interventions for family carers of people with long term illness: A promising approach? , 2018, Clinical psychology review.

[10]  Christopher Whitaker,et al.  The Use of Facebook in Recruiting Participants for Health Research Purposes: A Systematic Review , 2017, Journal of medical Internet research.

[11]  Steven H. Jones,et al.  Protocol for an online randomised controlled trial to evaluate the clinical and cost-effectiveness of a peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: Relatives Education And Coping Toolkit (REACT) , 2017, BMJ Open.

[12]  R. Cooke,et al.  Web-Based Alcohol Intervention: Study of Systematic Attrition of Heavy Drinkers , 2017, Journal of medical Internet research.

[13]  Aaron R Lyon,et al.  Accelerating Digital Mental Health Research From Early Design and Creation to Successful Implementation and Sustainment , 2017, Journal of medical Internet research.

[14]  Ann Blandford,et al.  Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis , 2016, Translational behavioral medicine.

[15]  S. Woolf,et al.  Methods for Evaluating Respondent Attrition in Web-Based Surveys , 2016, Journal of medical Internet research.

[16]  Ann Blandford,et al.  Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions. , 2016, American journal of preventive medicine.

[17]  Elizabeth Murray,et al.  Evaluating Digital Health Interventions: Key Questions and Approaches. , 2016, American journal of preventive medicine.

[18]  Peter Hanlon,et al.  Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies , 2016, BMC Medical Informatics and Decision Making.

[19]  Michael S. Amato,et al.  Engagement and attrition in Internet smoking cessation interventions: Insights from a cross-sectional survey of “one-hit-wonders” , 2016, Internet interventions.

[20]  Lucy L. Gu,et al.  Facebook, Twitter, & Qr codes: An exploratory trial examining the feasibility of social media mechanisms for sample recruitment , 2016, Comput. Hum. Behav..

[21]  R. Niaura,et al.  Baseline Characteristics and Generalizability of Participants in an Internet Smoking Cessation Randomized Trial , 2016, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[22]  E. Pedersen,et al.  Using Facebook for Health-related Research Study Recruitment and Program Delivery. , 2016, Current opinion in psychology.

[23]  P. Batterham,et al.  Recruiting for health, medical or psychosocial research using Facebook: Systematic review , 2016, Internet interventions.

[24]  Adam Gazzaley,et al.  Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense , 2016, BMJ Innovations.

[25]  G. Alkhaldi,et al.  The Effectiveness of Prompts to Promote Engagement With Digital Interventions: A Systematic Review , 2016, Journal of medical Internet research.

[26]  F. Lobban,et al.  What do relatives experience when supporting someone in early psychosis? , 2015, Psychology and psychotherapy.

[27]  R. Ware,et al.  Meta-analysis of randomized controlled trials supports the use of incentives for inducing response to electronic health surveys. , 2014, Journal of clinical epidemiology.

[28]  E. Eisner,et al.  An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia , 2014, Journal of mental health.

[29]  Jessica Kramer,et al.  Strategies to address participant misrepresentation for eligibility in Web‐based research , 2014, International journal of methods in psychiatric research.

[30]  G. Rait,et al.  Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis , 2014, BMJ Open.

[31]  G. Dunn,et al.  Feasibility of a supported self-management intervention for relatives of people with recent-onset psychosis: REACT study , 2013, British Journal of Psychiatry.

[32]  I. White,et al.  Attrition Revisited: Adherence and Retention in a Web-Based Alcohol Trial , 2013, Journal of medical Internet research.

[33]  Alexandra Barratt,et al.  Internet-based randomized controlled trials: a systematic review , 2013, J. Am. Medical Informatics Assoc..

[34]  Heather L. Coley,et al.  Who Participates in Web-Assisted Tobacco Interventions? The Quit-Primo and National Dental Practice-Based Research Network Hi-Quit Studies , 2013, Journal of medical Internet research.

[35]  E. Murray,et al.  The Sexunzipped Trial: Optimizing the Design of Online Randomized Controlled Trials , 2013, Journal of medical Internet research.

[36]  G. Walker,et al.  Peer support in adult mental health services: a metasynthesis of qualitative findings. , 2013, Psychiatric rehabilitation journal.

[37]  A. Barratt,et al.  Internet trials: participant experiences and perspectives , 2012, BMC Medical Research Methodology.

[38]  A. Faulkner,et al.  A helping hand: taking peer support into the 21st century , 2012 .

[39]  Marloes G. Postel,et al.  Attrition in Web-Based Treatment for Problem Drinkers , 2011, Journal of medical Internet research.

[40]  Zarnie Khadjesari,et al.  Impact of Length or Relevance of Questionnaires on Attrition in Online Trials: Randomized Controlled Trial , 2011, Journal of medical Internet research.

[41]  J. Morrow,et al.  Assessing Digital Literacy in Web-Based Physical Activity Surveillance: The WIN Study , 2011, American journal of health promotion : AJHP.

[42]  Tim Carter,et al.  A review of the literature on peer support in mental health services , 2011, Journal of mental health.

[43]  P. Sullivan,et al.  Bias in Online Recruitment and Retention of Racial and Ethnic Minority Men Who Have Sex With Men , 2011, Journal of medical Internet research.

[44]  Zarnie Khadjesari,et al.  Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials , 2011, Journal of medical Internet research.

[45]  Mick P Couper,et al.  Engagement and Retention: Measuring Breadth and Depth of Participant Use of an Online Intervention , 2010, Journal of medical Internet research.

[46]  M. Funk,et al.  Challenges of an Internet-based education intervention in a randomized clinical trial in critical care. , 2010, AACN advanced critical care.

[47]  Lorraine R. Buis,et al.  Barriers encountered during enrollment in an internet-mediated randomized controlled trial , 2009, Trials.

[48]  I. White,et al.  Methodological Challenges in Online Trials , 2009, Journal of medical Internet research.

[49]  M. Hadders‐Algra,et al.  How much loss to follow-up is acceptable in long-term randomised trials and prospective studies? , 2008, Archives of Disease in Childhood.

[50]  M. De Hert,et al.  A study of the family burden of 150 family members of schizophrenic patients , 2004, European Psychiatry.

[51]  E. Struening,et al.  Burden experienced by care-givers of persons with bipolar affective disorder , 1999, British Journal of Psychiatry.

[52]  H. Herrman,et al.  A controlled trial of a counselling intervention for caregivers of relatives with schizophrenia , 1996, Social Psychiatry and Psychiatric Epidemiology.

[53]  D. Goldberg,et al.  A scaled version of the General Health Questionnaire , 1979, Psychological Medicine.

[54]  Josée Poirier,et al.  Effectiveness of a multimodal online well-being intervention: a randomized controlled trial. , 2014, American journal of preventive medicine.

[55]  T. Buchanan,et al.  Ethics Guidelines for Internet-mediated Research , 2013 .

[56]  Nice Schizophrenia: core interventions in the treatment and management , 2005 .

[57]  B. Kable Mental health. , 2005, Australian family physician.

[58]  P. Wallace,et al.  Down your drink: a web-based intervention for people with excessive alcohol consumption. , 2004, Alcohol and alcoholism.