Model Formulation: Methodologic Issues in Health Informatics Trials: The Complexities of Complex Interventions

OBJECTIVE All electronic health (e-health) interventions require validation as health information technologies, ideally in randomized controlled trial settings. However, as with other types of complex interventions involving various active components and multiple targets, health informatics trials often experience problems of design, methodology, or analysis that can influence the results and acceptance of the research. Our objective was to review selected key methodologic issues in conducting and reporting randomized controlled trials in health informatics, provide examples from a recent study, and present practical recommendations. DESIGN For illustration, we use the COMPETE III study, a large randomized controlled clinical trial investigating the impact of a shared decision-support system on the quality of vascular disease management in Ontario, Canada. RESULTS We describe a set of methodologic, logistic, and statistical issues that should be considered when planning and implementing trials of complex e-health interventions, and provide practical recommendations for health informatics trialists. CONCLUSIONS Our recommendations emphasize validity and pragmatic considerations and would be useful for health informaticians conducting or evaluating e-health studies.

[1]  Gordon H. Guyatt,et al.  Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice , 2003, BMJ : British Medical Journal.

[2]  J K Iglehart.  The media as messenger. , 1998, Health affairs.

[3]  D G Altman,et al.  Concealing treatment allocation in randomised trials , 2001, BMJ : British Medical Journal.

[4]  M Eccles,et al.  Why does primary care need more implementation research? , 2001, Family practice.

[5]  Gordon H Guyatt,et al.  Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials , 2007, BMJ : British Medical Journal.

[6]  Steven Hawken,et al.  Preventive cardiologyAbstractsEffect of potentially modifiable risk factors associated with myocardial infarction in 52 countries (the INTERHEART study): Case-control study , 2004 .

[7]  J. Hanley,et al.  The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. , 2003, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[8]  D M Eddy,et al.  Performance measurement: problems and solutions. , 1998, Health affairs.

[9]  Hude Quan,et al.  Bmc Medical Research Methodology Open Access Dealing with Missing Data in a Multi-question Depression Scale: a Comparison of Imputation Methods , 2022 .

[10]  G. Guyatt,et al.  Double blind, you have been voted off the island! , 2002, Evidence-based mental health.

[11]  David J Torgerson,et al.  Contamination in trials: is cluster randomisation the answer? , 2001, BMJ : British Medical Journal.

[12]  Allan Donner,et al.  Design and Analysis of Cluster Randomization Trials in Health Research , 2001 .

[13]  Debbe Thompson,et al.  A framework for evaluating eHealth research. , 2006, Evaluation and program planning.

[14]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005, JAMA.

[15]  D. Torgerson,et al.  Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals , 2003, BMJ : British Medical Journal.

[16]  Peter C Gøtzsche,et al.  [Better reporting of harms in randomized trials: an extension of the CONSORT statement]. , 2005, Ugeskrift for laeger.

[17]  G. Guyatt,et al.  A INTRODUCTION : THE PHILOSOPHY OF EVIDENCE-BASED MEDICINE , 2001 .

[18]  D. Moher,et al.  The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration , 2001, Annals of Internal Medicine.

[19]  G. Guyatt,et al.  Validity of composite end points in clinical trials , 2005, BMJ : British Medical Journal.

[20]  C. Delpierre,et al.  A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[21]  S Trent Rosenbloom,et al.  Editorial Comments: Approaches to Evaluating Electronic Prescribing , 2006, J. Am. Medical Informatics Assoc..

[22]  Diane L. Seger,et al.  Application of Information Technology: Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care , 2006, J. Am. Medical Informatics Assoc..

[23]  Thomas K. Houston,et al.  Data Quality in the Outpatient Setting: Impact on Clinical Decision Support Systems , 2005, AMIA.

[24]  David Moher,et al.  Reporting Methods of Blinding in Randomized Trials Assessing Nonpharmacological Treatments , 2007, PLoS medicine.

[25]  Nick Freemantle,et al.  Composite outcomes in randomized trials: greater precision but with greater uncertainty? , 2003, JAMA.

[26]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[27]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[28]  P. Sandercock,et al.  Framework for design and evaluation of complex interventions to improve health , 2000, BMJ : British Medical Journal.

[29]  Edward H. Shortliffe,et al.  Evaluation Methods in Biomedical Informatics , 2000 .

[30]  J. Grimshaw,et al.  Research designs for studies evaluating the effectiveness of change and improvement strategies , 2003, Quality & safety in health care.

[31]  Kenneth F Schulz,et al.  Blinding in randomised trials: hiding who got what , 2002, The Lancet.

[32]  S. T. Rosenbloom,et al.  Comments Approaches to Evaluating Electronic Prescribing , 2006 .

[33]  D. Beaton,et al.  Many faces of the minimal clinically important difference (MCID): a literature review and directions for future research. , 2002, Current opinion in rheumatology.

[34]  Eli N. Perencevich,et al.  Impact of a computerized clinical decision support system on reducing inappropriate antimicrobial use: a randomized controlled trial. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[35]  E Coiera,et al.  Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems , 2006, Yearbook of Medical Informatics.

[36]  Kevin J. Anstrom,et al.  Economic Evaluation in Medical Information Technology: Why the Numbers Don't Add Up , 2006, AMIA.

[37]  George Hripcsak,et al.  Review Paper: Design and Analysis of Controlled Trials in Naturally Clustered Environments: Implications for Medical Informatics , 2002, J. Am. Medical Informatics Assoc..