A Meta-analysis of l,6 Randomized Controlled Trials to Evaluate Computer-based Clinical Reminder Systems for Preventive Care in the Ambulatory Setting

Ob’ t* let Ive: Computer-based reminder systems have the potential to change physician and patient behaviors and to improve patient outcomes. We performed a meta-analysis of published randomized controlled trials to assess the overall effectiveness of computer-based reminder systems in ambulatory settings directed at preventive care. Design: Meta-analysis. Search Strategy: Searches of the Medline (1966-1994), Nursing and Allied Health (1982-1994), and Health Planning and Administration (1975-1994) databases identified 16 randomized, controlled trials of computer-based reminder systems in ambulatory settings. Statistical Methods: A weighted mixed effects model regression analysis was used to estimate intervention effects for computer and manual reminder systems for six classes of preventive practices. Main Outcome Measure: Adjusted odds ratio for preventive practices. Results: Computer reminders improved preventive practices compared with the control condition for vaccinations (adjusted odds ratio [OR] 3.09; 95% confidence interval [CI] 2.39-4.00), breast cancer screening (OR 1.88; 95% CI 1.44-2.45), colorectal cancer screening (OR 2.25; 95% CI 1.74-2.91), and cardiovascular risk reduction (OR 2.01; 95% CI 1.55-2.61) but not cervical cancer screening (OR 1.15; 95% CI 0.89-1.49) or other preventive care (OR 1.02; 95% CI 0.79-1.32). For all six classes of preventive practices combined the adjusted OR was 1.77 (95% CI 1.38-2.27). Conclusion: Evidence from randomized controlled studies supports the effectiveness of data-driven computer-based reminder systems to improve prevention services in the ambulatory care setting. n JAMIA. 1996;3:399-409. Affiliations of the authors: Division of General Medicine, Department of Medicine, College of Physicians and Surgeons, Columbia University, New York, NY (SS, LB); Division of Epidemiology, Columbia University, School of Public Health, New York, NY (SS); Department of Medical lnformatics, College of Physicians and Surgeons, Columbia University, New York, NY (SS, WD); Division of Biostatistics, Columbia University School of Public Health, New York, NY (WD). Supported by a grant from The Commonwealth Fund. Correspondence and reprint requests to: Steven Shea, MD, Division of General Medicine, 622 West 168th Street, New York, NY, 10032. Received for publication: 5/21/96; accepted for publication: 7/15/96. Downloaded from https://academic.oup.com/jamia/article-abstract/3/6/399/823723 by guest on 27 July 2018 400 SHEA ET AL., Computer-based Clinical Reminder Systems The potential for computers to improve clinical care has been appreciated for many years, but the gradual development of effective technology has only recently made high-performance systems with rapid response times affordable and acceptable to potential users.1’2 One strategy for using computers to improve clinical care has relied on user-driven programs that provide consultation to clinicians seeking assistance with complex diagnostic or management issues.3-5 Users evoke these programs, input some or all of the necessary clinical data, and receive consultative output. A second strategy has been to develop data-driven programs that provide reminders in less complex but more frequently occurring clinical situations without requiring users to input new data or to evoke the program.6-8 These passive systems are triggered by conditions in a clinical database that satisfy specified logic.6-8 One data-driven reminder system, triggered by computer-based order entry, has been tested in the inpatient setting, where a controlled study found a reduction in length of stay, diagnostic test charges, pharmacy charges, and overall costs9 In the ambulatory setting, the same system, again triggered by computer-based order entry, reduced diagnostic test ordering and ancillary testing costs when physicians were shown prior test results,1D computer-generated predictions of abnormal results,” or test prices.12 An important clinical application of data-driven systems in the ambulatory setting is the area of preventive care. Evidence-based standards for preventive services have been published’3.‘4 and are widely accepted, but extensive data indicate that the use of mammography, Pap smears, vaccinations, and other preventive services falls short of their full public health potential.15 Interest in the potential of computer-based reminder systems to improve preventive services has led to a number of controlled trials testing the effects of these systems. These studies have been reviewed1.2 but not summarized using meta-analysis. We performed a meta-analysis of these studies in order to assess the current state of the evidence concerning the effectiveness of these systems and to provide a summary estimate of the magnitude of the effect of computer-generated reminders across studies. Two specific issues were of particular interest. First, we used an extension of conventional meta-analytic methods to combine different categories of preventive services to provide a pooled estimate using all available data of the effect of computer-based reminder systems on physicians’ actions in implementing preventive services. Second, we sought to identify sources of heterogeneity among these studies and to assess their effects on the findings.” Methods