Reviewing the impact of computerized provider order entry on clinical outcomes: The quality of systematic reviews

PURPOSE Computerized provider order entry (CPOE) is central to current efforts at improving clinical care. Understanding the quality of the evidence for CPOE is important to the practical decision of implementation, patient safety and future design efforts. This paper presents the results of a systematic analysis of the quality of systematic reviews of empirical CPOE research. METHODS The systematic search process included PubMed, CINAHL, Scopus, Cochrane, INSPEC, and PsychInfo databases from the years 1987-mid 2010 in English only. All reviews with a focus on CPOE, electronic ordering, Electronic Health Record, or Health Information Technology were included. Studies were excluded if they did not mention a systematic review in the title or text, report a formal search process, report results of the search, or specifically include a separate section on CPOE in the results. Quality was assessed using systematic criteria developed by Oxman and Guyatt, QUOROM, and PRISMA. All three authors conducted the reviews independently. Disagreements were resolved through discussion. Descriptive data was extracted. RESULTS The search process yielded 185 initial unique references with 13 final reviews meeting the inclusion criteria. The rating of overall quality in the Oxman and Guyatt scale averaged 4.9 out of a possible 7 and the average mean of the sum of the other questions was 5.69. The overall QUOROM/PRISMA ratings averaged 63% completion and ranging from 45% to 81%. CONCLUSIONS The quality of these reviews were moderate. Only one study conducted a full quantitative synthesis, and overall heterogeneity was reported as very high in the 3 studies that measured it. Recommendations emphasize clarifying the phenomenon of CPOE by avoiding reporting conclusions across sub-group analyses, increasing emphasis on the development of theoretical models, including more quantitative assessments, and increasing breadth of outcomes.

[1]  Wei Wu,et al.  The Effect of Computerized Physician Order Entry with Clinical Decision Support on the Rates of Adverse Drug Events: A Systematic Review , 2008, Journal of General Internal Medicine.

[2]  Elske Ammenwerth,et al.  Vision and challenges of Evidence-Based Health Informatics: A case study of a CPOE meta-analysis , 2010, Int. J. Medical Informatics.

[3]  J. Rothschild.  Computerized physician order entry in the critical care and general inpatient setting: a narrative review. , 2004, Journal of critical care.

[4]  Kaija Saranto,et al.  Definition, structure, content, use and impacts of electronic health records: A review of the research literature , 2008, Int. J. Medical Informatics.

[5]  E. Oren,et al.  Impact of emerging technologies on medication errors and adverse drug events. , 2003, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[6]  Youping Li,et al.  The reporting quality of meta-analyses improves: a random sampling study. , 2008, Journal of clinical epidemiology.

[7]  David W. Bates,et al.  Review Paper: What Evidence Supports the Use of Computerized Alerts and Prompts to Improve Clinicians' Prescribing Behavior? , 2009, J. Am. Medical Informatics Assoc..

[8]  Frank Sullivan,et al.  Has general practitioner computing made a difference to patient care? A systematic review of published reports , 1995, BMJ.

[9]  Nicola T. Shaw,et al.  The impact of a Critical Care Information System (CCIS) on time spent charting and in direct patient care by staff in the ICU: A review of the literature , 2009, Int. J. Medical Informatics.

[10]  Robert C. Wu,et al.  Evidence for handheld electronic medical records in improving care: a systematic review , 2006, BMC Medical Informatics Decis. Mak..

[11]  Martha L. Lehman,et al.  Physician Order Entry impact on drug turn-around times , 2001, AMIA.

[12]  Anne Holbrook,et al.  Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials , 2009, BMC Medical Informatics Decis. Mak..

[13]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[14]  Zvisinei Sandi.  DEFINITION , 1961, A Philosopher Looks at Sport.

[15]  George Davey Smith,et al.  Meta-analysis of randomised controlled trials , 1997, The Lancet.

[16]  A. Jamal,et al.  The Impact of Health Information Technology on the Quality of Medical and Health Care: A Systematic Review , 2009, Health information management : journal of the Health Information Management Association of Australia.

[17]  Johanna I Westbrook,et al.  Review Paper: Does Computerized Provider Order Entry Reduce Prescribing Errors for Hospital Inpatients? A Systematic Review , 2009, J. Am. Medical Informatics Assoc..

[18]  D. Moher.  Preferred reporting items for systematic reviews and meta-analyses [ressource électronique]: the PRISMA statement , 2009 .

[19]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[20]  Margaret Williamson,et al.  The impact of computerised physician order entry systems on pathology services: A systematic review , 2007, Int. J. Medical Informatics.

[21]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[22]  D. Coplen.  Types of Unintended Consequences Related to Computerized Provider Order Entry , 2007 .

[23]  D. Swinglehurst,et al.  Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. , 2009, The Milbank quarterly.

[24]  Shobha Phansalkar,et al.  The state of the evidence for computerized provider order entry: A systematic review and analysis of the quality of the literature , 2009, Int. J. Medical Informatics.

[25]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[26]  F. Sullivan,et al.  A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97 , 2001, BMJ : British Medical Journal.

[27]  G. Guyatt,et al.  The Science of Reviewing Research a , 1993, Annals of the New York Academy of Sciences.

[28]  Nicolette de Keizer,et al.  Review Paper: Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review , 2007, J. Am. Medical Informatics Assoc..

[29]  Y. Han,et al.  Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System , 2005, Pediatrics.

[30]  R. Kane,et al.  Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. , 2007, Health services research.

[31]  ELSKE AMMENWERTH,et al.  Review Paper: The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review , 2008, J. Am. Medical Informatics Assoc..

[32]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[33]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[34]  E. Keeler,et al.  Costs and benefits of health information technology. , 2006, Evidence report/technology assessment.

[35]  Marc Berg,et al.  Review Paper: The Impact of Computerized Provider Order Entry Systems on Inpatient Clinical Workflow: A Literature Review , 2009, J. Am. Medical Informatics Assoc..

[36]  A. Egberts,et al.  The Effect of Computerized Physician Order Entry on Medication Prescription Errors and Clinical Outcome in Pediatric and Intensive Care: A Systematic Review , 2009, Pediatrics.

[37]  B. Middel.  The impact of electronic health records on time efficiency of physicians and nurses: a systematic review1) , 2008 .

[38]  I. Olkin,et al.  Improving the quality of reports of meta‐analyses of randomised controlled trials: the QUOROM statement , 2000, Revista espanola de salud publica.

[39]  C. Delpierre,et al.  A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[40]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005 .

[41]  A. Peterson,et al.  Electronic prescribing in ambulatory practice: promises, pitfalls, and potential solutions. , 2001, The American journal of managed care.

[42]  S. Goldhaber,et al.  Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors , 2005 .

[43]  Abha Agrawal.  Medication errors: prevention using information technology systems. , 2009, British journal of clinical pharmacology.

[44]  J. Stockman,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[45]  R. Haynes,et al.  Effects of Computer-based Clinical Decision Support Systems on Clinician Performance and Patient Outcome: A Critical Appraisal of Research , 1994, Annals of Internal Medicine.

[46]  D. Bates,et al.  Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. , 2003, Archives of internal medicine.

[47]  G. Guyatt,et al.  Validation of an index of the quality of review articles. , 1991, Journal of clinical epidemiology.

[48]  Enrico Coiera,et al.  Decision support systems for antibiotic prescribing , 2008, Current opinion in infectious diseases.

[49]  Marc Berg,et al.  Implementing information systems in health care organizations: myths and challenges , 2001, Int. J. Medical Informatics.

[50]  R. Haynes,et al.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. , 1998, JAMA.

[51]  P. Rothwell,et al.  Meta-analysis of randomised controlled trials , 1997, The Lancet.

[52]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. , 2009, Journal of clinical epidemiology.

[53]  M. Pittler.  Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[54]  H S Sacks,et al.  Meta-analysis: an update. , 1996, The Mount Sinai journal of medicine, New York.

[55]  M Berg,et al.  The Search for Synergy: Interrelating Medical Work and Patient Care Information Systems , 2003, Methods of Information in Medicine.

[56]  Nicolette de Keizer,et al.  The impact of computerized physician medication order entry in hospitalized patients - A systematic review , 2008, Int. J. Medical Informatics.

[57]  I. Chalmers.  The Cochrane Collaboration: Preparing, Maintaining, and Disseminating Systematic Reviews of the Effects of Health Care , 1993, Annals of the New York Academy of Sciences.

[58]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[59]  A D Oxman,et al.  Systematic Reviews: Checklists for review articles , 1994, BMJ.

[60]  Kaija Saranto,et al.  Evaluating nursing documentation - research designs and methods: systematic review. , 2009, Journal of advanced nursing.

[61]  Russell E Glasgow,et al.  A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. , 2008, Joint Commission journal on quality and patient safety.

[62]  M J Schuemie,et al.  An Inventory of Publications on Electronic Medical Records Revisited , 2009, Methods of Information in Medicine.

[63]  A. Sutton,et al.  Comparison of two methods to detect publication bias in meta-analysis. , 2006, JAMA.

[64]  R Khajouei,et al.  The Impact of CPOE Medication Systems’ Design Aspects on Usability, Workflow and Medication Orders , 2010, Methods of Information in Medicine.

[65]  P. Shekelle,et al.  Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care , 2006, Annals of Internal Medicine.