The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

BackgroundComputerised decision support (CDS) based on trustworthy clinical guidelines is a key component of a learning healthcare system. Research shows that the effectiveness of CDS is mixed.Multifaceted context, system, recommendation and implementation factors may potentially affect the success of CDS interventions. This paper describes the development of a checklist that is intended to support professionals to implement CDS successfully.MethodsWe developed the checklist through an iterative process that involved a systematic review of evidence and frameworks, a synthesis of the success factors identified in the review, feedback from an international expert panel that evaluated the checklist in relation to a list of desirable framework attributes, consultations with patients and healthcare consumers and pilot testing of the checklist.ResultsWe screened 5347 papers and selected 71 papers with relevant information on success factors for guideline-based CDS. From the selected papers, we developed a 16-factor checklist that is divided in four domains, i.e. the CDS context, content, system and implementation domains. The panel of experts evaluated the checklist positively as an instrument that could support people implementing guideline-based CDS across a wide range of settings globally. Patients and healthcare consumers identified guideline-based CDS as an important quality improvement intervention and perceived the GUIDES checklist as a suitable and useful strategy.ConclusionsThe GUIDES checklist can support professionals in considering the factors that affect the success of CDS interventions. It may facilitate a deeper and more accurate understanding of the factors shaping CDS effectiveness. Relying on a structured approach may prevent that important factors are missed.

[1]  Gillian D Sanders,et al.  Enabling health care decisionmaking through clinical decision support and knowledge management. , 2012, Evidence report/technology assessment.

[2]  Adam Wright,et al.  Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study , 2012, BMC Medical Informatics and Decision Making.

[3]  Linn Brandt,et al.  A systematic review of trials evaluating success factors of interventions with computerised clinical decision support , 2018, Implementation Science.

[4]  Todd W. Rice,et al.  An Electronic Tool for the Evaluation and Treatment of Sepsis in the ICU: A Randomized Controlled Trial* , 2015, Critical care medicine.

[5]  K. Barnard,et al.  Effectiveness of a computerised assessment tool to prompt individuals with diabetes to be more active in consultations , 2007 .

[6]  R. Lau,et al.  Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) , 2016, Implementation Science.

[7]  J. Holmen,et al.  Implementing clinical guidelines in the treatment of hypertension in general practice. , 1998, Blood pressure.

[8]  Julie M. Fiskio,et al.  Technology Evaluation: A Randomized Trial of Electronic Clinical Reminders to Improve Quality of Care for Diabetes and Coronary Artery Disease , 2005, J. Am. Medical Informatics Assoc..

[9]  Ferdinand T. Velasco,et al.  Improving Outcomes with Clinical Decision Support: An Implementer's Guide , 2012 .

[10]  Stéphanie Bernonville,et al.  Human Factors Based Recommendations for the Design of Medication Related Clinical Decision Support Systems (CDSS) , 2011, MIE.

[11]  M. Petticrew,et al.  Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions , 2008, Journal of Epidemiology & Community Health.

[12]  C. McCowan, R. G. Neville, I. W. Ricketts, F. C. Warn,et al.  Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma , 2001 .

[13]  Michelle E. Kho,et al.  AGREE II: advancing guideline development, reporting and evaluation in health care , 2010, Canadian Medical Association Journal.

[14]  J Marc Overhage,et al.  Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. , 2005, Health services research.

[15]  Günter Ollenschläger,et al.  Guidelines International Network: Toward International Standards for Clinical Practice Guidelines , 2012, Annals of Internal Medicine.

[16]  Kensaku Kawamoto,et al.  Systematic review of clinical decision support interventions with potential for inpatient cost reduction , 2013, BMC Medical Informatics and Decision Making.

[17]  Abdul V. Roudsari,et al.  Automation bias: a systematic review of frequency, effect mediators, and mitigators , 2012, J. Am. Medical Informatics Assoc..

[18]  Tejal K. Gandhi,et al.  Actionable reminders did not improve performance over passive reminders for overdue tests in the primary care setting , 2011, J. Am. Medical Informatics Assoc..

[19]  Pavel S. Roshanov,et al.  Development of a Tailored Intervention With Computerized Clinical Decision Support to Improve Quality of Care for Patients With Knee Osteoarthritis: Multi-Method Study , 2018, JMIR research protocols.

[20]  R G Neville,et al.  Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma , 2001, Medical informatics and the Internet in medicine.

[21]  Walter Ricciardi,et al.  The effectiveness of computerized clinical guidelines in the process of care: a systematic review , 2010, BMC health services research.

[22]  Mor Peleg,et al.  Computer-interpretable clinical guidelines: A methodological review , 2013, J. Biomed. Informatics.

[23]  J. Ioannidis,et al.  Drivers of poor medical care , 2017, The Lancet.

[24]  G. Guyatt,et al.  Grading quality of evidence and strength of recommendations , 2004, BMJ : British Medical Journal.

[25]  G. Guyatt,et al.  Evidence for underuse of effective medical services around the world , 2017, The Lancet.

[26]  Owens Dk,et al.  Computer-based decision support: wishing on a star? , 2001 .

[27]  Elie A Akl,et al.  GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. , 2013, Journal of clinical epidemiology.

[28]  Monique W. M. Jaspers,et al.  Factors Known to Influence Acceptance of Clinical Decision Support Systems , 2011, MIE.

[29]  Aileen Clarke,et al.  Developing a quality criteria framework for patient decision aids: online international Delphi consensus process , 2006, BMJ : British Medical Journal.

[30]  Alissa L. Russ,et al.  Recommendations to improve the usability of drug-drug interaction clinical decision support alerts , 2015, J. Am. Medical Informatics Assoc..

[31]  Standards in health information technology: promise and challenges. , 2007, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[32]  Nicolette de Keizer,et al.  The role of standardized data and terminological systems in computerized clinical decision support systems: Literature review and survey , 2011, Int. J. Medical Informatics.

[33]  Barbara K. Burian,et al.  More Than a Tick Box: Medical Checklist Development, Design, and Use , 2018, Anesthesia and analgesia.

[34]  A. Oxman,et al.  Rational Prescribing in Primary care (RaPP): process evaluation of an intervention to improve prescribing of antihypertensive and cholesterol-lowering drugs , 2006, Implementation science : IS.

[35]  Ron A. G. Winkens,et al.  Design and Evaluation of a Computer Reminder System to Improve Prescribing Behaviour of GPs , 2006, MIE.

[36]  Eva Harris,et al.  Technology, innovation and health equity , 2015, Bulletin of the World Health Organization.

[37]  Jonathan M. Teich,et al.  Recommendations for Clinical Decision Support Deployment: Synthesis of a Roundtable of Medical Directors of Information Systems , 2007, AMIA.

[38]  Sandra F. Naoom,et al.  Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. , 2010 .

[39]  J Brender,et al.  STARE-HI – Statement on Reporting of Evaluation Studies in Health Informatics , 2013, Applied Clinical Informatics.

[40]  E Coiera,et al.  Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems , 2006, Yearbook of Medical Informatics.

[41]  M. Field,et al.  Clinical practice guidelines : directions for a new program , 1990 .

[42]  J. Weisz,et al.  Evidence-based psychotherapies for children and adolescents, 2nd ed. , 2010 .

[43]  Harlan M Krumholz,et al.  A randomized outpatient trial of a decision-support information technology tool. , 2005, Archives of internal medicine.

[44]  Christian Nøhr,et al.  Statement on Reporting of Evaluation Studies in Health Informatics , 2007 .

[45]  David W. Bates,et al.  Governance for clinical decision support: case studies and recommended practices from leading institutions , 2011, J. Am. Medical Informatics Assoc..

[46]  J. Metlay,et al.  One size does not fit all: evaluating an intervention to reduce antibiotic prescribing for acute bronchitis , 2013, BMC Health Services Research.

[47]  Julie M. Fiskio,et al.  Analysis of clinical decision support system malfunctions: a case series and survey , 2016, J. Am. Medical Informatics Assoc..

[48]  D. Ramaekers,et al.  Electronic Laboratory Medicine ordering with evidence-based Order sets in primary care (ELMO study): protocol for a cluster randomised trial , 2017, Implementation Science.

[49]  Monique W. M. Jaspers,et al.  Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis , 2017, Int. J. Medical Informatics.

[50]  Richard Baker,et al.  A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice , 2013, Implementation Science.

[51]  Bert Aertgeerts,et al.  Tailoring implementation strategies for evidence-based recommendations using computerised clinical decision support systems: protocol for the development of the GUIDES tools , 2015, Implementation Science.

[52]  G. Guyatt,et al.  Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial , 2017, BMJ Open.

[53]  H. Bloomfield,et al.  A trial of education, prompts, and opinion leaders to improve prescription of lipid modifying therapy by primary care physicians for patients with ischemic heart disease , 2005, Quality and Safety in Health Care.

[54]  A. Grant,et al.  Systematic review and economic modelling of the effectiveness and cost-effectiveness of non-surgical treatments for women with stress urinary incontinence. , 2010, Health technology assessment.

[55]  J. Grimshaw,et al.  Manual paper reminders: effects on professional practice and health care outcomes , 2014 .

[56]  Margaret Williamson,et al.  Review paper: Computerized clinical decision support for prescribing: provision does not guarantee uptake , 2010, J. Am. Medical Informatics Assoc..

[57]  Dean F Sittig,et al.  A new sociotechnical model for studying health information technology in complex adaptive healthcare systems , 2010, Quality and Safety in Health Care.

[58]  S. Brownlee,et al.  Evidence for overuse of medical services around the world , 2017, The Lancet.

[59]  Lucy Yardley,et al.  Process evaluation of a point-of-care cluster randomised trial using a computer-delivered intervention to reduce antibiotic prescribing in primary care , 2014, BMC Health Services Research.

[60]  David Newby,et al.  Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007) , 2009, BMC health services research.

[61]  J C Wyatt,et al.  Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: systematic reviews of the effects and cost-effectiveness of systems. , 2010, Health technology assessment.

[62]  Trudy van der Weijden,et al.  Exposure to and experiences with a computerized decision support intervention in primary care: results from a process evaluation , 2015, BMC Family Practice.

[63]  H. Clark,et al.  Screening for type 2 diabetes following gestational diabetes: family physician and patient perspectives. , 2010, Canadian family physician Medecin de famille canadien.

[64]  Albert Chan,et al.  Identifying Barriers to Hypertension Guideline Adherence Using Clinician Feedback at the Point of Care , 2006, AMIA.

[65]  A Haines,et al.  Evaluation of General Practice Computer Templates , 1999, Methods of Information in Medicine.

[66]  K. Wallston,et al.  Clinical inertia: a common barrier to changing provider prescribing behavior. , 2007, Joint Commission Journal on Quality and Patient Safety.

[67]  Elizabeth Murray,et al.  Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews , 2015, Implementation Science.

[68]  Arie Hasman,et al.  Evaluation of an automated test ordering and feedback system for general practitioners in daily practice , 2004, Int. J. Medical Informatics.

[69]  Marc Berg,et al.  Review Paper: Overriding of Drug Safety Alerts in Computerized Physician Order Entry , 2006, J. Am. Medical Informatics Assoc..

[70]  Anne Miller,et al.  Integrating computerized clinical decision support systems into clinical work: A meta-synthesis of qualitative research , 2015, Int. J. Medical Informatics.

[71]  David W. Bates,et al.  Synthesis of Research Paper: Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality , 2003, J. Am. Medical Informatics Assoc..

[72]  David F Lobach,et al.  The road to effective clinical decision support: are we there yet? , 2013, BMJ.

[73]  F. Alla,et al.  A tool to analyze the transferability of health promotion interventions , 2013, BMC Public Health.

[74]  P. Shekelle,et al.  The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis , 2014, Systematic Reviews.

[75]  Gert P. Westert,et al.  Implementation of multiple-domain covering computerized decision support systems in primary care: a focus group study on perceived barriers , 2015, BMC Medical Informatics and Decision Making.

[76]  McGinnis Jm,et al.  The learning healthcare system : workshop summary , 2007 .

[77]  Sarah Wordsworth,et al.  Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. , 2006, Family practice.

[78]  F D Hobbs,et al.  A prospective controlled trial of computerized decision support for lipid management in primary care. , 1996, Family practice.

[79]  Anne Holbrook,et al.  Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials , 2009, BMC Medical Informatics Decis. Mak..

[80]  Ida Sim,et al.  A taxonomic description of computer-based clinical decision support systems , 2006, J. Biomed. Informatics.

[81]  Charles Young,et al.  A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation. , 1996, Journal of the American Medical Informatics Association : JAMIA.

[82]  S. Brownlee,et al.  Addressing overuse and underuse around the world , 2017, The Lancet.

[83]  Arie Hasman,et al.  User satisfaction with a real-time automated feedback system for general practitioners: a quantitative and qualitative study. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[84]  Douglas S. Bell,et al.  Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions , 2012, J. Biomed. Informatics.

[85]  Michael Gillam,et al.  Computerized physician order entry and online decision support. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[86]  ISO / IEC 25010 : 2011 Systems and software engineering — Systems and software Quality Requirements and Evaluation ( SQuaRE ) — System and software quality models , 2013 .

[87]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[88]  R Brian Haynes,et al.  Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials , 2013, BMJ : British Medical Journal.

[89]  S. Flottorp,et al.  Process evaluation of a cluster randomized trial of tailored interventions to implement guidelines in primary care--why is it so hard to change practice? , 2003, Family practice.

[90]  James L. McCormack.  Clinician Perspectives on the Quality of Patient Data Used for Clinical Decision Support: A Qualitative Study , 2012, AMIA.

[91]  P S Heckerling,et al.  Effect of a fact sheet reminder on performance of the periodic health examination: a randomized controlled trial. , 1992, American journal of preventive medicine.

[92]  Adam Wright,et al.  A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success , 2013, J. Am. Medical Informatics Assoc..

[93]  Howard Balshem,et al.  GRADE guidelines: 3. Rating the quality of evidence. , 2011, Journal of clinical epidemiology.

[94]  Casimir A. Kulikowski,et al.  The dangerous decade , 2012, J. Am. Medical Informatics Assoc..

[95]  S. Brownlee,et al.  Levers for addressing medical underuse and overuse: achieving high-value health care , 2017, The Lancet.

[96]  David Parkin,et al.  Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial , 2002, BMJ : British Medical Journal.

[97]  Kaveh G Shojania,et al.  Effect of point-of-care computer reminders on physician behaviour: a systematic review , 2010, Canadian Medical Association Journal.

[98]  F. Griffiths,et al.  Changing Clinical Practice Through Patient Specific Reminders Available at the Time of the Clinical Encounter: Systematic Review and Meta-Analysis , 2012, Journal of General Internal Medicine.

[99]  Michelle Sweidan,et al.  Identification of features of electronic prescribing systems to support quality and safety in primary care using a modified Delphi process , 2010, BMC Medical Informatics Decis. Mak..

[100]  David W. Bates,et al.  Review Paper: What Evidence Supports the Use of Computerized Alerts and Prompts to Improve Clinicians' Prescribing Behavior? , 2009, J. Am. Medical Informatics Assoc..

[101]  W. Tierney,et al.  Provider Response to Computer-Based Care Suggestions for Chronic Heart Failure , 2005, Medical care.

[102]  Paul K. Davis,et al.  Advancing clinical decision support using lessons from outside of healthcare: an interdisciplinary systematic review , 2012, BMC Medical Informatics and Decision Making.

[103]  J. Wyatt,et al.  Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes. , 2017, The Cochrane database of systematic reviews.

[104]  Kensaku Kawamoto,et al.  Clinical Decision Support-based Quality Measurement (CDS-QM) Framework: Prototype Implementation, Evaluation, and Future Directions , 2014, AMIA.

[105]  Andrew Booth,et al.  Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual) , 2015, PLoS medicine.

[106]  Nicolette de Keizer,et al.  The effect of computerized decision support on barriers to guideline implementation: A qualitative study in outpatient cardiac rehabilitation , 2010, Int. J. Medical Informatics.

[107]  R. Watson.  European Union leads way on e-health, but obstacles remain , 2010, BMJ : British Medical Journal.

[108]  C. McDonald,et al.  Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. , 1993, JAMA.

[109]  J. Clarkson,et al.  Knowledge May Not Be the Best Target for Strategies to Influence Evidence-Based Practice: Using Psychological Models to Understand RCT Effects , 2009, International journal of behavioral medicine.

[110]  Jonathan M. Teich,et al.  Grand challenges in clinical decision support , 2008, J. Biomed. Informatics.

[111]  L. Trinquart,et al.  Computerized advice on drug dosage to improve prescribing practice. , 2013, The Cochrane database of systematic reviews.

[112]  Julia C. M. van Weert,et al.  The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature , 2012, J. Am. Medical Informatics Assoc..

[113]  R. Fassett,et al.  Improving effectiveness of clinical medicine: the need for better translation of science into practice , 2013, The Medical journal of Australia.

[114]  L. Moja,et al.  Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. , 2014, American journal of public health.

[115]  P. Pronovost,et al.  Clinical review: Checklists - translating evidence into practice , 2009, Critical care.

[116]  K. Grimmer-Somers,et al.  The effectiveness of clinical guideline implementation strategies--a synthesis of systematic review findings. , 2008, Journal of evaluation in clinical practice.

[117]  R. M. Vazquez.  The Checklist Manifesto How to Get Things Right , 2011 .

[118]  Neil M. Paige,et al.  Electronic health record-based interventions for improving appropriate diagnostic imaging: a systematic review and meta-analysis. , 2015, Annals of internal medicine.

[119]  Christine A. Sinsky,et al.  From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider , 2014, The Annals of Family Medicine.

[120]  Steven R. Simon,et al.  Reducing the Prescribing of Heavily Marketed Medications: A Randomized Controlled Trial , 2009, Journal of General Internal Medicine.

[121]  Jonathan M. Teich,et al.  AMIA Position Paper: Clinical Decision Support in Electronic Prescribing: Recommendations and an Action Plan: Report of the Joint Clinical Decision Support Workgroup , 2005, J. Am. Medical Informatics Assoc..

[122]  S. Flottorp,et al.  Effect of Interventions for Implementing Clinical Practice Guidelines , 2015 .

[123]  Kevin B. Johnson,et al.  Showing Your Work: Impact of annotating electronic prescriptions with decision support results , 2010, J. Biomed. Informatics.

[124]  Richard N. Shiffman,et al.  Bmc Medical Informatics and Decision Making the Guideline Implementability Appraisal (glia): Development of an Instrument to Identify Obstacles to Guideline Implementation , 2022 .

[125]  T. Ferris,et al.  Impact of a Computerized Template on Antibiotic Prescribing for Acute Respiratory Infections in Children and Adolescents , 2010, Clinical pediatrics.

[126]  Dean F. Sittig,et al.  Review of Reported Clinical Information System Adverse Events in US Food and Drug Administration Databases , 2011, Applied Clinical Informatics.