Clinical decision support software for diabetic foot risk stratification: development and formative evaluation

BackgroundIdentifying people at risk of developing diabetic foot complications is a vital step in prevention programs in primary healthcare settings. Diabetic foot risk stratification systems predict foot ulceration. The aim of this study was to explore the views and experiences of potential end users during development and formative evaluations of an electronic diabetic foot risk stratification tool based on evidence-based guidelines and determine the accuracy of the tool.MethodsFormative evaluation of the risk tool occurred in five stages over an eight-month period and employed a mixed methods research design consisting of semi-structured interviews, focus group and participant observation, online survey, expert review, comparison to the Australian Guidelines and clinical testing.ResultsA total of 43 healthcare practitioners trialled the computerised clinical decision support system during development, with multiple software changes made as a result of feedback. Individual and focus group participants exposed critical design flaws. Live testing revealed risk stratification errors and functional limitations providing the basis for practical improvements. In the final product, all risk calculations and recommendations made by the clinical decision support system reflect current Australian Guidelines.ConclusionsDevelopment of the computerised clinical decision support system using evidence-based guidelines can be optimised by a multidisciplinary iterative process of feedback, testing and software adaptation by experts in modern development technologies.

[1]  S. G. Axline,et al.  Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. , 1975, Computers and biomedical research, an international journal.

[2]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[3]  D. Spiegelhalter,et al.  Evaluating medical expert systems: what to test and how? , 1990, Medical informatics = Medecine et informatique.

[4]  G. Reiber,et al.  Pathways to Diabetic Limb Amputation: Basis for Prevention , 1990, Diabetes Care.

[5]  Clayton Lewis,et al.  Making usable, useful, productivity-enhancing computer applications , 1991, CACM.

[6]  S. Rith-Najarian,et al.  Identifying Diabetic Patients at High Risk for Lower-Extremity Amputation in a Primary Health Care Setting: A prospective evaluation of simple screening criteria , 1992, Diabetes Care.

[7]  R. Haynes,et al.  Effects of Computer-based Clinical Decision Support Systems on Clinician Performance and Patient Outcome: A Critical Appraisal of Research , 1994, Annals of Internal Medicine.

[8]  John Durkin,et al.  Expert systems - design and development , 1994 .

[9]  M J Lincoln,et al.  A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines. , 1995, Proceedings. Symposium on Computer Applications in Medical Care.

[10]  Bill Curtis,et al.  Applying Discount Usability Engineering , 1995, IEEE Softw..

[11]  C. Agardh,et al.  Decreasing Incidence of Major Amputation in Diabetic Patients: a Consequence of a Multidisciplinary Foot Care Team Approach? , 1995, Diabetic medicine : a journal of the British Diabetic Association.

[12]  D G Smith,et al.  The Independent Contributions of Diabetic Neuropathy and Yasculopatny in Foot Ulceration: How great are the risks? , 1995, Diabetes Care.

[13]  T. Greene,et al.  A Foot Risk Classification System to Predict Diabetic Amputation in Pima Indians , 1996, Diabetes Care.

[14]  F. Vinicor,et al.  Independent Physiological Predictors of Foot Lesions in Patients With NIDDM , 1997, Diabetes Care.

[15]  C. Komoltri,et al.  Factors Associated with Diabetic Foot Ulceration in Thailand: a Case‐control Study , 1997, Diabetic medicine : a journal of the British Diabetic Association.

[16]  J Wyatt,et al.  Quantitative evaluation of clinical software, exemplified by decision support systems. , 1997, International journal of medical informatics.

[17]  W. Hammond,et al.  Computerized decision support based on a clinical practice guideline improves compliance with care standards. , 1997, The American journal of medicine.

[18]  D. Armstrong,et al.  Practical criteria for screening patients at high risk for diabetic foot ulceration. , 1998, Archives of internal medicine.

[19]  L. Sanders,et al.  Reproducibility and accuracy among primary care providers of a screening examination for foot ulcer risk among diabetic patients. , 1998, Preventive medicine.

[20]  R. Haynes,et al.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. , 1998, JAMA.

[21]  Computer-supported identification and intervention for diabetic patients at risk for amputation. , 1998, M.D. computing : computers in medical practice.

[22]  Richard N. Shiffman,et al.  Review: Computer-based Guideline Implementation Systems: A Systematic Review of Functionality and Effectiveness , 1999, J. Am. Medical Informatics Assoc..

[23]  W G Henderson,et al.  Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. , 2000, JAMA.

[24]  L. Lavery,et al.  Effectiveness of the diabetic foot risk classification system of the International Working Group on the Diabetic Foot. , 2001, Diabetes care.

[25]  Bonnie Kaplan,et al.  Evaluating informatics applications - some alternative approaches: theory, social interactionism, and call for methodological pluralism , 2001, Int. J. Medical Informatics.

[26]  Bonnie Kaplan,et al.  Evaluating informatics applications - clinical decision support systems literature review , 2001, Int. J. Medical Informatics.

[27]  H. Lehmann,et al.  Clinical Decision Support Systems (cdsss) Have Been Hailed for Their Potential to Reduce Medical Errors Clinical Decision Support Systems for the Practice of Evidence-based Medicine , 2022 .

[28]  Tim Dornan,et al.  Involving users in the design and usability evaluation of a clinical decision support system , 2002, Comput. Methods Programs Biomed..

[29]  Victor M Montori,et al.  The impact of planned care and a diabetes electronic management system on community-based diabetes care: the Mayo Health System Diabetes Translation Project. , 2002, Diabetes care.

[30]  J. Griffiths,et al.  The North‐West Diabetes Foot Care Study: incidence of, and risk factors for, new diabetic foot ulceration in a community‐based patient cohort , 2002, Diabetic medicine : a journal of the British Diabetic Association.

[31]  C. Teddlie,et al.  SAGE Handbook of Mixed Methods in Social & Behavioral Research , 2010 .

[32]  J. Grimshaw,et al.  From best evidence to best practice: effective implementation of change in patients' care , 2003, The Lancet.

[33]  David M Nathan,et al.  A controlled trial of web-based diabetes disease management: the MGH diabetes primary care improvement project. , 2003, Diabetes care.

[34]  Elske Ammenwerth,et al.  Can evaluation studies benefit from triangulation? A case study , 2003, Int. J. Medical Informatics.

[35]  J M Grimshaw,et al.  Effectiveness and efficiency of guideline dissemination and implementation strategies , 2004, International Journal of Technology Assessment in Health Care.

[36]  S. Wild,et al.  Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. , 2004, Diabetes care.

[37]  George R. Franke,et al.  HANDBOOK OF MIXED METHODS IN SOCIAL & BEHAVIORAL RESEARCH (Book) , 2004 .

[38]  A. Boulton.  The diabetic foot: from art to science. The 18th Camillo Golgi lecture , 2004, Diabetologia.

[39]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[40]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[41]  A. Dhar,et al.  National Institute for Health and Clinical Excellence , 2005 .

[42]  Rob Procter,et al.  Clinical decision support software for management of chronic heart failure: Development and evaluation , 2006, Comput. Biol. Medicine.

[43]  G. Rugg,et al.  An investigation into the clinical reasoning of both expert and novice podiatrists , 2006 .

[44]  S Cunningham,et al.  Stratification of foot ulcer risk in patients with diabetes: a population‐based study , 2006, International journal of clinical practice.

[45]  B. Thiers.  Prediction of Diabetic Foot Ulcer Occurrence Using Commonly Available Clinical Information: The Seattle Diabetic Foot StudyBoyko EJ, Nelson KM, Ahroni JH, et al (VA Puget Sound Health Care System, Seattle; Univ of Washington, Seattle) Diabetes Care 29:1202–1207, 2006§ , 2007 .

[46]  T Fahey,et al.  Predicting foot ulcers in patients with diabetes: a systematic review and meta-analysis. , 2006, QJM : monthly journal of the Association of Physicians.

[47]  D. Murdoch,et al.  Reevaluating the Way We Classify the Diabetic Foot , 2008, Diabetes Care.

[48]  Richard Hellman,et al.  Comprehensive foot examination and risk assessment. A report of the Task Force of the Foot Care Interest Group of the American Diabetes Association, with endorsement by the American Association of Clinical Endocrinologists. , 2008, Physical therapy.

[49]  M. Hobbs,et al.  Validation of the basic foot screening checklist: a population screening tool for identifying foot ulcer risk in people with diabetes mellitus. , 2009, Journal of the American Podiatric Medical Association.

[50]  A. Sheikh,et al.  Evaluating eHealth Interventions: The Need for Continuous Systemic Evaluation , 2009, PLoS medicine.

[51]  A. Sheikh,et al.  General practitioners' and nurses' experiences of using computerised decision support in screening for diabetic foot disease: implementing Scottish Clinical Information - Diabetes Care in routine clinical practice. , 2010, Informatics in primary care.

[52]  C. Brand,et al.  The impact of socio-economic disadvantage on rates of hospital separations for diabetes-related foot disease in Victoria, Australia , 2011, Journal of foot and ankle research.

[53]  K. Brown,et al.  Measuring the accuracy of different ways to identify the ‘at‐risk’ foot in routine clinical practice , 2011, Diabetic medicine : a journal of the British Diabetic Association.

[54]  R Brian Haynes,et al.  Computerized clinical decision support systems for chronic disease management: A decision-maker-researcher partnership systematic review , 2011, Implementation science : IS.

[55]  N. Schaper,et al.  The development of global consensus guidelines on the management and prevention of the diabetic foot 2011 , 2012, Diabetes/metabolism research and reviews.

[56]  M. Dinis-Ribeiro,et al.  Validation and comparison of currently available stratification systems for patients with diabetes by risk of foot ulcer development. , 2012, European journal of endocrinology.

[57]  V. Hasselblad,et al.  Effect of Clinical Decision-Support Systems , 2012, Annals of Internal Medicine.

[58]  R Brian Haynes,et al.  Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials , 2013, BMJ : British Medical Journal.

[59]  R J Hinchliffe,et al.  Evidence-based management of PAD & the diabetic foot. , 2013, European journal of vascular and endovascular surgery : the official journal of the European Society for Vascular Surgery.

[60]  Monique W. M. Jaspers,et al.  From an expert-driven paper guideline to a user-centred decision support system: A usability comparison study , 2013, Artif. Intell. Medicine.

[61]  C. Dibben,et al.  Impact of health‐care accessibility and social deprivation on diabetes related foot disease , 2013, Diabetic medicine : a journal of the British Diabetic Association.

[62]  L. Moja,et al.  Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. , 2014, American journal of public health.

[63]  Colin Simpson,et al.  A systematic review and individual patient data meta-analysis of prognostic factors for foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS). , 2015, Health technology assessment.

[64]  T. Coleman,et al.  Evaluating Long-term Outcomes of NHS Stop Smoking Services (ELONS): a prospective cohort study. , 2015, Health technology assessment.