Safety and usability evaluation of a web-based insulin self-titration system for patients with type 2 diabetes mellitus

OBJECTIVE The rising incidence of type 2 diabetes mellitus (T2DM) induces severe challenges for the health care system. Our research group developed a web-based system named PANDIT that provides T2DM patients with insulin dosing advice using state of the art clinical decision support technology. The PANDIT interface resembles a glucose diary and provides advice through pop-up messages. Diabetes nurses (DNs) also have access to the system, allowing them to intervene when needed. The objective of this study was to establish whether T2DM patients can safely use PANDIT at home. To this end, we assessed whether patients experience usability problems with a high risk of compromising patient safety when interacting with the system, and whether PANDIT's insulin dosing advice are clinically safe. RESEARCH DESIGN AND METHODS The study population consisted of patients with T2DM (aged 18-80) who used a once daily basal insulin as well as DNs from a university hospital. The usability evaluation consisted of think-aloud sessions with four patients and three DNs. Video data, audio data and verbal utterances were analyzed for usability problems encountered during PANDIT interactions. Usability problems were rated by a physician and a usability expert according to their potential impact on patient safety. The usability evaluation was followed by an implementation with a duration of four weeks. This implementation took place at the patients' homes with ten patients to evaluate clinical safety of PANDIT advice. PANDIT advice were systematically compared with DN advice. Deviating advice were evaluated with respect to patient safety by a panel of experienced physicians, which specialized in diabetes care. RESULTS We detected seventeen unique usability problems, none of which was judged to have a high risk of compromising patient safety. Most usability problems concerned the lay-out of the diary, which did not clearly indicate which data entry fields had to be entered in order to obtain an advice. 27 out of 74 (36.5%) PANDIT advice differed from those provided by DNs. However, only one of these (1.4%) was considered unsafe by the panel. CONCLUSION T2DM patients with no prior experience with the web-based self-management system were capable of consulting the system without encountering significant usability problems. Furthermore, the large majority of PANDIT advice were considered clinically safe according to the expert panel. One advice was considered unsafe. This could however easily be corrected by implementing a small modification to the system's knowledge base.

[1]  Gilad J. Kuperman,et al.  Case Report: Comprehensive Analysis of a Medication Dosing Error Related to CPOE , 2005, J. Am. Medical Informatics Assoc..

[2]  Suzanne Bakken,et al.  Heuristic Evaluation of HIV-TIDES - Tailored Interventions for Management of Depressive Symptoms in HIV-infected Individuals , 2006, AMIA.

[3]  Stephanie J. Fonda,et al.  Usability test of an internet-based informatics tool for diabetes care providers: the comprehensive diabetes management program. , 2008, Diabetes technology & therapeutics.

[4]  J. Vermylen,et al.  The Belgian Improvement Study on Oral Anticoagulation Therapy: a randomized clinical trial. , 2005, European heart journal.

[5]  Stirling Bryan,et al.  A multicentre randomised assessment of the DAWN AC computer-assisted oral anticoagulant dosage program , 2009, Thrombosis and Haemostasis.

[6]  M. Davies,et al.  Improvement of glycemic control in subjects with poorly controlled type 2 diabetes: comparison of two treatment algorithms using insulin glargine. , 2005, Diabetes care.

[7]  Nancy L Wilczynski,et al.  Computerized clinical decision support systems for therapeutic drug monitoring and dosing: A decision-maker-researcher partnership systematic review , 2011, Implementation science : IS.

[8]  Norman Fleischer,et al.  The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. The Diabetes Control and Complications Trial Research Group. , 1993 .

[9]  H. Tymkew,et al.  A randomized study in diabetic patients undergoing cardiac surgery comparing computer-guided glucose management with a standard sliding scale protocol. , 2008, Journal of cardiothoracic and vascular anesthesia.

[10]  Brian Oldenburg,et al.  Web-based interventions for the management of type 2 diabetes mellitus: A systematic review of recent evidence , 2011, Int. J. Medical Informatics.

[11]  Arie Hasman,et al.  Design and implementation of a framework to support the development of clinical guidelines , 2001, Int. J. Medical Informatics.

[12]  L. L. Brown,et al.  A Review of Web-Assisted Interventions for Diabetes Management: Maximizing the Potential for Improving Health Outcomes , 2007, Journal of diabetes science and technology.

[13]  S. Genuth,et al.  The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. , 1993, The New England journal of medicine.

[14]  J. DeVries,et al.  Contact frequency determines outcome of basal insulin initiation trials in type 2 diabetes , 2009, Diabetologia.

[15]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[16]  Tosha B. Wetterneck,et al.  Technology Evaluation: Workarounds to Barcode Medication Administration Systems: Their Occurrences, Causes, and Threats to Patient Safety , 2008, J. Am. Medical Informatics Assoc..

[17]  Abdul V. Roudsari,et al.  Integrating model-based decision support in a multi-modal reasoning system for managing type 1 diabetic patients , 2003, Artif. Intell. Medicine.

[18]  T. Bodenheimer,et al.  Patient self-management of chronic disease in primary care. , 2002, JAMA.

[19]  R. Holman,et al.  Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). UK Prospective Diabetes Study (UKPDS) Group. , 1998 .

[20]  Ted Boren,et al.  Thinking aloud: reconciling theory and practice , 2000 .

[21]  V. Patel,et al.  Assessment of a computerized patient record system: a cognitive approach to evaluating medical technology. , 1996, M.D. computing : computers in medical practice.

[22]  Niels Peek,et al.  Formative usability evaluation of a web-based insulin self-titration system: preliminary results. , 2012, Studies in health technology and informatics.

[23]  Eldon D. Lehmann,et al.  Combining rule-based reasoning and mathematical modelling in diabetes care , 1994, Artif. Intell. Medicine.

[24]  Vimla L. Patel,et al.  A framework for analyzing the cognitive complexity of computer-assisted clinical ordering , 2003, J. Biomed. Informatics.

[25]  Y. Han,et al.  Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System , 2005, Pediatrics.

[26]  Marie-Catherine Beuscart-Zéphir,et al.  Usability assessment study of a web site displaying medical resources on line: the CISMeF. , 2002, Studies in health technology and informatics.

[27]  Matteo Cesari,et al.  Predictors of Combined Cognitive and Physical Decline , 2005, Journal of the American Geriatrics Society.

[28]  M. Mills,et al.  Usability of Health Web Sites for Older Adults: A Preliminary Study , 2004, Computers, informatics, nursing : CIN.

[29]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[30]  Harlan M. Krumholz,et al.  Optimizing Statin Treatment for Primary Prevention of Coronary Artery Disease , 2010, Annals of Internal Medicine.

[31]  K. Nørgaard,et al.  Use of an Automated Bolus Calculator in MDI-Treated Type 1 Diabetes , 2012, Diabetes Care.

[32]  Emmy Rood,et al.  Use of a computerized guideline for glucose regulation in the intensive care unit improved both guide-line adherence and glucose regulation , 2005 .

[33]  M. Moia,et al.  Effect of computer-aided management on the quality of treatment in anticoagulated patients: a prospective, randomized, multicenter trial of APROAT (Automated PRogram for Oral Anticoagulant Treatment). , 2001, Haematologica.

[34]  Katrina M. Romagnoli,et al.  Assessing the usability of a telemedicine-based Medication Delivery Unit for older adults through inspection methods. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[35]  Joseph L. Kannry,et al.  Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support , 2012, Int. J. Medical Informatics.

[36]  J. Eluf-Neto,et al.  A randomized controlled trial comparing a computer-assisted insulin infusion protocol with a strict and a conventional protocol for glucose control in critically ill patients. , 2009, Journal of critical care.

[37]  PD redaction,et al.  Implications of the Diabetes Control and Complications Trial , 1996, Diabetes Care.

[38]  Anne Holbrook,et al.  Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials , 2009, BMC Medical Informatics Decis. Mak..

[39]  R. Gabbay,et al.  Web-based management of diabetes through glucose uploads: has the time come for telemedicine? , 2009, Diabetes research and clinical practice.

[40]  Danielle M. Lottridge,et al.  Development and Iterative Refinement of an Internet-based Service for Chinese Family Caregivers of People with Alzheimer Disease , 2005, AMIA.

[41]  David Kerr,et al.  Diabetes Interactive Diary: A New Telemedicine System Enabling Flexible Diet and Insulin Therapy While Improving Quality of Life , 2009, Diabetes Care.

[42]  Bryant Thomas Karras,et al.  ActiveOptions: Leveraging existing knowledge and usability testing to develop a physical activity program website for older adults , 2007, AMIA.