Involving users in the design and usability evaluation of a clinical decision support system

AIM To design and evaluate a clinical decision support system (CDSS) to support cardiovascular risk prevention in type 2 diabetes. METHODS A preliminary requirements specification and three prototype CDSS interface designs were developed. Seven patients and seven clinicians conducted 'usability tests' on five different task scenarios with the CDSS prototypes to test its effectiveness, efficiency and 'user-friendliness'. Structured, qualitative questions explored their preferences for the different designs and overall impressions of clinical usefulness. RESULTS Patients and clinicians were enthusiastic about the CDSS and used it confidently after a short learning period. Some patients had difficulty interpreting clinical data, but most were keen to see the CDSS used to help them understand their diabetes, provided a clinician explained their results. Clinicians' main concern was that the CDSS would increase consultation times. Changes suggested by users were incorporated into the final interface design. CONCLUSION We have successfully incorporated patients' and clinicians' views into the design of a CDSS, but it was an arduous process.

[1]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[2]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[3]  William W. Stead.  Matching the Level of Evaluation to a project's Stage of Development , 1996, J. Am. Medical Informatics Assoc..

[4]  Enrico Coiera,et al.  Recent Advances: Medical informatics , 1995 .

[5]  R. D'Agostino,et al.  Effect of weight on cardiovascular disease. , 1997, The American journal of clinical nutrition.

[6]  Jakob Nielsen,et al.  Usability engineering at a discount , 1989 .

[7]  Vimla L. Patel,et al.  Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces , 1997, AMIA.

[8]  N. Hammond,et al.  The role of cognitive psychology in user-interface design , 1987 .

[9]  R. Holman,et al.  Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. UK Prospective Diabetes Study Group. , 1998 .

[10]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[11]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[12]  Deborah J. Mayhew,et al.  The usability engineering lifecycle , 1999, CHI Extended Abstracts.

[13]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[14]  Jakob Nielsen,et al.  Chapter 7 – Usability Assessment Methods beyond Testing , 1993 .

[15]  Clayton Lewis,et al.  Making usable, useful, productivity-enhancing computer applications , 1991, CACM.

[16]  R. Thomson,et al.  Decision aids for people facing health treatment or screening decisions. , 2003, The Cochrane database of systematic reviews.

[17]  R B Altman.  Informatics in the care of patients: ten notable challenges. , 1997, The Western journal of medicine.

[18]  D. Spiegelhalter,et al.  Evaluating medical expert systems: what to test and how? , 1990, Medical informatics = Medecine et informatique.

[19]  K. I. Jantunen,et al.  Guide to Medical Informatics, the Internet, and Telemedicine , 1998 .

[20]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[21]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[22]  J Wyatt,et al.  Quantitative evaluation of clinical software, exemplified by decision support systems. , 1997, International journal of medical informatics.

[23]  P Degoulet,et al.  Assessment of health information technology: which questions for which systems? Proposal for a taxonomy. , 1993, Medical informatics = Medecine et informatique.

[24]  W. Kannel.  Blood pressure as a cardiovascular risk factor: prevention and treatment. , 1996, JAMA.

[25]  John Karat,et al.  Software Evaluation Methodologies , 1988 .

[26]  M. Laakso,et al.  Mortality from coronary heart disease in subjects with type 2 diabetes and in nondiabetic subjects with and without prior myocardial infarction. , 1998, The New England journal of medicine.

[27]  J Wyatt,et al.  The NHS's new information strategy , 1998, BMJ.

[28]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[29]  Jane Grimson,et al.  A methodology for evaluation of knowledge-based systems in medicine , 1994, Artif. Intell. Medicine.

[30]  A. McCray,et al.  Yearbook of Medical Informatics , 2013, Yearbook of Medical Informatics.

[31]  Sara Jones,et al.  Graphical interfaces for knowledge engineering: an overview of relevant literature , 1988, The Knowledge Engineering Review.

[32]  H A Heathfield,et al.  Philosophies for the Design and Development of Clinical Decision-Support Systems , 1993, Methods of Information in Medicine.

[33]  Holly Jimison,et al.  Health care providers and purchasers and evaluation of interactive health communication applications , 1999 .

[34]  Patricia A. Billingsley,et al.  Taking Panes: Issues in the Design of Windowing Systems , 1988 .

[35]  R. Haynes,et al.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. , 1998, JAMA.

[36]  V. Entwistle,et al.  Decision aids for people facing health treatment or screening decisions. , 2018, The Cochrane database of systematic reviews.

[37]  James G. Anderson,et al.  Use and Impact of Computers in Clinical Medicine , 2011, Computers and Medicine.

[38]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[39]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[40]  Teven,et al.  MORTALITY FROM CORONARY HEART DISEASE IN SUBJECTS WITH TYPE 2 DIABETES AND IN NONDIABETIC SUBJECTS WITH AND WITHOUT PRIOR MYOCARDIAL INFARCTION , 2000 .

[41]  R. Holman,et al.  Risk factors for coronary artery disease in non-insulin dependent diabetes mellitus: United Kingdom prospective diabetes study (UKPDS: 23) , 1998, BMJ.

[42]  E. Shortliffe.  Computer programs to support clinical decision making. , 1990, JAMA.

[43]  Joint British recommendations on prevention of coronary heart disease in clinical practice: summary , 2001, BMJ : British Medical Journal.

[44]  Jakob Nielsen,et al.  The usability engineering life cycle , 1992, Computer.

[45]  E H Shortliffe,et al.  The evolution of electronic medical records. , 1999, Academic medicine : journal of the Association of American Medical Colleges.

[46]  Marita Franzke,et al.  Usability evaluation with the cognitive walkthrough , 1995, CHI 95 Conference Companion.

[47]  C. Aydin,et al.  Evaluating Health Care Information Systems: Methods and Applications , 1993 .

[48]  B. Kaplan.  The Influence of Medical Values and Practices on Medical Computer Applications , 1982 .

[49]  M. Gardiner,et al.  Applying cognitive psychology to user-interface design , 1987 .