Combining Usability Testing with Eye-Tracking Technology: Evaluation of a Visualization Support for Antibiotic Use in Intensive Care

This research work is an explorative study to measure efficiency, effectiveness and user satisfaction of a prototype called Infobiotika aiming to support antibiotic use in intensive care. The evaluation was performed by combining traditional usability testing with eye-tracking technology. The test was conducted with eight intensive care physicians whereof four specialists and four residents. During three test phases participants were asked to perform three types of tasks, namely navigational, clinical and tasks to measure the learning effect after 3-5 minutes free exploring time. A post-test questionnaire was used to explore user satisfaction. Based on the results and overall observations, Infobiotika seems to be effective and efficient in terms of supporting navigation and also a learnable product for intensive care physicians fulfilling their need to get an accurate overview of a patient status quickly. Applying eye-tracking technology during usability testing has shown to be a valuable complement to traditional methods that revealed many unexpected issues in terms of navigation and contributed a supplementary understanding about design problems and user performance.

[1]  Maria Hägglund,et al.  Integrated Care Plan for Shared Home Care : the technical architecture , 2005 .

[2]  Enrico Coiera,et al.  Decision support systems for antibiotic prescribing , 2008, Current opinion in infectious diseases.

[3]  Maria Hägglund,et al.  Health informatics and the delivery of care to older people. , 2009, Maturitas.

[4]  Alireza Kazemi,et al.  CPOE in Iran - A viable prospect?: Physicians' opinions on using CPOE in an Iranian teaching hospital , 2009, Int. J. Medical Informatics.

[5]  Maria Hägglund,et al.  A Case Study of a Service Oriented Architecture (SOA) Implementation using the , 2006 .

[6]  Torgny Groth,et al.  Methods for selection of adequate neural network structures with application to early assessment of chest pain patients by biochemical monitoring , 2000, Int. J. Medical Informatics.

[7]  U G Fors,et al.  Computer-aided risk management--a software tool for the Hidep model. , 2001, Quintessence international.

[8]  Uno Fors,et al.  Building a virtual patient commons , 2008, Medical teacher.

[9]  Qiang Fu,et al.  Anaesthesia record system on handheld computers - pilot experience and uses for quality control and clinical guidelines , 2005, Comput. Methods Programs Biomed..

[10]  Lars Bergman,et al.  Computer-aided DSM-IV-diagnostics – acceptance, use and perceived usefulness in relation to users' learning styles , 2005, BMC Medical Informatics Decis. Mak..

[11]  S Koch,et al.  Controlled diagnosis-oriented enhancement of automatically segmented radiographs in dentistry. , 1998, Computer methods and programs in biomedicine.

[12]  Nabil Zary,et al.  WASP -a generic web-based, interactive, patient simulation system , 2003, MIE.

[13]  Uno Fors,et al.  Evaluation of a web-based ECG-interpretation programme for undergraduate medical students , 2008, BMC medical education.

[14]  Torgny Groth,et al.  Dynamic decision support graph - Visualization of ANN-generated diagnostic indications of pathological conditions developing over time , 2008, Artif. Intell. Medicine.

[15]  Stefan Seipel,et al.  Rekonstruktion und dreidimensionale Visualisierung klinischer Krankheitsverläfe in der Parodontologie , 1994, GMDS.

[16]  Uno Fors,et al.  What they fill in today, may not be useful tomorrow: Lessons learned from studying Medical Records at the Women hospital in Tabriz, Iran , 2008, BMC public health.

[17]  Uno Fors,et al.  A new, interactive, computer simulation concept for treatment and evaluation of men convicted of domestic violence , 2010 .

[18]  Fabio Paternò,et al.  User Needs for Development of Context Dependent Devices in Mobile Home Care , 2003, Mobile HCI.

[19]  Maria Hägglund,et al.  Visualisation and interaction design solutions to address specific demands in Shared Home Care , 2006, MIE.

[20]  Sabine Koch.  Ubiquitous care in aging societies - a social challenge. , 2008, Studies in health technology and informatics.

[21]  S Koch.  Designing Clinically Useful Systems: Examples from Medicine and Dentistry , 2003, Advances in dental research.

[22]  Uno Fors,et al.  Bmc Medical Education Evaluation of an Interactive Case Simulation System in Dermatology and Venereology for Medical Students , 2022 .

[23]  Alireza Kazemi,et al.  The Effect of Computerized Physician Order Entry and Decision Support System on Medication Errors in the Neonatal Ward: Experiences from an Iranian Teaching Hospital , 2011, Journal of Medical Systems.

[24]  Parvati Dev,et al.  Interactive Simulated Patient: Experiences with Collaborative E-Learning in Medicine , 2003 .

[25]  Maria Hägglund,et al.  Integration Architecture of a Mobile Virtual Health Record for Shared Home Care , 2005, MIE.

[26]  Maria Hägglund,et al.  Heuristic Evaluation Extended by User Analysis: A Fast and Efficient Method to Identify Potential Usability Problems in Health Information Systems , 2006 .

[27]  W Schneider,et al.  Three-dimensional visualization of the mandible: a new method for presenting the periodontal status and diseases. , 1995, Computer methods and programs in biomedicine.

[28]  I V Wagner,et al.  Oral implant treatment planning in a virtual reality environment. , 1998, Computer methods and programs in biomedicine.

[29]  S Koch.  Ubiquitous computing in shared-care environments. , 2006, International journal of computerized dentistry.

[30]  Adrian David Cheok,et al.  22nd International Conference on Human-Computer Interaction with Mobile Devices and Services , 2007, Lecture Notes in Computer Science.

[31]  Martin Henkel,et al.  A New Approach for Goal-oriented Analysis of Healthcare Processes , 2010, MedInfo.

[32]  Sabine Koch,et al.  New Concept of an Integrated IT&T-based Dental Workstation for Quality Assurance in Oral Health Care , 1998, MedInfo.

[33]  Sabine Koch,et al.  Healthy ageing supported by technology – a cross-disciplinary research challenge , 2010, Informatics for health & social care.

[34]  Maria Hägglund,et al.  From user needs to system specifications: Multi-disciplinary thematic seminars as a collaborative design method for development of health information systems , 2008, J. Biomed. Informatics.

[35]  Andrzej A. Kononowicz,et al.  Enabling Interoperability, Accessibility and Reusability of Virtual Patients across Europe - Design and Implementation , 2009, MIE.

[36]  Sabine Koch,et al.  Factors influencing the quality of medical documentation when a paper-based medical records system is replaced with an electronic medical records system: An Iranian case study , 2008, International Journal of Technology Assessment in Health Care.

[37]  Uno Fors,et al.  Virtual patients for assessment of clinical reasoning in nursing -- a pilot study. , 2011, Nurse education today.

[38]  Maria Hägglund,et al.  Scenarios to capture work processes in shared homecare - From analysis to application , 2010, Int. J. Medical Informatics.

[39]  Sabine Koch,et al.  Computergestütztes Arbeiten und klinisches Dokumentieren am zahnärztlichen Arbeitsplatz (Teil 2): Qualitätssicherung in der digitalen intraoralen Radiographie - Methoden für die Praxis , 1996 .

[40]  Rong Chen,et al.  Modeling shared care plans using CONTsys and openEHR to support shared homecare of the elderly , 2011, J. Am. Medical Informatics Assoc..

[41]  Andrzej A. Kononowicz,et al.  Development and Validation of Strategies to Test for Interoperability of Virtual Patients , 2009, MIE.

[42]  Maria Hägglund,et al.  Bridging the gap: a virtual health record for integrated home care , 2007, International journal of integrated care.

[43]  T Groth,et al.  Early assessment of patients with suspected acute myocardial infarction by biochemical monitoring and neural network analysis. , 1997, Clinical chemistry.

[44]  Sabine Koch,et al.  Knowledge-based diagnosis-oriented improvement of automatically segmented intraoral radiographs , 1996 .

[45]  W Schneider,et al.  Effective and quality-controlled use of digital radiography in dental practice. , 2000, International journal of computerized dentistry.

[46]  Jeremiah Scholl,et al.  Challenges and opportunities for the adoption of telemedicine in India , 2011, Journal of telemedicine and telecare.

[47]  Andreas Hamfelt,et al.  A Web-based User Adaptive Learning Environment , 2000 .

[48]  Elizabeth M Borycki,et al.  Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application , 2005, Int. J. Medical Informatics.

[49]  Maria Hägglund,et al.  Specific demands for developing ICT systems for shared home care - a user centred approach , 2005 .

[50]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[51]  Maria Hägglund,et al.  Towards a virtual health record for mobile home care of elderly citizens , 2004, MedInfo.

[52]  Sabine Koch,et al.  Home telehealth - Current state and future trends , 2006, Int. J. Medical Informatics.

[53]  N Zary,et al.  Web-based virtual patients in dentistry: factors influencing the use of cases in the Web-SP system. , 2009, European journal of dental education : official journal of the Association for Dental Education in Europe.

[54]  Stefan Seipel,et al.  Bildverarbeitende Methoden zur Qualitätsbeurteilung und automatischen Optimierung intraoraler Röntgenaufnahmen , 1994, GMDS.

[55]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[56]  Håkan Hult,et al.  Virtual patient simulation for learning and assessment: Superior results in comparison with regular course exams , 2010, Medical teacher.

[57]  Torgny Groth,et al.  Transferability of neural network-based decision support algorithms for early assessment of chest-pain patients , 2000, Int. J. Medical Informatics.

[58]  Matthew H. Samore,et al.  Antimicrobial Resistance Determinants and Future Control , 2005, Emerging infectious diseases.

[59]  T Groth,et al.  Neural network analysis of biochemical markers for early assessment of acute myocardial infarction. , 1997, Studies in health technology and informatics.

[60]  Jonathan M. Teich,et al.  Grand challenges in clinical decision support , 2008, J. Biomed. Informatics.

[61]  S. Koch,et al.  Heuristic evaluation performed by usability-educated clinicians: education and attitudes. , 2007, Studies in health technology and informatics.

[62]  S Koch,et al.  An object-relational model for structured representation of medical knowledge. , 2006, International journal of computerized dentistry.

[63]  A. So,et al.  Meeting the challenge of antibiotic resistance , 2008, BMJ : British Medical Journal.

[64]  Alireza Kazemi,et al.  Physician Order Entry Or Nurse Order Entry? Comparison of Two Implementation Strategies for a Computerized Order Entry System Aimed at Reducing Dosing Medication Errors , 2010, Journal of medical Internet research.

[65]  Maria Hägglund,et al.  A user-centred deployment process for ICT in health care teams - Experiences from the OLD@HOME project , 2006, MIE.

[66]  Sabine Koch,et al.  Social Care Informatics - The Missing Partner in eHealth , 2009, MIE.

[67]  Isabella Scandurra,et al.  Using scenarios to capture work processes in shared home care. , 2007, Studies in health technology and informatics.

[68]  Sabine Koch.  Meeting the Challenges - the Role of Medical Informatics in an Ageing Society , 2006, MIE.

[69]  Uno Fors,et al.  Interactive Simulation of Patients – An advanced tool for student activating learning in medicine , 2002 .

[70]  R Haux,et al.  On health-enabling and ambient-assistive technologies. What has been achieved and where do we have to go? , 2009, Methods of information in medicine.

[71]  Nabil Zary,et al.  Development, implementation and pilot evaluation of a Web-based Virtual Patient Case Simulation environment – Web-SP , 2006, BMC medical education.

[72]  Uno Fors,et al.  A pilot for a computer-based simulation system for risk estimation and treatment of mentally disordered offenders , 2009, Informatics for health & social care.

[73]  W Schneider,et al.  IT-based evaluation and automatic improvement of the quality of intraoral radiographs. , 1995, Computer methods and programs in biomedicine.

[74]  P C Tang,et al.  Major issues in user interface design for health professional workstations: summary and recommendations. , 1994, International journal of bio-medical computing.

[75]  Maria Hägglund,et al.  Integrated Care Plan and Documentation on Handheld Devices in Mobile Home Care , 2004, Mobile HCI.

[76]  Lars Bergman,et al.  Decision support in psychiatry – a comparison between the diagnostic outcomes using a computerized decision support system versus manual diagnosis , 2008, BMC Medical Informatics Decis. Mak..