Usability in healthcare : overcoming the mismatch between information systems and clinical work

Aalto University, P.O. Box 11000, FI-00076 Aalto www.aalto.fi Author Johanna Kaipio Name of the doctoral dissertation Usability in Healthcare: Overcoming the Mismatch between Information Systems and Clinical Work Publisher School of Science Unit Department of Computer Science and Engineering Series Aalto University publication series DOCTORAL DISSERTATIONS 105/2011 Field of research T011Z Usability Research Manuscript submitted 14 June 2011 Manuscript revised 5 October 2011 Date of the defence 4 November 2011 Language English Monograph Article dissertation (summary + original articles)

[1]  Carla Simone,et al.  PRODOC: an Electronic Patient Record to Foster Process-Oriented Practices , 2009, ECSCW.

[2]  Vimla L. Patel,et al.  Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces , 1997, AMIA.

[3]  Petra Wilson,et al.  Mapping the Potential of eHealth: Empowering the Citizen through eHealth Tools and Services. Research Report presented at the eHealth Conference, Cork, Ireland, 5-6 May 2004 , 2004 .

[4]  Karel Vredenburg,et al.  A survey of user-centered design practice , 2002, CHI.

[5]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[6]  Moira C. Norrie,et al.  Physicians' and Nurses' Documenting Practices and Implications for Electronic Patient Record Design , 2007, USAB.

[7]  François Sainfort,et al.  Evaluating usability of a commercial electronic health record: A case study , 2008, Int. J. Hum. Comput. Stud..

[8]  Morten Hertzum,et al.  Images of Usability , 2010, Int. J. Hum. Comput. Interact..

[9]  J. Preece,et al.  The Human-Computer Interaction Handbook , 2003 .

[10]  H. Hyppönen.  eHealth services and technology: Challenges for co-development , 2007 .

[11]  B. Nardi.  Activity theory and human-computer interaction , 1995 .

[12]  Jan Stage,et al.  A longitudinal study of usability in health care: Does time heal? , 2010, Int. J. Medical Informatics.

[13]  Jesper Simonsen,et al.  Positive effects of electronic patient records on three clinical activities , 2008, Int. J. Medical Informatics.

[14]  M. Kyng,et al.  Participatory Design in Health Sciences: Using Cooperative Experimental Methods in Developing Health Services and Computer Technology , 2007, Qualitative health research.

[15]  Alex Mihailidis,et al.  Pervasive Computing in Healthcare , 2006 .

[16]  Kaija Saranto,et al.  Evaluation of electronic nursing documentation - Nursing process model and standardized terminologies as keys to visible and transparent nursing , 2010, Int. J. Medical Informatics.

[17]  Tom Delbanco,et al.  Healthcare in a land called PeoplePower: nothing about me without me , 2001, Health expectations : an international journal of public participation in health care and health policy.

[18]  E. B. Steen,et al.  The Computer-Based Patient Record: An Essential Technology for Health Care , 1992, Annals of Internal Medicine.

[19]  Sameer Malhotra,et al.  Physicians' potential use and preferences related to health information exchange , 2011, Int. J. Medical Informatics.

[20]  Steven K. Feiner,et al.  Physician-driven management of patient progress notes in an intensive care unit , 2010, CHI.

[21]  Tinja Lääveri,et al.  Physicians' experiences of participation in healthcare IT development in Finland: Willing but not able , 2012, Int. J. Medical Informatics.

[22]  Yngve Dahl,et al.  Toward a best practice for laboratory-based usability evaluations of mobile ICT for hospitals , 2008, NordiCHI.

[23]  André Kushniruk,et al.  Evaluation in the design of health information systems: application of approaches emerging from usability engineering , 2002, Comput. Biol. Medicine.

[24]  Charles Safran,et al.  Health care in the information society , 2002, Int. J. Medical Informatics.

[25]  Ian Robinson,et al.  Benefits of and barriers to involving users in medical device technology development and evaluation , 2007, International Journal of Technology Assessment in Health Care.

[26]  H. Jaap van den Herik,et al.  Towards Patient-Related Information Needs , 2005, MIE.

[27]  Philipp Neuhaus,et al.  Adapting Usability Testing Techniques to Gather User Requirements: An Illustrative Proposal , 2011, ITCH.

[28]  Wolfgang Lauer,et al.  Cognitive Task Analysis for Prospective Usability Evaluation in Computer-Assisted Surgery , 2007, USAB.

[29]  Jakob E. Bardram,et al.  Activity-based computing for medical work in hospitals , 2009, TCHI.

[30]  Lacramioara Stoicu-Tivadar,et al.  Human-computer interaction reflected in the design of user interfaces for general practitioners , 2006, Int. J. Medical Informatics.

[31]  Jos Aarts,et al.  User centred networked health care , 2011 .

[32]  Susanna Martikainen,et al.  Participatory interaction design in user requirements specification in healthcare , 2010, MedInfo.

[33]  Blackford Middleton,et al.  Barriers to Electronic Health Record Use during Patient Visits , 2006, AMIA.

[34]  Elske Ammenwerth,et al.  Health care in the information society. A prognosis for the year 2013 , 2002, Int. J. Medical Informatics.

[35]  Berthel Sutter,et al.  What does it take to replace an old functioning information system with a new one?: A case study , 2007, Int. J. Medical Informatics.

[36]  Dag Svanæs,et al.  Fidelity Considerations for Simulation-Based Usability Assessments of Mobile ICT for Hospitals , 2010, Int. J. Hum. Comput. Interact..

[37]  Rollin J Fairbanks,et al.  Poor interface design and lack of usability testing facilitate medical error. , 2004, Joint Commission journal on quality and safety.

[38]  Mikko Korpela,et al.  A participatory assessment of IS integration needs in maternity clinics using activity theory , 2007, Int. J. Medical Informatics.

[39]  Dag Svanæs,et al.  Usability testing of mobile ICT for clinical settings: Methodological and practical challenges , 2010, Int. J. Medical Informatics.

[40]  M G Kahn,et al.  Contextual inquiry: discovering physicians' true needs. , 1995, Proceedings. Symposium on Computer Applications in Medical Care.

[41]  Elizabeth Murphy,et al.  Capturing user requirements in medical device development: the role of ergonomics , 2006, Physiological measurement.

[42]  Pirkko Nykänen,et al.  Hoitotyön kansallisen kirjaamismallin ja hoitokertomusten käytettävyys , 2010 .

[43]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[44]  Nicolette de Keizer,et al.  Introducing Guidelines for Good Evaluation Practice in Health Informatics , 2009, MIE.

[45]  André Kushniruk,et al.  Cognitive Analysis of a Medication Reconciliation Tool: Applying Laboratory and Naturalistic Approaches to System Evaluation , 2011, ITCH.

[46]  Kenton O'Hara,et al.  Collaboration Within Different Settings: A Study of Co-located and Distributed Multidisciplinary Medical Team Meetings , 2010, Computer Supported Cooperative Work (CSCW).

[47]  Roger Watson,et al.  A quantitative analysis of the impact of a computerised information system on nurses' clinical practice using a realistic evaluation framework , 2009, Int. J. Medical Informatics.

[48]  Jan Gulliksen,et al.  Key principles for user-centred systems design , 2003, Behav. Inf. Technol..

[49]  Jakob E. Bardram,et al.  Temporal Coordination –On Time and Coordination of CollaborativeActivities at a Surgical Department , 2000, Computer Supported Cooperative Work (CSCW).

[50]  Mario Beyer,et al.  IT support for clinical pathways - Lessons learned , 2006, Int. J. Medical Informatics.

[51]  Ping Yu,et al.  Health IT acceptance factors in long-term care facilities: A cross-sectional survey , 2009, Int. J. Medical Informatics.

[52]  Kasper Hornbæk,et al.  Work-domain knowledge in usability evaluation: Experiences with Cooperative Usability Testing , 2010, J. Syst. Softw..

[53]  Kaija Saranto,et al.  Definition, structure, content, use and impacts of electronic health records: A review of the research literature , 2008, Int. J. Medical Informatics.

[54]  Wendy Chan,et al.  Increasing the success of physician order entry through human factors engineering. , 2002, Journal of healthcare information management : JHIM.

[55]  Hallvard Lærum,et al.  Experiences with the Implementation of Computerized Tools in Health Care Units: A Review Article , 2008, Int. J. Hum. Comput. Interact..

[56]  Marko Nieminen,et al.  Usability Evaluation of Digital Dictation Procedure - An Interaction Analysis Approach , 2011, USAB.

[57]  David W. Bates,et al.  White Paper: Personal Health Records: Definitions, Benefits, and Strategies for Overcoming Barriers to Adoption , 2006, J. Am. Medical Informatics Assoc..

[58]  Anne Kuusisto,et al.  Experiences with the Electronic Nursing Discharge Summary , 2009, Nursing Informatics.

[59]  Sameer Malhotra,et al.  Designing the design phase of critical care devices: a cognitive approach , 2005, J. Biomed. Informatics.

[60]  Richard I. Cook,et al.  Getting to the point: developing IT for the sharp end of healthcare , 2005, J. Biomed. Informatics.

[61]  Douglas A. Talbert,et al.  Clinicians' perceptions of clinical decision support integrated into computerized provider order entry , 2004, Int. J. Medical Informatics.

[62]  Tinja Lääveri,et al.  National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability , 2011, Int. J. Medical Informatics.

[63]  Sari Kauppinen,et al.  Yksityinen palvelutuotanto sosiaali- ja terveydenhuollossa , 2003 .

[64]  Robert L Edsall,et al.  An EHR user-satisfaction survey: advice from 408 family physicians. , 2005, Family practice management.

[65]  Jeffery L. Belden,et al.  Defining and testing EMR usability: Principles and proposed methods of EMR usability evaluation and rating. , 2009 .

[66]  Andrea Giordano,et al.  How can usability measurement affect the re-engineering process of clinical software procedures? , 1998, Int. J. Medical Informatics.

[67]  Yan Xiao,et al.  Artifacts and collaborative work in healthcare: methodological, theoretical, and technological implications of the tangible , 2005, J. Biomed. Informatics.

[68]  P. Ruotsalainen,et al.  Finland's strategy and implementation of citizens' access to health information. , 2008, Studies in health technology and informatics.

[69]  Nigel Bevan,et al.  Usability is Quality of Use , 1995 .

[70]  Kerstin Öhrn,et al.  Using system dynamics for collaborative design: a case study , 2007, BMC Health Services Research.

[71]  Arild Faxvaag,et al.  Norwegians GPs' use of electronic patient record systems , 2009, Int. J. Medical Informatics.

[72]  N. D. de Keizer,et al.  An Inventory of Evaluation Studies of Information Technology in Health Care , 2005, Methods of Information in Medicine.

[73]  An Jacobs,et al.  A methodology for shifting the focus of e-health support design onto user needs: A case in the homecare field , 2008, Int. J. Medical Informatics.

[74]  Jarmo Reponen,et al.  Current status of national eHealth and telemedicine development in Finland. , 2008, Studies in health technology and informatics.

[75]  Andre Kushniruk,et al.  Integrating technology-centric and user-centric system testing methods: ensuring healthcare system usability and safety. , 2010, Studies in health technology and informatics.

[76]  Marinos Themistocleous,et al.  Evaluating Integration Approaches Adopted by Healthcare Organizations , 2006, J. Comput. Inf. Syst..

[77]  Vimla L. Patel,et al.  A framework for analyzing the cognitive complexity of computer-assisted clinical ordering , 2003, J. Biomed. Informatics.

[78]  Monique W. M. Jaspers,et al.  A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence , 2009, Int. J. Medical Informatics.

[79]  Nicolette de Keizer,et al.  UvA-DARE ( Digital Academic Repository ) An inventory of evaluation studies of information technology in health care trends in evaluation research 1982-2002 , 2005 .

[80]  Robyn Tamblyn,et al.  Review Paper: The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review , 2005, J. Am. Medical Informatics Assoc..

[81]  Johanna Viitanen.  Redesigning digital dictation for physicians: A user-centred approach , 2009, Health Informatics J..

[82]  Cornelia M. Ruland,et al.  A survey about the usefulness of computerized systems to support illness management in clinical practice , 2004, Int. J. Medical Informatics.

[83]  P Nykänen,et al.  Success and failure factors in the regional health information system design process--results from a constructive evaluation study. , 2006, Methods of information in medicine.

[84]  K. Davis,et al.  A 2020 vision of patient-centered primary care , 2005, Journal of General Internal Medicine.

[85]  Karen Davis,et al.  Health information technology and physician perceptions of quality of care and satisfaction. , 2009, Health policy.

[86]  Kaarina Tanttu,et al.  Nationally standardized electronic nursing documentation in Finland by the year 2007. , 2006, Studies in health technology and informatics.

[87]  Álvaro Alesanco Iglesias,et al.  Organizational, Contextual and User-Centered Design in e-Health: Application in the Area of Telecardiology , 2007, USAB.

[88]  Hans-Ulrich Prokosch,et al.  Visions and strategies to improve evaluation of health information systems: Reflections and lessons based on the HIS-EVAL workshop in Innsbruck , 2004, Int. J. Medical Informatics.

[89]  Wilhelm Hasselbring,et al.  The SI challenge in health care , 2000, CACM.

[90]  Rebecca Randell,et al.  Editorial — Evaluating New Interactions in Health Care: Challenges and Approaches , 2010, Int. J. Hum. Comput. Interact..

[91]  Ilaria Redaelli,et al.  Knowledge Management in Locating the Patient in an Emergency Medical Service in Italy , 2010, Computer Supported Cooperative Work (CSCW).

[92]  U. Josefsson,et al.  Patients creating self-help on the internet -lessons for future design of internet based healthcare resources , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[93]  K. Kuhn,et al.  From Hospital Information Systems to Health Information Systems , 2001, Methods of Information in Medicine.

[94]  Gregory D. Abowd,et al.  Human-Computer Interaction (3rd Edition) , 2003 .

[95]  T. Norris.  The Computer-Based Patient Record: An Essential Technology for Health Care. Revised Edition , 1998, The Journal of the American Board of Family Medicine.

[96]  ZhangJiajie,et al.  A user-centered framework for redesigning health care interfaces , 2005 .

[97]  M. Conrick.  Health Informatics: Transforming Healthcare With Technology , 2006 .

[98]  M. Sheelagh T. Carpendale,et al.  InfoFlow Framework for Evaluating Information Flow and New Health Care Technologies , 2010, Int. J. Hum. Comput. Interact..

[99]  Reinhold Haux,et al.  Health information systems - past, present, future , 2006, Int. J. Medical Informatics.

[100]  Craig E. Kuziemsky,et al.  A grounded theory guided approach to palliative care systems design , 2007, Int. J. Medical Informatics.

[101]  Richard Heeks,et al.  Health information systems: Failure, success and improvisation , 2006, Int. J. Medical Informatics.

[102]  Clay Spinuzzi,et al.  Context and consciousness: Activity theory and human-computer interaction , 1997 .

[103]  Monique W. M. Jaspers,et al.  Usability Studies on Interactive Health Information Systems; Where Do We Stand? , 2008, MIE.

[104]  Peter R. Croll,et al.  Investigating risk exposure in e-health systems , 2007, Int. J. Medical Informatics.

[105]  Jens H. Weber,et al.  Engineering Medical Information Systems: Architecture, Data and Usability & Security , 2007, 29th International Conference on Software Engineering (ICSE'07 Companion).

[106]  Sandra Martelli,et al.  Criteria of interface evaluation for computer assisted surgery systems , 2003, Int. J. Medical Informatics.

[107]  Pekka Ruotsalainen,et al.  Sharable EHR systems in Finland. , 2006, Studies in health technology and informatics.

[108]  Johanna Viitanen,et al.  Contextual Inquiry Method for User-Centred Clinical IT System Design , 2011, MIE.

[109]  J Brender,et al.  Cognitive evaluation: how to assess the usability of information technology in healthcare. , 1997, Computer methods and programs in biomedicine.

[110]  A. Hassey.  The National Programme for IT in the NHS. , 2005, The British journal of general practice : the journal of the Royal College of General Practitioners.

[111]  R. Weber.  Basic Content Analysis , 1986 .

[112]  Elizabeth Keay,et al.  Methods to assess the safety of health information systems. , 2010, Healthcare quarterly.

[113]  E BardramJakob.  Temporal Coordination On Time and Coordination of CollaborativeActivities at a Surgical Department , 2000 .

[114]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[115]  Morten Hertzum,et al.  Acceptance of speech recognition by physicians: A survey of expectations, experiences, and social influence , 2009, Int. J. Hum. Comput. Stud..

[116]  Vimla L. Patel,et al.  Interface design for health care environments: the role of cognitive science , 1998, AMIA.

[117]  Elizabeth M. Borycki,et al.  Use of Clinical Simulations to Evaluate the Impact of Health Information Systems and Ubiquitous Computing Devices Upon Health Professional Work , 2011 .

[118]  Marc Berg,et al.  Accumulating and Coordinating: Occasions for Information Technologies in Medical Work , 1999, Computer Supported Cooperative Work (CSCW).

[119]  Vimla L. Patel,et al.  Cognitive and usability engineering methods for the evaluation of clinical information systems , 2004, J. Biomed. Informatics.

[120]  What next for electronic communication and health care? , 2004, BMJ : British Medical Journal.

[121]  P. Shekelle,et al.  Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care , 2006, Annals of Internal Medicine.

[122]  Robert Nguyen,et al.  Evidence on the Costs and Benefits of Health Information Technology , 2008 .

[123]  Ian Robinson,et al.  User involvement in healthcare technology development and assessment: structured literature review. , 2006, International journal of health care quality assurance incorporating Leadership in health services.

[124]  André Kushniruk,et al.  Who's Users? Participation and Empowerment in Socio-Technical Approaches to Health IT Developments , 2011, ITCH.

[125]  David W. Bates,et al.  Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study , 2005, J. Biomed. Informatics.

[126]  R. Grol,et al.  Which data source in clinical performance assessment? A pilot study comparing self-recording with patient records and observation. , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[127]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[128]  J. M. Christian Bastien,et al.  Usability testing: a review of some methodological and technical aspects of the method , 2010, Int. J. Medical Informatics.

[129]  Johanna I. Westbrook,et al.  Allied health professionals' use of online evidence: a survey of 790 staff working in the Australian public hospital system , 2004, Int. J. Medical Informatics.

[130]  C. Anandan,et al.  The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview , 2011, PLoS medicine.

[131]  S. Davidson,et al.  Physician and nurse satisfaction with an Electronic Medical Record system. , 2004, The Journal of emergency medicine.

[132]  Timo Jokela,et al.  To what extent usability truly matters?: a study on usability requirements in call-for-tenders of software systems issued by public authorities , 2010, NordiCHI.

[133]  Dag Svanæs,et al.  The Contextual Nature of Usability and its Relevance to Medical Informatics , 2008, MIE.

[134]  Sampsa Hyysalo,et al.  VERSIONS OF CARE TECHNOLOGY , 2007 .

[135]  Elske Ammenwerth,et al.  PLANNING TO TEST SUCCESS - EMPLOYING THE GUIDELINES FOR GOOD EVALUATION PRACTICE IN HEALTH INFORMATICS (GEP-HI) , 2009 .

[136]  Elske Ammenwerth,et al.  Effect of a nursing information system on the quality of information processing in nursing: An evaluation study using the HIS-monitor instrument , 2011, Int. J. Medical Informatics.

[137]  Jarmo Reponen,et al.  Large-scale eHealth Systems: Providing Information to Support Evidence-based Management , 2011 .

[138]  Margit Pohl,et al.  Usability and Transferability of a Visualization Methodology for Medical Data , 2007, USAB.

[139]  Johanna Kaipio,et al.  Evaluation of the national nursing model and four nursing documentation systems in Finland - Lessons learned and directions for the future , 2012, Int. J. Medical Informatics.

[140]  Elizabeth M Borycki,et al.  Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application , 2005, Int. J. Medical Informatics.

[141]  Robert L. Wears,et al.  Health information technology: fallacies and sober realities , 2010, J. Am. Medical Informatics Assoc..

[142]  Annalu Waller,et al.  Participatory design of a text message scheduling system to support young people with diabetes , 2006, Health Informatics J..

[143]  Ting-Ting Lee,et al.  Two-stage evaluation of the impact of a nursing information system in Taiwan , 2008, Int. J. Medical Informatics.

[144]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[145]  Colin G. Ellard,et al.  Context and consciousness , 1995, Behavioral and Brain Sciences.

[146]  Karen Davis,et al.  Continuous innovation in health care: implications of the Geisinger experience. , 2008, Health affairs.

[147]  Michael Boland,et al.  What factors affect the use of electronic patient records by Irish GPs? , 2009, Int. J. Medical Informatics.

[148]  Timo Jokela,et al.  Determining usability requirements into a call-for-tenders: a case study on the development of a healthcare system , 2010, NordiCHI.

[149]  Kari Mäkelä,et al.  Management of electronic patient record systems in primary healthcare in a Finnish county. , 2010, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[150]  Toomas Timpka,et al.  Third generation participatory design in health informatics - Making user participation applicable to large-scale information system projects , 2008, J. Biomed. Informatics.

[151]  Stephanie M. Wilson,et al.  Cognitive Artefacts In Support Of Medical Shift Handover: An In-Use, In-Situ Evaluation , 2019 .

[152]  Torsten Gruchmann,et al.  The Effect of New Standards on the Global Movement Toward Usable Medical Devices , 2007, USAB.

[153]  Tapio Ala-Hiiro,et al.  Adopting the National Structure of Nursing Documentation is Consequential in the Development of Care , 2010, MedInfo.

[154]  Marc Berg.  Patients and professionals in the information society: what might keep us awake in 2013 , 2002, Int. J. Medical Informatics.

[155]  R. Hanka,et al.  Evaluating information technology in health care: barriers and challenges , 1998, BMJ.

[156]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[157]  J. T. Martin.  Report to the Congress , 1964 .

[158]  Monique W. M. Jaspers,et al.  The significance of a usability evaluation of an emerging laboratory order entry system , 2007, Int. J. Medical Informatics.

[159]  Craig E. Kuziemsky,et al.  Use of Qualitative Methods Across the Software Development Lifecycle in Health Informatics , 2011, ITCH.

[160]  Madhu C. Reddy,et al.  Coordinating heterogeneous work: Information and representation in medical care , 2001, ECSCW.

[161]  Jesús Favela,et al.  Ecological Validity and Pervasiveness in the Evaluation of Ubiquitous Computing Technologies for Health Care , 2010, Int. J. Hum. Comput. Interact..

[162]  Lou S. Davis.  Problems facing large Health Information Systems , 1973, ACM Annual Conference.

[163]  Nicolette de Keizer,et al.  Guideline for good evaluation practice in health informatics (GEP-HI) , 2011, Int. J. Medical Informatics.

[164]  David W. McDonald,et al.  Participatory design of a collaborative clinical trial protocol writing system , 2007, Int. J. Medical Informatics.

[165]  Russell E Glasgow,et al.  eHealth evaluation and dissemination research. , 2007, American journal of preventive medicine.

[166]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[167]  Hans-Bernd Bludau,et al.  Questionnaire Based Usability Evaluation of Hospital Information Systems , 2004 .

[168]  Elske Ammenwerth,et al.  Can evaluation studies benefit from triangulation? A case study , 2003, Int. J. Medical Informatics.

[169]  Arie Hasman,et al.  Review Paper: Determinants of Success of Inpatient Clinical Information Systems: A Literature Review , 2003, J. Am. Medical Informatics Assoc..

[170]  Peter G. Goldschmidt,et al.  HIT and MIS , 2005, Commun. ACM.

[171]  J A Jacko,et al.  Framework for usability: healthcare professionals and the Internet , 2001, Ergonomics.

[172]  Elizabeth M Borycki,et al.  Low-cost rapid usability engineering: designing and customizing usable healthcare information systems. , 2006, Healthcare quarterly.

[173]  Madhu C. Reddy,et al.  Participatory design and an eligibility screening tool , 2000, AMIA.

[174]  Beach Christina Thielst,et al.  Clinical documentation systems: another link between technology and quality. , 2008, Journal of healthcare management / American College of Healthcare Executives.

[175]  Blackford Middleton,et al.  Complementary methods of system usability evaluation: Surveys and observations during software design and development cycles , 2010, J. Biomed. Informatics.

[176]  Helena Karasti,et al.  Bridging Work Practice and System Design: Integrating Systemic Analysis, Appreciative Intervention and Practitioner Participation , 2001, Computer Supported Cooperative Work (CSCW).

[177]  Susanna Martikainen,et al.  Tampereen kaupungin Pegasos-järjestelmän käytettävyystutkimus murtumapotilaan hoitoketjussa , 2007 .

[178]  Pirkko Nykänen,et al.  Usability of Electronic Nursing Record Systems: Definition and Results from an Evaluation Study in Finland , 2011, ITCH.

[179]  Elizabeth M. Borycki,et al.  Usability Analysis of the Tele-nursing Call Management Software at HealthLink BC , 2011, ITCH.

[180]  M. Sheelagh T. Carpendale,et al.  Supporting Nurses' Information Flow by Integrating Paper and Digital Charting , 2009, ECSCW.

[181]  Marc Berg,et al.  Considerations for sociotechnical design: experiences with an electronic patient record in a clinical context , 1998, Int. J. Medical Informatics.

[182]  Stefan Hunziker,et al.  Nurses' Working Practices: What Can We Learn for Designing Computerised Patient Record Systems? , 2007, USAB.

[183]  D. Brailer.  Interoperability: the key to the future health care system. , 2005, Health affairs.

[184]  Elizabeth M. Borycki,et al.  Combining two forms of simulation to predict the potential impact of interface design on technology-induced error in healthcare , 2008, SpringSim '08.

[185]  Robert M. Schumacher,et al.  How to Select an Electronic Health Record System that Healthcare Professionals can Use , 2009 .

[186]  Marc Berg,et al.  Implementing information systems in health care organizations: myths and challenges , 2001, Int. J. Medical Informatics.

[187]  Toni Granollers,et al.  Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use , 2008, Inf. Softw. Technol..

[188]  Andre Kushniruk,et al.  Predicting changes in workflow resulting from healthcare information systems: ensuring the safety of healthcare. , 2006, Healthcare quarterly.