Evaluation frameworks for nursing informatics

Rigorous evaluation of informatics applications in healthcare is important so that the impact of such systems can be understood. Although many quantitative methods have been employed to evaluate informatics systems, there is a growing trend to utilized qualitative methods during both the formative and summative phases of research. Several evaluation frameworks that have been proposed highlight the need for both qualitative and quantitative evaluation methods in the development and post-implementation phases of informatics systems development. Recommendations regarding the timing, type and use of qualitative methods differ for each of these frameworks. This paper examines the strengths and weaknesses of the published evaluation frameworks and enumerates the qualitative research methods in use in each of these frameworks.

[1]  J. S. Briggs,et al.  Designing the user interface: strategies for effective human-computer interaction: by Ben Schneiderman. Published by Addison-Wesley. 463pp. £17.95 , 1987 .

[2]  Robert A. Greenes,et al.  Methods of Cognitive Analysis to Support the Design and Evaluation of Biomedical Systems: The Case of Clinical Practice Guidelines , 2001, J. Biomed. Informatics.

[3]  P. Munhall,et al.  Nursing Research: A Qualitative Perspective , 1987 .

[4]  Edward H. Shortliffe,et al.  System Design and Engineering , 2001 .

[5]  Reed M. Gardner,et al.  White Paper: Designing Medical Informatics Research and Library-Resource Projects to Increase What Is Learned , 1994, J. Am. Medical Informatics Assoc..

[6]  Allen C. Smith,et al.  Design and Conduct of Subjectivist Studies , 1997 .

[7]  Johanna I. Westbrook,et al.  Evaluating the impact of information communication technologies on complex organizational systems: a multi-disciplinary, multi-method framework , 2004, MedInfo.

[8]  C. Waltz,et al.  Measurement in nursing research , 1984 .

[9]  Jeremy C. Wyatt,et al.  When and how to evaluate health information systems? , 2003, Int. J. Medical Informatics.

[10]  Charles P. Friedman,et al.  Evaluation Methods in Medical Informatics , 1997, Computers and Medicine.

[11]  Gerhard Fischer,et al.  Transcending the individual human mind—creating shared understanding through collaborative design , 2000, TCHI.

[12]  C. Boyd.  Philosophical foundations of qualitative research. , 1993, NLN publications.

[13]  Jochen R. Moehr.  Evaluation: salvation or nemesis of medical informatics? , 2002, Comput. Biol. Medicine.

[14]  James G. Anderson,et al.  Evaluation in health informatics: social network analysis , 2002, Comput. Biol. Medicine.

[15]  Judith A. Effken,et al.  Different lenses, improved outcomes: a new approach to the analysis and design of healthcare information systems , 2002, Int. J. Medical Informatics.

[16]  M. Sandelowski.  Rigor or rigor mortis: the problem of rigor in qualitative research revisited. , 1993, ANS. Advances in nursing science.

[17]  Karel Vredenburg,et al.  The state of user-centered design practice , 2005, IEEE Engineering Management Review.

[18]  N. D. de Keizer,et al.  An Inventory of Evaluation Studies of Information Technology in Health Care , 2005, Methods of Information in Medicine.

[19]  Bonnie Kaplan,et al.  Review: Addressing Organizational Issues into the Evaluation of Medical Systems , 1997, J. Am. Medical Informatics Assoc..

[20]  Lawrence M. Fagan,et al.  Medical informatics: computer applications in health care and biomedicine (Health informatics) , 2003 .

[21]  Marc Berg,et al.  Patient care information systems and health care work: a sociotechnical approach , 1999, Int. J. Medical Informatics.

[22]  André Kushniruk,et al.  Evaluation in the design of health information systems: application of approaches emerging from usability engineering , 2002, Comput. Biol. Medicine.

[23]  Bonnie Kaplan,et al.  Evaluating informatics applications - some alternative approaches: theory, social interactionism, and call for methodological pluralism , 2001, Int. J. Medical Informatics.

[24]  D. R. Dixon,et al.  The behavioral side of information technology , 1999, Int. J. Medical Informatics.

[25]  Hans-Ulrich Prokosch,et al.  Visions and strategies to improve evaluation of health information systems: Reflections and lessons based on the HIS-EVAL workshop in Innsbruck , 2004, Int. J. Medical Informatics.

[26]  Nicola T. Shaw,et al.  'CHEATS': a generic information communication technology (ICT) evaluation framework , 2002, Comput. Biol. Medicine.

[27]  Vimla L. Patel,et al.  Cognitive and usability engineering methods for the evaluation of clinical information systems , 2004, J. Biomed. Informatics.

[28]  Charles P. Friedman,et al.  Evaluation and Technology Assessment , 2001 .

[29]  J. Simpson,et al.  The Oxford English Dictionary , 1884 .

[30]  Andrew Grant,et al.  The TEAM methodology for the evaluation of information systems in biomedicine , 2002, Comput. Biol. Medicine.

[31]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[32]  R A Greenes,et al.  Toward a Framework for Computer-Mediated Collaborative Design in Medical Informatics , 1999, Methods of Information in Medicine.