Integrating Quantitative and Qualitative Methods in Patient Care Information System Evaluation

OBJECTIVE The aim of this paper is twofold. First, we describe two important dimensions of patient care information systems (PCIS) evaluation: the domain of evaluation and the different phases of the PCIS implementation. Second, we claim that, though Randomized Controlled Trials (RCTs) are often still seen as the standard approach, this type of design hardly generates relevant information for the organizational decision maker. METHOD Interpretive study of evaluation literature. RESULTS AND CONCLUSIONS The field of evaluation is scattered and the types of questions that can be asked and methods that can be used seem infinite and badly demarcated. Different stakeholders, moreover, often have different priorities in evaluating ICT. The most important reason for the lack of relevance of RCTs is that they are ill suited for investigating why and how a PCIS is being used, or not, and what the (often unplanned) effects and consequences are. Subsequently, our aim is to contribute to the discussion about the viability of qualitative versus quantitative methods in PCIS evaluation, by arguing for a specific integration of quantitative and qualitative research methods. The joint utilization of these methods, we claim, yields the richest results.

[1]  Steve Smithson,et al.  Information systems evaluation in practice: a case study of organizational change , 2000, J. Inf. Technol..

[2]  J Savage,et al.  Ethnography and health care , 2000, BMJ : British Medical Journal.

[3]  Frank F. Land,et al.  The moving staircase - Problems of appraisal and evaluation in a turbulent environment , 1999, Inf. Technol. People.

[4]  Frank Charles Gray Southon,et al.  IT, Change and evaluation: an overview of the role of evaluation in health services , 1999, Int. J. Medical Informatics.

[5]  Charles P. Friedman,et al.  Evaluation Methods in Medical Informatics , 1997, Computers and Medicine.

[6]  D. Bates,et al.  Improving response to critical laboratory results with automation: results of a randomized controlled trial. , 1999, Journal of the American Medical Informatics Association : JAMIA.

[7]  John Øvretveit,et al.  Evaluating Health Interventions: An Introduction to Evaluation of Health Treatments, Services, Policies, and Organizational Interventions , 1998 .

[8]  Donald Berwick,et al.  Developing and Testing Changes in Delivery of Care , 1998, Annals of Internal Medicine.

[9]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[10]  John Mingers,et al.  Combining IS Research Methods: Towards a Pluralist Methodology , 2001, Inf. Syst. Res..

[11]  Steve Smithson,et al.  Information systems evaluation in practice: a case study of organizational change , 2000, J. Inf. Technol..

[12]  Marc Berg,et al.  Patient care information systems and health care work: a sociotechnical approach , 1999, Int. J. Medical Informatics.

[13]  Bonnie Kaplan,et al.  Review: Addressing Organizational Issues into the Evaluation of Medical Systems , 1997, J. Am. Medical Informatics Assoc..

[14]  Timothy M. Franz,et al.  Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. , 1999, JAMA.

[15]  Lorraine Nicholson,et al.  Evaluating large scale health information systems: from practice towards theory , 1997, AMIA.

[16]  Matthew R. Jones Mission impossible? Pluralism and 'multi-paradigm' IS research , 1999 .

[17]  Bonnie Kaplan,et al.  Evaluating informatics applications - clinical decision support systems literature review , 2001, Int. J. Medical Informatics.

[18]  E. Murphy,et al.  Evaluating medical information systems: ethnomethodological and interactionist approaches , 2001, Health services management research.

[19]  Marilynne Hebert Professional and Organizational Impact of Using Patient Care Information Systems , 1998, MedInfo.

[20]  Jan L. Talmon,et al.  The VATAM guidelines , 1999, Int. J. Medical Informatics.

[21]  C. Pope,et al.  Qualitative Research: Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research , 1995 .

[22]  H A Heathfield,et al.  Current evaluations of information technology in health care are often inadequate , 1996, BMJ.

[23]  Bonnie Kaplan,et al.  Qualitative Research Methods for Evaluating Computer Information Systems , 2005 .