Learning from Experience: A New Approach to Evaluating Health Information Systems

Multi-method evaluations that include both quantitative and qualitative methods are being increasingly advocated in health informatics. The work presented here focuses on the methodology of conducting applied evaluation research on the role and impact of primary care based electronic medical records (EMRs). This mixed methods study is being conducted in the real world, as opposed to a laboratory or manipulated setting, and as such the research methodology itself is not following a traditional model but rather being developed iteratively in a process 1 am calling agile research. This paper explains the genesis of agile research and how it has been applied in the context of applied evaluation research in health informatics.

[1]  N T Ellis,et al.  When protocols fail: technical evaluation, biomedical knowledge, and the social production of 'facts' about a telemedicine clinic. , 2001, Social science & medicine.

[2]  C. R. May,et al.  Psychiatry By Videophone: A Trial Service In North West England , 1999, MIE.

[3]  Bonnie Kaplan,et al.  Qualitative Research Methods for Evaluating Computer Information Systems , 2005 .

[4]  Bonnie Kaplan,et al.  Fitting System Design to Work Practice:Using Observation in Evaluating A Clinical Imaging System , 1995 .

[5]  F S Mair,et al.  Factors Influencing the Evaluation of Telehealth Interventions: Preliminary Results from a Qualitative Study of Evaluation Projects in the UK , 2002, Journal of telemedicine and telecare.

[6]  Bonnie Kaplan,et al.  New Approaches To Evaluation: Alternatives To The Randomized Controlled Trial Qualitative Approaches to Design and Evaluation: Theory and Practice , 2001, MedInfo.

[7]  Nicola T. Shaw,et al.  'CHEATS': a generic information communication technology (ICT) evaluation framework , 2002, Comput. Biol. Medicine.

[8]  Bonnie Kaplan,et al.  Ethnographic interviews to elicit patients' reactions to an intelligent interactive telephone health behavior advisor system , 1999, AMIA.

[9]  B Kaplan,et al.  People, Organizational, and Social Issues: Evaluation as an exemplar , 2002, Yearbook of Medical Informatics.

[10]  F. Mair,et al.  Telepsychiatry evaluation in the north-west of England: Preliminary results of a qualitative study , 2000, Journal of telemedicine and telecare.

[11]  C. May,et al.  Evaluation of new technologies in health-care systems: what’s the context? , 2000 .

[12]  Frances S. Mair,et al.  Understanding the evaluation of telemedicine: the play of the social and the technical, and the shifting sands of reliable knowledge , 2001 .

[13]  Bonnie Kaplan,et al.  Evaluating informatics applications - some alternative approaches: theory, social interactionism, and call for methodological pluralism , 2001, Int. J. Medical Informatics.

[14]  F. Mair,et al.  Resisting and promoting new technologies in clinical practice: the case of telepsychiatry. , 2001, Social science & medicine.

[15]  Barry Boehm,et al.  Balancing Agility and Dis-cipline: A Guide for the Perplexed , 2003 .

[16]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..