Evaluating information technology in health care: barriers and challenges

There is strong push for clinical leadership in the development and procurement of information technology in health care.1 The lack of clinical input to date has been cited as a major factor in the failure of information technology in health services2 and has prompted many clinicians to become involved in such endeavours. Furthermore, there are various clinical decision support systems available, the merits of which clinicians are expected to judge (such as Prodigy3 and Capsule4). It is essential that clinicians have a knowledge of evaluation issues in order that they can assess the strengths and weaknesses of evaluation studies and thus interpret their results meaningfully, and also contribute to the design and implementation of such studies to provide them with useful information. #### Summary points Clinicians are becoming increasingly involved in the development and procurement of information technology in health care, yet evaluation studies have provided little useful information to assist them Evaluations by means of randomised controlled trials have not yet provided any major indication of improved patient outcomes or cost effectiveness, are difficult to generalise, and do not provide the scope or detail necessary to inform decision making Clinical information systems are a different kind of intervention from drugs, and techniques used to evaluate drugs (particularly randomised controlled trials) are not always appropriate The challenge for clinical informatics is to develop multi-perspective evaluations that integrate quantitative and qualitative methods Evaluation is not just for accountability but to improve our understanding of the role of information technology in health care and our ability to deliver systems that offer a wide range of clinical and economic benefits Decision makers may be …