Mapping the Territory: issues in evaluating large-scale learning technology initiatives

This article details the challenges that the authors faced in designing and carrying out two recent large-scale evaluations of programmes designed to foster the use of ICT in UK higher education. Key concerns that have been identified within the evaluation literature are considered and an account is given of how these concerns were addressed within the two studies. A detailed examination is provided of the general evaluative strategies of employing a multi-disciplinary team and a multi-method research design and of how the research team went about: tapping into a range of sources of information, gaining different perspectives on innovation, tailoring enquiry to match vantage points, securing representative ranges of opinion, coping with changes over time, setting developments in context and dealing with audience requirements. Strengths and limitations of the general approach and the particular tactics that were used to meet the specific challenges posed within these two evaluation projects are identified.