Building on strength, understanding weakness: realistic evaluation and program review.

The implementation of the EWB Challenge as a part of a compulsory first year subject for a large cohort of engineering students at the University of Queensland has had a mixed response from both students and staff. In evaluating "what worked for whom under what circumstances" we have begun to disentangle the ways in which disparate factors interacted to produce the observed responses. This will allow us to articulate a theoretical model to explain the mechanisms leading to those outcomes and to demonstrate how much could be attributed to the Challenge and how much to other factors. This paper describes this realist approach to evaluation and offers a model for the evaluation of innovations that goes well beyond whether the students liked it or not.