Evaluation of community-based injury prevention programmes: methodological issues and challenges

The evaluation of comprehensive community-based injury prevention programmes is complex and poses many methodological challenges. There is little consensus in contemporary literature about the most appropriate methods of evaluating these programmes. This study employed a systematic literature review to examine evaluations of 16 community-based injury prevention programmes with regard to key methodological issues and challenges. Three aspects of the evaluated programmes were analysed: assessed elements (context, structure, process, impact, and outcome); study design; and methodological issues addressed. The results showed that context, structure and process assessments were the most neglected aspects of the evaluation studies. The programmes were typically described with minimal discussion of how the context may have influenced the effectiveness. The process (activities) was described rather than evaluated against appropriate standards of comparisons. Impact evaluations adhered more closely to documented guidelines, but half of the evaluations did not include impact variables. Outcome evaluations focused on injury incidence. Most evaluations employed some qualitative methods, but the vast majority of methods used were quantitative. This study indicated that the quasi-experimental study design has become an accepted norm for the evaluation of community-based injury prevention programmes. Most of the evaluations contained explicit details of the methodology used and of the choices related to the methodology. While threats to internal validity were identified in most studies, problems related to external validity and construct validity were largely overlooked by the evaluators.