The Use of Trained Observers as an Evaluation Tool for a Multi-Hospital Bioterrorism Exercise

Abstract Introduction: Evidence suggests that regular disaster exercises have beneficial effects on subsequent mock and actual disaster responses. The purpose of this report is to describe a multiple hospital, bioterrorism exercise, evaluated by independent observers who used an evaluation template. Methods: The overall tabletop exercise design included participation from 23 Joint Commission Accreditation of Healthcare Organizations hospitals, four health departments, and a representative from one federal agency. The exercise was evaluated by trained exercise observers utilizing an independently prepared, evaluation protocol. Results: All exercise sites successfully identified the bio-agent involved and answered after-exercise debriefing questions without much difficulty. Evaluations, in the form of an after-action report by the independent observers, commented upon the many limitations to the construct of the exercise. Conclusion: Having an independent observer group at the exercise appeared to provide a value-added benefitfor capturing subjective information and data. However, these data were not in a form conducive to statisticalanalysis. Further work is needed to create an evaluation tool that would allow for statistical analysis so that exercises can be compared and improvements can be objective.