Action research: Overcoming the Sports Mentality Approach to Assessment/Evaluation

Each time we make significant changes in what we teach or how we teach we are faced with the same question: how can we find out whether the innovation we have brought into our classroom is worthwhile? Chemists, familiar as they are with the criteria for decision-making adopted by physical scientists, find this question so difficult to answer that they often avoid doing the experiment that might provide evidence on which to base an answer. Let us therefore build a metaphor on a recent example from medical research. In 1997, Bailar and Gornik reported an analysis of ageadjusted mortality rates due to cancer from 1950 to 19941. This paper was picked up by the popular press, who reported that the war on cancer had been a failure2. Bailar and Gornik chose to analyze age-adjusted mortality rates because they regarded it as “the most basic measure of progress against cancer” and because it “focuses attention on the outcome that is most reliably reported”1. The question before us is simple: would they have reached the same conclusions if they had examined changes in the length of the patient’s survival, or changes in the quality of life after cancer had been diagnosed? Bailar and Gornik’s paper provides a metaphor on which discussions of the evaluation of instructional innovation can be based because it illustrates the role that the choice of methodology for evaluation has on the conclusions that are reached. Chemists concerned with improving the way they teach chemistry need to recognize this and act accordingly.