The future of natural selection knowledge measurement: A reply to Anderson et al. (2010)

The development of rich, reliable, and robust measures of the composition, structure, and stability of student thinking about core scientific ideas (such as natural selection) remains a complex challenge facing science educators. In a recent article (Nehm & Schonfeld 2008), we explored the strengths, weaknesses, and insights provided by a detailed exploration of three commonly used measures of student thinking about natural selection in a large sample ( > 100) of underrepresented minority students. One of our core findings was that all of the tools we studied—including the CINS—have strengths and weaknesses that must be carefully taken into consideration by those who employ, interpret, and act upon their outcomes. The continuous reevaluation and improvement of measurement instruments is a fundamental component of test development because of the inherent limitations of all methods at our disposal for capturing and quantifying student knowledge. Exploring the efficacy and generalizability of measures requires the repeated study of students from different racial and ethnic groups, geographic regions, socioeconomic and language backgrounds, and content preparations. Additionally, new methods (such as Rasch analysis) allow more accurate and precise evaluations of instrument properties. Furthermore, many science assessment developers have ignored the Standards for Educational and Psychological Testing (AERA/APA/NCME, 1999), which should be applied to all measures. We view Anderson, Fisher, and Smith’s (AFS) (2010) defense of the CINS as sacrosanct to be antithetical to the spirit and reality of instrument development, evaluation, and improvement.