A Model for Teaching Materials Evaluation: Development and Testing of Interactive Computer Simulations Modules for Undergraduate Education

A comprehensive evaluation program was developed as part of an NSF Course Curriculum, and Laboratory Improvement grant to develop three undergraduate computer simulation modules. Aerosol science and technology is generally taught at the graduate level and the goal of this program was to develop materials that would bring this subject to the undergraduate level. To achieve this goal, an evaluation plan was developed that included formative and summative, and cognitive and affective measures. This was a collaboration between content and evaluation experts who were involved from the proposal stage. The program included formative evaluation of pilot versions of the materials. Both undergraduate students and faculty were involved and provided significant feedback on how the materials could be modified to be more effective. The modified versions were formally tested in classroom settings to determine if students could master the material and if they enjoyed using the modules. Students made statistically significant gains in knowledge as a result of the modules and appreciated the ability to go through the simulations at their own pace. The evaluation program used here was instrumental in ensuring high quality products and can serve as a model easily exportable to other educational product development projects. The full model and the lessons learned will be described.