Clicker Score Trajectories and Concept Inventory Scores as Predictors for Early Warning Systems for Large STEM Classes

Increasing the retention of STEM (science, technology, engineering, and mathematics) majors has recently emerged as a national priority in undergraduate education. Since poor performance in large introductory science and math courses is one significant factor in STEM dropout, early detection of struggling students is needed. Technology-supported “early warning systems” (EWSs) are being developed to meet these needs. Our study explores the utility of two commonly collected data sources—pre-course concept inventory scores and longitudinal clicker scores—for use in EWS, specifically, in determining the time points at which robust predictions of student success can first be established. The pre-course diagnostic assessments, administered to 287 students, included two concept inventories and one attitude assessment. Clicker question scores were also obtained for each of the 37 class sessions. Additionally, student characteristics (sex, ethnicity, and English facility) were gathered in a survey. Our analyses revealed that all variables were predictive of final grades. The correlation of the first 3 weeks of clicker scores with final grades was 0.53, suggesting that this set of variables could be used in an EWS starting at the third week. We also used group-based trajectory models to assess whether trajectory patterns were homogeneous in the class. The trajectory analysis identified three distinct clicker performance patterns that were also significant predictors of final grade. Trajectory analyses of clicker scores, student characteristics, and pre-course diagnostic assessment appear to be valuable data sources for EWS, although further studies in a diversity of instructional contexts are warranted.