A Framework to Characterize Student Difficulties in Learning Inference from a Simulation-Based Approach.

Although hypothesis testing is ubiquitous in data analysis, research suggests it is commonly misunderstood. Simulation-based inference methods have potential to make student thinking visible, thus providing a valuable lens to analyze developing conceptions about inference. This paper identifies difficulties made visible through simulation-based methods and introduces a framework to characterize the conceptions behind those difficulties. Using the framework, difficulties can be described largely in terms of two challenges. First, students struggle to coordinate the multi-level scheme, which includes the population or true underlying relationship, the distribution of a single sample, and the distribution of statistics collected from multiple samples. Second, students struggle to coordinate two perspectives: the real world where the sample data were collected, and the hypothetical perspective where the null hypothesis is assumed to be true.