Implementing a Layered Analytic Approach For Real-Time Modeling of Students' Scientific Understanding

We have developed layered analytic models of how high school and university students construct, modify and retain problem solving strategies as they learn to solve science problems online. First, item response theory modeling is used to provide continually refined estimates of problem solving ability as students solve a series of simulations. In parallel, the strategies students apply are modeled by self-organizing artificial neural network analysis, using the actions that students take during problem solving as the classifying inputs. This results in strategy maps detailing the qualitative and quantitative differences among problem solving approaches. Learning trajectories across sequences of student performances are developed by applying Hidden Markov Modeling to stochastically model problem solving progress through the strategic stages in the learning process. Using this layered analytical approach we have found that students quickly adopt preferential problem solving strategies, and continue to use them up to four months later. Furthermore, the approach has shown that students working in groups solve a higher percentage of the problems, stabilize their strategic approaches quicker, and use a more limited repertoire of strategies than students working alone. In this paper, we also describe our ongoing and future work in developing an online collaborative learning environment that both models the group interaction and identifies which individual student contributions might contribute to increased achievement.