Measuring the Effect of ITS Feedback Messages on Students' Emotions

When an ITS gives supportive, empathetic, or motivational feedback messages to the learner, does it alter the learner’s emotional state, and can the ITS detect the change? We investigated this question on a dataset of n = 36 African-American undergraduate students who interacted with iPad-based cognitive skills training software that issued various feedback messages. Using both automatic facial expression recognition and heart rate sensors, we estimated the effect of the different messages on short-term changes to students’ emotions. Our results indicate that, except for a few specific messages (“Great Job”, and “Good Job”), the evidence for the existence of such effects was meager, and the effect sizes were small. Moreover, for the “Good Job” and “Great Job” actions, the effects can easily be explained by the student having recently scored a point, rather than the feedback itself. This suggests that the emotional impact of such feedback, at least in the particular context of our study, is either very small, or it is undetectable by heart rate or facial expression sensors.