Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance

With the gradual adoption of multimedia technologies in curriculum designs, the need has increased for in-depth studies that explore how different presentation techniques for multimedia materials affect learner emotions and learner performance. This study employed the emWave system, a stress detector for emotional states that was developed by the Institute of HeartMath for measuring changes in learner emotional states when presented with different multimedia materials with the same learning content. By analyzing the collected emotional data and assessment of learning performance, this study explores how different multimedia learning materials affect learning emotions, and ultimately, learning performance. Preliminary results show that the video-based multimedia material generates the best learning performance and most positive emotion among three types of multimedia materials assessed in the study. Moreover, a partial correlation exists between negative learning emotion and learning performance. This study confirms that simultaneously considering pretest score and negative emotion can predict learning performance of learners who use video-based multimedia material for learning. It also finds significant gender difference in learner emotional states while using different multimedia materials for learning: female learners in this study are more easily affected by different multimedia material than male learners.