Student Engagement Dataset

A major challenge for online learning is the inability of systems to support student emotion and to maintain student engagement. In response to this challenge, computer vision has become an embedded feature in some instructional applications. In this paper, we propose a video dataset of college students solving math problems on the educational platform MathSpring.org with a front facing camera collecting visual feedback of student gestures. The video dataset is annotated to indicate whether students’ attention at specific frames is engaged or wandering. In addition, we train baselines for a computer vision module that determines the extent of student engagement during remote learning. Baselines include state-of-the-art deep learning image classifiers and traditional conditional and logistic regression for head pose estimation. We then incorporate a gaze baseline into the MathSpring learning platform, and we are evaluating its performance with the currently implemented approach.

[1]  A. Graesser,et al.  Confusion can be beneficial for learning. , 2014 .

[2]  A. Graesser.  Deeper Learning With Advances in Discourse Science and Technology , 2015 .

[3]  Salma Kammoun Jarraya,et al.  Student behavior analysis to measure engagement levels in online learning environments , 2021, Signal Image Video Process..

[4]  Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning , 2019, ArXiv.

[5]  M. Ali Akber Dewan,et al.  Engagement detection in online learning: a review , 2019, Smart Learning Environments.

[6]  Deborah Rivas‐Drake,et al.  Transformative Social and Emotional Learning (SEL): Toward SEL in Service of Educational Equity and Excellence , 2019, Educational Psychologist.

[7]  Kasia Muldner,et al.  The Impact of Animated Pedagogical Agents on Girls' and Boys' Emotions, Attitudes, Behaviors and Learning , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[8]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[9]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[10]  Abhinav Dhall,et al.  Prediction and Localization of Student Engagement in the Wild , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[11]  Gengming Zhu,et al.  Joint Face Detection and Facial Expression Recognition with MTCNN , 2017, 2017 4th International Conference on Information Science and Control Engineering (ICISCE).

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Guoying Zhao,et al.  Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Yung-Yu Chuang,et al.  FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Beverly Park Woolf,et al.  Ella Me Ayudó (She Helped Me): Supporting Hispanic and English Language Learners in a Math ITS , 2018, AIED.

[17]  Kasia Muldner,et al.  Gender Differences in the Use and Benefit of Advanced Learning Technologies for Mathematics. , 2013 .

[18]  Abhay Gupta,et al.  DAiSEE: Towards User Engagement Recognition in the Wild. , 2016, 1609.01885.

[19]  Guoying Zhao,et al.  Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond , 2018, International Journal of Computer Vision.

[20]  Kasia Muldner,et al.  A Multimedia Adaptive Tutoring System for Mathematics that Addresses Cognition, Metacognition and Affect , 2014, International Journal of Artificial Intelligence in Education.

[21]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[22]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[23]  Christoph Lofi,et al.  Webcam-based Attention Tracking in Online Learning: A Feasibility Study , 2018, IUI.

[24]  S. D’Mello.  Gaze-Based Attention-Aware Cyberlearning Technologies , 2018, Mind, Brain and Technology.

[25]  Ali Abedi,et al.  Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network , 2021, 2021 18th Conference on Robots and Vision (CRV).

[26]  Sanya Liu,et al.  Fine-grained Engagement Recognition in Online Learning Environment , 2019, 2019 IEEE 9th International Conference on Electronics Information and Emergency Communication (ICEIEC).

[27]  Dinesh Babu Jayagopi,et al.  Predicting student engagement in classrooms using facial behavioral cues , 2017, MIE@ICMI.

[28]  Daniel McDuff,et al.  Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected "In-the-Wild" , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[29]  L. S. Vygotskiĭ,et al.  Mind in society : the development of higher psychological processes , 1978 .

[30]  Christoph Lofi,et al.  IntelliEye: Enhancing MOOC Learners' Video Watching Experience through Real-Time Attention Tracking , 2018, HT.

[31]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Guoying Zhao,et al.  Recognition of Affect in the Wild Using Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Angela Stewart,et al.  Gaze-based Detection of Mind Wandering during Lecture Viewing , 2017, EDM.

[34]  W. Kintsch,et al.  Are Good Texts Always Better? Interactions of Text Coherence, Background Knowledge, and Levels of Understanding in Learning From Text , 1996 .