Detection of Subject Attention in an Active Environment Through Facial Expressions Using Deep Learning Techniques and Computer Vision

This research aims for investigation of workers in an industrial environment and can be used as an alternate for monitoring an attention of operator in real-time. Detection of attentiveness and non-attentiveness of people working in an industry could help to identify the weaknesses and strengths of any industrial organization. Human factor is the main and the most critical part of any industrial organization. As a special case, we have established how to detect student attention in the classroom using deep learning techniques along with computer vision. Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) are used to extract the percentage of attentiveness and non-attentiveness of students based on the student emotions in the classroom. We used the FER-2013 data set for this paper. As per the study, human has finite number of emotions. So, it is easy if we include some emotions in an attentive (Happy, Anger, Surprised and Neutral) domain and some emotions in non-attentive (Sad, Fear and Disgust) domain. This will help the teacher in a way that he can easily evaluate his class attentiveness. On another side, it is also the evaluation of the teacher’s teaching methodology because if the students are engaged in his lecture it means his teaching methodology is good and if most of the students are not engaged then the teacher needs to revise his methodology of teaching in order to engage his class during the lecture.

[1]  Ahmad Salman,et al.  Real-time fish detection in complex backgrounds using probabilistic background modelling , 2019, Ecol. Informatics.

[2]  Andrej Kosir,et al.  Predicting students’ attention in the classroom from Kinect facial and body features , 2017, EURASIP J. Image Video Process..

[3]  Wei Wang,et al.  Instance-Aware Image and Sentence Matching with Selective Multimodal LSTM , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Lakshmi Priya Gg,et al.  Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric , 2016 .

[5]  David C. Novak,et al.  Examining the Affects of Student Multitasking with Laptops During the Lecture , 2010, J. Inf. Syst. Educ..

[6]  Marcus Liwicki,et al.  Document Image Binarization using LSTM: A Sequence Learning Approach , 2015, HIP@ICDAR.

[7]  Bernard Ghanem,et al.  Representation learning with deep extreme learning machines for efficient image set classification , 2016, Neural Computing and Applications.

[8]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[9]  Charles B. Hodges,et al.  Modeling Students' Attention in the Classroom using Eyetrackers , 2019, ACM Southeast Regional Conference.

[10]  So-Young Park,et al.  Student engagement and classroom variables in improving mathematics achievement , 2005 .

[11]  E. Skinner,et al.  Developmental Dynamics of Student Engagement, Coping, and Everyday Resilience , 2012 .

[12]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[13]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[14]  Muhammad Imran Malik,et al.  Automatic fish species classification in underwater videos: exploiting pre-trained deep neural network models to compensate for limited labelled data , 2018 .