An Automatic Recognition Method for Students' Classroom Behaviors Based on Image Processing

Received: 17 December 2019 Accepted: 29 March 2020 The classroom behaviors of students are the key objects in teaching analysis. It is very important to quantify such behaviors in an intuitive and dynamic manner. This paper summarizes the typical classroom behaviors of students, and specifies the steps to preprocess the collected sample images on these behaviors. Then, it is decided to discriminate students’ classroom behaviors by head poses and facial expressions. Next, a positioning method for facial feature points was developed based on deep convolutional neural network (D-CNN) and cascading, and the head poses and facial expressions were analyzed and recognized. Our method was compared with other facial expression recognition algorithms. The results show that our method is more robust and accurate than the contrastive algorithms.

[1]  Andrea F. Abate,et al.  Near Real-Time Three Axis Head Pose Estimation Without Training , 2019, IEEE Access.

[2]  Alexandrina Rogozan,et al.  Driver head pose estimation using efficient descriptor fusion , 2016, EURASIP J. Image Video Process..

[3]  Martin Kampel,et al.  Facial Expression Recognition using Convolutional Neural Networks: State of the Art , 2016, ArXiv.

[4]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[5]  Anagha Kulkarni,et al.  Real Time Facial Expression Recognition using Deep Learning , 2019, SSRN Electronic Journal.

[6]  Rodrigo Ferreira Berriel,et al.  Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network , 2017, 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[7]  Simone Calderara,et al.  Face-from-Depth for Head Pose Estimation on Depth Images , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Angelo Cangelosi,et al.  Head pose estimation in the wild using Convolutional Neural Networks and adaptive gradient methods , 2017, Pattern Recognit..

[9]  Massimo Mauro,et al.  Head pose estimation through multi-class face segmentation , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[10]  Maozhen Li,et al.  Real time facial expression recognition app development on mobile phones , 2016, 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[11]  James M. Rehg,et al.  Fine-Grained Head Pose Estimation Without Keypoints , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Jiri Matas,et al.  Cumulative attribute space regression for head pose estimation and color constancy , 2019, Pattern Recognit..

[13]  Debora Gil,et al.  Continuous Head Pose Estimation Using Manifold Subspace Embedding and Multivariate Regression , 2018, IEEE Access.

[14]  Je-Won Kang,et al.  Combining random forest with multi-block local binary pattern feature selection for multiclass head pose estimation , 2017, PloS one.

[15]  David Declercq,et al.  3D facial expression recognition using kernel methods on Riemannian manifold , 2017, Eng. Appl. Artif. Intell..

[16]  Benyamin Kusumoputro,et al.  Fuzzy vector implementation on manifold embedding for head pose estimation with degraded images using fuzzy nearest distance , 2017, ICCIP '17.

[17]  Mohammad H. Mahoor,et al.  Going deeper in facial expression recognition using deep neural networks , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  P. J. Narayanan,et al.  Nose, Eyes and Ears: Head Pose Estimation by Locating Facial Keypoints , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Yong Man Ro,et al.  Multi-Objective Based Spatio-Temporal Feature Representation Learning Robust to Expression Intensity Variations for Facial Expression Recognition , 2019, IEEE Transactions on Affective Computing.