Students’ behavior mining in e-learning environment using cognitive processes with information technologies

Rapid growth and recent developments in education sector and information technologies have promoted E-learning and collaborative sessions among the learning communities and business incubator centers. Traditional practices are being replaced with webinars (live online classes) E-Quizes (online testing) and video lectures for effective learning and performance evaluation. These E-learning methods use sensors and multimedia tools to contribute in resource sharing, social networking, interactivity and corporate trainings. While, artificial intelligence tools are also being integrated into various industries and organizations for students’ engagement and adaptability towards the digital world. Predicting students’ behaviors and providing intelligent feedbacks is an important parameter in the E-learning domain. To optimize students’ behaviors in virtual environments, we have proposed an idea of embedding cognitive processes into information technologies. This paper presents hybrid spatio-temporal features for student behavior recognition (SBR) system that recognizes student-student behaviors from sequences of digital images. The proposed SBR system segments student silhouettes using neighboring data points observation and extracts co-occurring robust spatio-temporal features having full body and key body points techniques. Then, artificial neural network is used to measure student interactions taken from UT-Interaction and classroom behaviors datasets. Finally a survey is performed to evaluate the effectiveness of video based interactive learning using proposed SBR system.

[1]  Hongtao Huang.  A Simplified Image Recognition Algorithm Based on Simple Scenarios , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[2]  Tae-Seong Kim,et al.  Human Activity Recognition via Recognized Body Parts of Human Depth Silhouettes for Residents Monitoring Services at Smart Home , 2013 .

[3]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Joris De Schutter,et al.  An adaptable system for RGB-D based human body detection and pose estimation , 2014, J. Vis. Commun. Image Represent..

[5]  Ken Chen,et al.  Research on Speech Emotion Recognition System in E-Learning , 2007, International Conference on Computational Science.

[6]  Ahmad Jalal,et al.  Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[7]  Gang Xu,et al.  A new image recognition algorithm based on skeleton , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Ahmad Jalal,et al.  Security Enhancement for E-Learning Portal , 2008 .

[9]  Maulahikmah Galinium,et al.  Implementation of face recognition algorithm for biometrics based time attendance system , 2014, 2014 International Conference on ICT For Smart Society (ICISS).

[10]  Xiaodong Yang,et al.  Super Normal Vector for Activity Recognition Using Depth Sequences , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Daijin Kim,et al.  Robust human activity recognition from depth video using spatiotemporal multi-fused features , 2017, Pattern Recognit..

[12]  Song Wang,et al.  Local-based active classification of test report to assist crowdsourced testing , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[13]  Hashim Yasin,et al.  Human movement Recognition using Euclidean Distance: A tricky approach , 2010, 2010 3rd International Congress on Image and Signal Processing.

[14]  A. Kolarov,et al.  Threshold based admission policies for multi-rate services in the DECT system , 1995, Proceedings of 6th International Symposium on Personal, Indoor and Mobile Radio Communications.

[15]  Wei Liang,et al.  Recognising human interaction from videos by a discriminative model , 2014, IET Comput. Vis..

[16]  I-Cheng Chang,et al.  Pictorial Structures Model based human interaction recognition , 2014, 2014 International Conference on Machine Learning and Cybernetics.

[17]  I. D. Longstaff,et al.  Recent developments in the use of the co-occurrence matrix for texture recognition , 1997, Proceedings of 13th International Conference on Digital Signal Processing.

[18]  He Zhang,et al.  Fast image segmentation method based on threshold , 2008, 2008 Chinese Control and Decision Conference.

[19]  D. S. Yeung,et al.  A weighted fuzzy reasoning and its corresponding neural network , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[20]  Alois Knoll,et al.  Social behavior recognition using body posture and head pose for human-robot interaction , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Tijo Thomas,et al.  Video summarization by clustering using euclidean distance , 2011, 2011 International Conference on Signal Processing, Communication, Computing and Networking Technologies.

[22]  Peter Willett,et al.  Classification with a combined information test , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[23]  Kaiming He,et al.  Detecting and Recognizing Human-Object Interactions , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Ken Eguro,et al.  Random decision tree body part recognition using FPGAs , 2012, 22nd International Conference on Field Programmable Logic and Applications (FPL).

[25]  Janne Heikkilä,et al.  Body part segmentation of noisy human silhouette images , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[26]  Tatsuya Fujii,et al.  Gesture recognition system for Human-Robot Interaction and its application to robotic service task , 2014 .

[27]  Unsang Park,et al.  Compositional interaction descriptor for human interaction recognition , 2017, Neurocomputing.

[28]  Mala John,et al.  Human interaction recognition through deep learning network , 2016, 2016 IEEE International Carnahan Conference on Security Technology (ICCST).

[29]  M. Mejia-Lavalle,et al.  Feature Selection with Interactions for Continuous Attributes and Discrete Class , 2007, Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007).

[30]  Othman O. Khalifa,et al.  Automated daily human activity recognition for video surveillance using neural network , 2017, 2017 IEEE 4th International Conference on Smart Instrumentation, Measurement and Application (ICSIMA).

[31]  Weitao Lu,et al.  A Histogram Descriptor Based on Co-occurrence Matrix and its Application in Cloud Image Indexing and Retrieval , 2009, 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[32]  Meinard Müller,et al.  Motion templates for automatic classification and retrieval of motion capture data , 2006, SCA '06.

[33]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[34]  Gwo-Jen Hwang,et al.  A context-aware ubiquitous learning environment for conducting complex science experiments , 2009, Comput. Educ..

[35]  Jake K. Aggarwal,et al.  Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Jing Liu,et al.  A improved silhouette tracking approach integrating particle filter with graph cuts , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Andrej Kosir,et al.  Predicting students’ attention in the classroom from Kinect facial and body features , 2017, EURASIP J. Image Video Process..

[38]  Chiranjoy Chattopadhyay,et al.  Supervised framework for automatic recognition and retrieval of interaction: a framework for classification and retrieving videos with similar human interactions , 2016, IET Comput. Vis..

[39]  Lahcen Oughdir,et al.  A fuzzy classification approach for learning style prediction based on web mining technique in e-learning environments , 2018, Education and Information Technologies.

[40]  M. Turcanik,et al.  Network routing by artificial neural network , 2012, 2012 Military Communications and Information Systems Conference (MCC).

[41]  Hadi Moradi,et al.  An empirical study of using sequential behavior pattern mining approach to predict learning styles , 2017, Education and Information Technologies.

[42]  Daijin Kim,et al.  Ridge body parts features for human pose estimation and recognition from RGB-D video data , 2014, Fifth International Conference on Computing, Communications and Networking Technologies (ICCCNT).

[43]  Ahmad Jalal,et al.  Facial Expression Recognition in Image Sequences Using 1D Transform and Gabor Wavelet Transform , 2018, 2018 International Conference on Applied and Engineering Mathematics (ICAEM).

[44]  Yannick Benezeth,et al.  Human Interaction Recognition Based on the Co-occurrence of Visual Words , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[45]  Xin Zhao,et al.  Human action recognition based on semi-supervised discriminant analysis with global constraint , 2013, Neurocomputing.

[46]  Michael A. Shanblatt,et al.  A two-phase optimization neural network , 1992, IEEE Trans. Neural Networks.

[47]  Bin Fan,et al.  Pose estimation of human body based on silhouette images , 2004, International Conference on Information Acquisition, 2004. Proceedings..

[48]  Osman Sabancı,et al.  The Recognition and Behavior Management of Students With Talented and Gifted in an Inclusive Education Environment , 2018, Journal of Education and Training Studies.

[49]  Daijin Kim,et al.  A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments , 2014, Sensors.

[50]  Zhiyu Qin,et al.  A kind of method for selection of optimum threshold for segmentation of digital color plane image , 2008, 2008 9th International Conference on Computer-Aided Industrial Design and Conceptual Design.

[51]  Hyo-Sung Ahn,et al.  Distance-based formation control using euclidean distance dynamics matrix: Three-agent case , 2011, Proceedings of the 2011 American Control Conference.

[52]  Li Fei-Fei,et al.  Scaling Human-Object Interaction Recognition Through Zero-Shot Learning , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[53]  Jun Zhang,et al.  Semi-supervised and Compound Classification of Network Traffic , 2012, 2012 32nd International Conference on Distributed Computing Systems Workshops.

[54]  Jirí Síma,et al.  Neural networks between integer and rational weights , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[55]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering , 2013 .

[56]  Tao Lu,et al.  Activity Recognition in Ubiquitous Learning Environment , 2012 .

[57]  K. Fazekas,et al.  Threshold procedures and image segmentation , 2005, 47th International Symposium ELMAR, 2005..