Prediction of Confusion Attempting Algebra Homework in an Intelligent Tutoring System through Machine Learning Techniques for Educational Sustainable Development

Incorporating substantial, sustainable development issues into teaching and learning is the ultimate task of Education for Sustainable Development (ESD). The purpose of our study was to identify the confused students who had failed to master the skill(s) given by the tutors as homework using the Intelligent Tutoring System (ITS). We have focused ASSISTments, an ITS in this study, and scrutinized the skill-builder data using machine learning techniques and methods. We used seven candidate models including: Naïve Bayes (NB), Generalized Linear Model (GLM), Logistic Regression (LR), Deep Learning (DL), Decision Tree (DT), Random Forest (RF), and Gradient Boosted Trees (XGBoost). We trained, validated, and tested learning algorithms, performed stratified cross-validation, and measured the performance of the models through various performance metrics, i.e., ROC (Receiver Operating Characteristic), Accuracy, Precision, Recall, F-Measure, Sensitivity, and Specificity. We found RF, GLM, XGBoost, and DL were high accuracy-achieving classifiers. However, other perceptions such as detecting unexplored features that might be related to the forecasting of outputs can also boost the accuracy of the prediction model. Through machine learning methods, we identified the group of students that were confused when attempting the homework exercise, to help foster their knowledge and talent to play a vital role in environmental development.

[1]  Vernon C. Smith,et al.  Predictive Modeling to Forecast Student Outcomes and Drive Effective Interventions in Online Community College Courses , 2012 .

[2]  Emily R. Fyfe,et al.  Providing feedback on computer-based algebra homework in middle-school classrooms , 2016, Comput. Hum. Behav..

[3]  V. Aleven,et al.  Rapid Authoring of Intelligent Tutors for Real-World and Experimental Use , 2006, Sixth IEEE International Conference on Advanced Learning Technologies (ICALT'06).

[4]  Reva Freedman Atlas: A Plan Manager for Mixed-Initiative, Multimodal Dialogue , 1999 .

[5]  Neil T. Heffernan,et al.  Towards an Understanding of Affect and Knowledge from Student Interaction with an Intelligent Tutoring System , 2013, AIED.

[6]  Robert Gilmore Pontius,et al.  The total operating characteristic to measure diagnostic ability for multiple thresholds , 2014, Int. J. Geogr. Inf. Sci..

[7]  E. Zaluska,et al.  Predicting attrition from Massive Open Online Courses in FutureLearn and edX , 2017 .

[8]  Mingyu Feng,et al.  Online Mathematics Homework Increases Student Achievement , 2016 .

[9]  Ryan Shaun Joazeiro de Baker,et al.  Exploring the Relationship between Novice Programmer Confusion and Achievement , 2011, ACII.

[10]  Cristina Conati,et al.  Procedural Help in Andes: Generating Hints Using a Bayesian Network Student Model , 1998, AAAI/IAAI.

[11]  Wu Zhang,et al.  Using machine learning to predict student difficulties from learning session data , 2018, Artificial Intelligence Review.

[12]  Ming Yang,et al.  Detection of Left-Sided and Right-Sided Hearing Loss via Fractional Fourier Transform , 2016, Entropy.

[13]  Zachary A. Pardos,et al.  Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes , 2013, LAK '13.

[14]  Vincent Aleven,et al.  Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. , 2012, EDM 2012.

[15]  Keeley A. Crockett,et al.  On predicting learning styles in conversational intelligent tutoring systems using fuzzy decision trees , 2017, Int. J. Hum. Comput. Stud..

[16]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[17]  Wanli Xing,et al.  Dropout Prediction in MOOCs: Using Deep Learning for Personalized Intervention , 2019 .

[18]  Xinyu Zhang,et al.  Cloud Model Approach for Lateral Control of Intelligent Vehicle Systems , 2016, Sci. Program..

[19]  N. Heffernan,et al.  Tutor Modeling vs . Student Modeling , 2012 .

[20]  Jianqiang Wang,et al.  Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment , 2018, IEEE Transactions on Industrial Informatics.

[21]  Kinshuk,et al.  Computer aided learning for entry level accountancy students , 1996 .

[22]  C. Y. Peng,et al.  An Introduction to Logistic Regression Analysis and Reporting , 2002 .

[23]  D. H. Vu,et al.  A variance inflation factor and backward elimination based robust regression model for forecasting monthly electricity demand using climatic variables , 2015 .

[24]  Hua Li,et al.  Dropout prediction in MOOCs using behavior features and multi-view semi-supervised learning , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[25]  Yudong Zhang,et al.  Dual-Tree Complex Wavelet Transform and Twin Support Vector Machine for Pathological Brain Detection , 2016 .

[26]  Cristina Conati,et al.  Empirically building and evaluating a probabilistic model of user affect , 2009, User Modeling and User-Adapted Interaction.

[27]  Arthur C. Graesser,et al.  Interventions to Regulate Confusion during Learning , 2012, ITS.

[28]  C. Metz Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[29]  Deyi Li,et al.  A Hardware Platform Framework for an Intelligent Vehicle Based on a Driving Brain , 2018, Engineering.

[30]  Neil T. Heffernan,et al.  The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching , 2014, International Journal of Artificial Intelligence in Education.

[31]  Matthew D. Dailey,et al.  Feedback during Web-Based Homework: The Role of Hints , 2011, AIED.

[32]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[33]  Naman K. Gupta,et al.  Understanding Instructional Support Needs of Emerging Internet Users for Web-based Information Seeking , 2010, EDM 2010.

[34]  A. Graesser,et al.  Confusion can be beneficial for learning. , 2014 .

[35]  Neil T. Heffernan,et al.  Estimating the Effect of Web-Based Homework , 2013, AIED Workshops.

[36]  Naomi J. Aldrich,et al.  Does Discovery-Based Instruction Enhance Learning?. , 2011 .

[37]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[38]  Ryan Shaun Joazeiro de Baker,et al.  Classroom activities and off-task behavior in elementary school children , 2013, CogSci.

[39]  T. Goetz,et al.  Academic Emotions in Students' Self-Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research , 2002 .

[40]  J. Hattie,et al.  The Power of Feedback , 2007 .

[41]  Olusola O. Adesope,et al.  Intelligent tutoring systems and learning outcomes: A meta-analysis , 2014 .

[42]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[43]  Robert A. Cribbie,et al.  The gamma generalized linear model, log transformation, and the robust Yuen-Welch test for analyzing group means with skewed and heteroscedastic data , 2019, Commun. Stat. Simul. Comput..

[44]  R. S. Bichkar,et al.  Performance Prediction of Engineering Students using Decision Trees , 2011 .

[45]  Mary A. Mark,et al.  An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City” , 2015, International Journal of Artificial Intelligence in Education.