Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses

The data about high students' failure rates in introductory programming courses have been alarming many educators, raising a number of important questions regarding prediction aspects. In this paper, we present a comparative study on the effectiveness of educational data mining techniques to early predict students likely to fail in introductory programming courses. Although several works have analyzed these techniques to identify students' academic failures, our study differs from existing ones as follows: (i) we investigate the effectiveness of such techniques to identify students likely to fail at early enough stage for action to be taken to reduce the failure rate; (ii) we analyse the impact of data preprocessing and algorithms fine-tuning tasks, on the effectiveness of the mentioned techniques. In our study we evaluated the effectiveness of four prediction techniques on two different and independent data sources on introductory programming courses available from a Brazilian Public University: one comes from distance education and the other from on-campus. The results showed that the techniques analyzed in our study are able to early identify students likely to fail, the effectiveness of some of these techniques is improved after applying the data preprocessing and/or algorithms fine-tuning, and the support vector machine technique outperforms the other ones in a statistically significant way. Display Omitted We evaluated effectiveness of mining techniques to early predict students' failures.We collected data from two independent introductory programming courses.The results showed that the Support Vector Machine reached highest effectiveness.

[1]  Carlos Márquez-Vera,et al.  Predicting School Failure and Dropout by Using Data Mining Techniques , 2013, IEEE Journal of Latin-American Learning Technologies.

[2]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[3]  Sven F. Crone,et al.  The impact of preprocessing on data mining: An evaluation of classifier sensitivity in direct marketing , 2006, Eur. J. Oper. Res..

[4]  Eliseo Reategui,et al.  Detection and assistance to students who show frustration in learning of algorithms , 2013, 2013 IEEE Frontiers in Education Conference (FIE).

[5]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[6]  Li Zhu,et al.  Data Mining on Imbalanced Data Sets , 2008, 2008 International Conference on Advanced Computer Theory and Engineering.

[7]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[8]  Carlos R. Minussi,et al.  Prediction of school dropout risk group using Neural Network , 2013, 2013 Federated Conference on Computer Science and Information Systems.

[9]  Hana Bydzovská,et al.  A Comparative Analysis of Techniques for Predicting Student Performance , 2016, EDM.

[10]  Frederick W. B. Li,et al.  Failure rates in introductory programming revisited , 2014, ITiCSE '14.

[11]  Steven L. Salzberg,et al.  Book Review: C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993 , 1994, Machine Learning.

[12]  Frederick W. B. Li,et al.  Predicting Performance in an Introductory Programming Course by Logging and Analyzing Student Programming Behavior , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[13]  Jens Bennedsen,et al.  Failure rates in introductory programming , 2007, SGCS.

[14]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[15]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[16]  Sérgio Manuel Serra da Cruz,et al.  WAVE: an architecture for predicting dropout in undergraduate courses using EDM , 2014, SAC.

[17]  Ian Witten,et al.  Data Mining , 2000 .

[18]  Xiaohua Hu.  DB-HReduction: A data preprocessing algorithm for data mining applications , 2003, Appl. Math. Lett..

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[20]  Stefan Müller,et al.  Pentaho Data Integration , 2014 .

[21]  Abhishek Singhal,et al.  PREDICTION & WARNING: a method to improve student's performance , 2014, SOEN.

[22]  Margaret Hamilton,et al.  Predictors of success in a first programming course , 2006 .

[23]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[24]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[25]  Lubos Popelínský,et al.  Predicting drop-out from social behaviour of students , 2012, EDM.

[26]  Hemerson Pistori,et al.  SVM with Stochastic Parameter Selection for Bovine Leather Defect Classification , 2007, PSIVT.

[27]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[28]  F. Imbault,et al.  A stochastic optimization approach for parameter tuning of support vector machines , 2004, ICPR 2004.

[29]  Witold Pedrycz,et al.  Data mining tasks and methods: Classification: neural network approaches , 2002 .

[30]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[31]  Erman Yukselturk,et al.  Predicting Dropout Student: An Application of Data Mining Methods in an Online Education Program , 2014 .

[32]  Fadhilah Ahmad,et al.  The Prediction of Students' Academic Performance Using Classification Data Mining Techniques , 2015 .

[33]  Steven Salzberg,et al.  Programs for Machine Learning , 2004 .

[34]  Erkan Er.  Identifying At-Risk Students Using Machine Learning Techniques: A Case Study with IS 100 , 2012 .

[35]  Lindawati,et al.  Fine-Tuning Algorithm Parameters Using the Design of Experiments Approach , 2011, LION.

[36]  David L. Olson,et al.  Advanced Data Mining Techniques , 2008 .

[37]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[38]  Choo-Yee Ting,et al.  Learning Difficulties in Programming Courses: Undergraduates' Perspective and Perception , 2009, 2009 International Conference on Computer Technology and Development.

[39]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[40]  Mohammed J. Zaki.  Data Mining and Analysis: Fundamental Concepts and Algorithms , 2014 .

[41]  F. Hutter,et al.  ParamILS: an automatic algorithm configuration framework , 2009 .